The Deseret News, a local Utah newspaper, today published a story titled Study: Majority of U.S. charter schools perform equal or worse than traditional schools, accompanied by the following infographic:
What’s wrong with this story? While the information conveyed by the headline is, strictly speaking, an accurate reflection of the data, the DesNews is using the headline to seriously mislead the public. Let’s explore an alternate, accurate headline the DesNews could have run to see how they’re misinforming the public with this story.
While the story’s headline is accurate given the data shown in the infographic, the opposite headline is also true. The story could just as accurately have been titled:
“Study: Majority of U.S. charter schools perform equal or better than traditional schools”
How can both these statements be true? The answer is in the statistics. 56% of the charter schools in the study are not significantly different from other public schools in their local market when it comes to student performance in reading. 40% are not significantly different from other public schools in their local market when it comes to student performance in math. To say “majority” in the headline, we only have to get to 51%. The reading scores are already above 51%, and math only needs a few percentage points to reach majority status. So by including the 19% and 31% of charter schools that were significantly lower in reading and math, we get totals of 75% and 71% for schools with “equal or worse performance” in reading or math. So the DesNews’ headline is accurate.
However, the math works the other direction as well. By including the 25% and 29% of charter schools that were significantly better in reading and math, we get the totals of 81% and 69% for schools with “equal or better performance” in reading or math. So the opposite, positive headline would also have been accurate.
When there are two equally accurate – yet opposite – ways of interpreting data, the choice one makes clearly reveals one’s bias. It’s unclear whether the anti-charter school bias in the DesNews story belongs to the reporter or to the paper’s editors. Regardless of the source of the bias, the choice – and it this case, it is clearly a conscious choice – to portray the data in a negative way is disappointing.
Interestingly, you seem to have picked the one anti-charter story out there. Some more:
>>New York Times June 25, 2013 Charter Schools Are Improving, a Study Says
>>Wall Street Journal June 25,2013 Charter Schools Receive a Passing Grade
>>Associated Press June 25, 2013 Study: Minority, Poor Students Gain From Charters
>>Education Week June 25, 2013 Charters Show “Slow and Steady Progress,” Multistate Study Finds
>>Bloomberg June 25, 2013 Stanford Study Says Charter School Children Outperform
>>Huffington Post June 25, 2013 Charter School Performance Study Finds Small Gains
Source: http://credo.stanford.edu/
I have my own suspicions about the study (specifically, the definition of the ‘TPS’ (which would seem to exclude specialized and advanced schools, or magnet schools)) and the ‘virtual twin’ methodology (which has the effect of counting charter school data twice), the statement that “Most researchers agree that the best method of measuring school effectiveness is to look at how much schools aid student academic growth, independent of other possible influences,” and the conversion of this data to ‘days of student growth’. Not that any traditional media would look at any such issues either way.
Interesting article. It shows the inexactness of language and the specious of statistics.