The Deseret News, a local Utah newspaper, today published a story titled Study: Majority of U.S. charter schools perform equal or worse than traditional schools, accompanied by the following infographic:
What’s wrong with this story? While the information conveyed by the headline is, strictly speaking, an accurate reflection of the data, the DesNews is using the headline to seriously mislead the public. Let’s explore an alternate, accurate headline the DesNews could have run to see how they’re misinforming the public with this story.
While the story’s headline is accurate given the data shown in the infographic, the opposite headline is also true. The story could just as accurately have been titled:
“Study: Majority of U.S. charter schools perform equal or better than traditional schools”
How can both these statements be true? The answer is in the statistics. 56% of the charter schools in the study are not significantly different from other public schools in their local market when it comes to student performance in reading. 40% are not significantly different from other public schools in their local market when it comes to student performance in math. To say “majority” in the headline, we only have to get to 51%. The reading scores are already above 51%, and math only needs a few percentage points to reach majority status. So by including the 19% and 31% of charter schools that were significantly lower in reading and math, we get totals of 75% and 71% for schools with “equal or worse performance” in reading or math. So the DesNews’ headline is accurate.
However, the math works the other direction as well. By including the 25% and 29% of charter schools that were significantly better in reading and math, we get the totals of 81% and 69% for schools with “equal or better performance” in reading or math. So the opposite, positive headline would also have been accurate.
When there are two equally accurate – yet opposite – ways of interpreting data, the choice one makes clearly reveals one’s bias. It’s unclear whether the anti-charter school bias in the DesNews story belongs to the reporter or to the paper’s editors. Regardless of the source of the bias, the choice – and it this case, it is clearly a conscious choice – to portray the data in a negative way is disappointing.