(Mis)Understanding Educational Stats

Over at The Anova, Freddie deBoer has a knack for writing about statistical questions and making them not just readable but interesting.

Case in point: he recently explored the New York Times feature about school choice.

Although careful to praise the Times authors for their genuine concern and dedication, he thoughtfully explicates the numerous ways in which their article gets important questions wrong because it doesn’t think its way through statistics carefully enough.

For example: when we say we want students to do better, does that mean we want individual students to rise above the average, or that we want to raise the average for students overall?

As deBoer sees the field, we typically say we want the latter, but focus on (and tell stories about) the former.

DeBoer’s article doesn’t express an opinion about school choice (I’m sure he has one, but he doesn’t tip his hand here). But, it’s an excellent reminder that statistics can help us only so long as we are clear-minded about what they really measure.

As he glumly says in his final paragraph:

It’s not just that we can’t get what we want. It’s that nobody really knows what they’re trying to accomplish.