Skip to main content

Can You Rely on Meta-analysis? Can You Doubt It?

Over at his blog Filling the Pail, Greg Ashman likes challenging popular ideas. In a recent post, he takes issue with meta-analysis as a way of analyzing educational research.

In the first place, Ashman argues — in effect —  “garbage in, garbage out.” Combining badly-designed studies with well-designed studies still gives some weight to the badly-designed ones.

Of course, Ashman has some thoughtful suggestions as well.

Why Does It Matter?

Why should we care about such an obscure and complicated statistical technique?

Meta-analysis matters because we pay so much attention to it.

For instance: just a month ago, a pair of meta-analyses about Mindset Theory set off another round of anxiety. Edu-twitter lit right up with thoughtful scholars wondering if we should stop focusing so much on the right kind of praise.

Or: I frequently rebut claims about working memory training by citing this well-known meta-analysis by Melby-Lervag and Hulme.

If we’re going to rely so much on this technique, we should be clear-minded about its strengths and its weaknesses.


Recent Blogs

As We Begin: Dispositions of Mind, Learning, and the Brain in Early Childhood by Tia Henteleff
Erik Jahner, PhD
Erik Jahner, PhD

There’s a line quoted in Tia Henteleff’s As We Begin...

Too Good to be True? The Effect of Tetris on PTSD Symptoms
Andrew Watson
Andrew Watson

Teachers and school leaders turn to research -- in part...

Will Chess Make Me Better at Baseball?
Andrew Watson
Andrew Watson

Imagine for a moment that I'm coaching college baseball. I've...