Can You Rely on Meta-analysis? Can You Doubt It? – Education & Teacher Conferences Skip to main content

Can You Rely on Meta-analysis? Can You Doubt It?

Over at his blog Filling the Pail, Greg Ashman likes challenging popular ideas. In a recent post, he takes issue with meta-analysis as a way of analyzing educational research.

In the first place, Ashman argues — in effect —  “garbage in, garbage out.” Combining badly-designed studies with well-designed studies still gives some weight to the badly-designed ones.

Of course, Ashman has some thoughtful suggestions as well.

Why Does It Matter?

Why should we care about such an obscure and complicated statistical technique?

Meta-analysis matters because we pay so much attention to it.

For instance: just a month ago, a pair of meta-analyses about Mindset Theory set off another round of anxiety. Edu-twitter lit right up with thoughtful scholars wondering if we should stop focusing so much on the right kind of praise.

Or: I frequently rebut claims about working memory training by citing this well-known meta-analysis by Melby-Lervag and Hulme.

If we’re going to rely so much on this technique, we should be clear-minded about its strengths and its weaknesses.


Recent Blogs

Executive Functions: Setting the Record Straight
Guest Post

If you’ve been in education long enough, you’ve probably heard...

ChatGPT and the Future of AI—The Deep Language Revolution by Terrence Sejnowski
Erik Jahner, PhD
Erik Jahner, PhD

Every semester, a few essays arrive so polished they almost...

Interleaving Meets Prequestioning: A Powerful Combination for Learning?
Andrew Watson
Andrew Watson

A few weeks ago, I wrote about an intriguing research...