Can You Rely on Meta-analysis? Can You Doubt It? – Education & Teacher Conferences Skip to main content

Can You Rely on Meta-analysis? Can You Doubt It?

Over at his blog Filling the Pail, Greg Ashman likes challenging popular ideas. In a recent post, he takes issue with meta-analysis as a way of analyzing educational research.

In the first place, Ashman argues — in effect —  “garbage in, garbage out.” Combining badly-designed studies with well-designed studies still gives some weight to the badly-designed ones.

Of course, Ashman has some thoughtful suggestions as well.

Why Does It Matter?

Why should we care about such an obscure and complicated statistical technique?

Meta-analysis matters because we pay so much attention to it.

For instance: just a month ago, a pair of meta-analyses about Mindset Theory set off another round of anxiety. Edu-twitter lit right up with thoughtful scholars wondering if we should stop focusing so much on the right kind of praise.

Or: I frequently rebut claims about working memory training by citing this well-known meta-analysis by Melby-Lervag and Hulme.

If we’re going to rely so much on this technique, we should be clear-minded about its strengths and its weaknesses.


Recent Blogs

Interleaving Meets Prequestioning: A Powerful Combination for Learning?
Andrew Watson
Andrew Watson

A few weeks ago, I wrote about an intriguing research...

From Facts to Application: The Surprising Power of Repeated Retrieval
Andrew Watson
Andrew Watson

The more time you've spent in Learning and the Brain...

From Lab to Life: Testing Study Strategies with 2,500+ Real Students
Andrew Watson
Andrew Watson

Research-based conclusions often appeal to us because of their tidy...