Can You Rely on Meta-analysis? Can You Doubt It? – Education & Teacher Conferences Skip to main content

Can You Rely on Meta-analysis? Can You Doubt It?

Over at his blog Filling the Pail, Greg Ashman likes challenging popular ideas. In a recent post, he takes issue with meta-analysis as a way of analyzing educational research.

In the first place, Ashman argues — in effect —  “garbage in, garbage out.” Combining badly-designed studies with well-designed studies still gives some weight to the badly-designed ones.

Of course, Ashman has some thoughtful suggestions as well.

Why Does It Matter?

Why should we care about such an obscure and complicated statistical technique?

Meta-analysis matters because we pay so much attention to it.

For instance: just a month ago, a pair of meta-analyses about Mindset Theory set off another round of anxiety. Edu-twitter lit right up with thoughtful scholars wondering if we should stop focusing so much on the right kind of praise.

Or: I frequently rebut claims about working memory training by citing this well-known meta-analysis by Melby-Lervag and Hulme.

If we’re going to rely so much on this technique, we should be clear-minded about its strengths and its weaknesses.


Recent Blogs

The Neuroscience Trap: When Skeptics Stop Being Skeptical
Andrew Watson
Andrew Watson

I collaborated on this post with Dr. Cindy Nebel. Her...

I Take It All Back: Research on “Expressive Writing” Has Evolved
Andrew Watson
Andrew Watson

I spend my days giving research-informed advice to teachers and...

The Creativity Choice: The Science of Making Decisions to Turn Ideas into Action by Zorana Ivcevic Pringle
Erik Jahner, PhD
Erik Jahner, PhD

It happened just this morning! I was watching my favorite...