Can You Rely on Meta-analysis? Can You Doubt It? – Education & Teacher Conferences Skip to main content

Can You Rely on Meta-analysis? Can You Doubt It?

Over at his blog Filling the Pail, Greg Ashman likes challenging popular ideas. In a recent post, he takes issue with meta-analysis as a way of analyzing educational research.

In the first place, Ashman argues — in effect —  “garbage in, garbage out.” Combining badly-designed studies with well-designed studies still gives some weight to the badly-designed ones.

Of course, Ashman has some thoughtful suggestions as well.

Why Does It Matter?

Why should we care about such an obscure and complicated statistical technique?

Meta-analysis matters because we pay so much attention to it.

For instance: just a month ago, a pair of meta-analyses about Mindset Theory set off another round of anxiety. Edu-twitter lit right up with thoughtful scholars wondering if we should stop focusing so much on the right kind of praise.

Or: I frequently rebut claims about working memory training by citing this well-known meta-analysis by Melby-Lervag and Hulme.

If we’re going to rely so much on this technique, we should be clear-minded about its strengths and its weaknesses.


Recent Blogs

Meaningful and Relevant: Engaging Learners in an Era of Distraction by A.J. Juliani
Erik Jahner, PhD
Erik Jahner, PhD

Oh man! Student responses sometimes punch me in the gut!...

Executive Functions “Debunked”? [Reposted]
Andrew Watson
Andrew Watson

A recent study claims to have "debunked" the idea of...

Do Fidget Spinners Help Children with ADHD? [Reposted]
Andrew Watson
Andrew Watson

We've been hearing A LOT about fidget spinners and squishy...