Can You Rely on Meta-analysis? Can You Doubt It? – Education & Teacher Conferences Skip to main content

Can You Rely on Meta-analysis? Can You Doubt It?

Over at his blog Filling the Pail, Greg Ashman likes challenging popular ideas. In a recent post, he takes issue with meta-analysis as a way of analyzing educational research.

In the first place, Ashman argues — in effect —  “garbage in, garbage out.” Combining badly-designed studies with well-designed studies still gives some weight to the badly-designed ones.

Of course, Ashman has some thoughtful suggestions as well.

Why Does It Matter?

Why should we care about such an obscure and complicated statistical technique?

Meta-analysis matters because we pay so much attention to it.

For instance: just a month ago, a pair of meta-analyses about Mindset Theory set off another round of anxiety. Edu-twitter lit right up with thoughtful scholars wondering if we should stop focusing so much on the right kind of praise.

Or: I frequently rebut claims about working memory training by citing this well-known meta-analysis by Melby-Lervag and Hulme.

If we’re going to rely so much on this technique, we should be clear-minded about its strengths and its weaknesses.


Recent Blogs

The Shelf-Life of Research: When Education Theories Evolve…
Andrew Watson
Andrew Watson

To most of us, research conclusions have an air of...

Enjoyment or Skill? The Case of Reading [Repost]
Andrew Watson
Andrew Watson

Student motivation has always been a concern; never more so...

Book Review: 10 to 25, by David Yeager
Andrew Watson
Andrew Watson

As long as humans have lived into their teens, adults...