Do We Actually Know What We Think We Know?

Teachers worry a lot about stress. For that reason, this blog regularly scans research updates for useful news about stress and stress reduction techniques.

replication crisis

(In fact, I posted about potential benefits of stress just two weeks ago.)

One of our favorite studies offers a surprisingly simple approach to stress reduction: writing about it.

We like this study so much, we’ve posted about it twice: here and here.

So here’s an important question: do we actually know that the strategy works?

Put in a different way: how do psychologists and neuroscientists know what they claim to know?

Replication, Replication Crisis

To answer this question, we can describe research methodology. The ritual of science publication, in fact, requires almost gruesomely soporific descriptions of EXACTLY what the scientists did.

(What computer model did they use? What software version did they use? What size was the font? How far away from the computer did the participants sit? You get the idea…)

Here’s another answer: to be sure we know what we know, we double check. When one group of researchers arrives at a conclusion — especially an exciting or novel conclusion — other researchers see if they get the same result.

To be fancy, we can say they try to “replicate” the original result. As we get more replications, we feel more confident that we know.

Here’s the problem: psychology is facing a replication crisis.

A Case in Point: Writing About Stress

In recent years, more and more psychology studies just don’t replicate. We thought we knew, but now we’re not so sure.

For example: that study we love? The one about “reducing stress by writing about it?” A new replication crisis project tried to replicate it, and got no results.

In this replication, the benefits of writing about stress were…nada.

In fact, this group of researchers tried to replicate all 21 studies published in Nature and Science between 2010 and 2015. Depending on how you count, 13 of them replicated. That means that eight of them did not replicate.

YIKES.

What’s a Teacher to Do?

This news might seem like a disaster. If so much research doesn’t replicate, we might be tempted to stop relying on science in our teaching.

Although understandable, this dis-spirited conclusion goes too far. Instead, the replication crisis should remind us of two essential points:

First: teachers should adopt teaching ideas that have lots of research support.

If you see one study suggesting that (say) chewing gum helps students convert fractions to decimals, DO NOT adopt that technique. At a minimum, you should look to see if other researchers have replicated that finding.

In fact, you’d like several researchers to have explored this idea — preferably in different grades and schools, with different research paradigms.

(You should also weigh the other pluses and minuses. All that gum chewing might be really distracting.)

Second: we should be ready to change our minds. If a wave of research points one direction, we might adopt a particular strategy.

However, if better research over longer periods of time discounts those findings, then we have to be willing to try something else.

The best-known example of this problem: “learning styles.” Early research suggested that they might exist, but the vast weight of evidence shows clearly that they don’t.

In brief: we should adopt evidence-based theories only if we’re willing to let go of them as further evidence dictates.

Final Thoughts

But what about that stress-reduction technique? Do we have to give it up, now that it didn’t replicate?

That’s a surprisingly complicated question.

True enough: this attempt at replication failed.

However, in the original study, researchers Ramirez and Beilock tried their technique twice — just to be sure they had it right.

In fact, they tried it twice in the psychology lab and twice more in a local high school.

And, a similar research paradigm arrived at similar results.

We can say that the study has an uneven replication record. That’s not the same thing as “didn’t replicate.”

For these reasons, I think we should put the “writing about stress” strategy in the “we just don’t know” category. It might work. It might not.

It might depend on circumstances that our research paradigms haven’t yet revealed.

When psychology research offers you contradictory advice, think about your own classroom circumstances and make the best decision you can.

That advice is complicated, but it’s certainly not a crisis.

 

tags: / / category: L&B Blog

Leave a Reply

Your email address will not be published. Required fields are marked *