Author Archives: Andrew Watson

Here’s a remarkable story about potentially falsified research data.

The short version: researchers James Heathers and Nick Brown thought that Nicolas Guéguen’s research findings were both too sexy and too tidy.

Too sexy: Guéguen’s findings regularly made great headlines. For instance, his research shows that men voluntarily assist women who wear their hair loose much more than those who wear it in a ponytail or a bun.

Talk about your gender-roles clickbait!

Too tidy: As Heathers and Brown considered Guéguen’s math, they realized that his numbers were…too round. When Guéguen calculated averages, he had to divide by 30 — because his study groups had 30 people in them.

But, his results often ended in improbably round figures:  1.60, 1.80, 2.80.

Mathematically speaking, that’s possible. But, when you’re dividing by 30, it’s wildly unlikely.

Heathers and Brown have now spent quite a long time — years, in fact — trying to get Guéguen to share his data and/or answer their questions.

As far as I know, they haven’t gotten answers yet. (Here’s a somewhat more recent article.)

What Teachers Should Do

First, keep in mind that scientists are people too. While most strive honorably to follow procedures and discover meaningful findings, a few will always cheat the system and abuse our trust.

That’s true in any profession. It’s true in psychology and neuroscience research. (It might even be true of teachers.)

Second, let this knowledge energize your skepticism.

Here’s my suggestion: never make changes to your classroom work without asking hard questions.

Think about it this way: if someone wanted to sell you something for $10,000, you’d needs lots of confirmation. You wouldn’t take the seller’s word that the product was worth $10k. You’d investigate on your own.

For instance, ask…

Have other researchers gotten similar findings?

How aggressively have they tried to disprove them?

Is the suggestion strongly confirming a bias that everyone already holds?

So: your classroom is worth more than $10,000. You wouldn’t buy an expensive product without asking hard questions. Don’t adopt research suggestions without asking hard questions.

In the meanwhile, I’m going to put my hair up in a bun.

Let’s imagine that you show me research suggesting that students remember the words they draw better than the words they write down.

After some thought…perhaps some experimentation on my own…I decide not to follow this research advice.

Why did I “resist” these research findings? What prompted me to do so?

Education researcher Tim Cain divides possible answers into four categories. The verbs he uses to describe each one are all synonyms. But, he gives each one distinct meaning to distinguish among the possibilities.

And, as you’ll see, three of the four choices sound really bad.

3 Bad Choices

Denial: denialists basically pretend that there is reasonable disagreement on a topic where none really exits. Example: companies that say smoking isn’t bad for your lungs, or historians who pretend the Holocaust didn’t happen.

For the most part, in Cain’s analysis, deniers strive to “protect self-esteem and status.”

Opposition: whereas denialists typically have power and want to protect it, oppositionists don’t have much power, and reject scientific findings that might continue their subjugation.

For instance, I might have rejected the drawing strategy because I didn’t think it worked (see below). But, I might reject it because – as a teacher with relatively little cultural power – I don’t want to be bossed around by scientific researchers – who have more cultural standing than I do.

Rejection: Rejections gets a little complicated. In this model, I accept research findings only if they BOTH help students learn AND make me look good. But, if they don’t hit both targets, I’m not interested.

So, for example, if drawing does help students remember, but doesn’t win me esteem in the faculty room, then I’m just not interested.

As you can see, these first three choices don’t seem very appealing. I’m oversimplifying a bit – but not a lot – to say that teachers who resist research for these reasons are being jerks.

Frankly, I’m feeling a bit stressed right now. Does Cain acknowledge that teachers have any good reasons to resist research findings?

One More?

Indeed, Cain does give us one more choice.

Dissent: if teachers think critically about research, we might see gaps, flaws, or logical leaps. Rather than being driven by the sinister motives outlined above, we might honestly – even insightfully – disagree with the arguments put before us.

Being a researcher, Cain wanted to know: which is it? Why do teachers ultimately decide not to follow researchers’ advice?

Are we protecting the power we have (“denial”)? Fighting to prevent others from getting even more power over us (“opposition”)? Focusing on prestige more than usefulness (“rejection”)?

Or, are we enhancing educational debate by thinking critically (“dissent”)?

The Big Reveal

I’ll cut to the chase: for the most part, Cain finds that we’re in the critical thinking business.

To arrive at this conclusion, Cain worked with several teachers at two schools in northern England. He gave them some research articles, and asked them to try out the researchers’ findings. He then met with them to talk over their work, and interviewed them about their conclusions.

Here’s what he found:

First: teachers ultimately agreed with and accepted significant chunks of the researchers’ conclusions and advice. There didn’t simply reject everything they read and undertook.

Second: at the same time, teachers didn’t see researchers’ conclusions as more important than their own. As Cain puts it:

Essentially, almost all the teachers saw the authority of the published research reports as provisional. They did not see the research as having greater authority than their own experience or other forms of information.

Third: when teachers did resist researchers’ conclusions, they did so for entirely plausibly reasons.

They (plausibly) thought some of the studies contained contradictions. They (plausibly) saw some findings as out of date. And, they (plausibly) raised objections to research methodology.

They also – and I think this is very good news – emphasized the narrow particularity of research findings. As one teacher said:

If you researched in different schools, it would be different. If you had an inner-city school, a wealthy middle-class school, a private school, every one would be totally, totally different.

And another:

Does anything work for every single person? No, I don’t think there’s anything that will work exactly the same. It’s finding what’s right for your group: the age, the personalities.

(Regular readers of the blog know that I bang on about this point all the time, so I’m DELIGHTED to see it articulated so clearly here.)

Closing Thoughts

Cain (rightly) emphasizes that his study is early and exploratory. He worked with volunteers: that is, people who are likely to be interested in research in the first place. (If they weren’t interested, they wouldn’t have volunteered.)

And, like any study, this one has lots of limitations. For instance: these teachers worked in “Gifted and Talented” programs. Findings in other settings might be different.

But, at least initially, Cain’s finding shows that teachers can be great partners for researchers. We’re not resisting for the sake of resisting.

Instead, we’re thinking critically about the limits of research, and the goodness of fit for our particular classrooms.

Which is exactly what we should do.