{"id":5835,"date":"2020-09-29T08:00:04","date_gmt":"2020-09-29T13:00:04","guid":{"rendered":"https:\/\/braindevs.net\/blog\/blog\/?p=5835"},"modified":"2020-09-27T14:52:52","modified_gmt":"2020-09-27T19:52:52","slug":"how-psychologists-and-teachers-can-talk-about-research-most-wisely","status":"publish","type":"post","link":"https:\/\/www.learningandthebrain.com\/blog\/how-psychologists-and-teachers-can-talk-about-research-most-wisely\/","title":{"rendered":"How Psychologists and Teachers Can Talk about Research Most Wisely"},"content":{"rendered":"

\"\"<\/a><\/p>\n

Dr. Neil Lewis thinks a lot about\u00a0science communication<\/strong>: in fact, his appointment at Cornell is in both the Psychology AND the Communications departments. (For a complete bio, click here<\/a>.)<\/p>\n

He and Dr. Jonathan Wai recently posted an article<\/a> focusing on a troubling communication paradox<\/em>:<\/p>\n

Researchers are encouraged to “give science away”; however, because of the “replication crisis,” it’s hard to know what science is worth being given.<\/p>\n

Here at Learning and the Brain, we think about that question frequently — so I was delighted that Dr. Lewis agreed to chat with me about his article.<\/p>\n

In this conversation, we talk about…<\/p>\n

… how teachers can ask psychologists good questions<\/p>\n

… the dangers of “eminence”<\/p>\n

… what we should think about\u00a0growth mindset<\/strong> research<\/p>\n

… the research “hype cycle.”<\/p>\n

I hope you enjoy this conversation as much as I did.<\/p>\n

\n

Andrew Watson:<\/strong><\/p>\n

Thank you, Dr. Lewis, for sharing your ideas with our readers.<\/p>\n

In your recent article, you and Dr. Wai write about tensions between two imperatives in the field of psychology.<\/p>\n

First<\/em>, psychologists are being asked to \u201cgive research away.\u201d And second<\/em>, our field worries about the \u201creplication crisis.\u201d<\/p>\n

Both of those phrases mean more or less what they say. Could you define them a little more precisely, and talk about the tensions that these imperatives are creating?<\/p>\n

Dr. Lewis:<\/strong><\/p>\n

There has been a long-standing call in psychology\u2014going back, really, to the 60\u2019s when George Miller first issued this call\u2014to \u201cgive psychology away.\u201d<\/p>\n

As scholars, we spend our time doing all this research: we should try to communicate it with the world<\/em> so that people can use it and improve lives.<\/p>\n

Professional psychology societies and organizations really encourage researchers to \u201cget our work out there.\u201d<\/p>\n

But at the same time, over the past decade or so, there has been a movement to reflect on what we really know in psychology.<\/p>\n

A \u201creplication crisis\u201d has occurred\u2014not only in psychology, it\u2019s been happening in many areas.<\/p>\n

We are having a hard time replicating many research findings. And that [failure] is making us, the scientists, wrestle with: what do we know? How do we know it? How robust are some of our findings?<\/p>\n

And so there\u2019s a tension here. We\u2019re supposed to be \u201cgiving our findings away,\u201d but at the same time we\u2019re not sure which ones are robust enough to be worth giving away.<\/p>\n

Andrew Watson:<\/strong><\/p>\n

That does sound like a problem. In that tension, do you see any special concerns about the field of education?<\/p>\n

Dr. Lewis:<\/strong><\/p>\n

One of the things I\u2019ve been thinking about for education researchers is: how do we know what we know? We have to look very closely at the details of the paper to figure those things out.<\/p>\n

Which students are being studied in the papers you\u2019re reading?<\/p>\n

What kinds of schools?<\/p>\n

What kind of teachers?<\/p>\n

At least in the US, there\u2019s so much segregation in our school systems that schools look very different.<\/p>\n

If studies are run\u2014let\u2019s say\u2014with kids in the Ithaca school district where I live in upstate New York: those kids, those parents, those schools are very different than studies run\u2014let\u2019s say\u2014in the Detroit public school district, which is the district I thought a lot about during my graduate training when I lived in Michigan.<\/p>\n

There are big differences between these districts. We have to figure out: are the schools that we\u2019re trying to intervene in, similar to the studies that were run? Or are they different?<\/p>\n

Andrew Watson:<\/strong><\/p>\n

I have a question about that process.<\/p>\n

Here\u2019s a problem: to know what questions teachers ought to be asking, we need expert knowledge. Because we\u2019re teachers<\/em> not psychologists, it\u2019s hard to know the right questions.<\/p>\n

So: what\u2019s the best question that a nonspecialist teacher can ask of a researcher, in order to get an answer that we can genuinely understand?<\/p>\n

Dr. Lewis:<\/strong><\/p>\n

I think there are some basic things that teachers can ask of researchers.<\/p>\n

The teachers can ask what kinds of schools were these studies run in<\/em>. Are they urban schools, rural schools?<\/p>\n

What percentage of the students are on free lunch? (That\u2019s an indicator of poverty levels of the school. Research findings are often influenced by background characteristics about the students.)<\/p>\n

What do we know about the kinds of students that were involved in studies?<\/p>\n

What do we know about the teachers?<\/p>\n

Those are basic things that the researchers should be able to tell you. And then you can figure out whether those are similar to:<\/p>\n

the students that you\u2019re working with,<\/p>\n

the kinds of schools that you have,<\/p>\n

the kind of leadership in your school district, and the like.<\/p>\n

Those basic characteristics about how the study was done will help you figure out whether or not you can use it.<\/p>\n

Andrew Watson:<\/strong><\/p>\n

I spend a lot of time talking with teachers about this concern. Most psychology research is done with college undergraduates. That research is obviously important. But if you\u2019re teaching reading to third graders, maybe that research translates to your context and maybe it doesn\u2019t.<\/p>\n

Dr. Lewis:<\/strong><\/p>\n

Right.<\/p>\n

Andrew Watson:<\/strong><\/p>\n

One of the more intriguing points you made in the article has to do with the idea of eminence<\/em>.<\/p>\n

In the world of education, we\u2019re often drawn to Big Names. You argue that the things scholars do to achieve eminence<\/em> don\u2019t necessarily help them produce high quality research<\/em>.<\/p>\n

As teachers, how do we sort through this paradox? How can we be wise when we think about that?<\/p>\n

Dr. Lewis:<\/strong><\/p>\n

We brought up eminence to reinforce what I just noted. Look at the details of the study and don\u2019t rely on the \u201ccue\u201d of eminence as your signal that research must be good.<\/p>\n

Researchers are judged by many metrics. Once you put those metrics in place, people do what they can to… I hesitate to use the word \u201cgame,\u201d but to optimize their standing in those metrics.<\/p>\n

Andrew Watson:<\/strong><\/p>\n

Which is a lot like \u201cgaming,\u201d isn\u2019t it?<\/p>\n

Dr. Lewis:<\/strong><\/p>\n

Yes. In the research world, there are a few metrics that don\u2019t necessarily help [produce meaningful results]. One of them, for instance, is that researchers are incentivized to publish as much as we can<\/em>.<\/p>\n

Unfortunately, publishing fast is the way to rise up the ranks. But sometimes figuring out these differences that I have been talking about\u2014like, between contexts and samples\u2014it takes some time. It slows you down from churning out papers; and unfortunately, researchers often aren\u2019t incentivized to take that slower, more careful approach.<\/p>\n

And so there\u2019s that tension again too. I don\u2019t want to leave the impression that we just shouldn\u2019t trust eminent people. That\u2019s not the point I want to make.<\/p>\n

The point is: eminence in and of itself is not a useful signal of quality. You have to look very closely at the details of the studies in front of you. Then compare those details to your own situation and judge the work on that. Judge the work<\/em>, don\u2019t judge based on how famous the person is.<\/p>\n

Andrew Watson:<\/strong><\/p>\n

It occurs to me as you\u2019re explaining this, there\u2019s a real problem with the emphasis on rapid publication. One of the consistent findings in education research is that short-term performance<\/em> isn\u2019t a good indicator of long-term learning<\/em>.<\/p>\n

But if scholars are incentivized to publish quickly, they\u2019re incentivized to the study short-term, which doesn\u2019t tell us much about what we really want to know: learning that lasts<\/em>.<\/p>\n

Dr. Lewis:<\/strong><\/p>\n

Absolutely right. As I\u2019ve written in other articles, we don\u2019t have enough longitudinal studies<\/em> for the very reasons we\u2019re talking about: longitudinal studies take forever\u2014and, again, the incentive is to publish fast, publish often.<\/p>\n

The outcomes that are often measured in psychology studies are these shorter term things. You have the student do something, and you measure at the end of the session<\/em>. Maybe you look again at the end of the semester<\/em>.<\/p>\n

But [we should] look next<\/em> year<\/em>, two <\/em>years, three <\/em>years, because we know some of these effects take time to accumulate.<\/p>\n

Some older studies have looked at long-term outcomes. I\u2019ve seen a few fascinating studies showing, initially<\/em>, no significant findings. But if you look far enough down the road<\/em>, you start to see meaningful effects. It just takes time for the benefits to accumulate.<\/p>\n

In education, we shouldn\u2019t assume that research results \u201cgeneralize.\u201d\u00a0[Editor: That is, we shouldn\u2019t assume that research with 1st<\/sup> graders applies to 10th<\/sup> graders; or that short term findings will also be true in the long term.]\n

Now, until I see more evidence, I assume findings are context-specific.\u00a0[Editor: That is, research with 1st<\/sup> graders applies to 1st<\/sup> graders\u2014but not much beyond that age\/grade. Research from the United States applies to the US cultural context, but not\u2014perhaps\u2014to Korea.]\n

For instance: \u201cgrowth mindset.\u201d In recent studies, authors have been looking at how much the effect varies by context and by population. Those details matter in thinking about mindset studies.<\/p>\n

Andrew Watson:<\/strong><\/p>\n

Yes, I think mindset is a really interesting case study for the topic we\u2019re talking about. My impression is that teachers got super excited about growth mindset. We went to a highly simplistic \u201cposter-on-the-wall\u201d version of the theory.<\/p>\n

And in the last 18 months or so, there has been a real backlash. Now we hear: \u201cgrowth mindset means nothing whatsoever! Why are you wasting your time?\u201d<\/p>\n

We need to find our way to a nuanced middle ground. No, growth mindset is not a panacea. But nothing<\/em> is a panacea. At the same time, in a specific set of circumstances, mindset can help certain students in specific ways.<\/p>\n

That balanced conclusion can be a hard place to get the conversation to go.<\/p>\n

Dr. Lewis<\/strong><\/p>\n

Yes, issues like that motivated us to write our paper.<\/p>\n

If we [researchers] are able to communicate those nuances clearly, then I think we avoid these misunderstandings. It\u2019s not that mindset is useless<\/em>; instead, mindset will have a small effect<\/em> under certain conditions. We should just say that.<\/p>\n

We have a problem with the \u201chype cycle.\u201d<\/p>\n

If something is over-hyped one day, then you\u2019re really setting people\u2019s expectations unreasonably high. Later, when the research doesn\u2019t meet those expectations, teachers are disappointed.<\/p>\n

And so researchers should set expectations appropriately. Mindset is not a panacea. We shouldn\u2019t expect enormous impacts. And that\u2019s fine. Let\u2019s just say that.<\/p>\n

Andrew Watson:<\/strong><\/p>\n

I think this \u201chype cycle\u201d is part of the challenge that we\u2019re facing.<\/p>\n

For instance, with learning styles<\/em>, teachers thought that it had a lot of scientific backing. We embraced it because it was \u201cresearch based.\u201d<\/p>\n

Now the message is: \u201cno, research got that wrong; learning styles aren\u2019t a thing. But here\u2019s another research-based thing instead.\u201d<\/p>\n

And teachers are saying: \u201cwait, if I shouldn\u2019t have followed research about learning styles, why should I believe new research about new teaching suggestions?\u201d<\/p>\n

Dr. Lewis:<\/strong><\/p>\n

That\u2019s a tricky problem.<\/p>\n

One way to think about science is: science is a way of reducing uncertainty<\/em>.<\/p>\n

We had this idea about learning styles. We gathered some initial evidence about it. It seemed like a good idea for a while.<\/p>\n

But as we continued studying it, we realized, well, maybe there is not as much good evidence as we thought.<\/p>\n

And that\u2019s part of the scientific process. I think it\u2019s important to explain that.<\/p>\n

But: that shift without an explanation naturally leads teachers to be suspicious.<\/p>\n

Teachers think: \u201cwhy are you telling me, just make this change<\/em>. You have to explain to me what is going on and why<\/em> should I make that change.\u201d<\/p>\n

This explanation does take more time. But that\u2019s what is necessary to get people to update their understanding of the world.<\/p>\n

Something that we all have to keep in mind: just as every year teachers<\/em> are learning new ways to teach the new generations of students, scientists<\/em> are doing the same thing too. We\u2019re constantly trying to update our knowledge.<\/p>\n

So there will be changes in the recommendations over time. If there weren\u2019t changes, none of us would be doing our best. So we\u2019re learning and improving constantly.<\/p>\n

But we have to have that conversation. How<\/em> are we updating our knowledge? And what are ways<\/em> that we can implement that new knowledge into curriculum?<\/p>\n

And, the conversation has to go both ways. Researchers communicate things to teachers, but teachers also need to be telling things to researchers. So we can keep that real classroom context in mind as we\u2019re developing research advice.<\/p>\n

Andrew Watson:<\/strong><\/p>\n

In your article, you and Dr. Wai remind researchers that they\u2019re not communicating with one undifferentiated public. They are talking with many distinct, smaller audiences\u2014audiences which have different interests and needs.<\/p>\n

Are there difficulties that make it especially hard to communicate with teachers about psychology research? Is there some way that we\u2019re an extra challenging audience? Or maybe, an especially easy audience?<\/p>\n

Dr. Lewis:<\/strong><\/p>\n

I think what\u2019s hard for presenters is not knowing details about the audience, where they\u2019re coming from. That section of the paper is about is really getting to know your audience, and tailoring your message from there.<\/p>\n

If I\u2019m going to go explain psychology findings to a group of STEM teachers, that talk might be different than if the audience is a broader cross-section of teachers.<\/p>\n

In the university setting, it\u2019s easier to figure out those distinctions because you know which department invited you to speak.<\/p>\n

In broader K-12 settings you don\u2019t always know. A school district invites you. You can do some Googling to try to figure something out about the district. But you don\u2019t know who\u2019s going to be in the room, and what is happening [in that district]. So you might end up giving too broad a talk, that might be less informative than if you did get some more information.<\/p>\n

Andrew Watson:<\/strong><\/p>\n

Are there questions I haven\u2019t asked that I ought to have asked?<\/p>\n

Dr. Lewis:<\/strong><\/p>\n

The key point for me is: when we communicate about science in the world, we really have to look at key research details and have serious conversations about them. Nuances matter, and we just can\u2019t gloss over them.<\/p>\n

Andrew Watson:<\/strong><\/p>\n

Dr. Lewis, I very much appreciate your taking the time to talk with me today.<\/p>\n

Dr. Lewis:<\/strong><\/p>\n

Thank you.<\/p>\n","protected":false},"excerpt":{"rendered":"

Dr. Neil Lewis thinks a lot about\u00a0science communication: in fact, his appointment at Cornell is in both the Psychology AND the Communications departments. (For a complete bio, click here.) He and Dr. Jonathan Wai recently posted an article focusing on a troubling communication paradox: Researchers are encouraged to “give science away”; however, because of the […]<\/p>\n","protected":false},"author":18,"featured_media":5842,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[19],"class_list":["post-5835","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-lb-blog","tag-skepticism"],"_links":{"self":[{"href":"https:\/\/www.learningandthebrain.com\/blog\/wp-json\/wp\/v2\/posts\/5835","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.learningandthebrain.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.learningandthebrain.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.learningandthebrain.com\/blog\/wp-json\/wp\/v2\/users\/18"}],"replies":[{"embeddable":true,"href":"https:\/\/www.learningandthebrain.com\/blog\/wp-json\/wp\/v2\/comments?post=5835"}],"version-history":[{"count":8,"href":"https:\/\/www.learningandthebrain.com\/blog\/wp-json\/wp\/v2\/posts\/5835\/revisions"}],"predecessor-version":[{"id":5846,"href":"https:\/\/www.learningandthebrain.com\/blog\/wp-json\/wp\/v2\/posts\/5835\/revisions\/5846"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.learningandthebrain.com\/blog\/wp-json\/wp\/v2\/media\/5842"}],"wp:attachment":[{"href":"https:\/\/www.learningandthebrain.com\/blog\/wp-json\/wp\/v2\/media?parent=5835"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.learningandthebrain.com\/blog\/wp-json\/wp\/v2\/categories?post=5835"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.learningandthebrain.com\/blog\/wp-json\/wp\/v2\/tags?post=5835"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}