classroom advice – Page 17 – Education & Teacher Conferences

Benefiting from Retrieval Practice: Get the Timing Just Right

Andrew Watson

January 02, 2018

Benefiting from Retrieval Practice: Get the Timing Just Right

I’ve posted a lot here recently about retrieval practice: the practice of reviewing material by pulling it OUT of the student’s head rather than trying to put it back IN.

For example: if I ask my students to write down the 5 main points from yesterday’s class about the Buddha, that’s retrieval practice. They have to get info out of their heads.

If, instead, I remind them of the 5 main points from yesterday’s class, that’s not retrieval practice. After all, I’m putting information back in.

The short sales pitch for retrieval practice is: it works for all students in all subjects, all the time. (Ask Dr. Pooja Agarwal.)

Unless…

Of course, all students in all subjects all the time is quite a grand claim. It’s rare for any teaching practice to work all the time, so we should be on the lookout for boundary conditions.

And, indeed, one has recently jumped out at me.

The story is interestingly complicated. I promise, however, that a close study of this complexity leads to specific and useful teaching advice. So: hang in there!

When Retrieval Practice Timing Might Be Bad

Imagine that, in yesterday’s class, we went over ten definitions for key economics terms. I want to begin today’s class with a quick review, so we go back over five of those terms.

My assumption is that, by reviewing five, I’m actually helping you to remember all ten.

Here’s the surprising research finding: by practicing some of the terms, I actually make it LESS LIKELY that you’ll remember the unpracticed terms.

In other words: recalling some of the words prompts you to forget the unpracticed words.

Psychologists call this bizarre result retrieval-induced forgetting. After all, the retrieval — that is, the practice — induced you to forget.

When Research Fields Contradict

So: the retrieval practice research says that retrieval is beneficial for memory.

And: the retrieval-induced forgetting research says that retrieval is detrimental for memory.

What happens when teachers do both? Does one cancel out the other? Can Superman defeat Iron Man?

Research done by Jason CK Chan helps answer this intriguing question.

The short answer is: in the short term, retrieval-induced forgetting is stronger. So: if I quiz you on five of those economics terms, and then give you the final test on those terms an hour later, you’re more likely to forget the five unpracticed words.

However, in the longer term, retrieval practice is stronger. So: the quiz on five terms will benefit you if you take that final test 24 hours later.

This result is especially likely if my quiz encourages you to think about how these five words connect conceptually to the other words.

Practical Advice

Although these research findings can be difficult to follow, they do all lead to a specific suggestion.

Retrieval practice is an excellent study strategy for students more than 24 hours ahead of a test. However, within that 24 hour window, teachers and students should focus more on connecting ideas rather than recalling them.

To update Dr. Agarwal’s guidance: retrieval practice works for all students in all subjects, (almost) all the time.

Enhance Memory by Saying Important Words Aloud

Andrew Watson

December 27, 2017

Enhance Memory by Saying Important Words Aloud

You’d like to remember a list of words better? Here’s a simple trick: say them aloud to yourself.

According to recent research by Forrin and MacLeod, the benefits of both reading and saying words out loud are greater than either reading or saying the words.

When going over flashcards of essential chemistry terms — for example — students might say the definitions as they review them. This strategy should help them learn those definitions better.

Practical limitations of saying words aloud

As I think about the teaching implications of this research, I don’t think we should encourage students to read everything aloud. (Except, of course, students who are learning to read.)

Instead, we should suggest this technique as a study supplement for a few key concepts: the definitions or formulas that we most want them to learn.

This strategy takes little time and costs nothing. In other words, it’s perfect for the world of education.

Andrew Watson

December 13, 2017

Highlighting Retrieval Practice

The Effortful Educator describes his fun system for using highlighters during retrieval practice. He teaches AP Psychology in high school, but I suspect this system could be easily used with younger students as well.

EE’s lesson plan stands out for two reasons.

First: it’s a great example of retrieval practice — asking students to pull information out of their brains rather than trying to put more information in.

Second: it’s a great example of translation. EE knows the research about retrieval practice–he’s a psychology teacher after all. In this case, he’s gone well beyond simply replicating methods used by psychology teachers. Instead, he’s thought carefully about the uses of that idea in his particular context, and he’s translated the research to make it work for his students.

In other words: you might emulate the Effortful Educator’s specific strategy of using different colored highlighters. You should emulate his general strategy of adapting psychology to your classroom, your students, and your own approaches to teaching.

Andrew Watson

December 09, 2017

Advice for College Students

This brief (and admirably clear) article offers guidance to college students on the study strategies that have research support — and, helpfully, those that don’t.

The authors offer a few sources to verify their claims, explain why some counter-intuitive strategies work better that more traditional ones, and even toss in a few un-researched but entirely plausible suggestions.

(One minor disagreement: the authors cite the Mueller & Oppenheimer study to discourage laptop note-taking. Regular readers of the blog know I think that study doesn’t support its own conclusions.)

Andrew Watson

November 29, 2017

Welcome to “the Messiness”

In a recent interview on this blog, Dr. Pooja K. Agarwal spoke about the benefits of retrieval practice: a study technique that–in her words–focuses on pulling information OUT of students’ brains rather than getting it back IN.

For example: if I begin today’s class by having my students write down three things they remember from yesterday’s lesson on the Han dynasty, that’s retrieval practice. After all, they’re going back into their memories and drawing OUT facts and ideas we discussed.

If, however, I begin by briefly summarizing yesterday’s class, well, then I’m trying to put information back IN. That’s not retrieval practice.

Dr. Agarwal summarizes the benefits of retrieval practice thus: “it works for all students in all subjects, all the time.”

Sounds tempting, no?

Pushing Boundaries

In one part of our conversation, Dr. Aragwal notes that she likes doing research in actual classrooms with actual students–rather than in psychology labs in highly controlled conditions–because “I really like the messiness of of doing scientific research in classrooms. The fire alarms, and school assemblies, and kids who are out sick, I really enjoy it because it pushes boundaries.”

In the spirit of messiness, here’s a recent post from the Learning Scientists about using retrieval practice in elementary school to learn vocabulary.

The good news about this study:

First: it took place in a real school with real students, not in a psychology lab. That means its results are likelier to be meaningful to teachers.

Second: the participants were 9-year-olds, not college students. So, we can be more confident that retrieval practice works with…say…4th graders.

Third: the study took place in the Netherlands, so we’ve got reason to believe that the benefits go beyond a North American cultural context.

So far, so good.

Let the Messiness Begin

At the same time, this particular study revealed a few muddles as well.

Muddle #1: the size of the benefit was relatively small. Retrieval practice produced more learning than simple restudy, and more than “elaborative retrieval,” but statistically speaking that difference was harder to find than in a psychology lab.

Muddle #2: Dr. Agarwal’s research shows that fill-in-the-blank retrieval practice and multiple-choice retrieval practice are equally effective. This study, however, contradicts that finding; multiple-choice retrieval didn’t produce more learning than pure restudy.

Muddle #3: believe it or not, muddle #3 contradicts muddle #2. Because of the study design, the authors acknowledge that their own findings about multiple-choice tests aren’t fully persuasive. For example: because the average score on the multiple-choice tests was above a 90%, there wasn’t enough difference among the students’ scores to calculate meaningful effects.

What should teachers do with all this contradictory information?

My advice: Embrace the muddle.

Teachers should expect that different studies produce muddled–and occasionally contradictory–results.

No one study tells us everything we need to know about retrieval practice. Instead, we’re looking for patterns of findings.

If we do ten studies, and eight of them show that retrieval practice helps learning, that’s impressive. We don’t need to be thrown off by one study that shows no effect–or, as in this case, a relatively smaller effect than in a psych lab.

The Quiet Finding

Although the authors don’t dwell on this point, one finding jumped out at me.

In one of the restudy conditions, students were asked to “elaborate” on the meaning of the word. For example, as they tried to remember “compost pile,” they were asked to circle the words relating to a compost pile on this list: manure, plastic, delicious, orange-peels, mailbox, dead leaves.

My teacherly instincts tell me that this restudy condition ought to help students. After all, to circle the correct words, they have to think a bit harder about the meaning of the phrase “compost pile.” That additional thought strikes me as a desirable difficulty, and ought to produce more learning.

But–at least in this one study–it didn’t. Students who “elaboratively restudied” scored between the “pure restudy” group and the “retrieval practice” group–and their scores weren’t significantly different from either.

The Take-Aways…

I myself reach three conclusions based on this research:

A) Yup: retrieval practice still works, even with 4th graders, even with vocabulary learning, even in the Netherlands.

B) My instincts about elaborative restudy might be off. I should keep my eyes peeled for further research.

C) The muddle isn’t disheartening, it’s enjoyable. Jump in–the water’s warm!

Andrew Watson

November 26, 2017

Now Even the New York Times Has It Wrong

Here’s a hypothetical situation:

Let’s say that psychology researchers clearly demonstrate that retrieval practice helps students form long-term memories better than rereading the textbook does.

However, despite this clear evidence, these researchers nonetheless emphatically recommend that students avoid retrieval practice and instead reread the textbook. These researchers have two justifications for their perverse recommendation:

First: students aren’t currently doing retrieval practice, and

Second: they can’t possibly learn how to do so.

Because we are teachers, we are likely to respond this way: “Wait a minute! Students learn how to do new things all the time. If retrieval practice is better, we should teach them how to do it, and then they’ll learn more. This solution is perfectly obvious.”

Of course it is. It’s PERFECTLY OBVIOUS.

Believe It Or Not…

This hypothetical situation is, in fact, all too real.

In 2014, Pam Mueller and Dan Oppenheimer did a blockbuster study comparing the learning advantages of handwritten notes to laptop notes.

Their data clearly suggest that laptop notes ought to be superior to handwritten notes as long as students learn to take notes the correct way.

(The correct way is: students should reword the professor’s lecture, rather than simply copy the words down verbatim.)

However–amazingly–the study concludes

First: students aren’t currently rewording their professor’s lecture, and

Second: they can’t possibly learn how to do so.

Because of these two beliefs, Mueller and Oppenheimer argue that–in the witty title of their article–“The Pen is Mightier than the Laptop.”

But, as we’ve seen in the hypothetical above, this conclusion is PERFECTLY OBVIOUSLY incorrect.

Students can learn how to do new things. They do so all the time. Learning to do new things is the point of school.

If students can learn to reword the professor’s lecture when taking notes on a laptop, then Mueller and Oppenheimer’s own data suggest that they’ll learn more. And yes, I do mean “learn more than people who take handwritten notes.”

(Why? Because laptop note-takers can write more words than handwriters, and in M&O’s research, more words lead to more learning.)

And yet, despite the self-evident logic of this argument, the belief that handwritten notes are superior to laptop notes has won the day.

That argument is commonplace is the field of psychology. (Here‘s a recent example.)

Even the New York Times has embraced it.

The Fine Print

I do need to be clear about the limits of my argument:

First: I do NOT argue that a study has been done supporting my specific hypothesis. That is: as far as I know, no one has trained students to take reworded laptop notes, and found a learning benefit over reworded handwritten notes. That conclusion is the logical hypothesis based on Mueller and Oppenheimer’s research, but we have no explicit research support yet.

Second: I do NOT discount the importance of internet distractions. Of course students using laptops might be easily distracted by Twinsta-face-gram-book. (Like everyone else, I cite Faria Sana’s research to emphasize this point.)

However, that’s not the argument that Mueller and Oppenheimer are making. Their research isn’t about internet distractions; it’s about the importance of reworded notes vs. verbatim notes.

Third: I often hear the argument that the physical act of writing helps encode learning more richly than the physical act of typing. When I ask for research supporting that contention, people send me articles about 1st and 2nd graders learning to write.

It is, I suppose, possible that this research about 1st graders applies to college students taking notes. But, that’s a very substantial extrapolation–much grander than my own modest extrapolation of Mueller and Oppenheimer’s research.

And, again, it’s NOT the argument that M&O are making.

To believe that the kinesthetics of handwriting make an essential difference to learning, I want to find a study showing that the physical act of writing helps high school/college students who are taking handwritten notes learn more. Absent that research, this argument is even more hypothetical than my own.

Hopeful Conclusion

The field of Mind, Brain, & Education promises that the whole will be greater than the sum of the parts.

That is: if psychologists and neuroscientists and teachers work together, we can all help each other understand how to do our work better.

Frequently, advice from the world of psychology gives teachers wise guidance. (For example–retrieval practice.)

In this case, we teachers can give psychology wise guidance. The founding assumption of the Mueller and Oppenheimer study–that students can’t learn to do new things–simply isn’t true. No one knows that better than teachers do.

If we can keep this essential truth at the front of psychology and neuroscience research, we can benefit the work that they do, and improve the advice that they give.

Andrew Watson

November 25, 2017

Good News about Concept Mapping

This meta-analysis, which looks at studies including almost 12,000 students, concludes that creating concept maps does indeed promote learning.

Specifically, it’s better than simply looking at concept maps, or listening to lectures, or participating in discussions, or even writing summaries.

The article summarizes several hypotheses to explain the benefits of concept mapping: it reduces working memory load by using both visual and verbal channels, it requires greater cognitive elaboration, and so forth.

So, let’s hear it: how do you get your students to map concepts? What successes have you had? Let me know in the comments…

(h/t IQ’s Corner)

Andrew Watson

November 08, 2017

Does project-based learning work?

The answer to the titular question depends on a) your definition of “project-based learning,” and b) your methodology for measuring success.

In a just-published, comprehensive literature review, MDRC takes 84 pages to say: “we can’t really answer the question, because we don’t have consistent definitions or consistent methodologies.”

For example:

Without a common set of PBL design principles, it is difficult to use the existing body of research to draw conclusions about PBL’s effectiveness. (p. 53)

More rigorous evidence is needed to confirm whether PBL is a better approach to prepare students for college and career than traditional teacher-directed methods. (p. 55)

That’s a frustrating answer.

If you love and believe in PBL–and, more than most pedagogical theories, PBL really has true believers–you’d rather have a ringing endorsement.

If you’re a skeptic–check out Kirschner’s emphatic rejection here–you’d like this idea put to bed once and for all.

In this review, however, the authors make clear that until we agree what PBL really is (and, what it isn’t), we can’t coherently measure its effectiveness.

What Should Teachers Do?

In the absence of a clear research answer to this question, I have two suggestions.

First: teacher experience matters. If you and your colleagues have experience teaching both PBL and direct-instruction curricula, and you’ve had good success with one or the other, then draw on that experience. As long as you’re being honest with yourselves, and keeping good records, then your experience is–for now–at least as good as any other information we’ve got.

Second: rely on useful principles from cognitive science. Does PBL help your students pay attention? If yes, that’s good. Does PBL decrease their motivation? If yes, that’s bad.

Quite often, for instance, I find that PBL curricula overwhelm students’ working memory limits. If so, then it doesn’t matter that the curriculum ought to work, or was designed by experts, because it’s overwhelming working memory.

In other words: if the curriculum sounds upliftingly progressive, but it violates basic principles of cognition, then put the rubric down and step away from the authentic question.

Every curriculum must fit with the way that students’ brains work–including a PBL curriculum.

(In case you’re wondering, “MDRC” stands for “Manpower Demonstration Research Corporation.” It was created by the Ford Foundation; its lumpy name was simplified to MDRC in 2003. You can read its history here.)

Multiple-Choice Tests Are A) Good or B) Bad

Andrew Watson

November 02, 2017

Multiple-Choice Tests Are A) Good or B) Bad

Teachers hate (and love) multiple-choice tests. On the one hand, they seem dreadfully reductive. On the other, they’re blissfully easy to grade — and easy grading is never to be belittled.

In our recent conversation, Pooja Agarwal recommended multiple-choice tests as one kind of retrieval practice. Inspired by her guidance, you might be asking yourself: “what can researchers tell me about the best kind of multiple-choice test to give?”

If you’re asking yourself that question, look no further: the estimable Andrew Butler is on the case.

(For example: if you want to know how many distractors to include on your test, you should see what Butler has to say…)