Skip to main content

Tag: retrieval practice

Older posts
Newer posts
To Grade or Not to Grade: Should Retrieval Practice Quizzes Be Scored?
Andrew Watson
Andrew Watson
March-30-2021

To Grade or Not to Grade: Should Retrieval Practice Quizzes Be Scored?

Read More

We’ve seen enough research on retrieval practice to know: it rocks.

When students simply review material (review their notes; reread the chapter), that mental work doesn’t help them learn.

However, when they try to remember (quiz themselves, use flashcards), this kind of mental work does result in greater learning.

In Agarwal and Bain’s elegant phrasing: don’t ask students to put information back into their brains. Instead, ask them to pull information out of their brains.

Like all teaching guidance, however, the suggestion “use retrieval practice!” requires nuanced exploration.

What are the best methods for doing so?

Are some retrieval practice strategies more effective?

Are some frankly harmful?

Any on-point research would be welcomed.

On-Point Research

Here’s a simple and practical question. If we use pop quizzes as a form of retrieval practice, should we grade them?

In other words: do graded pop quizzes result in more or less learning, compared to their ungraded cousins?

This study, it turns out, can be run fairly easily.

Dr. Maya Khanna taught three sections of an Intro to Psychology course. The first section had no pop quizzes. In the second section, Khanna gave six graded pop quizzes. In the third, six ungraded pop quizzes.

Students also filled out a questionnaire about their experience taking those quizzes.

What did Khanna learn? Did the quizzes help? Did grading them matter?

The Envelope Please

The big headline: the ungraded quizzes helped students on the final exam.

Roughly: students who took the ungraded pop quizzes averaged a B- on the final exam.

Students in the other two groups averaged in the mid-to-high C range. (The precise comparisons require lots of stats speak.)

An important note: students in the “ungraded” group scored higher even though the final exam did not repeat the questions from those pop quizzes. (The same material was covered on the exam, but the questions themselves were different.)

Of course, we also wonder about our students’ stress. Did these quizzes raise anxiety levels?

According to the questionnaires, nope.

Khanna’s students responded to this statement: “The inclusion of quizzes in this course made me feel anxious.”

A 1 meant “strongly disagree.”

A 9 meant “strongly agree.”

In other words, a LOWER rating suggests that the quizzes didn’t increase stress.

Students who took the graded quizzes averaged an answer of 4.20.

Students who took the ungraded quizzes averaged an answer of 2.96.

So, neither group felt much stress as a result of the quizzes. And, the students in the ungraded group felt even less.

In the Classroom

I myself use this technique as one of a great many retrieval practice strategies.

My students’ homework sometimes includes retrieval practice exercises.

I often begin class with some lively cold-calling to promote retrieval practice.

Occasionally — last Thursday, in fact — I begin class by saying: “Take out a blank piece of paper. This is NOT a quiz. It will NOT be graded. We’re using a different kind of retrieval practice to start us off today.”

As is always true, I’m combining this research with my own experience and classroom circumstances.

Khanna gave her quizzes at the end of class; I do mine at the beginning.

Because I’ve taught high school for centuries, I’m confident my students feel comfortable doing this kind of written work. If you teach younger grades, or in a different school context, your own experience might suggest a different approach.

To promote interleaving, I include questions from many topics (Define “bildungsroman.” Write a sentence with a participle. Give an example of Janie exercising agency in last night’s reading.) You might focus on one topic to build your students’ confidence.

Whichever approach you take, Khanna’s research suggests that retrieval practice quizzes don’t increase stress and don’t require grades.

As I said: retrieval practice rocks!

Retrieval Practice and Metacognition: What and How Do Students Think about This Powerful Learning Strategy?
Andrew Watson
Andrew Watson
March-09-2021

Retrieval Practice and Metacognition: What and How Do Students Think about This Powerful Learning Strategy?

Read More

Ask almost anyone in Learning and the Brain world, they’ll tell you: retrieval practice benefits students.

More than most any other technique we have, this one both has lots of research support and can easily be integrated into our classrooms. (For a handy review of its benefits, check out Agarwal and Bain’s great book, Powerful Teaching.)

Of course, because retrieval practice offers so many benefits, we want our students to use it.

Do they?

Metacognition and Retrieval Practice

The short answers are: not enough, and not very well.

Michelle Rivers wanted to know why, and so explored research into students’ metacognitive beliefs about retrieval practice. That is:

What do they believe about retrieval practice?

How do they use it to monitor their learning?

How do they use its insights to control their learning activities?

The more we understands students’ metacognitive perspective on their learning, the more wisely we can guide them.

What did she find?

Beliefs about Retrieval Practice

Sadly, most students don’t understand RP’s benefits.

In 10 studies that asked “why do you quiz yourself,” only 26% of students say they do so in order to learn more.

Instead, most students (52%) do so “to figure out how well I’ve learned the information I’m studying.”

In other words: even the students who use RP most often do so for the wrong reasons.

Of course: they’re not harming themselves by using retrieval practice this way. But — and this is a big but — they’re not getting the benefits that RP can offer.

In fact, Rivers’s survey suggests one reasons students might not use retrieval practice to help themselves learn. Studies suggest that when students try both methods, they don’t predict that they’ll remember more after retrieval practice. (Check out this study by Kornell and Son.)

I find this research pool baffling, even disheartening. Even when students experience greater success with RP than with simple rereading, they don’t internalize the broader lesson that active retrieval helps them learn.

Little wonder, then, that most students review material (43%) or copy their notes (11%) as a go-to strategy, rather than self-testing (8%).

Uses of Retrieval Practice

Given these flawed beliefs, how do students use RP?

Among Rivers’s findings: students try retrieval practice …

… when the questions are easy

… right before a test

… relatively late in the learning process.

… relatively few times for any given pool of information.

Of course, retrieval practice benefits students when they do so…

… with questions that are challenging

… well before a test (in fact, RP immediately before a test might be counterproductive)

… throughout the learning process

… several times for any given pool of information.

Simply put: even when students use this excellent study strategy, they do so in less-than-optimal ways.

Next Steps: Learning How to Learn

So far, this is quite the glum post. A potentially powerful learning strategy is largely going to waste.

What can we teachers do?

I’ve got two suggestions.

First, this recent post summarizes a promising approach from Mark McDaniel and Gilles Einstein. Their multi-step process not only works to persuade students of RP’s benefits; it encourages them to make specific retrieval practice plans and to follow through on them.

In other words: we shouldn’t just tell our students about its benefits. We shouldn’t just tell them to do it. We should go the next steps to create plans and structures.

Second, I’ve seen LOTS of online programs to help teachers and students with their retrieval practice.

For instance, Adam Boxer has created a program called Carousel. This program allows teachers to create retrieval questions, and to jumble them together in useful ways. It allows students to self-score their work (and teachers to correct any errors). It keeps track of right and wrong answers, so we can see how well our students are learning specific topics and questions.

I have used Carousel enough to find it intriguing; I haven’t used it enough to make strong claims about it. (Given responses to it on Twitter, however, it seems that teachers kind of love it.)

Whichever program you choose, I think students will learn how to learn more effectively if we build tools like these into our teaching practice.

In Sum

A: We know that retrieval practice can help students learn, but only if they use it correctly.

B: We know that, for the most part, they don’t.

A + B = We should focus more on helping students use this strategy wisely. And, we’ve got the tools to do so!

 

Two New Ways of Thinking About Memory
Andrew Watson
Andrew Watson
December-01-2020

Two New Ways of Thinking About Memory

Read More

In our classroom work, we teachers focus on learning; in their research, psychologists and neuroscientists often focus on memory. We have, in other words, different frameworks for talking about the same topic.

Photo by Vlad Tchompalov on Unsplash

When I find one review article that provides TWO fresh ways to understand memory and learning, well, that’s worth sharing.

Humans have MANY memory systems with many daunting (and overlapping) names: working memory, declarative memory, semantic memory, and so forth.

In our day-to-day lives, we often focus on episodic memory. As the name suggests, this memory strand acquires vividly detailed pictures of specific events:

My birthday party (I can describe the cinnamon in the chocolate cake, and why my brother was looking so grouchy).

The time you found a stranger’s wallet (You can remember the chilly, opaque puddle from which you plucked the wallet, and the stranger’s shocked gratitude when you sleuthed down his phone number to return it).

The day the principal literally dropped the mic (Students still talk about the hollow bang and the agonizing reverb when the mic hit the stage floor in the gym).

Episodic memories fill our scrapbooks and dinner-table stories.

Over time, episodic memories gradually turn into semantic memories: general knowledge of abstract facts.

For instance:

At one point, probably in school, you learned that Abraham Lincoln had been assassinated. That evening, you had an episodic memory of learning that truth. You could say which teacher told you; you might wince at the sound of squeaking chalk as s/he wrote on the board.

Over time, however, that detailed episodic memory has become semantic memory. You know the abstract fact (Booth killed Lincoln), but not the rich details of when you learned it.

You no longer remember — episodically — when you learned that fact, but you remember the fact itself — semantically.

In psychology language, your brain semanticized this episodic memory.

Too Much of a Good Thing

Truthfully, we want our students to semanticize most of their learning.

For instance: I don’t want them to know that 3 + 4 = 7 at this specific moment.

I want them to abstract a general, semantic rule: three of something, combined with four more of the somethings, add up to seven somethings. (Unless those somethings are rumors, in which case they add up to a billion.)

I want them to know that the pen is mightier than the sword is an example of metonymy. But I don’t want them to limit their knowledge to that one example.

Instead, I want them to recognize other metonyms — which they can do if they semanticize that example.

At times, however, students can abstract too far.

If they conclude that a roundish number (like 3) plus a pointy number (like 4) add up to seven, then they might conclude that a roundish 8 plus a sharpish 1 add up to seven. In this case, they over-generalized: that is, over-semanticized.

If they conclude that the words pen and sword are always metonyms — that they never mean literally “pen” and “sword” — then they have gone too far.

When learning to speak, children pick up the abstract rule that “-ed” makes words past tense in English. But, they over-semanticize when they say “goed” instead of “went.”

As teachers, we want students to get the balance just right. We want them to translate individual examples into abstract rules.

But: we don’t want them to over-apply those abstract rules to inappropriate situations.

Teaching Implications?

At this point, you might worry: gosh, ANOTHER set of teaching techniques I have to master.

This research team has good news for you: the techniques you’ve heard of at Learning and the Brain conferences help students get this balance right.

That is: retrieval practice helps students get the episodic/semantic balance right.

So do spacing and interleaving.

So does sleep, and (probably) mindfulness and mindful rest.

This episodic/semantic balance is a new way of thinking about old teaching techniques, not a call for new teaching techniques.

Second “New Way”

Authors van Kesteren and Meeter also offer a neuroscientific account of long-term memory formation.

The (very) brief summary goes like this.

We know that both the prefrontal cortex (PFC) and the hippocampus participate in new learning.

In their framework, the PFC helps connect new information with pre-existing mental models (often called schema). And the hippocampus helps organize new information that doesn’t align with a pre-existing mental model/schema.

That is: the brain’s response to new information depends on our current knowledge of it.

If we know something, the PFC does lots of the memory work.

If we know very little, the hippocampus does lots of the memory work.

As always, this neurobiological account wildly oversimplifies a hugely complicated series of events. (This is a blog, after all.)

And, it doesn’t provide new teaching strategies. We don’t “teach the PFC this way, and teach the hippocampus this other way.” (If you hear someone say that, be SUPER skeptical.)

However, it does offer a fascinating theory about the brain activity underlying our amazing mental abilities.

Putting It Together

This post’s title offered “two new ways to think about memory.”

First, teachers can think about converting episodic memories into semantic memories (without going too far).

Second, we can think about the PFC’s role in adding to existing schema, and the hippocampus’s role in developing new schema.

Neither new framework changes your teaching — assuming you’re already using the strategies that you hear about at LatB conferences so frequently. But, both offer us new ways to view our teaching from new perspectives — that is, to use both our PFCs and our hippocampi at the same time.

For earlier thoughts on episodic (also called “autobiographical”) memory vs. semantic memory, click here. And here for Clare Sealy’s discussion of the topic.

“But How Do We Know If It Works in the Classroom?”: The Latest on Retrieval Practice
Andrew Watson
Andrew Watson
November-03-2020

“But How Do We Know If It Works in the Classroom?”: The Latest on Retrieval Practice

Read More

We’ve heard so much about retrieval practice in the last two years that it seems like we’ve ALWAYS known about its merits.

But no: this research pool hasn’t been widely known among teachers until recently.

We can thank Agarwal and Bain’s wonderful Powerful Teaching for giving it a broad public audience. (If you had been attending Learning and the Brain conferences, of course, you would have heard about it a few years before that.)

Of course, we should stop every now and then to ask ourselves: how do we know this works?

In this case, we’ve got several answers.

In addition to Agarwal and Bain’s book, both Make it Stick (by Brown, Roediger, and McDaniel) and How We Learn (by Benedict Carey) offer helpful surveys of the research.

You could also check out current research. Ayanna Kim Thomas recently published a helpful study about frequent quizzing in college classrooms. (It helps!)

All these ways of knowing help. Other ways of knowing would be equally helpful.

For instance: I might want to know if retrieval practice helps in actual classrooms, not just in some psychology lab somewhere.

Yes, yes: Agarwal and Bain’s research mostly happened in classrooms. But if you’ve met them you know: it might work because they’re such engaging teachers! What about teachers like me — who don’t quite live up to their energy and verve?

Today’s News

A recent meta-analysis looked at the effect on retrieval practice in actual classrooms with actual students. (How many students? Almost 8000 of them…)

Turns out: retrieval practice helps when its studied in psychology labs.

And, it helps when vivacious teachers (like Agarwal and Bain) use it.

And, it helps when everyday teachers (like me) use it.

It really just helps. As in: it helps students learn.

A few interesting specifics from this analysis:

First: retrieval practice quizzes helped students learn more when they were counted for a final grade than when they weren’t. (Although: they did help when not counted toward the grade.)

Second: they helped more when students got feedback right away than when feedback was delayed. (This finding contradicts the research I wrote about last week.)

Third: short answer quizzes helped learning more than multiple choice (but: multiple choice quizzes did produce modest benefits).

Fourth: announced quizzes helped more than unannounced quizzes.

and, by the way

Fifth: retrieval practice helped middle-school and high-school students more than college students. (Admittedly: based on only a few MS and HS studies.)

In brief: all that good news about retrieval practice has not been over sold. It really is among the most robustly researched and beneficial teaching strategies we can use.

And: it’s EASY and FREE.

A Final Note

Because psychology research can be — ahem — written for other psychology researchers (and not for teachers), these meta-analyses can be quite daunting. I don’t often encourage people to read them.

In this case, however, authors Sotola and Crede have a straightforward, uncomplicated prose style.

They don’t hold back on the technical parts — this is, after all, a highly technical kind of writing.

But the explanatory paragraphs are unusually easy to read. If you can get a copy — ask your school’s librarian, or see if it shows up on Google Scholar — you might enjoy giving it a savvy skim.

“Successive Relearning”: 1 + 1 = +10%
Andrew Watson
Andrew Watson
September-15-2020

“Successive Relearning”: 1 + 1 = +10%

Read More

We teachers get LOTS of advice from cognitive science. Research tells us to…

…monitor and manage our students’ stress levels.

…use mid-class exercise to enhance attention.

…interleave topics to create desirable difficulties.

Each individual suggestion has lots of research behind it, and we’re glad to get these ideas.

But: what happens when we start thinking about combinations?

If we use more than one technique at a time, do the benefits add onto each other? Do they conflict with each other? How can we link up research-informed strategies to create the best overall learning experience?

Focus on Memory

In the last ten years, we’ve seen a real explosion in research about long-term memory formation (aka, learning).

We’ve seen that retrieval practice takes no more time than simple review, but results in lots more learning.

We’ve seen that spreading practice out (aka, spacing) helps students learn better than bunching practice together.

So, here’s the big question: what happens if we do both? Is retrieval practice + spacing more powerful than each technique by itself? Is 1+1 greater than 2?

A research team at Kent State recently explored this question.

In this study, researchers developed a complex study paradigm that created both retrieval practice and spacing. Unlike some retrieval practice exercises — which simply require students to try to remember the answer — this paradigm required students to get questions correct before they were done.

Researchers called this combination successive relearning. Students used successive relearning for some of the topics they learned in an advanced biopsychology course. They used their regular (“business-as-usual”) study techniques for the others.

Did successive relearning help students learn?

Answers, and More Questions

To some degree, the answer to that question is: it depends on what you compare to what.

Should the researchers compare this year’s students to last year’s students in the same course?

Should they compare students’ memory of topics where they did use successive relearning to topics where they didn’t?

Being thorough, this research team compared lots of variables to lots of other variables. Quite consistently, they found that “successive relearning” helped.

That is: this year’s students learned more than last year’s. Among this year’s students, successive relearning helped students remember more than their “business-as-usual” strategies.

Roughly speaking, students remembered at least 10% more using successive relearning than other strategies.

So: 1 + 1 = +10%

Case closed.

Case Reopened?

But wait just a minute here.

If you’ve got a good memory, this blog post might be ringing some bells.

Back in February of this year, I wrote about a study in which spacing helped students learn, but — in the long run — retrieval practice didn’t.

If you’ve got an AMAZING memory, you might recall a blog post from June of 2018. Researchers polled students about their study habits. They found that students did benefit from their own retrieval practice, but did not benefit from their spacing — the exact opposite result.

What’s going on here? Why did the two studies differ from each other? Why are they different from this study I’ve just described?

My hypothesis: specifics matter.

In those previous studies, the classes already included one of these techniques.

That is: the study I wrote about in February looked at a math class. Math classes already include lots of retrieval practice, because problem solving is a kind of RP. So, additional retrieval practice didn’t help. But the spacing did.

And, in the class I wrote about in 2018, the professor’s syllabus already included lots of spacing: cumulative review sheets and cumulative exams. So additional spacing done by the students  didn’t help. But their retrieval practice did.

In this most recent study, students benefitted from both because the biopsychology course didn’t include either.

In other words: the best combination of retrieval practice and spacing will depend — in part — on the structure and content of the course you’re teaching.

Final Thoughts

Here’s how I concluded my post back in February:

In my own view, we can ask/expect our students to join us in retrieval practice strategies. Once they reach a certain age or grade, they should be able to make flashcards, or use quizlet, or test one another.

However, I think spacing requires a different perspective on the full scope of a course. That is: it requires a teacher’s perspective. We have the long view, and see how all the pieces best fit together.

For those reasons, I think we can (and should) ask students to do retrieval practice (in addition to the retrieval practice we create). But, we ourselves should take responsibility for spacing. We — much more than they they — have the big picture in mind. We should take that task off their to do list, and keep it squarely on ours.

That’s an opinion, not a research conclusion. But I still think it’s true.

Retrieval Practice is GREAT. Can We Make It Better?
Andrew Watson
Andrew Watson
July-15-2020

Retrieval Practice is GREAT. Can We Make It Better?

Read More

By now you know that retrieval practice has lots (and lots) (and LOTS) of research behind it. (If you’d like a handy comprehensive resource, check out this website. Or this book.)

The short version: don’t have students review by putting information back into their brains — say, by rereading a chapter. Instead, have them pull information out of their brains — say, by quizzing themselves on that chapter.

It’s REALLY effective.

When we know that a technique works in general, we start asking increasingly precise questions about it.

Does it work for children and adult learners? (Yes.)

Does it work for facts and concepts? (Yes.)

Does it work for physical skills? (Yes.)

Does it work when students do badly on their retrieval practice exercises? Um. This is awkward. Not so much…

That is: when students score below 50% on a retrieval practice exercise, then retrieval practices is less helpful than simple review.

How do we fix this problem?

“Diminishing Cues” and Common Sense

Let’s say I want to explain Posner and Rothbart’s “Tripartite Theory of Attention.” In their research, attention results from three cognitive sub-processes: “alertness,” “orienting,” and “executive attention.”

Depending on the complexity of the information I provide, this explanation might get confusing. If a retrieval practice exercise simply asks students to name those three processes, they might not do very well.

Common sense suggests a simple strategy: diminishing cues.

The first time I do a retrieval practice exercise on this topic, I provide substantial cues:

“Fill in these blanks: Posner and Rothbart say that attention results from al______, or_____, and ex_______ at______.”

A few days later, I might ask:

“Fill in these blanks: Posner and Rothbart say that attention results from ______, _____, and _______  ______.”

A week later:

“What three sub-processes create attention, in Posner and Rothbart’s view?”

And finally:

“Describe how attention works.”

The first instance requires students to retrieve, but offers lots of support for that retrieval. Over time, they have to do more and more of the cognitive work. By the end, I’m asking a pure retrieval question.

“Diminishing Cues” and Research

So, common sense tells us this strategy might work. In fact, I know teachers who have stumbled across this approach on their own.

Here at Learning and the Brain, we like common sense and we REALLY like research. Do we have research to support our instincts?

Yes.

In 2017, two researchers put together an impressive combination of studies.

They looked at different study strategies: review, retrieval practice, diminishing-cues retrieval practice.

They tested participants after different lengths of time: right away, 24 hours later, a week later.

They tested different amounts of studying: 3 sessions, 6 sessions…

You get the idea.

Because they ran SO MANY studies, they’ve got LOTS of data to report.

The short version: “diminishing cues retrieval practice” ALWAYS helped more than traditional review (rereading the chapter). And it OFTEN helped more than plain-old retrieval practice (self-quizzing on the chapter).

If you want the details, you can check out the study yourself; it’s not terribly jargony. The process is a bit complicated, but the key concepts are easy to grasp.

To Sum Up

Retrieval practice helps students learn.

If we want to ensure that it works optimally, we should use it multiple times — and successively remove more and more scaffolding from the retrieval practice questions we ask.

Common sense and research agree.

Beyond Retrieval Practice: The Benefits of Student-Generated Questions
Andrew Watson
Andrew Watson
April-18-2020

Beyond Retrieval Practice: The Benefits of Student-Generated Questions

Read More

Retrieval Practice has gotten a lot of press in recent years — especially at our conference last fall on Deeper Learning.

The short version: students don’t benefit much from simple review — say, rereading a passage. But, they benefit a lot from actively trying to recall information — say, answering questions about that passage.

Dr. Pooja Agarwal puts it this way: Students should practice not by trying to put information into their brains, but by trying to take information out.

(She and Patrice Bain have written a great book on the topic: Powerful Teaching.)

We have LOTS of research showing that retrieval practice yields great benefits. Can other strategies match it?

Here’s an idea: maybe instead of having students answer questions (retrieval practice), we should have them create questions to be answered. Just perhaps, generating questions might boost learning more than simple review. Or — let’s get crazy: maybe generating questions boosts learning as much as retrieval practice? Even more?

Generating Research

Over the years, the “generation effect” has been studied occasionally — alas, not as much as retrieval practice. Often, research in this area includes a training session where students learn how to ask good questions. That step makes sense … but it might discourage teachers from adopting this strategy. Who has the time?

Researchers in Germany had three groups of college students read slides from a lecture about infant developmental psychology.

The first group practiced the information by rereading it. Specifically, the were instructed to memorize the content of those slides.

Group two practiced by answering questions on each slide. They if they couldn’t remember the answer, they were allowed to go back and review the slide. In effect, this was “open-book retrieval practice.”

In group three,

“students were instructed to formulate one exam question in an open response format for the content of each slide [,] and also to provide an answer to that question.”

That is: they generated questions.

So, here’s the big question: when they took a surprise quiz, how did students in each group do?

Drum Roll Please…

First: Students who generated questions scored ~10% higher on that surprise quiz than those who tried to memorize information.

Second: Students who generated questions did as well as those who used retrieval practice.

Third: Questioners got these benefits even without explicit training in how to ask good questions.

Fourth: Question generators (and retrieval practicers) scored higher than mere reviewers on both factual question and transfer questions.

Fifth: Researchers got these impressive results even though the surprise quiz took place one week later. (In research like this, those quizzes often happen right away. Of  course, a week’s delay looks a lot more like genuine learning.)

We could hardly ask for better results than these. In this research paradigm, question generation worked as well as retrieval practice — which works better than almost anything else we know of to help students learn.

Explaining Amazing Results

Why would this be? Why does generating questions help students as much as answering them?

This study doesn’t answer that question directly, but it suggests a rough-n-ready answer.

Both common sense and lots o’ research tell us: students learn more when they think hard about something. (Obvi.)

If we increase the challenge of the thinking task, we prompt students to think harder and therefore to learn better.

Psychologists talk about “desirable difficulties”: a level of mental challenge that forces students to work their synapses but doesn’t overtax them.

In this case, we can reasonably hypothesize that students who must create a question on a topic have to think hard about it. To come up with a good question, they have to think at least as hard as students answering questions on that topic.

And, they have to think considerably harder than students who simply reread a passage.

Voila! Generating questions help students learn.

A Few Caveats

As always, research provides teachers with helpful guidance. But: we need to adapt it to our own circumstances.

First: this study took place with college students. We should take care that our students can — in fact — come up with good questions.

For instance, I’m a high-school English teacher. I would use this technique with Their Eyes Were Watching God or Passing or Sula. But I don’t think I’d use it with The Scarlet Letter or Hamlet. My students struggle to understand the basics with those texts; I’m not sure they’d do a good job coming up with resonant exam questions.

More precisely: I’d structure those assignments quite differently. I suspect I could be open-ended with an assignment to create Passing questions, but would offer a lot more guidance for Scarlet Letter questions.

Second: yes, this study found that retrieval practice and question generation resulted in additional learning. And, we have a reasonably hypothesis about why that might be so.

But, we have MUCH more research about retrieval practice. Before we invest too heavily in question generation, we should keep our eyes peeled for more studies.

Third: In this paradigm, trying to memorize resulted in less learning. However, we shouldn’t conclude that students should never try to memorize. At times, “overleaning” is essential for reducing working memory load — which facilitates learning.

As long as we keep these caveats in mind, we can be excited about trying out a new review technique.

And: this can work in online settings as well!

The Neuroscience of Retrieval Practice
Andrew Watson
Andrew Watson
March-27-2020

The Neuroscience of Retrieval Practice

Read More

What’s the best way for students to practice? Should they review information or procedures? Or, should they try to remember or enact them?

We’ve got scads of research showing that retrieval practice helps brains learn.

That is: if I want to learn the definition of a word I’ve studied, I should try to recall it before I look it up again. (For a handy review, check out RetrievalPractice.org.)

So, we know that retrieval practice works. But: why? What’s happening in the brain that makes it work?

Two Possibilities

We’ve got several possible answers, but let’s focus conceptually on two of them.

Increased neural connections

Reduced neural connections

That is: when I engage in retrieval practice, I push myself to remember X. But it takes me a while to get to X. I might start with S, and then wonder about Y. Perhaps I’ll take a detour to gamma. Eventually, I figure out X.

During this mental work, I both remember X and connect X to all those other (rejected) possibilities: S and Y and gamma. By increasing connections among all these topics, I make it easier to remember X later on. If I accidentally think about S, I can quickly get to X.

Or, maybe the opposite process happens.

The first time I try to remember X, I waste mental time with S and gamma. But, the next time, I’ve gotten better at remembering X, and so I take less time to get there. I can “prune away” extraneous mental connections and thereby simplify the remembering process.

In this account, by reducing the steps involved in remembering X, I see the benefits of retrieval practice.

We Have a Winner (?)

A research team in Europe took on this question, and looked at several studies in this field.

Whenever you start looking at neuroscience research, you should brace yourself for complexity. And, this research is no exception. It’s REALLY complicated.

The short version goes like this. Van den Broek and colleagues identify several brain regions associated with memory formation and retrieval. You might have heard of the angular gyrus. You might not have heard of the inferior parietal lobe. Anyway, they’ve got a list of plausible areas to study.

They then asked: did retrieval practice produce more activity in those regions (compared to review)? If yes, that finding would support the “increased connection” hypothesis.

Or, did retrieval practice result in less activity in those regions? That finding would support the “reduced connection” hypothesis.

The answer? Less activity. At least in the studies van den Broek’s team analyzed, the “reduced connection” hypothesis makes better predictions than the “increased connection hypothesis.”

To be clear: I’ve left out a few other explanations they consider. And: I’ve simplified this answer a bit. If you’re intrigued, I encourage you to look at the underlying review: it’s FASCINATING.

To Sum Up

We have at least a tentative idea about why retrieval practice works.

And: we have SUPER PERSUASIVE evidence that retrieval practice works.

Even though we’re not 100% sure about the why, we should — as teachers — give our students as many opportunities as we can to retrieve.

What Do Teachers Get Right About Cognitive Science?
Andrew Watson
Andrew Watson
March-16-2020

What Do Teachers Get Right About Cognitive Science?

Read More

Here’s a chance to test your knowledge about the teaching implications of cognitive science. Which answer would you pick to this question?

After teaching students the names of the branches of the US government and what each does, which would be the most effective way a teacher could help their students remember this information?

A) Have students read the facts for 10 days at the beginning of class.

B) Have students copy the facts into a notebook where they can reference them as needed.

C) Have students take a once-a-week quiz for 10 weeks where they recall the facts from memory.

D) Have students participate in a review game where they have to recall the facts from memory several times in one class period.

As you think about that question — which I’ll answer later in the post — ask yourself: what basic principle of learning informs your choice?

How Can We Discover What Teachers Know?

For several years now, Deans for Impact have worked to improve teacher education. In particular, they want schools of education to emphasize well-established principles from cognitive science.

They have done lots of great work to further this mission — including publishing this invaluable resource on the science of learning. (Quick: download it now!)

Of course, if they — and we — are going to help teachers improve, we have to know what teachers already believe and do. If teachers don’t believe in learning styles theory, then we don’t have to debunk it. (Alas, lots of teachers do.)

To answer that question, Deans for Impact developed a 54 question assessment of teacher beliefs, and administered it to 1000+ teachers in the fall of 2019. The question you answered above is one of those 54 questions.

Based on the answers they got, they now have a much better idea of typical beliefs and misunderstandings. As they note, however, these teachers are enrolled in education schools that are interested in cognitive science. So:

“the data generated from this assessment is more likely to overstate what most teacher-candidates know about learning science.”

With that caveat in mind, what did they learn?

What Do Teachers Know about Cognitive Science?

Unsurprisingly, D4I found a mixed bag.

In some categories, teachers-in-training did quite well. In particular, they had good information about the importance of building, and the right ways to build, feedback loops.

That’s really good news, of course, because feedback is so important.

In general, teachers also had a clear understanding that prior knowledge matters a lot. When students lack relevant background knowledge, they struggle mightily to learn.

Sadly, teachers overestimated the possibility of critical thinking.

Of course we want our students to have strong critical thinking skills. But, for the most part, those skills don’t exist generically. That is: I must have a great deal of specific content knowledge before I can think critically about a particular topic.

If that claim seems surprising or suspect, try to answer this question: are Dreiser’s novels more like Wharton’s or Dos Passos’s? Unless you know A LOT about Dreiser and Wharton and Dos Passos (and novels), you’ll struggle to have much to say.

Needs Improvement

Alarmingly, teachers-in-training scored only 33% on questions relating to “practicing with a purpose.” We learn almost everything by practicing in the right way, so this finding should encourage us to focus quite emphatically on this research field.

To do that, let’s return to the question at the top of this post. What kind of practice would help students remember information about branches of the US government?

A) Have students read the facts for 10 days at the beginning of class.

This choice spaces practice out. That’s good. But, it doesn’t allow for active recall. As we know from the world of retrieval practice, recall creates more lasting memories than mere review.

B) Have students copy the facts into a notebook where they can reference them as needed.

This choice is a dud. It requires students to do minimal processing (“copying”!), and to do it once. Nothing to see here. Move along.

C) Have students take a once-a-week quiz for 10 weeks where they recall the facts from memory.

Choice C requires recall (a quiz). And, it includes spacing (over 10 weeks!). Spacing + retrieval looks great!

D) Have students participate in a review game where they have to recall the facts from memory several times in one class period.

This option sounds fun — it’s a game! And, it includes active recall. But, alas, active recall combined with fun isn’t as beneficial as active recall combined with spacing.

So, we might be tempted by option D — in fact, 60% of teachers-in-training chose it. Only 13% opted for choice C: the one best supported by cognitive science. (By the way: if you’re interested in combining retrieval practice with spacing, check out this research.)

In Sum

Generally speaking: keep Deans for Impact on your radar. They’re a GREAT (and greatly reliable) resource for our work.

Specifically speaking: this most recent report lets us know where we should focus most urgently as we help teachers improve our profession.

Are “Retrieval Practice” and “Spacing” Equally Important? [Updated]
Andrew Watson
Andrew Watson
February-28-2020

Are “Retrieval Practice” and “Spacing” Equally Important? [Updated]

Read More

If you follow research in the world of long-term memory, you know you’ve got SO MANY GOOD STRATEGIES.

Agarwal and Bain’s Powerful Teaching, for instance, offers a delicious menu: spacing, interleaving, retrieval practice, metacognition.

Inquiring minds want to know: how do we best choose among those options? Should we do them all? Should we rely mostly on one, and then add in dashes of the other three? What’s the idea combination?

One Important Answer

Dr. Keith Lyle and his research team wanted to know: which strategy has greater long-term impact in teaching college math: retrieval practice or spacing?

That is: in the long term, do students benefit from more retrieval? From greater spacing? From both?

To answer this really important question, they carefully designed weekly quizzes in a college precalculus class. Some topics, at “baseline,” were tested with three questions at the end of the week. That’s a little retrieval practice, and a few days of spacing.

Some topics were tested with six quiz questions at the end of the week. That’s MORE retrieval practice, but the same baseline amount of spacing.

Some topics were tested with three quiz questions spread out over the semester. That’s baseline retrieval practice, but MUCH GREATER spacing.

And, some topics were tested with six quiz questions spread out over the semester. That’s extra retrieval AND extra spacing.

They then measured: how did these precalculus students do when tested on those topics on the final exam? And — hold on you hats — how did they do when tested a month later, when they started taking the follow-up class on calculus?

Intriguing Answers…

Lyle and Co. found that — on the precalculus final exam…

…extra retrieval practice helped (about 4% points), and

…extra spacing helped (about 4% points), and

…combining extra retrieval with extra spacing helped more (about 8% points).

So, in the relatively short term, both strategies enhance learning. And, they complement each other.

What about the relatively longer term? That is, what happened a month later, on the pre-test for the calculus class? In that case…

…extra retrieval practice didn’t matter

…extra spacing helped (about 4% points).

…combining extra retrieval with extra spacing produced no extra benefit (still about 4% points).**

For enduring learning, then, extra spacing helped, but extra retrieval practice didn’t.

…Important Considerations

First: as the researchers note, it’s important to stress that this research comes from the field of math instruction. Math — more than most disciplines — already has retrieval practice built into in.

That is: when I do math homework, every problem I solve requires me (to some degree) to recall the math task at hand. (And, probably, lots of other relevant math info as well.)

But, when I do my English homework, the paper I’m writing about Macbeth might not remind me about Grapes of Wrath. Or, when I do my History homework, the time I spend studying Aztec civilization doesn’t necessarily require me to recall facts or concepts from the Silk Road unit. (It might, but might not.)

So, this study shows that extra retrieval practice didn’t help over and above the considerable retrieval practice the math students were already doing.

Second: notice that the “spacing” in this case was a special kind of spacing. It was, in fact, spacing of retrieval practice. Of course, that counts as spacing.

But, we have lots of other ways to space as well. For instance, Dr. Rachael Blasiman testing spacing by taking time in lectures to revisit earlier concepts. That strategy did create spacing, but didn’t include retrieval practice.

So, this research doesn’t necessarily apply to other kinds of spacing. It might, but we don’t yet know.

Practical Classroom Applications

Lyle & Co.’s study gives us three helpful classroom reminders.

First: as long as we’ve done enough retrieval practice to establish ideas (as math homework does almost automatically), we can redouble our energies to focus on spacing.

Second: Lyle mentions in passing that students do (very slightly) worse on quizzes that include spacing — because spacing is harder. (Regular readers know, we call this “desirable difficulty.”)

This reminder gives us an extra reason to be sure that quizzes with spacing are low-stakes or no-stakes. We don’t want to penalize students for participating in learning strategies that benefit them.

Third: In my own view, we can ask/expect our students to join us in retrieval practice strategies. Once they reach a certain age or grade, they should be able to make flashcards, or use quizlet, or test one another.

However, I think spacing requires a different perspective on the full scope of a course. That is: it requires a teacher’s perspective. We have the long view, and see how all the pieces best fit together.

For those reasons, I think we can (and should) ask students to do retrieval practice (in addition to the retrieval practice we create). But, we ourselves should take responsibility for spacing. We — much more than they they — have the big picture in mind. We should take that task off their to do list, and keep it squarely on ours.

** This post has been revised on 3/7/30. The initial version did not include the total improvement created by retrieval practice and spacing one month after the final exam.

Older posts
Newer posts