Retrieval Practice and Feedback: Do Students Know What They Know?

When I talk with teachers about retrieval practice, one question inevitably arises:

“When students try to retrieve information, do they need to know if they got the answer right? Should I give them feedback after retrieval practice exercises?”

The short answer to that question is:

“Retrieval practice helps students more than simple review, even without feedback.”

But that short answer simplifies a complex story. In this blog post, I want to explore an important follow-up question:

“Does retrieval practice help students evaluate their knowledge more accurately? Does it help them know what they really know?”

The Longest River in the World

Back in 2008, Dr. Andrew Butler and colleagues explored this question in a pair of studies.

In the first study, several college undergraduates answered 40 general knowledge questions. For example: “What is the longest river in the world?” They got feedback on 20 of those questions. For instance, anyone who said the Mississippi is the world’s longest river learned that the Nile currently has that title. Of course, students did not get feedback on the other 20 questions.

After a quick distraction task, they took a second general knowledge test. This one repeated those 40 questions and also asked 20 more. This test design lets us ask:

Did students score higher on the questions they had already answered (the initial 40) than ones they hadn’t seen before (the new 20)?
Did feedback matter?

The data make the answers very clear:

On the new 20 questions, students averaged 24% correct.
On the previously asked questions, students averaged 41% correct even if they hadn’t gotten any feedback.
On the previously asked questions, they averaged a whopping 87% correct if they did get feedback.

So, the chance to answer questions once helped a lot. And that chance with feedback made an enormous difference: from 24% to 41% to 87%.

Enter Metacognition

This first study shows that — under specific conditions — feedback helps students get answers right. We want to know: does feedback help them accurately gauge their understanding? That is: are they correctly confident in their right answers, and appropriately uncertain about their wrong ones?

Butler repeated his study, with two important changes. First, the follow-up test came two days after the initial test. This time gap means that he can evaluate longer term effects of feedback. Second, the students both answered the questions and rated their level of confidence: from 0% to 100%.

In addition to the previous questions we asked, we want to know:

Are students appropriately confident in their answers?
Are they appropriately uncertain about incorrect answers?

Once again, Butler found that recalling information once helped students recall a second time. And, the corrective feedback helped substantially. In addition, he found that feedback brought confidence into alignment with correctness.

Butler offers a lot of data to support this interpretation; I’ll keep it simple by offering three pairs of data.

On the questions that students hadn’t answered before, they got 33% correct, and believed they had gotten 38% correct.
- In other words: they got most answers wrong, and were a bit overconfident.
On the no-feedback questions they had answered once before, they got 47% correct, and believed they had gotten 56% correct.
- In other words: they got roughly half of the answers wrong, and were still a bit overconfident.
On the feedback questions they had answered once before, they got 83% correct, and believed they had gotten 83% correct.
- In other words, they got most answers right, and were appropriately confident.

One additional point. Butler calculated that students who got feedback were better at distinguishing between their right and wrong answers — not just that they were appropriately confident overall.

I started this post by asking: “does feedback after retrieval practice help students recognize what they really know?” Butler’s study gives us a reason to say “it seems so, yes.”

From Days to Months

Butler’s study offers us reason to hope, but it’s not a slam dunk. At least two concerns suggest we shouldn’t take its conclusions as final.

First: the study lasted either two hours, or two days. Such a short period of time doesn’t tell us much about long-term learning.

Second: Butler quizzed students about general knowledge. He didn’t teach them new knowledge and then study the benefits of retrieval practice and feedback.

We’re glad to have his study, but we’d like more research please!

Happily, a different kind of study — led by Dr. Nick Naujoks — helps fill in those blanks. I won’t go into all the details; this blog post already has plenty of research methodology in it. But the relevant headlines are:

Once again, this study compared students who did NOT get feedback after retrieval practice with those who DID get feedback.
Naujoks worked with students in college classes who learned new material. In other words, the study does a better job looking at actual classroom conditions.
He measured students’ performance on final exams — taken weeks or months (not days) after initial learning.

In this study, both students who DID get feedback and those who DID NOT learned more after retrieval practice. And: both groups accurately predicted which answers they got right. But — here’s the big takeaway — students who did not get feedback after retrieval practice got worse at predicting which answers they got wrong.

Here again — in this different set of circumstances — we have reason to believe that feedback after retrieval practice helps students evaluate their own level of understanding.

I do want to be cautious here. For stats-y reasons, we can’t use these two studies to make absolute claims. But we can reasonably say that both studies support these hypotheses:

retrieval practice helps students remember, and
feedback after retrieval practice helps them understand what they don’t yet know.

A Final Note

This blog routinely warns against large uplifting claims with simple catch phrases. “Embodied cognition” sounds great, but I honestly don’t know what it means. (Any word that encompases everything from mindfulness to direct instruction strikes me as too vague a term to be useful.)

For this reason, I’ve been hesitant to join Team Metacognition. Yes, “thinking about thinking” sounds like a good idea. But SO MANY different teaching practices fall under this heading that I’m unwilling to sign on to all of them.

In this case, reasonable people might say: “retrieval practice followed by feedback supports students’ metacognition!” I myself prefer a narrower claim: “when teachers provide feedback after retrieval practice, we can help students distinguish between what they truly do know and what they think they know.”

One of the unarguable benefits of this kind of “thinking about thinking”: students discover what they should think about next.

A Modest Addendum

In researching this blog post, I’ve discovered there’s some controvery about which river is the world’s longest: the Nile or the Amazon. Butler’s study says the Nile is the correct answer to that question…but not everyone agrees.

Butler, A. C., Karpicke, J. D., & Roediger III, H. L. (2008). Correcting a metacognitive error: feedback increases retention of low-confidence correct responses. Journal of Experimental Psychology: Learning, Memory, and Cognition, 34(4), 918.