Ask almost anyone in Learning and the Brain world, they’ll tell you: retrieval practice benefits students.
More than most any other technique we have, this one both has lots of research support and can easily be integrated into our classrooms. (For a handy review of its benefits, check out Agarwal and Bain’s great book, Powerful Teaching.)
Of course, because retrieval practice offers so many benefits, we want our students to use it.
Do they?
Metacognition and Retrieval Practice
The short answers are: not enough, and not very well.
Michelle Rivers wanted to know why, and so explored research into students’ metacognitive beliefs about retrieval practice. That is:
What do they believe about retrieval practice?
How do they use it to monitor their learning?
How do they use its insights to control their learning activities?
The more we understands students’ metacognitive perspective on their learning, the more wisely we can guide them.
Sadly, most students don’t understand RP’s benefits.
In 10 studies that asked “why do you quiz yourself,” only 26% of students say they do so in order to learn more.
Instead, most students (52%) do so “to figure out how well I’ve learned the information I’m studying.”
In other words: even the students who use RP most often do so for the wrong reasons.
Of course: they’re not harming themselves by using retrieval practice this way. But — and this is a big but — they’re not getting the benefits that RP can offer.
In fact, Rivers’s survey suggests one reasons students might not use retrieval practice to help themselves learn. Studies suggest that when students try both methods, they don’t predict that they’ll remember more after retrieval practice. (Check out this study by Kornell and Son.)
I find this research pool baffling, even disheartening. Even when students experience greater success with RP than with simple rereading, they don’t internalize the broader lesson that active retrieval helps them learn.
Little wonder, then, that most students review material (43%) or copy their notes (11%) as a go-to strategy, rather than self-testing (8%).
Uses of Retrieval Practice
Given these flawed beliefs, how do students use RP?
Among Rivers’s findings: students try retrieval practice …
… when the questions are easy
… right before a test
… relatively late in the learning process.
… relatively few times for any given pool of information.
Of course, retrieval practice benefits students when they do so…
… several times for any given pool of information.
Simply put: even when students use this excellent study strategy, they do so in less-than-optimal ways.
Next Steps: Learning How to Learn
So far, this is quite the glum post. A potentially powerful learning strategy is largely going to waste.
What can we teachers do?
I’ve got two suggestions.
First, this recent post summarizes a promising approach from Mark McDaniel and Gilles Einstein. Their multi-step process not only works to persuade students of RP’s benefits; it encourages them to make specific retrieval practice plans and to follow through on them.
In other words: we shouldn’t just tell our students about its benefits. We shouldn’t just tell them to do it. We should go the next steps to create plans and structures.
Second, I’ve seen LOTS of online programs to help teachers and students with their retrieval practice.
For instance, Adam Boxer has created a program called Carousel. This program allows teachers to create retrieval questions, and to jumble them together in useful ways. It allows students to self-score their work (and teachers to correct any errors). It keeps track of right and wrong answers, so we can see how well our students are learning specific topics and questions.
I have used Carousel enough to find it intriguing; I haven’t used it enough to make strong claims about it. (Given responses to it on Twitter, however, it seems that teachers kind of love it.)
Whichever program you choose, I think students will learn how to learn more effectively if we build tools like these into our teaching practice.
In Sum
A: We know that retrieval practice can help students learn, but only if they use it correctly.
B: We know that, for the most part, they don’t.
A + B = We should focus more on helping students use this strategy wisely. And, we’ve got the tools to do so!
Charles Fadel will be speaking on “Creativity, Curiosity, and Collaboration” at the upcoming Learning & the Brain conference.
He is the founder of the Center for Curriculum Redesign at Harvard’s Graduate School of Education.
The Innovative Mission of the Center for Curriculum Redesign
Andrew Watson:
Most educational reform focuses on the way the teaching happens: project based learning, or flipped classrooms, or technology, or that sort of thing. Your focus is more on curriculum, which is to say, what it is that teachers are actually teaching. Why have you chosen that focus, instead of the method of teaching?
Charles Fadel:
When I was at Cisco, we started the Partnership for 21st Century Skills; we started looking at the skills that make people successful all their life.
But as I started getting more and more into this question, I realized—wow!—in the past 50 or so years everyone has been so focused on how to teach better, because somehow we assumed that what we taught was adequate. And it was probably adequate since the Industrial revolution, to a point. But nowadays, it’s really not adequate anymore.
There are several difficulties with this question.
First of all, if the curricula are already chock-full of knowledge items, there is no time and space available to teach skills, and character qualities, and meta-learning abilities. And systems are very, very resistant to removing some things so that there might be time and space for these new competencies.
Second, there are all sort of human dynamics at play. Do you think that a literature expert will say, “Well, that’s fine, we can just read two books this year, and the other eight we’re just going to do cliff notes.” That’s never going to happen. Right?
Naturally enough, everyone considers that their discipline is the most important thing, and they’re not going to let go of it. We call this “expert (confirmation) bias.”
Third you have the politics. As I go around the world, I realize to what extent the curriculum is prisoner of parliaments and politicians. And they, for political expediency’s sake, decide some of the things that are taught. Whether it’s history, or languages, or so on—and these decisions also crowd the curriculum.
Funny enough, parliaments don’t have too many opinions about maths…
Andrew Watson:
People are generally in favor of math.
Charles Fadel:
Well, yes, we are too, but should that mean that we abdicate our responsibility entirely to the hands of maths academics? Who have no idea about how math is used in the real world.
So, for all of these reasons, I realized—wow!—the curriculum is in trouble. What should we be learning?
Do we need to learn the multiplication tables? Yes, no, and why?
Do we need to learn long division? Yes, no, and why?
How do we decide? And how can we remove obsolete and extraneous items to make time and space for
deepening understanding of traditional disciplines (Maths, History etc.),
adding modern branches to the traditional disciplines, like say adding algorithmics to mathematics, not just algebra and geometry,
adding modern disciplines, like entrepreneurship, robotics, etc. and
very importantly, combining all these disciplines with the competencies that need to be developed. By “competencies” I mean skills, character, and meta-learning.
How in the world are we going to do all this unless we curate what’s already on the dockets?
Andrew Watson:
How hard can that be?
Charles Fadel:
The current curriculum a bit like a budget deficit, you know. Every single line item you can justify in insolation from the rest. You know this restaurant you went to with your significant other, that was a good thing to do in isolation. But, you do this too many times and you’ve broken your budget.
So every single line item in the curriculum needs to be scrutinized, and that’s not something that disciplinary expert groups are particularly well suited to do left on their own—because of the “expert bias” I described before.
Andrew Watson:
I can tell you from my own school experience that the battles fought over curricular changes are quite fierce. So, it’s a good thing the Center for Curriculum Redesign exists.
Charles Fadel:
We at the Center are non-political and non-dogmatic. We don’t have an axe to grind; we can look at all these things more cold-bloodedly and say “you know what, after analyzing and polling, we haven’t heard a profound justification for why long division should stay, therefore it goes …”
Curriculum Redesign to Foster Creativity, Curiosity, Collaboration
Andrew Watson:
Okay. I’m going to change gears to introduce your talk at the up-coming Learning and the Brain conference. That talk, as I understand it, focuses on creativity, curiosity and collaboration, which are in the Skills portion of your book.
Charles Fadel:
Actually “creativity” and “collaboration” are Skills, but “curiosity” is in the Character portion.
Andrew Watson:
Oh right, okay. I want to think about those topics in reverse order.
Starting with collaboration: Is the way to teach collaboration just say, “Hey everybody, you have to work together to perform this task” and assume they’ll figure out how to collaborate? Or is it a skill we can teach with a collaboration curriculum? If “yes,” how do we do that?
Charles Fadel:
Any single one of these 12 competencies that we have identified in the book’s framework can be taught.
It’s not enough to say “Here, we’re going to throw you in a collaborative situation, good luck to you.”
The same way that there are better techniques for how to deploy projects, there are better techniques for how to collaborate. For instance, by understanding the psychological make up of your team members.
In the corporate world, for example, we use the Myers-Briggs Type Indicator. I’m not saying that the Myers-Briggs has huge science behind it, but I’m saying that it’s a facilitator of the conversations of team members.
Andrew Watson:
Understood.
Charles Fadel:
The Myers-Briggs test might say, this person has a certain dominant personality characteristic, and that’s why they are not comfortable making decisions right away. Whereas I’ll be tempted to just jump to a conclusion and make a decision early on.
The test helps me take a deep breath about my own behavior, about their behavior. It forces certain metacognitive views of people and the world.
So that’s the sort of thing you can do to understand yourself and others better. That’s part of what you would want to learn to better collaborate.
You can also teach certain group principles, such as respect. Making sure that everyone has a chance to speak. Even that shy introvert in the corner. Make sure the team leader puts the spotlight on them and says, “you can you speak now, or if you’d be more comfortable you could send me an email.” Et cetera.
So, there are all sorts of techniques. To use communication analogy: the same way that they tell you “don’t wear a striped shirt when you go on camera,” you know there are plenty of tips and tricks that one needs to know to improve collaboration.
And so all of these things need to be taught. In fact, they should first be taught to teachers, because they have not necessarily been in a collaborative environment themselves. It’s such an isolated profession!
Andrew Watson:
Got it. Let’s talk a little bit about curiosity. Schools have a really bad reputation about curiosity and are often are accused of interfering with curiosity. Can you think of ways we would be better at promoting curiosity?
Charles Fadel:
Right, so first let’s have a balanced conversation about how schools are “bad at creativity.”
Remember, schools have to cover a certain amount of material in a certain period of time. Yes, we want to give students enough agency and free will to learn things their own way. But at the same time, there’s not an infinite time to allow them to experiment forever.
And so that’s the complexity of the balance to strike.
Now can traditional schooling do a far better job? Absolutely.
But I’m saying schools don’t have to be entirely one way or the other. Just as research shows that we need to have the right balance between didactic instruction and constructivism/projects, we also have to have the right balance of student choice and student non-choice.
Now, should we err in the direction of student choice right now? Yes, because we’ve gone too far in the wrong direction of non-choice.
With that said, there are ways to bring these approaches together.
For instance, let’s say that the curriculum says you’ve got to learn robotics. The choices could be: well I’m going to do a swimming robot, and you’re going to do a flying robot, then the third person is going to do an underwater robot. But Robotics are important so everyone has to learn that – their own way.
And you can have finer-grade choices as well.
Everyone does need to understand robotics, but we don’t all need to go beyond basic skills in programming.
If that is not where my forte is, I’m going to devote more of my time to making the robot emote and mimic emotion—because that’s what I’m attracted to. I want to go into anthropology later, so emotions are my interest.
So you see, there’s a blend of requirements and choices: meaning, even the anthropologist has to understand how to code. But, the anthropologist doesn’t have to be the best possible coder.
You’d say “okay anthropologist, you got passing grades in coding, move on. You don’t have to be the best one, but at least you understand the principles of coding, and now we can have you focus on the thing that jazzes you up – and know how to interface with the coder when needed later.”
So we recommend a constant blend of choice and prescription.
There’s an important graph we have in the book. The ratio between choice and prescription changes over time.
When they’re young kids, most of the curriculum is going to be prescribed. We’re not going to let you decide whether or not you want to learn how to read and write and compute. It’s just not a choice. Otherwise you’re not going to be a functional member of modern society, your chances will be severely truncated…
But, by high school, it’s certainly clear that if you want to do a given type of project, you should have some latitude. And so the ratio of prescription to choice changes over time.
Andrew Watson:
In some ways it sounds like curiosity and collaboration work well together. Because I’m curious about anthropology, I’m going to collaborate with someone who has more technical skills in this robot building process. The fact that I’m working collaboratively in a team means I get the focus on the thing I’m most particularly curious about.
Charles Fadel:
Yes. These 12 competencies are not isolated; they’re not silos in our brains. All 12 interplay with each other.
Andrew Watson:
Let’s change gears and think about creativity. What strategies do you see out there that are going to be helpful in teaching students creativity?
Charles Fadel:
Remember that CCR focuses on the what—the curriculum—not the how—the teaching methods. If your questions relate to classroom techniques, I’m really not the right person to ask.
But, we have to be clear about the where. These competencies must be practiced in as many disciplines as possible.
We might be tempted to say: “arts teach creativity.” Well for creativity to stick – to transfer – it has to be activated in several different disciplines. We would want as many disciplines as possible to develop as many competencies as possible.
At the same time, let’s not fool ourselves. It may be a bit exaggerated to say that every discipline can help with every competency.
Andrew Watson:
So for all three of these—for collaboration, curiosity and creativity—you’re not at this point advocating teaching strategies. You’re making an argument that we have to teach these competencies, and you’re putting together a cognitive framework to help teachers think about how they inter-relate with each other.
Charles Fadel:
We’re trying to make sure that—at a minimum—each discipline is tasked or not tasked in developing them. Could you teach collaboration in every single discipline? Yes. Are some—like engineering and art—more conducive to that? Yes.
Part of the complexity is that teachers want to say: yes, I agree with you. Let’s all teach collaboration. However, perhaps some disciplines are better suited for some competencies than others. It’s exaggerated for every discipline to think that they’re going to do everything.
“Incremental Ambition”
Andrew Watson:
In the book, you describe your work as “incrementally ambitious,” which is a very fun phrasing. Can you unpack that a little bit?
Charles Fadel:
Sure. If you use a word like “radical,” it sounds good for getting on the New York Times front page. But, you know, that’s really not the point.
“Radical” implies you throw away the existing system. You “disrupt,” which is a term that is so over-used.
We want to profoundly change, but in an evolutionary, not revolutionary sort of way.
Now I hear the irritation of those who say, “to hell with the system. It’s not listening; therefore I’m going to take it down.”
But it’s a lot easier to take down than to build. And so the challenge for the CCR is change management. That’s why we have to be ambitious, but incrementally so.
Andrew Watson:
The people who are at Learning and the Brain conferences, by definition, those are people who want to change what we’re doing. But, we don’t want the change to mean that everything we’ve ever worked for goes away. We want to be better at what we’re doing. Which is why the ambition has to be incremental.
Charles Fadel:
It’s a lot easier to hope for some magic solution, perhaps using technology, some extraordinary e-tutor. You just hope, oh if I only had a magic wand. That’s not the way the world works.
Andrew Watson:
If a magic wand exists, we haven’t found it in the teaching world yet.
Charles Fadel:
Or in the technology world.
Andrew Watson:
Thank you for taking the time to talk with me today.
Charles Fadel:
You’re welcome.
Charles Fadel is a global education thought leader and author, futurist and inventor; founder and chairman of the Center for Curriculum Redesign; visiting scholar at Harvard GSE; chair of the education committee at BIAC/OECD; co-author of “Four-Dimensional Education” (available in nine languages) and best-selling “21st Century Skills”; founder and president of the Fondation Helvetica Educatio (Geneva, Switzerland); senior fellow at The Conference Board. He has worked with education systems and institutions in more than thirty countries. He was formerly Global Education Lead at Cisco Systems, visiting scholar at MIT ESG, and angel investor with Beacon Angels. He holds a BSEE, an MBA, and seven patents. Full Bio at: http://curriculumredesign.org/about/team/#charles
Center for Curriculum Redesign
The mission of the Center for Curriculum Redesign (CCR) is to answer this timely question, and openly propagate its recommendations and frameworks on a worldwide basis. The CCR brings together non-governmental organizations, jurisdictions, academic institutions, corporations, and non-profit organizations including foundations.
The last major changes to curriculum were effected in the late 1800’s as a response to the sudden growth in societal and human capital needs. As the world of the 21st century bears little resemblance to that of the 19th century, education curricula need to be deeply redesigned for the four dimensions of Knowledge, Skills, Character and Meta-Learning. Adapting to 21st century needs means revisiting each dimension and the interplay between them.
This paper outlines the CCR model for curriculum redesign.
Working with Charles Atwood at the University of Utah, Brock Casselman tried an idea:
He had students in a general chemistry class do weekly online problems and practice tests; after completing that work, the students received detailed feedback.
In addition to this online practice, half of the students also predicted their scores before they took the tests; they then made study plans after they received the feedback.
Did this additional work help?
Indeed it did. On average, it raised grades on the final exam by 4%.
Even more impressively, those in the bottom quartile of the class raised their exam grade by 10%.
Especially for those who struggled with the material, making predictions and updating study plans boosted learning.
Reasons to Celebrate; Reasons to Pause
Of course, this research is quite helpful in giving us specific teaching advice. The more we can encourage our students to stop and predict their success, the more we can prompt them to make thoughtful study plans, the more that they’re likely to learn.
So far, so good.
However, I do see two reasons to add a note of caution.
First, this study was done in a difficult college class; according to this interview, only 2/3 of the students who take the class ever pass it.
A study technique that helps in such a difficult class might be beneficial to students in less rigorous classes…but, we can’t be sure based on this research.
Second, I do worry about the broad vocabulary used to describe this study technique: “metacognition.”
No doubt you’ve heard of metacognition: it means “thinking about thinking.” When I stop and ask myself, “now, why did I get that problem wrong? What patterns do I notice with other mistakes I made?” I’ve engaged in metacognition.
Here’s the potential danger. While it is true that Casselman’s particular set of metacognitive strategies helped these students, that doesn’t mean that ALL metacognitive strategies will help ALL students.
For instance, you might read that “using context clues” is a metcognitive strategy. It certainly is. And, of course, using context clues might well help students to important discoveries.
However: that’s not the metacognitive strategy that was used in this case. So, this study doesn’t show that using context clues would help students in this chemistry class.
Or that it would help your students.
Boundaries Matter
In a recent post, I encouraged teachers to look for boundary conditions. In other words: we’re interested in researchers’ general findings, but we want to be sure that they apply specifically to our students.
To do so, check out the “participants” section of the research you’re reading. If the students who participated in the research resemble your students, then you’re good to go. If not, use your own best judgment about the applicability of that research.
Equally important: be sure that the specific techniques described as “metacognition” are in fact the ones that you’re using. If not, you should look for more research to be sure you’re on the right track.
After all, my predictions about the benefits of metacognition might be correct–but if my results show that a particular metacognitive strategy didn’t work, I need to develop a new study plan.
Ok: you’ve taught your students a particular topic, and you’ve provided them with lots of ways to review and practice for the upcoming test. But, will they do so?
How can you ensure that they prepare most effectively?
Patricia Chen’s research team studied a surprisingly simple answer to this question. You might help your students study by asking them to think about the approaches that they will use–and, to make specific plans.
Chen & Co. asked students to follow a four step process:
Step 1: students wrote about the kind of questions they expected on the test.
Step 2: they then chose the resources they wanted to use to prepare for those questions. The checklist from which they chose included 15 options, such as “go over practice exam questions,” “go to professor’s office hours,” and “work with a peer study group.”
Step 3: they wrote why and how they thought each of the resources they selected might be helpful.
Step 4: they made specific and realistic plans about where and when they would use those resources.
Compared to a control group–who were simply reminded that they should study for the upcoming exam–students in this group averaged 1/3 of a letter grade higher.
For example, students in the control group had an average class grade of 79.23. Those who went through these 4 steps had an average grade of 83.44.
That’s a lot of extra learning from asking four basic questions.
What Should We Do?
Chen’s research team worked with college students studying statistics. Do their conclusions apply to–say–5th graders studying history? Or, 10th graders learning chemistry?
As is so often the case, individual teachers will make this judgment call on their own. Now that you’ve got a good study suggesting that this method might work, you can think over your own teaching world–your students, your curriculum, your approach to teaching–and see if this technique fits.
In case you decide to do so, I will offer three additional suggestions.
First: check out Gollwizer’s work on “implementation intentions.” His idea overlaps with Chen’s work, and would pair with it nicely.
Second: I’m a little concerned that Chen’s list of proposed study strategies included two options we know don’t help–reviewing notes and rereading the text. (If my skepticism about those two methods surprises you, check out Ian Kelleher’s post here.) Your list of study strategies should NOT include those suggestions.
Third: as always, keep working memory limitations in mind. The kind of meta-cognition that Chen outlines can clearly benefit students, but it also might overwhelm their ability to keep many ideas in mind at the same time.
However, if we can prevent working memory overload, this strategy just might help bridge the gap between “I taught it” and “they learned it.” As is so often the case, a key plank in that bridge is: asking students to think just a little bit more..
Research into the benefits of bilingualism has gotten lots of attention in recent years. For example, some scholars argue that being bilingual protects our cognitive dexterity as we age.
However, a recent study suggests a potential downside for bilinguals. Folke et. al. find that, compared to their monolingual peers, young bilingual adults have a harder time with metacognitive processing — that is, analyzing their own cognitive performance.
If further research supports this finding, then teachers and scholars will have to add this potential short-term cognitive detriment to their calculus as they consider long-term cognitive benefits.
To be clear: this research does not show that being bilingual is cognitively bad, or that bilingual education is a bad idea. Instead, it offers one potentially interesting data point for a complex discussion — a discussion that must consider benefits, detriments, and many, many unknowns.
