A few weeks ago, I wrote about an intriguing research question. We know that retrieval practice helps students learn. And we know that generative drawing helps students learn. So: what happens if we combine those two research-informed strategies?
As I wrote back then, I think we have three answers to that question:
- Answer #1: combining strategies didn’t help. Students remembered the same amount whether or not they retrieved as they drew.
- Answer #2: Wait a minute: I’m actually not so sure about answer #1. The “retrieval” in this study didn’t really match the typical definition of “retrieval practice.” The students recalled something they had just read, not something they had learned a few days before. In other words: they were “retrieving” from working memory, not from long-term memory. That’s not really retrieval practice.
- Answer #3: although we haven’t learned the answer to the question–at least in my opinion–we have learned that research into combining strategies gets tricky.
I’ve just come across another study asking a similar “is two better than one?” question. Here goes:
- We know that jumbling practice on related topics together helps students learn. That is: rather than practice “adjectives” and “adverbs” separately, students benefit from doing practice questions on both topics together. (We call that approach “interleaving.”)
- And we know that asking students to answer questions before they have studied the answer helps students learn. (We call that technique “prequestioning” or “pretesting.”)
- Dr. Steven Pan and his colleagues wanted to know: does interleaving + prequestioning result in more learning than either strategy on its own? Or, the same amount of learning? Less learning?
Let’s explore…
“What Is Bipolar Disorder?”
Using an online research platform, Pan and his colleagues asked adults to learn about six different mental conditions: ADHD, bipolar disorder, etc.. To learn this information, participants read short “case study” passages about three people with each condition.
- Half of the participants read three consecutive passages about any one condition. That is: they read 3 passages about a person with schizophrena; then 3 about someone with autism; and so forth. (This structure is the opposite of “interleaving”; we call it “blocking.”)
- The other half of the participants read those same passages–but all jumbled together. So, they might read a passage about someone with ADHD, then another about someone with schizophrenia, then another about someone with bipolar disorder. (Yup: this is “interleaving.”)
So: all the participants read the same 18 passages in groups of 3 passages. But half of the participants read them in “blocked” groups; the other half read them jumbled up in an “interleaving” structure.
So far, this research explores the relative benefits of blocking or interleaving. But what about the “prequestions“?
- Within each of those groups, half of the participants saw the list of technical names for the six conditions that they’re learning about: “cyclothymic affect disorder,” “resonance development disorder,” “schismic cognition disorder,” and so forth. (The researchers deliberately chose obscure technical names so that participants couldn’t rely on prior knowledge—they had to learn the associations from scratch.) Participants had to guess: which one of these six is the passage about?
- They had to answer these questions without having yet learned about the conditions. That’s a “prequestion.”
- The other half saw the technical name at the top of each passage they were reading. So: a case study about bipolar disorder would be labeled “cyclothymic affect disorder.”
- No prequestion here.
With these groups and subgroups, Pan and Co. can now discover which combination helps students learn the most:
- It could be BLOCKING, with our without PREQUESTIONS
- It could be INTERLEAVING, with our without PREQUESTIONS
To be extra careful, the researchers checked to see how much the participants learned five minutes later. And then — with a completely different group of participants — they checked to see how much they learned 48 hours later.
We’ve Got Answers (I’ve Got Questions)
Team Pan wanted to know: does combining interleaving (a structure) with prequestioning (a technique) increase learning?
In a word: YES.
- Both right away and 48 hours later, these adults learned more about identifying cognitive disorders if they read jumbled case studies (“interleaving) and had to guess what each one was (“prequestioning”).
- The group that read the case studies all clumped together (“blocking”) without prequestions learned the least.
- And the groups that either studied with interleaved case studies or answered prequestions scored in the middle — roughly the same as each other.
On the one hand, this helpfully direct answer gives teachers useful guidance. Sure enough, using an interleaving structure with a prequestioning technique helps students learn.
I do, however, have two questions.
First: A quiz after two days gives us more useful information than a quiz right away. But “two days” isn’t yet learning. In fact, we’ve got all sorts of reasons to worry that “short term performance does not accurately predict long-term learning.” (Check out this research review.) I will feel MUCH more confident offering this guidance if we have studies confirming these results after noticeably longer periods of time.
While I’m at it, I should say: adult learners might benefit from different combinations than K-12 learners. For this reason, I’d also like to see replications with different age groups.
To be clear, Pan and his colleagues haven’t done anything wrong by limiting their study to two days, and working with adults. No one study answers all questions. They’ve given us a helpful set of data points. I myself think we need several more collections of data points before we offer strong advice. They themselves raise this point in their study.
Second: The procedure described above uses a version of “prequestioning”; but it’s not exactly “prequestioning.” With typical prequestions, students almost certainly don’t know the answer because they haven’t yet studied the material. In this case, the students (probably) don’t know the answer at the beginning of the study. However, over time, they have more and more reason to know the answer.
To my way of thinking, this study STARTS by combining interleaving with prequestioning, and ENDS UP combining interleaving with a version of retrieval practice. After all: by the final round or two, the students can start recalling the correct answers that they’ve been studying up to now.
Here again, I’m not describing a flaw in the study. At the same time, I do think it’s helpful to have our terminology as clear as possible.
To Combine Or Not
Based on this research, we have a tentative reason to believe that starting with questions that students can’t yet answer (“prequestioning”) and jumbling practice problems together (“interleaving”) helps students learn.
As teachers, we can start trying out this combination of research-supported strategies.
And: we can keep an eager eye out for more research into strategy combinations. A whole new fruitful field awaits…
Pan, S. C., Selvarajan, G., & Murphy, C. S. (2024). Interleaved pretesting enhances category learning and classification skills. Journal of Applied Research in Memory and Cognition, 13(3), 393.
