The ever provocative Freddie deBoer explores the relationship between correlation and causation.
You know, of course, that the one does not imply the other.
DeBoer, however, wants to push your certainty on this point.
Are there circumstances under which proving causation would be immoral, and therefore correlation is a useful placeholder? (Do you really want to run the double-blind study about smoking cigarettes?)
Are there circumstances under which the causal chain is wildly complicated, and so correlation is an important clue?
In other words: while correlation doesn’t prove causation, common sense tells us that it’s an interesting starting point. And: we often face circumstances where causal proof is hard to come by, and so correlation gets our attention as a useful indicator of potential causation.
As long as we’re careful about these subtleties, we can allow ourselves to notice correlation, and speculate (humbly) about causation.
Here’s how deBoer concludes his article:
What we need, I think, is to contribute to a communal understanding of research methods and statistics, including healthy skepticism. […] Reasonable skepticism, not unthinking rejection; a critical utilization, not a thoughtless embrace.
That’s a hard logical place to find; here’s hoping we can find it together.
____________________
Update: I wrote the piece above on 11/8. Today (11/16), Greg Ashman posted a thoughtful piece making very similar arguments. I wonder what coincidence implies about causation…
Parents—a noun, something an individual may be—have existed for as long as there have been children. The idea of “parenting” as a verb, something one does, is a new, odd, and problematic cultural change for parents and children alike. Alison Gopnik, a self-described “bubbe at Berkeley” or mother and grandmother, and University of California, Berkeley professor of psychology and philosophy, argues that parents can aspire to love their children better without thinking of their role as making their child into a certain kind of adult. Her most recent book, The Gardener and the Carpenter: What the New Science of Child Development Tells Us about the Relationship Between Parents and Children, references the idea that parents should act like a gardener, providing a rich and safe environment in which flowers (or people) can grow into the best version of themselves and thrive. Gopnik advises that Parents should not mold children into a particular and unchanging kind of being the way a carpenter molds furniture into its specific form. tThroughout the book Gopnik explores parenting paradoxes and the implications for how to support children. These paradoxes include the fact that children must transform from complete dependence to complete autonomy; children must transform from people who mostly play to people who mostly work; parents need both to pass on knowledge and traditions and to allow kids to innovate and differentiate; and parents’ love is specific to their individual children, which can conflict with the job of supporting positive development for a community’s children.
The verb of “parenting” only emerged towards the end of the 1950s and then spiked in popular usage in the 1970s. Rather than parenting, we need to let kids learn and grow by providing them a safe and protected period to do so. Tragically, even though the U.S. is the richest nation, we do not do a good job as a society of protecting children (e.g., 1 out of 5 kids lives in poverty, we do not have guaranteed paid leave for parents to care for kids).
Gopnik argues that an evolutionary perspective on the parent-child relationship can help us understand how to be parents today. We are unique among species in our long period of childhood, and across species longer average childhoods are associated with greater parental involvement, fewer children per mother, longer lives of offspring, higher survival rates of offspring, more intelligence, and larger brains. The tendency of our species to form long-term bonds with a co-parent, invest significantly in raising children, live long enough to become grandparents, and rely on other adults to assist with child care have all contributed to our longer childhoods.
Our longer childhoods provide a protected period of observation, learning, and play, which contributes to our ability to become flexible, innovative adults. Children learn from observing actions and intentions of other people, especially individuals similar to them, who have proven themselves to be good teachers, and who convey confidence. Apprenticeships, in which students see skills modeled and in which their learning process and production of products are entwined, is an effective way to learn. Children learn from what adults say and from answers to the questions children ask adults. Play helps kids engage in counterfactual thinking, develop theory of mind, and develop cognitive flexibility, but kids must feel safe to engage in play. We do not need to formally instruct in order to support young children in learning; rather, we should involve them in the activities of our daily lives and make them feel secure. Gopnik argues that wealthy children often have overly scheduled and controlled environments, while poor children often have chaotic lives in which they are neglected. Few children have the ideal environment that feels both safe and free.
Very young children show tremendous versatility in their learning and plasticity in their brain. By about school age kids become a bit less flexible but more efficient, both in how they learn and in how their brain is connected within itself. Adolescence brings a reemergence of neural plasticity and a desire to innovate. During this period of innovation, parents serve as a bridge to the past as they watch their teens enter into the future.
Parents who are turned off by the pressure to prepare kids for the Ivy League before they have started Little League, educators who are concerned by a school culture or the broader culture of overly involved parenting, policy makers interested in creating scientifically informed policies to support the next generation’s development, and anyone interested in how the relationship between parents and children make us uniquely human will find Gopnik’s book informative and compelling.
Gopnik, A. (2016). The Gardener and the Carpenter: What the New Science of Child Development Tells Us About the Relationship Between Parents and Children. New York, NY: Farrar, Straus and Giroux. Macmillan.
Dr. Daniel T. Willingham from the University of Virginia was presented with the “2017 Transforming Education Through Neuroscience Award” for his contributions to the field of Mind, Brain, and Education during the Learning & the Brain® educational conference in Boston, MA.
A groundbreaking researcher whose work lies at the intersection of education and cognitive neuroscience was awarded the tenth annual prize for “Transforming Education Through Neuroscience.” The award was established to honor individuals who represent excellence in bridging neuroscience and education and is funded by the Learning & the Brain® Foundation. The $2,500 award will be used to support translational efforts bridging scientific findings and classroom practice.
Dr. Willingham is being honored for his work on learning and memory and the applications of cognitive psychology to education. He has devoted much of his career leveraging scientific findings to address important educational issues. Dr. Willingham received his BA from Duke and his PhD in Cognitive Psychology from Harvard University and is now a Professor of Psychology at the University of Virginia. He writes the “Ask the Cognitive Scientist” column for American Educator magazine. Dr. Willingham is also the author of the following books Why Don’t Students Like School?, When Can You Trust the Experts?, Raising Kids Who Read, and The Reading Mind. His writing on education has appeared in fifteen languages.
David B. Daniel, PhD, Professor of Psychology at James Madison University and the 2013 winner of the award, had praise for the new recipient. “Dr. Willingham is one of the nation’s most responsible and effective translators of psychological science to educational practice. He is adept at synthesizing seemingly divergent literatures and working across multiple levels-of-analysis to construct evidence-based, usable knowledge for educational practices. Dr. Willingham’s impact across the field, from the classroom to policy, is both important and influential.”
According to John D.E. Gabrieli, PhD, the Grover Hermann Professor of Health Sciences and Technology and Professor of Cognitive Neuroscience at the Massachusetts Institute of Technology, “Daniel Willingham has been the leader in communicating concepts and advances in the science of learning to a broad community of educators and parents. He has created a bridge between the best of science and the most important challenges that teachers and students face in the classroom.”
Dr. Daniel presented the prize to Dr. Willingham at the Learning & the Brain® educational conference in Boston, MA on Sunday, November 12, held at the Westin Copley Hotel. The Learning & the Brain® Foundation wishes Dr. Willingham our heartiest congratulations.
In the excitement of this weekend’s Learning and the Brain conference, I overlooked my own one-year anniversary as editor of this blog.
I’ve enjoyed the chance to think aloud with you about teaching, psychology, neuroscience, research–and all the odd topics that come across my desk. (Squirrels, anyone?)
I’ve particularly enjoyed the chance to interact with so many of you. Please keep your emails, questions, suggestions, criticisms, and experiences coming: [email protected].
Presumably, your eyes automatically turn to the part of the visual field that stands out the most: the bright red door, the tower jutting up from the cliff, the sharp angle against all the curves.
However, it’s also possible that your eyes naturally turn to the part of the visual field that means the most: the subtle-but-essential clue, the small-but-important face, the mundane-but-crucial key that unlocks the mystery.
In the first hypothesis, our visual systems immediately process information without taking meaning into account; in the second, those systems take enough time to include meaning in their guidance.
John Henderson‘s team developed quite an intricate system for testing these possibilities. (You can read the full description here.)
The short version is: they used images where the part that stood out was distinct from the part that meant the most. And, they used an eye-tracking gizmo to see where people looked first.
The answer: eyes focus first on meaning.
Even at the most basic level of processing, our brains prioritize meaningful information over flashy information.
What Henderson’s Research Means for Teachers
This study reminds me of Daniel Willingham’s response to learning styles theory.
In Why Don’t Students Like School, Willingham argues that–for example–visual processing differences don’t make much difference for most students because–most of the time–we don’t want our students to think about what something looks like, we want them to think about what that something means.
Henderson’s study suggests that, even at the moment of initial processing, our eyes prioritize meaning.
For this reason, it can be true that some people remember visuals better than others, but they still aren’t “visual learners.” All of us are “visual learners” because all of our eyes focus on meaning more than on purely visual salience.
How do psychologists know what they know about human mental processes?
Quite often, they run studies to see how people behave: what do they remember? where do they look? what do they choose? how do they describe their thoughts?
If they run those studies just right, psychologists can test a very small number of people, and reach conclusions about a very large number of people.
Perhaps they can reach conclusions about all 7,400,000,000 of us.
Unless…
What if that small group of people being studied isn’t even remotely a representative sample of the world’s population. What if almost all of them are psychology majors at American colleges and universities?
What if they are–almost exclusively–from countries that are Western, Educated, Industrial, Rich, and Democratic?
(Notice that, cleverly, those adjectives acronym up to the word WEIRD.)
Here’s an example of the problem. Last year, I spoke about Mindset at the African Leadership Academy in South Africa: a school that draws students from all across the African continent.
And yet, I know of no research at all that studies Mindset in an African cultural context. I could share with them research from the US, and from Hong Kong, and from France, and from Taiwan. But Africa? Nothing.
How valid are Mindset conclusions for their students? We don’t really know–at least, “know” in the way that psychologists want to know things–until we do research in Africa.
(By the way: if you know of some Mindset research done in Africa, please send it my way…)
Beyond Psychology
This article over at The Atlantic does a good job of describing this problem in neuroscience.
Because the sample of the population included in neuroscience studies is so skewed, the conclusions we reach about…say…typical brain development schedules are simply wrong.
Better said: those conclusions are correct about the subset of the population being studied, but not necessarily correct for everyone else.
And, of course, most people are “everyone else.”
What Does This Problem Mean for Teachers?
Here’s my advice to teachers:
When a researcher gives you advice, find out about the participants included in their study. If those participants resemble your students, that’s good. But if not, you needn’t be too quick to adopt this researcher’s advice.
For example: if a study of college students shows that a particular kind of challenging feedback promotes a growth mindset, that information is very helpful for people who teach college.
But, if you teach 3rd grade, you might need to translate that challenging feedback to fit your students’ development. In fact, you might need to set it aside altogether.
Because participants in these studies are often so WEIRD, we should beware extrapolating results to the rest of the world’s students, including our own.
The answer to the titular question depends on a) your definition of “project-based learning,” and b) your methodology for measuring success.
In a just-published, comprehensive literature review, MDRC takes 84 pages to say: “we can’t really answer the question, because we don’t have consistent definitions or consistent methodologies.”
For example:
Without a common set of PBL design principles, it is difficult to use the existing body of research to draw conclusions about PBL’s effectiveness. (p. 53)
or
More rigorous evidence is needed to confirm whether PBL is a better approach to prepare students for college and career than traditional teacher-directed methods. (p. 55)
That’s a frustrating answer.
If you love and believe in PBL–and, more than most pedagogical theories, PBL really has true believers–you’d rather have a ringing endorsement.
If you’re a skeptic–check out Kirschner’s emphatic rejection here–you’d like this idea put to bed once and for all.
In this review, however, the authors make clear that until we agree what PBL really is (and, what it isn’t), we can’t coherently measure its effectiveness.
What Should Teachers Do?
In the absence of a clear research answer to this question, I have two suggestions.
First: teacher experience matters. If you and your colleagues have experience teaching both PBL and direct-instruction curricula, and you’ve had good success with one or the other, then draw on that experience. As long as you’re being honest with yourselves, and keeping good records, then your experience is–for now–at least as good as any other information we’ve got.
Second: rely on useful principles from cognitive science. Does PBL help your students pay attention? If yes, that’s good. Does PBL decrease their motivation? If yes, that’s bad.
Quite often, for instance, I find that PBL curricula overwhelm students’ working memory limits. If so, then it doesn’t matter that the curriculum ought to work, or was designed by experts, because it’s overwhelming working memory.
In other words: if the curriculum sounds upliftingly progressive, but it violates basic principles of cognition, then put the rubric down and step away from the authentic question.
Every curriculum must fit with the way that students’ brains work–including a PBL curriculum.
(In case you’re wondering, “MDRC” stands for “Manpower Demonstration Research Corporation.” It was created by the Ford Foundation; its lumpy name was simplified to MDRC in 2003. You can read its history here.)
Regular readers of this blog—and, people who have even a glimpse of common sense—already know that mobile devices distract college students during lectures.
(If you’d like a review of research on this topic, you can check out The Distracted Mind by Gazzaley and Rosen.)
You can picture college students now: squinting at small screens, thumbing away at tiny keyboards, chuckling at oddly inappropriate moments of the lecture.
How can there possibly be any reason to research this question further?
One Reason to Research this Question Further
When scientists discover any kind of principal, they quickly start investigating the specific conditions under which it applies.
For example: we know that retrieval practice is—generally speaking— a great way to review. But, does it work equally well for 8-year-olds, 18-year-olds, and 80-year-olds?
We know that a growth mindset—generally speaking—enhances motivation. But, does it work for athletic as well as academic endeavors?
We know that—generally speaking—stress is bad for learning. But: how much stress is bad? Is there a low level of stress that might be good? Or, are there some tasks that benefit from high levels of stress during learning?
Researchers call these boundary conditions: a finding applies under these particular circumstances, but notthose particular circumstances.
And so, we might want to investigate use of mobile devices during lectures even further to discover their boundary conditions. Are there ages at which cell-phone use matters less? Are there class lengths where it matters more? Are there personality types who learn more while surfing away?
A Surprising Answer
A group of researchers in South Africa wanted to find boundary conditions for the harm done by mobile devices in college lectures. In particular, they wanted to know: do cell phones lower grades equally in all disciplines?
Perhaps history students are more distractible than classics students. Or, perhaps physics concepts can be obscured more readily than biology concepts.
By surveying students and by doing a meta-analysis of other studies, le Roux & Parry found that mobile phones did less harm in Engineering classes than in Arts and Social Sciences classes.
So: cell phones distract students during lectures, but they don’t distract students equally during lectures on different topics.
Teaching Implications
I, for one, wouldn’t encourage my Engineering students to break out the iPads during class. Those devices might not be as distracting as in other classes, but they’re still distracting.
(And: they’re probably distracting to other students: see Faria Sana’s research.)
Here’s what I would do: follow le Roux’s example and look for boundary conditions.
If a speaker says “working memory limits preclude students from remembering more than 2 instructions,” ask if that rule applies to your 11th graders. Ask if it applies to written instructions as well as verbal instructions. Ask if it applies to instructions given in a foreign language class. Ask if it applies to instructions that students must follow over the next 30 minutes.
Look for boundary conditions.
(By the way, the answer to those questions are:
Because WM capacity increases with age, most 11th graders can recall more than 2 instructions.
Written instructions don’t take up much working memory capacity at all.
Because foreign language instruction is VERY WM taxing, students might struggle to remember even a small number of instructions.
The longer students have to remember instructions, the harder that effort becomes. That’s why you make shopping lists: it’s hard to remember what you want at the store when it’s 30 minutes away.)
In Sum…
Cognitive sciences offers teachers general principles—and those principles can be mightily helpful. (For instance: retrieval practice DOES work well for 8, 18, and 80 year olds.)
But, most of those principles do have important boundaries. Your students, your class size, your discipline, your age group, your personality—all these variables just might be outside those boundaries.
And so: be curious about the general principles. And, be equally curious about their boundaries.
Editor’s note: Dr. Agarwal will be speaking at next week’s Learning and the Brain conference. Here’s your chance to get to know her and her work better…
Andrew Watson:
I understand that you worked as a teacher before you started training as a scientist, so I’m curious about that experience. More specifically, how did your classroom experience shape your research interests?
Dr. Pooja K. Agarwal:
I completed Elementary Education Certification in undergrad, and I taught 4th and 5th grade. And throughout college I also developed curriculum and taught in summer camps: grades 3 through 12, mostly in science curriculum. So I was majoring in Elementary Education, and—of course—I was also taking Psychology courses.
It was fascinating to be taking my Education classes—where we’re learning pedagogy and teaching methods—simultaneously with taking Psychology classes with my mentor Roddy Roediger on the science of learning and cognitive psychology.
And the two fields just seemed completely disconnected. It was one of those “Aha!” moments for me where I didn’t understand why these two departments, and these two approaches, were so different—because they could really benefit from each other. And that’s what led to my combination of teaching—and my teaching approaches—and being a scientist.
Most of my research in the past 12 years has been in classrooms, as opposed to in laboratory settings. I really enjoy the messiness of doing scientific research in classrooms. The fire alarms, and school assemblies, and kids who are out sick, I really enjoy it because it pushes boundaries. We find all these great things about learning in the psychology laboratory with college students, but do they actually work in the real world?
Watson:
Your research focuses particularly on retrieval practice. Can you define that for us?
And, perhaps you could give an example of something that is retrieval practice, and something that isn’t: “If you do it this way, that’s retrieval practice. If you do it that way, it isn’t retrieval practice.”
Agarwal:
In a laboratory, in one of the comparisons we use most often, students read a passage: let’s say about sea otters.
We might have students read it once, and then have them write down everything they can remember about the passage three times. That would be retrieval practice. They had to bring the material about sea otters back to mind and retrieve what they learned.
On the other hand, after students read the passage, we might have them simply read it three more times. This rereading does not involve any retrieval.
Another example would be a student who re-reads their textbook; or who takes notes, and re-reads their notes; that’s not retrieval practice. As opposed to a student who uses flashcards: that’s retrieval practice.
What I like to say is: it’s the difference between trying to get information INTO your head by re-reading stuff over and over, versus trying to get information OUT of your head by using flashcards or writing down what you can remember.
Watson:
That’s a helpful and a clear way to explain the difference.
Agarwal:
Retrieval practice is actually more intuitive and used more often in K-12 than it is in college. In college, students are very used to sitting down, being talked at in a lecture, and then having to output things only on an exam. In K-12, I think we engage in retrieval much more often. There’s a lot more questioning, or cold calling, or discussion.
In K-12, we should be a little more mindful about, “If I’m going to review something as the teacher, can I change that to a retrieval activity?”
For instance, instead of saying, “Last week we learned about King Tut, and how old he was, and when he died, and what he did as Pharaoh,” a different angle to involve retrieval in a K-12 classroom would be: “All right, write down two things you remember from our discussion about King Tut.” And then have a brief discussion to remind everyone, “Oh, Andrew picked up on something that Pooja did not. And Emily remembered something that Thomas remembered as well. This is cool.”
That sort of retrieval activity could involve “think, pair, share,” where students are thinking, they’re writing down what they learned about King Tut, and then, of course, talking in pairs, and then sharing with the class.
It’s important to know that retrieval practice can take one minute or less, where students think about or write down what they remembered, and then you move on with your lesson – with or without discussion or grading. It’s also important to keep in mind that classroom retrieval activities should be low stakes. Make sure to emphasize retrieval as a learning strategy, not an assessment.
Watson:
You mentioned the difference between doing research in the lab—where most psychology research happens—and doing research in actual schools and classrooms. Can you talk more about that?
Agarwal:
A lot of research on retrieval practice has been in laboratory settings. Typically with lab research, all of the participants are college students. They come in, and we have them look at stuff on the computer: like foreign language word pairs, or brief reading passages: 500 words, maybe 1,000 words.
And then we distract them—we might have them play Tetris. And then, for instance, we ask students: “Okay, write down everything you remember from that passage about sea otters that you just read.”
And then we have students come back, let’s say a week later, and we ask them to do the same thing. “Hey, you read this passage about sea otters. It’s been a week. Your life went on. And now, write down everything you can still remember.”
And so we’ll ask: “How much do students remember a week later [after retrieval practice] compared to if they just re-read that passage over and over again without engaging in retrieval practice?”
And when it comes to classroom research, my colleagues and I have done similar research. We’ve played around with retrieval practice approaches and research questions like
Are multiple-choice quizzes versus short-answer quizzes more beneficial for learning?
Do online websites or retrieval programs like Kahoot! or Quia also boost student learning?
How many times do students have to retrieve in classroom settings to get the “biggest bang for your buck?”
Do pre-quizzes help, or is it better to quiz after a lesson?
Feedback: does it have to explain why students got something wrong, or simply indicate a correct or incorrect answer?
There are definitely advantages to extending lab research into classrooms. Not just to see if lab findings work in the real world, but there are some logistical things that are actually easier.
For instance, it’s hard to get college students to come back into a lab one week later. They forget. Even if we pay them and send email reminders, they don’t show up. They’re college students.
But in K-12, students go to school every day, and so we can look at rich learning in a real environment, and students come back a week later, or three months later, or seven months later. And the vast majority of students are still in school at the end of the school year. Laboratory research looking at learning and forgetting rarely goes beyond one week.
Watson:
Can you think of something in the world of retrieval practice that looked promising when it was researched in a lab, but when you tried it out in a classroom it just didn’t have the same effect there?
Agarwal:
Oh, that’s a good question. There are some lab findings that disappear in a classroom.
Watson:
I’m intrigued. “Disappear” sounds dramatic.
Agarwal:
Let’s look at multiple choice versus short answer retrieval in the form of low-stakes quizzes.
Based on 100 plus years of research, we’ve found that the more challenging the learning strategy, the more robust the learning over the long term. One researcher, Robert Bjork, coined the term “desirable difficulty.”
In line with that theory, from laboratory research it appeared that retrieval practice with short answer questions really boosted long term learning, compared to multiple-choice quizzes. Again, that seems pretty intuitive; there’s more of a desirable difficulty in writing a short answer as opposed to just being able to choose one of four multiple-choice options.
My colleagues (Kathleen McDermott, Roddy Roediger, and Mark McDaniel) and I did pretty much the exact same research in classrooms – 7th grade science and high school history. Students learned normal materials from their classroom teacher. Then we gave them either multiple-choice or short-answer quizzes. After a few days on a unit exam and even months later at the end of the semester, we saw a large benefit of retrieval practice, but the difference between short-answer quizzes and multiple-choice quizzesdisappeared.
It wasn’t as though short-answer quizzes helped students learn more than multiple-choice. That simple act of retrieval improved learning more than not engaging in retrieval at all.
For me that’s an example where findings from the lab don’t appear to apply in classroom settings.
And so I typically recommend to teachers, “You know what? Do what’s logistically easier.” Short-answer quizzes can take more time to grade. Of course, multiple-choice might take more time to develop, because you want to come up with really good alternatives. Either way, retrieval practice boosts long-term student learning, regardless of the low stakes quiz format.
Watson:
Is retrieval practice beneficial for all ages? Is it beneficial for all subjects that are taught?
Agarwal:
Patrice Bain has been a collaborator with me and my colleagues for more than 10 years. She and I like to say that retrieval practice works for all students, all subjects, all the time.
Our research in the Columbia School District outside of St. Louis in Illinois has included special ed. students, gifted students, students in pull-out tutoring programs. We still see the same benefits of retrieval practice.
In terms of all subjects: we’ve seen that retrieval benefits history, science, Spanish, vocabulary learning, a few others.
A subject area that we haven’t done too much research on is math, because—as you can imagine—in math they do retrieval all the time.
Watson:
Right, pretty much all math homework is retrieval practice. Does it work equally well for learning facts and learning skills?
Agarwal:
Good question. There’s more research on fact learning than skill learning, especially in laboratories.
In my dissertation, I focused on fact learning versus higher order learning. In one experiment, in the sixth-grade social studies classroom with my collaborator Patrice Bain, she was teaching chapters on World War II and the Russian Revolution.
I looked at how retrieval practice can improve fact learning—again, we already have lab and classroom research showing that it improves fact learning— sometimes even doubles learning compared to non-retrieval lessons.
But can we use higher order quizzes to improve higher order learning? For instance: questions that go beyond what is explicitly stated in the lesson.
I found that, excitingly, yes, especially with these complex materials, retrieval does improve higher order learning. (I based those materials on Bloom’s taxonomy.)
Watson:
Up to this point our conversation has focused primarily on psychology: which is to say, how minds work. I want to change gears and talk about neuroscience: which is to say how brains work. Do we, as researchers, have any understanding why retrieval practice helps new neural networks form to create long-term memories?
Agarwal:
A bit. That kind of work is recent, and so there’s still a lot more to be done.
The predominant research in neuroscience related to retrieval practice is about a process we call consolidation. The basic idea is that by retrieving, we’re recreating memories, and strengthening neural networks.
Exactly how that happens in terms of synapses or neurotransmitters is something we’re still trying to figure out. There’s not a whole lot known about it.
Watson:
That was my impression.
One of the points we emphasize on the blog is that, in the world of science, skepticism is key for what we do. Scientists know more today than they did yesterday because we’re a skeptical bunch.
I’m hoping that there is skepticism about retrieval practice, and I’m curious to know what you think is the most valid skepticism about it?
Agarwal:
Two things.
One of them is that retrieval practice is just a lot of tests: “Well, this just sounds like a lot of testing, and teaching to the test.”
Retrieval practice used to be referred to as “the testing effect.” Our field has moved away from that, especially because it’s not tests or quizzes that improve learning. It’s the actual process of retrieval. Retrieval is, in many ways, not even related to assessments.
The other main skeptical response is exactly what you asked about with fact learning versus higher-order learning, or more complex skills.
I agree that we should be a bit skeptical. There isn’t much research yet on extending retrieval practice to applied settings, let alone higher-order complex materials. Part of that reason, I think, is that in laboratory settings we like things really controlled so we can make sure A causes B. And when it comes to complex materials it’s very hard to do that.
For instance, it took me a year to develop the higher order complex materials for my dissertation.
I think that skepticism for higher order learning is warranted in that there isn’t yet much research yet. That being said, the research that is there—including my own research—shows that if we engage in complex skills during retrieval, then that will improve complex skills down the road.
Watson:
If I’m hearing you right, it sounds like the concept of retrieval practice itself is pretty well settled, and there aren’t people out there who say, “You did the math wrong. If you studied correctly, the effect goes away.”
Agarwal:
Correct. There is so much reliable research both for labs and in classrooms— especially from the past 15 years—showing that time and time again, huge effect sizes. I don’t think any skeptics still argue that we should be re-reading our textbooks instead of engaging in retrieval.
Watson:
Okay. Are there specific questions I should have asked that I haven’t yet asked you?
Agarwal:
Again, I’ll name two things.
One is: for more research, resources, and a down-loadable retrieval practice guide, go to retrievalpractice.org. We also send out weekly email updates with research summaries and evidence-based recommendations, too.
Second: I like to highlight what someone can do tomorrow. What can a teacher do tomorrow in their classroom to use this powerful strategy?
Depending on the subject area and the grade level, I like to highlight a technique called a “brain dump” which is simply asking students to write down everything they can remember about a class topic, lesson, unit, etc.
And that can take less than a minute. Also, teachers can include a “think, pair, share” after the brain dump or just move on.
So using retrieval practice doesn’t require redoing someone’s teaching approach, or curricula, or anything like that. It can just be as simple as, “Think about this question, or turn and talk to someone about what you learned.”
Watson:
Those are both very helpful.
One last question, which has nothing to do with science. A lot of people who will be attending the conference aren’t from Boston. Do you have any recommendations—restaurants, or museums, or pubs, or parks you think conference goers should see?
Agarwal:
I always recommend one thing that is never mentioned on tourist websites, called the Mapparium. I think it’s fantastic, and I’ve never seen something like it in any other city.
_____________________________
You can listen to an interview with Dr. Agarwal over at Cult of Pedagogy by clicking this link.
_____________________________
Biography:
Pooja K. Agarwal, Ph.D. is an expert in the field of cognitive science. She has conducted learning and memory research in a variety of classroom settings for more than 10 years. Passionate about evidence-based education, Pooja has extensive teaching experience in K-12 and higher education, as well as expertise in education policy at state and national levels. Currently, she is an Assistant Professor at the Berklee College of Music in Boston, teaching psychological science to exceptional undergraduate musicians.
To advance the use of scientifically-based learning strategies, Pooja founded RetrievalPractice.org, a hub of cognitive science research, resources, and tips for educators. Pooja’s research is supported by grants from the National Science Foundation and the U.S. Department of Education. In addition, her work has been featured in the New York Times, Education Week, and Scientific American, as well as academic journals, books, and podcasts. For more information, visit poojaagarwal.com and retrievalpractice.org.
Teachers hate (and love) multiple-choice tests. On the one hand, they seem dreadfully reductive. On the other, they’re blissfully easy to grade — and easy grading is never to be belittled.
In our recent conversation, Pooja Agarwal recommended multiple-choice tests as one kind of retrieval practice. Inspired by her guidance, you might be asking yourself: “what can researchers tell me about the best kind of multiple-choice test to give?”
If you’re asking yourself that question, look no further: the estimable Andrew Butler is on the case.
(For example: if you want to know how many distractors to include on your test, you should see what Butler has to say…)
