Let’s say I go out to lunch with two friends. The total bill–including tip–is $87 dollars. To keep things simple, we decide to divide the bill evenly.
I say, “well, I guess this is about my share” and put $50 on the table.
If one of my two friends is a math teacher, she’s likely to say that I don’t have good number sense. That is: I don’t have a quick, intuitive sense of magnitude; a gut-level sense of mathematical operations. After all, $87/3 is less than $30, not $50. I should be able to get that math sorted without breaking out a calculator.
For over a decade now, some math scholars have promoted number talks as a research-based strategy to help students develop number sense. In a number talk, students solve math problems in their heads. They then discuss the various strategies that they used to arrive at an answer. A skilled teacher should ask probing questions and explore different avenues–being sure that students who arrived at the wrong answer know where they went astray.
Longtime readers know that I hold two strong–often competing–opinions.
- Teachers should be open to the idea that research can strengthen our practice and benefit our students.
- Scholars shouldn’t boss teachers around until they have lots of good research to support their claims.
In this case:
- If research shows that number talks improve number sense (or provide other benefits), then math teachers should use them!
- If we don’t have several well-designed studies reaching that conclusion, then we shouldn’t speak the magic words: “research based.”
So: what happens when we look for research in on this topic?
A Potential Winner
When I started searching for research into this topic, one study came up most often: “Number Talks Benefit Fifth Graders’ Numeracy.” (Link)
In this study, published in 2020, a 5th grade teacher followed a number-talks protocol with 11 students twice a week for six weeks. A second class of 11 students served as the control group; their math class continued as usual without math talks.
The teacher, Pamela May, measured three variables: the students’ speed, their flexibility, and their accuracy with mental math. When she compared their results before and after the six-week period, sure enough, she found a statistically significant improvement in their speed. (Students did not get measurably more accurate; they got more flexible on one measurement, but not on another.)
Before I explore my concerns with this study and its conclusion, I do want to emphasize a key point: we all benefit when teachers undertake research in their classrooms. Although I’m about to argue that this study doesn’t support the use of number talks, I don’t want that argument to imply that May did something wrong, or shouldn’t be doing research. Quite the contrary: May did a lot of hard work, published her results, and let us learn from those results. These contributions merit our respect and honor.
At the same time, I do have concerns with this study and its conclusions. I’ll start with the small stuff, and then step up to my more substantial concerns.
The Small Stuff
First: this study includes twenty-two students. Typically, a study of this size encourages researchers to do another, much larger study. It does not allow us to draw any meaningful conclusions. After all: only eleven students undertook those number talks. No one should change their teaching based on eleven students.
Second: May compared students who did something special with students who did the normal thing. We call this a “business as usual” control group, and they rarely fill researchers with enthusiasm. Instead, a business-as-usual control group usually encourages researchers to develop a study where both groups do something special. That study–if replicated several times–might start to sound persuasive.
Third: Whenever possible, we want the people enacting the experiment to know as little as possible about the experiment’s hypothesis. If they know the goal of the study, that knowledge might subtly influence them to teach one group slightly more effectively than the other. In this case, of course, May did all the work–and so she knows her own hypothesis!
As I wrote above, May did lots of hard work to publish this study, and none of these concerns means that she “did something wrong.” In the absence of larger replications–with more rigorous procedures–they do mean that we shouldn’t change our teaching because of this one study.
Beyond Procedure
These merely procedural concerns introduce more substantive hesitations.
In this study, eleven students practiced doing mental math. Result: they got faster at doing mental math, compared to other students who didn’t practice. This conclusion is entirely unsurprising—practice effects alone would predict faster performance. Honestly, I’d be more surprised if they didn’t get faster.
Note, too, that the students got faster, but they didn’t get more accurate.
The results about flexibility require more parsing. In theory, number talks encourage students to understand and use more math strategies. In this study, students knew more strategies, but they didn’t use more strategies unprompted. As May herself writes:
“Students in fifth grade had solidified their preferences in mental calculation strategies. They did not appear to spend time evaluating various possible strategies before trying to solve the problem.”
This statement undermines one key argument in favor of number talks.
To introduce the final substantive concern, let’s consider this hypothetical claim:
My business got more profitable. You know that’s true because I hired more employees!
In response to this claim, you might reasonably say:
Hiring more employees might suggest greater profits–but I’ll need many more data points. What were their salaries? What were your revenues and expenses? How’s your tax situation shaping up?
In other words: “number of employees” might tell us something about “profit,” but not the whole story.
This study is used to support the idea that number talks increase number sense, but the study itself doesn’t directly measure number sense. It reports an increased speed in solving mental math problems (like the “number of employees” in my analogy), but doesn’t give data about all the other elements of number sense (“salaries, net revenue, expenses” in my analogy). This study doesn’t support a hypothesis about number sense, because it doesn’t include the right information to make such a claim.
In brief: my concerns about this study aren’t merely procedural. Given this limited set of findings, I just don’t think this study should change a math teacher’s practice.
Zooming Back the Camera
If this one study doesn’t support the claim that number talks benefit students, what about other research?
We just don’t have much, at least not that I could find. A 2020 review famously describes number talks as a black hole of research. Another study that gets some enthusiastic attention doesn’t include a traditional control group–so we can’t know how much effect number talks have compared to something else.
In sum:
- If “research-based” means “the theory behind this idea has some research associated with it,” then we can loosely describe number talks as research-based.
- If “research-based” means “someone tested this specific idea with students, and it helped!” I don’t think we’re anywhere close to that claim being true.
Number talks might work wonderfully. But right now, we need research more before we believe that claim to have merit.
May, P. L. (2020). Number Talks Benefit Fifth Graders’ Numeracy. International Journal of Instruction, 13(4), 361-374.
