{"id":5417,"date":"2020-02-04T08:00:16","date_gmt":"2020-02-04T13:00:16","guid":{"rendered":"https:\/\/braindevs.net\/blog\/blog\/?p=5417"},"modified":"2020-01-30T19:43:45","modified_gmt":"2020-01-31T00:43:45","slug":"a-fresh-approach-to-evaluating-working-memory-training","status":"publish","type":"post","link":"https:\/\/www.learningandthebrain.com\/blog\/a-fresh-approach-to-evaluating-working-memory-training\/","title":{"rendered":"A Fresh Approach to Evaluating Working Memory Training"},"content":{"rendered":"

Because working memory is SO IMPORTANT for learning<\/a>, we would love to enhance our students’ WM capacity.<\/p>\n

\"\"<\/a><\/p>\n

Alas, over and over, we find that WM training programs just don’t work (here<\/a> and here<\/a> and here<\/a>). I’ve written about this question so often that I’ve called an informal moratorium. Unless there’s something new to say, or a resurgence of attempts to promote such products, I’ll stop repeating this point.<\/p>\n

Recently I’ve come across a book chapter that does<\/em> offer something new. A research team led by\u00a0Claudia C. von Bastian used a very powerful statistical method to analyze the effectiveness of WM training programs.<\/p>\n

This new methodology (which I’ll talk about below) encourages us to approach the question with fresh eyes. That is: before I read von Bastian’s work, I reminded myself that it might well contradict<\/em> my prior beliefs.<\/p>\n

It might show that WM training does<\/em> work. And, if it shows that, I need to announce that conclusion as loudly as I’ve announced earlier doubts.<\/p>\n

In other words: there’s no point in reading this chapter simply to confirm what I already believe. And, reader, the same applies for you<\/strong>. I hereby encourage you: prepare to have your beliefs about WM training challenged. You shouldn’t read the rest of this post unless you’re open to that possibility.<\/p>\n

New Methodology<\/h2>\n

One problem with arguments about WM training is that sample sizes are so small<\/em>. In one recent meta-analysis, the average sample size per study was 20 participants<\/strong>.<\/p>\n

In a recent book<\/a> on cognitive training, von Bastian, Guye, and De Simoni note that small sample sizes lead to quirky p-values. In other words, we struggle to be sure that the findings of small studies don’t result from chance or error.<\/p>\n

Instead, von Bastian & Co. propose using Bayes factors: an alternate technique for evaluating the reliability of a finding, especially with small sample sizes. The specifics here go WAY beyond the level of this blog, but the authors summarize handy tags for interpreting Bayes factors:<\/p>\n

1-3\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0Ambiguous<\/p>\n

3-10\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 Substantial<\/p>\n

10-30\u00a0 \u00a0 \u00a0 \u00a0 \u00a0Strong<\/p>\n

30-100\u00a0 \u00a0 \u00a0 Very Strong<\/p>\n

100+\u00a0 \u00a0 \u00a0 \u00a0 \u00a0Decisive<\/p>\n

They then calculate Bayes factors for 28 studies of WM training.<\/p>\n

Drum Roll, Please…<\/h2>\n

We’ve braced ourselves for the possibility that a new analytical method will overturn our prior convictions. Does it?<\/p>\n

Well, two of the 28 studies “very strongly” suggest WM training works. 1 of the 28 “substantially” supports WM training. 19 are “ambiguous.” And 6 “substantially” suggest that WM training has no effect.<\/p>\n

In other words: 3 of the 28 show meaningful support of the hypothesis. The other 25 are neutral or negative.<\/p>\n

So, in a word: “no.” Whichever method you use to evaluate the success of WM training, we just don’t have good reason to believe that it works.<\/p>\n

Especially when such training takes a long time, and costs lost of money, schools should continue to be wary.<\/p>\n

Three Final Notes<\/h2>\n

First<\/strong>: I’ve focused on p-values and Bayes factors in this blog post. But, von Bastian’s team emphasizes a number of problems in this field. For instance: WM training research frequently lacks an “active” control group. And, it often lacks a substantial theory, beyond “cognitive capacities should be trainable.”<\/p>\n

Second<\/strong>: This research team is itself working on an intriguing hypothesis right now. They wonder if working memory\u00a0capacity<\/strong> cannot <\/em>be trained, but working memory\u00a0efficiency<\/strong>\u00a0can<\/em> be trained. That’s a subtle but meaningful distinction, and I’m glad to see they’re exploring this question.<\/p>\n

So far they’re getting mixed results, and don’t make strong claims. But, I’ll keep an eye on this possibility — and I’ll report back if they develop helpful strategies.<\/p>\n

Third<\/strong>: I encouraged you to read von Bastian’s chapter because it might change your mind. As it turns out, the chapter probably didn’t. Instead it confirmed what you (and certainly I) already thought.<\/p>\n

Nonetheless, that was an important mental exercise. Those of us committed to relying on research for teaching guidance should be prepared to change our approach when research leads us in a new direction.<\/p>\n

Because, you know, some day a new WM training paradigm just might work.<\/p>\n

\n

von Bastian, C. C., Guye, S., & De Simoni, C. (2019). How strong is the evidence for the effectiveness of working memory training? In M. F. Bunting, J. M. Novick, M. R. Dougherty & R. W. Engle (Eds.), Cognitive and Working Memory Training: Perspectives from Psychology, Neuroscience, and Human Development<\/i> (pp. 58\u201375). Oxford University Press.<\/p>\n","protected":false},"excerpt":{"rendered":"

A new method for evaluating working memory training raises an intriguing possibility: despite all our skepticism, might that training work after all?<\/p>\n","protected":false},"author":18,"featured_media":5424,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[19,30],"class_list":["post-5417","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-lb-blog","tag-skepticism","tag-working-memory"],"_links":{"self":[{"href":"https:\/\/www.learningandthebrain.com\/blog\/wp-json\/wp\/v2\/posts\/5417","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.learningandthebrain.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.learningandthebrain.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.learningandthebrain.com\/blog\/wp-json\/wp\/v2\/users\/18"}],"replies":[{"embeddable":true,"href":"https:\/\/www.learningandthebrain.com\/blog\/wp-json\/wp\/v2\/comments?post=5417"}],"version-history":[{"count":7,"href":"https:\/\/www.learningandthebrain.com\/blog\/wp-json\/wp\/v2\/posts\/5417\/revisions"}],"predecessor-version":[{"id":5426,"href":"https:\/\/www.learningandthebrain.com\/blog\/wp-json\/wp\/v2\/posts\/5417\/revisions\/5426"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.learningandthebrain.com\/blog\/wp-json\/wp\/v2\/media\/5424"}],"wp:attachment":[{"href":"https:\/\/www.learningandthebrain.com\/blog\/wp-json\/wp\/v2\/media?parent=5417"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.learningandthebrain.com\/blog\/wp-json\/wp\/v2\/categories?post=5417"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.learningandthebrain.com\/blog\/wp-json\/wp\/v2\/tags?post=5417"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}