Tag: technology

Our blogger is off this week. He asked us to repost this piece, because he’ll be chatting with these researchers again soon!

---

 

A friend recently sent me a link to an article with a click-baity headline: something like “Virtual Reality Will Change Education Forever.”

Her pithy comment: “This is obviously nonsense.” (It’s possible she used a spicier word that ‘nonsense.’)

On the one hand, I’m skeptical that ANYTHING will change education forever. Heck, if Covid didn’t transform education, who knows what will.

More specifically, ed-tech claims about “transforming education” have been around for a long time. Their track record doesn’t dazzle. (Smart boards, anyone?)

On the other hand, I always like to find reserch that challenges my long-held beliefs. After all, if I can’t learn from people who disagree with me, who can I learn from?

So, I followed my usual process.

In essence, I switched into Detective Mode, and started asking lots of questions.

If I ask the right questions, I thought, I’ll get a much clearer picture of potential educational benefits of VR.

Act I: The Investigation Begins

When I reviewed the article my friend sent, I noticed a troubling gap: the article didn’t link to underlying research.

As I’ve written in the past, this absence creates a red flag. If the article champions “research-based innovation,” why not link to the research?

So, I asked my first detective question. I emailed the author of the article and asked to see the research.

How simple is that?

Obviously, any resistance to this request — “sorry, we can’t share that at this moment” — would underline my friend’s skeptical verdict: “nonsense.”

However, the author responded immediately with a link to a research summary.

A promising development…

The Plot Thickens

This research summary showed real promise.

In brief:

Some college students in an introductory Biology course followed the typical path — readings, lectures, labs. (That’s the “control group.”)

Other students in the same course followed an alternative path: readings, lectures, supplementary Virtual Reality experience, alternative labs based on the VR experience.

When researchers looked at all sorts of results, they found that students on the alternative VR path did better.

That is: not only did the students enjoy the VR experiences; not only did they engage more with the material; they (on average) learned more.

However — and this is a BIG however — this research didn’t look like it was published.

In fact, when I asked that direct question, the article author confirmed that the research hadn’t yet been published in a peer-reviewed journal.

Now, the topic of peer review creates LOTS of controversy. The peer-review system has MANY troubling flaws.

However, that system probably reduces the amount of deceptive nonsense that gets published.

I almost never blog about research that hasn’t been peer reviewed, and so I thought my detecting was at its logical end. The VR claim might not be ‘nonsense,’ but it didn’t yet have enough published evidence to strengthen it.

And then, an AMAZING thing happened: the lead researcher emailed me to say she would be happy to talk with me about the study.

Over the years, I have occasionally reached out to researchers to be sure I understand their arguments.

But no researcher has EVER straight-up volunteered for such a meeting. And I mean: EVER.

The Payoff

Honestly, I’d love to transcribe my conversation with Dr. Annie Hale and Lisa Fletcher (“Chief of Realm 4”) — both at Arizona State University because it was both fascinating and inspiring.

Because you’re busy, I will instead boil it into three key points:

First:

Hale and Fletcher have done — and continue to do — incredibly scrupulous research.

For instance, in the description above, I put the words “control group” in quotations marks.

I did so because of Hale and Fletcher’s insistance. The two groups of Biology students had somewhat similar academic experiences.

But the research paradigm required enough differences to make the words “control group” technically inappropriate.

Hale and Fletcher insisted on this precision throughout our discussion. For instance, they regularly pointed out that a particular calculation suggested a positive result, but didn’t reache statistical significance.

In other words, they highlighted both the strengths and weaknesses of their own argument.

This habit, it my view, makes them MUCH more reliable guides in this field.

Second:

Here’s a shocker: Hale and Fletcher do not claim that virtual reality will transform education.

No, really, they don’t.

The headline of the article my friend sent me made that claim, but the researchers themselves don’t.

Instead, they make a very different claim. The alternative Biology path included at least three big changes from the typical path:

Change #1: students had the VR experience (and their lab was based on that experience)

Change #2: the key underlying biology concepts had been translated into stories. For instance, this “narratively-driven virtual reality” includes an imaginary species called the Astelar. (Some of the students got QUITE protective of these imaginary creatures.)

Change #3: the TAs in these alternative path classes got special training, inspired by Doug Lemov’s Teach Like a Champion.

We can’t know — and, Hale and Fletcher don’t say they know — which of these three parts made the biggest difference.

We can tentatively suspect that these three elements working together produced all those learning benefits. And, Hale and Fletcher are planning lots of further research to confirm this tentative belief.

But, they’re not trying to get VR goggles on every forehead.

Key Point #3

Here’s one of my mantras:

Researchers isolate variables. Teachers combine variables.

In other words: research — as much as possible — looks at the effect of just one thing.

For instance: “mid-lecture aerobic movement improves learning in college students.”

However, teachers juggle hundreds of variables at every second. All those isolated variables studied by researchers might not provide me with useful guidance.

For instance: if I teach in a business school, my formally-dressed students might not appreciate my insistance that they do jumping jacks in the middle of the lecture hall.

My particular combination of variables doesn’t helpfully align with that isolated exercise variable.

Here’s my point: Hale and Fletcher seem to be changing the research half of this paradigm.

In their research, notice that they aren’t isolating variables. They are, instead, looking at combinations of variables.

VR + stories + Lemov training –> more learning

In fact, if I understand their argument right, they don’t really think that isolating variables can produce the most useful results — at least not in education research.

After all (and here I’m adding my own perspective), if teachers combine variables, shouldn’t research also look at combinations?

An Early Verdict

I set out on this detective adventure feeling quite skeptical. Both the initial claim (“transform education!”) and the absence of links made me all-but-certain that the strong claim would implode. (Example here.)

However, by persistently asking reasonable detective questions, I’ve arrived at a very different place:

VR + [concepts as stories] + [Lemov-inspired TA training] just might produce big learning gains, at least for some students.

And — crucially — a thoughtful, precise, imaginative, and cautious group of scholars is exploring this possibility in detail.

As I said back at the beginning, I’ve always got something to learn.

---

This post was edited on April 7, 2023 to correct Lisa Fletcher’s title.

A friend recently sent me a link to an article with a click-baity headline: something like “Virtual Reality Will Change Education Forever.”

Her pithy comment: “This is obviously nonsense.” (It’s possible she used a spicier word that ‘nonsense.’)

On the one hand, I’m skeptical that ANYTHING will change education forever. Heck, if Covid didn’t transform education, who knows what will.

More specifically, ed-tech claims about “transforming education” have been around for a long time. Their track record doesn’t dazzle. (Smart boards, anyone?)

On the other hand, I always like to find reserch that challenges my long-held beliefs. After all, if I can’t learn from people who disagree with me, who can I learn from?

So, I followed my usual process.

In essence, I switched into Detective Mode, and started asking lots of questions.

If I ask the right questions, I thought, I’ll get a much clearer picture of potential educational benefits of VR.

Act I: The Investigation Begins

When I reviewed the article my friend sent, I noticed a troubling gap: the article didn’t link to underlying research.

As I’ve written in the past, this absence creates a red flag. If the article champions “research-based innovation,” why not link to the research?

So, I asked my first detective question. I emailed the author of the article and asked to see the research.

How simple is that?

Obviously, any resistance to this request — “sorry, we can’t share that at this moment” — would underline my friend’s skeptical verdict: “nonsense.”

However, the author responded immediately with a link to a research summary.

A promising development…

The Plot Thickens

This research summary showed real promise.

In brief:

Some college students in an introductory Biology course followed the typical path — readings, lectures, labs. (That’s the “control group.”)

Other students in the same course followed an alternative path: readings, lectures, supplementary Virtual Reality experience, alternative labs based on the VR experience.

When researchers looked at all sorts of results, they found that students on the alternative VR path did better.

That is: not only did the students enjoy the VR experiences; not only did they engage more with the material; they (on average) learned more.

However — and this is a BIG however — this research didn’t look like it was published.

In fact, when I asked that direct question, the article author confirmed that the research hadn’t yet been published in a peer-reviewed journal.

Now, the topic of peer review creates LOTS of controversy. The peer-review system has MANY troubling flaws.

However, that system probably reduces the amount of deceptive nonsense that gets published.

I almost never blog about research that hasn’t been peer reviewed, and so I thought my detecting was at its logical end. The VR claim might not be ‘nonsense,’ but it didn’t yet have enough published evidence to strengthen it.

And then, an AMAZING thing happened: the lead researcher emailed me to say she would be happy to talk with me about the study.

Over the years, I have occasionally reached out to researchers to be sure I understand their arguments.

But no researcher has EVER straight-up volunteered for such a meeting. And I mean: EVER.

The Payoff

Honestly, I’d love to transcribe my conversation with Dr. Annie Hale and Lisa Fletcher (“Chief of Realm 4”) — both at Arizona State University because it was both fascinating and inspiring.

Because you’re busy, I will instead boil it into three key points:

First:

Hale and Fletcher have done — and continue to do — incredibly scrupulous research.

For instance, in the description above, I put the words “control group” in quotations marks.

I did so because of Hale and Fletcher’s insistance. The two groups of Biology students had somewhat similar academic experiences.

But the research paradigm required enough differences to make the words “control group” technically inappropriate.

Hale and Fletcher insisted on this precision throughout our discussion. For instance, they regularly pointed out that a particular calculation suggested a positive result, but didn’t reache statistical significance.

In other words, they highlighted both the strengths and weaknesses of their own argument.

This habit, it my view, makes them MUCH more reliable guides in this field.

Second:

Here’s a shocker: Hale and Fletcher do not claim that virtual reality will transform education.

No, really, they don’t.

The headline of the article my friend sent me made that claim, but the researchers themselves don’t.

Instead, they make a very different claim. The alternative Biology path included at least three big changes from the typical path:

Change #1: students had the VR experience (and their lab was based on that experience)

Change #2: the key underlying biology concepts had been translated into stories. For instance, this “narratively-driven virtual reality” includes an imaginary species called the Astelar. (Some of the students got QUITE protective of these imaginary creatures.)

Change #3: the TAs in these alternative path classes got special training, inspired by Doug Lemov’s Teach Like a Champion.

We can’t know — and, Hale and Fletcher don’t say they know — which of these three parts made the biggest difference.

We can tentatively suspect that these three elements working together produced all those learning benefits. And, Hale and Fletcher are planning lots of further research to confirm this tentative belief.

But, they’re not trying to get VR goggles on every forehead.

Key Point #3

Here’s one of my mantras:

Researchers isolate variables. Teachers combine variables.

In other words: research — as much as possible — looks at the effect of just one thing.

For instance: “mid-lecture aerobic movement improves learning in college students.”

However, teachers juggle hundreds of variables at every second. All those isolated variables studied by researchers might not provide me with useful guidance.

For instance: if I teach in a business school, my formally-dressed students might not appreciate my insistance that they do jumping jacks in the middle of the lecture hall.

My particular combination of variables doesn’t helpfully align with that isolated exercise variable.

Here’s my point: Hale and Fletcher seem to be changing the research half of this paradigm.

In their research, notice that they aren’t isolating variables. They are, instead, looking at combinations of variables.

VR + stories + Lemov training –> more learning

In fact, if I understand their argument right, they don’t really think that isolating variables can produce the most useful results — at least not in education research.

After all (and here I’m adding my own perspective), if teachers combine variables, shouldn’t research also look at combinations?

An Early Verdict

I set out on this detective adventure feeling quite skeptical. Both the initial claim (“transform education!”) and the absence of links made me all-but-certain that the strong claim would implode. (Example here.)

However, by persistently asking reasonable detective questions, I’ve arrived at a very different place:

VR + [concepts as stories] + [Lemov-inspired TA training] just might produce big learning gains, at least for some students.

And — crucially — a thoughtful, precise, imaginative, and cautious group of scholars is exploring this possibility in detail.

As I said back at the beginning, I’ve always got something to learn.

---

This post was edited on April 7, 2023 to correct Lisa Fletcher’s title.