Brain Game Claims Fail A Big Scientific Test

Oct 3, 2016
Originally published on October 4, 2016 5:01 pm

Want to be smarter? More focused? Free of memory problems as you age?

If so, don't count on brain games to help you.

That's the conclusion of an exhaustive evaluation of the scientific literature on brain training games and programs. It was published Monday in the journal Psychological Science in the Public Interest.

"It's disappointing that the evidence isn't stronger," says Daniel Simons, an author of the article and a psychology professor at the University of Illinois at Urbana-Champaign.

"It would be really nice if you could play some games and have it radically change your cognitive abilities," Simons says. "But the studies don't show that on objectively measured real-world outcomes."

The evaluation, done by a team of seven scientists, is a response to a very public disagreement about the effectiveness of brain games, Simons says.

In October 2014, more than 70 scientists published an open letter objecting to marketing claims made by brain training companies. Pretty soon, another group, with more than 100 scientists, published a rebuttal saying brain training has a solid scientific base.

"So you had two consensus statements, each signed by many, many people, that came to essentially opposite conclusions," Simons says.

In an effort to clarify the issue, Simons and six other scientists reviewed more than 130 studies of brain games and other forms of cognitive training. The evaluation included studies of products from industry giant Lumosity, which has been a prominent sponsor of NPR and other public radio programming.

"We went through each paper and tried to look at the kind of evidence it provided," Simons says.

That meant asking questions like: How big was the study? Did it have an appropriate control group? Do the results support the marketing claims made by companies?

The scientists found that "many of the studies did not really adhere to what we think of as the best practices," Simons says.

Some of the studies included only a few participants. Others lacked adequate control groups or failed to account for the placebo effect, which causes people to improve on a test simply because they are trying harder or are more confident.

There were some good studies, Simons says. And they showed that brain games do help people get better at a specific task.

"You can practice, for example, scanning baggage at an airport and looking for a knife," he says. "And you get really, really good at spotting that knife."

But there was less evidence that people got better at related tasks, like spotting other suspicious items, Simons says. And there was no strong evidence that practicing a narrow skill led to overall improvements in memory or thinking.

That's disappointing, Simons says, because "what you want to do is be better able to function at work or at school."

The evaluation got a warm reception from at least some of the scientists who had signed the 2014 letter defending the science behind brain training.

"The evaluation was very even-handed and raised many excellent points," says George Rebok, a psychologist at Johns Hopkins University who has been involved in brain training research for the past 20 years. "It really helped raise the bar in terms of the level of science that we must aspire to."

Rebok, who says he has no ties to brain training companies, remains optimistic that the right program of brain exercises can improve mental functioning and delay the effects of aging.

One reason brain games haven't shown a clear benefit so far, he says, may be that they don't work the brain hard enough or over a long enough time period.

"It takes mental effort and practice to be able to see results," Rebok says. "If we can implement that long range, I think that there will be a big dividend eventually."

In the meantime, the brain training industry is facing scrutiny from the Federal Trade Commission.

In January, the company behind Lumosity agreed to pay a $2 million fine to settle FTC charges that it made unfounded claims about its brain training program.

And in May, a smaller brain training company called LearningRx agreed to pay a $200,000 fine to settle similar charges. Even so, the Learning Rx website still promises "A better brain at any age."

Copyright 2017 NPR. To see more, visit http://www.npr.org/.

RENEE MONTAGNE, HOST:

It's an appealing idea - spend a few minutes a day playing one of those brain-training games, and you'll be smarter and more focused. You'll even stave off memory problems as you age. The question is, can brain games really do all of that? A new evaluation of the scientific evidence finds little reason to believe they can. NPR's Jon Hamilton reports.

JON HAMILTON, BYLINE: The evaluation is a response to a very public disagreement. In 2014, a group of scientists published an open letter objecting to the marketing claims made by companies that offer brain training. A few weeks later, another group of scientists published a rebuttal. They said there was good science showing that brain games work.

DAN SIMONS: So you had two consensus statements, each signed by many, many people that came to essentially opposite conclusions.

HAMILTON: That's Dan Simons, a psychology professor at the University of Illinois. Simons decided to take a close look at the research on brain training, so he and a handful of other scientists began reviewing more than 130 studies.

SIMONS: And what we did was went through each paper and tried to look at the kind of evidence it provided.

HAMILTON: How big was the study? Did it have an appropriate control group? The results appear in the journal Psychological Science in the Public Interest. And Simon says what the team found was disappointing.

SIMONS: Many of the studies did not really adhere to what we think of as the best practices for intervention designs.

HAMILTON: Some were really small. Others compared the people who played games with people who did nothing. Simons says there were some studies that showed brain games can have an effect.

SIMONS: You can practice, for example, scanning baggage at an airport and looking for a knife. And you get really, really good at spotting that knife, but not necessarily spotting other things.

HAMILTON: Or at solving math problems or remembering your grocery list. And Simons says those are the improvements many people hope for when they start playing brain games.

SIMONS: We don't care if you practice these games and then you get better at doing some weird computer task. What you want to do is be better able to function at work or at school.

HAMILTON: Simon says that's not a realistic expectation - at least not now.

SIMONS: It'd be really nice if you could play some games and have it radically change your cognitive abilities. And it's still possible that it might be. It's just that the studies up to this point really don't provide strong evidence.

HAMILTON: George Rebok is a psychologist at Johns Hopkins who has spent two decades researching brain training. He says the evaluation is fair, and he agrees with many of its criticisms.

GEORGE REBOK: We've all been aware of some of the methodological shortcomings of the studies, and I think it really helped raise the bar in terms of the level of science that we must aspire to.

HAMILTON: But Rebok remains optimistic that brain-training programs based on rigorous science will prove their value someday. He says there have already been some hints of success, including one of his own studies that looked at thinking and learning in older people.

REBOK: We gave people only 10 sessions of initial training, and we were seeing effects 10 years down the road.

HAMILTON: Now, Rebok would like to try a training program that lasts for years and pushes people much harder than most brain games do.

REBOK: If we can implement that long range, I think there will be, you know, a big dividend eventually.

HAMILTON: In the meantime, the Federal Trade Commission is going after brain-training companies that make scientifically unsupportable claims. In January, the company behind Luminosity (ph), a sponsor of NPR programs, agreed to pay a $2 million fine and revise its marketing practices. Jon Hamilton, NPR News. Transcript provided by NPR, Copyright NPR.