Man Who Is Paralyzed Communicates By Imagining Handwriting

An experimental device that turns thoughts into text has allowed a man who was left paralyzed by an accident to construct sentences swiftly on a computer screen.

The man was able to type with 95% accuracy just by imagining he was handwriting letters on a sheet of paper, a team reported Wednesday in the journal Nature.

"What we found, surprisingly, is that [he] can type at about 90 characters per minute," says Krishna Shenoy of Stanford University and the Howard Hughes Medical Institute.

The device would be most useful to someone who could neither move nor speak, says Dr. Jaimie Henderson, a neurosurgeon at Stanford and co-director, with Shenoy, of the Stanford Neural Prosthetics Translational Laboratory.

"We can also envision it being used by someone who might have had a spinal cord injury who wants to use email," Henderson says, "or, say, a computer programmer who wants to go back to work."

Both Henderson and Shenoy have a proprietary interest in commercializing the experimental approach used to decode brain signals.

The idea of decoding the brain activity involved in handwriting is "just brilliant," says John Ngai, who directs the National Institutes of Health's BRAIN Initiative, which helped fund the research.

"But it was only on one subject in a laboratory setting," Ngai says. "So at the moment it's a great demonstration of proof of principle."

The man who agreed to test the device is unable to move his arms and legs as the result of a freak accident.

"He was taking out the garbage, slipped, fell and instantly became quadriplegic," Henderson says. "So he's essentially completely paralyzed."

A few years ago, the man agreed to take part in a study of an experimental system called BrainGate2. It allows people who are paralyzed to control computers and other devices using only their thoughts.

The system relies on electrodes surgically implanted near the part of the brain that controls movement. In previous studies, participants had learned to control a computer cursor or robotic arm by imagining they were moving their hands.

This time, Henderson, Shenoy and a team of scientists had the man imagine he was writing individual letters by hand while a computer monitored the electrical activity in his brain.

Eventually, the computer learned to decode the distinct pattern of activity associated with every letter of the alphabet as well as several symbols.

Once that process is complete, Shenoy says, "We can determine if the letter you wrote is an A or a B or a C and then plop that up on the screen and you're able to spell out words and sentences and so forth one letter at a time."

In previous experiments, participants had been able to use their thoughts to "point and click" at letters on a screen. But that approach was much slower than imagined handwriting.

Also, because the new system relies on familiar thoughts, the participant was able to use it almost immediately.

"He was very happy when he was able to write out messages in response to the questions we asked him." Henderson says. "He was pretty excited about this."

The team's success decoding imagined handwriting is just the latest advance in efforts to link computers to the human brain, Ngai says.

"I was introduced to this concept over 10 years ago, and I thought it was quite a bit of science fiction," he says. "Then roughly about five years later it was shown to be not to be such science fiction after all. So I think we're seeing a progression. It's really quite exciting."

An editorial accompanying the study shares that view.

The handwriting approach "has brought neural interfaces that allow rapid communication much closer to a practical reality," wrote Pavithra Rajeswaran and Amy L. Orsborn of the University of Washington. [Copyright 2021 NPR]