The Wildly Ambitious Quest to Build a Mind-Controlled Exoskeleton by 2014

• By Greg Miller
• 02.27.13

The feet of a monkey-sized prototype of the exoskeleton in Nicolelis’s lab at Duke. Photo: Nick Pironio/Wired

Neuroscientist Miguel Nicolelis went on The Daily Show in 2011 and told Jon Stewart that he would develop a robotic body suit that would allow paralyzed people to walk again simply by thinking about it — and he’d do it in just 3 or 4 years.
It was an audacious, some might say reckless, claim. But two years later, Nicolelis insists he’s on track. And he hopes to prove it in brazen fashion in front of billions of people during one of the world’s most-watched events: the World Cup.
The tournament, which will be held in his native Brazil, is less than 16 months away. If all goes according to plan, during the opening ceremony, a young paralyzed person will step onto the field in a robotic exoskeleton operated by electrodes implanted in his or her brain, walk about 20 steps, and kick a soccer ball.
This may sound incredible, but in recent years, research on using signals from the brain to operate machines has taken great strides. Scientists have developed brain-machine interfaces that allow paralyzed humans to move a computer cursor or even use a robotic arm to pick up a piece of chocolate or touch a loved one for the first time in years. Nicolelis has set his sights even higher: He wants to get paralyzed people up and walking around. If he succeeds it could be a tremendous advance. Right now he’s still developing this technology in monkeys. There’s a long way to go.
But Nicolelis was brimming with confidence in January when I visited his lab at Duke University to see how his work is progressing. “We’re getting close to making wheelchairs obsolete,” he said.

Miguel Nicolelis. Photo: Nick Pironio/Wired

Such proclamations don’t sit well with everyone. In the Brazilian media, some scientists have criticized Nicolelis’ plan as premature, an expensive stunt, funded with scarce federal research money and aimed more at creating a spectacle than advancing science. Meanwhile, some U.S. researchers fear he could deal a setback to the fast-moving field of brain-machine interfaces by promising too much, too soon.
“Nicolelis may enjoy being provocative, and certainly that could strike many people as not being as cautious as one could be,” said Krishna Shenoy, who studies brain-machine interfaces at Stanford. But Shenoy doesn’t necessarily take it as a sign of recklessness. “I think he may tend to over-promise as a way to motivate himself and his crew,” he said.
Brain-controlled prostheses are one of the hottest areas in neuroscience. In December, researchers at the University of Pittsburgh published a case study in The Lancet of a 53-year-old woman named Jan Scheuermann who was paralyzed from the neck down by a genetic neurodegenerative condition. Scheuermann learned to control a nearby robotic arm after surgeons implanted a small grid of electrodes in her brain.
In videos released with the paper and aired on 60 Minutes, she moves the arm in 3 dimensions and uses it to grasp and move objects, stacking several plastic cones, for example. The arm itself is a marvel of engineering: It cost DARPA more than $100 million to develop, and its hand and fingers can do almost everything the real deal can. Scheuermann’s movements are slow and sometimes faltering, but they are astonishing nonetheless. After all, she’s controlling the arm just by thinking about it. And she’s making the most sophisticated movements yet made by a human being with a brain-controlled prosthetic.
Nicolelis thinks he can do much better.
As a boy growing up in São Paulo, he was inspired by the Apollo program to become a scientist. Now he sees neural prostheses that free people from paralyzed bodies as a 21st century moonshot. He also feels compelled to give something back to his native country, which he left at the age of 27 to study in the U.S.
The giving goes both ways. Nicolelis says the Brazilian government has awarded him $20 million to pursue his grand plan. Only a small part of that will go toward the demo at the World Cup, which he says was approved in a meeting with the secretary general of FIFA, the world governing body of soccer. The rest will be used to establish a neurorobotic rehabilitation and research center at a hospital in São Paulo.

An electrode array. Photo: Nick Pironio/Wired

Nicolelis thinks the next big leap in the performance of neural prosthetics will come from two types of advances. One is using information from a much larger number of neurons to enable faster, more natural movements. So far, electrode grids used in human patients can capture the electric blips of about 100 neurons. Nicolelis and colleagues at Duke have pushed that number to 500, and they’ve implanted up to four of these electrode arrays in a single monkey, enabling them to record from nearly 2,000 neurons simultaneously.
And there’s no reason to stop there, especially in the much larger brain of a human patient, Nicolelis says. With 20,000 or 30,0000 neurons, the fluidity of movements would be even better.
“I could get them to kick Brazilian style,” he said. “Not British, Brazilian.”
The other key, in his view, is incorporating tactile feedback. In 2011, his team broke new ground by demonstrating a neural prosthesis with an artificial sense of touch in monkeys. Electrodes implanted in a brain region responsible for the feeling texture enabled the monkeys to identify different virtual objects by “feel.”
Sensors on the exoskeleton will eventually feed directly into the brain in a similar manner to provide crucial feedback on the position of the limbs and when the feet hit the ground, Nicolelis says. “None of these robotic devices will work for real without tactile feedback,” he said. “You can’t walk without knowing where the floor is.“ The extent to which sensory feedback will be ready for the World Cup demo remains to be seen.
And with less than a year and a half to go, Nicolelis is still working exclusively with monkeys.

Shankari Rajangam monitors an experiment with a monkey in an adjacent room. Photo: Nick Pironio/Wired

In a small control room at Duke during my visit in January, a young woman dressed hairnet to booties in blue surgical garb monitors an experiment on several screens. She’s training a monkey in an adjacent room to control an avatar with its mind. Small grids of electrodes record signals from the animal’s primary motor cortex, creating softly crackling background noise on an audio monitor. A computer translates those signals into commands that control the avatar. What the real monkey thinks, the virtual monkey does. Or that’s the idea. For now the computer is doing most of the work.
On one screen, a cartoonish monkey avatar can be seen from the back, ambling slowly down what looks like a bowling lane towards a ghostly, translucent cube. The monkey sees the same thing on another screen inside its room. When the avatar monkey’s arms touch the cube, the real monkey gets a drop of juice, and the routine starts over. The juice reward teaches her that good things happen when the avatar touches the block. This monkey is just starting to learn the task, but with time the researchers will dial down the computer’s contribution to controlling the avatar and the monkey’s brain will take over, telling each leg when and how to move.
This animal is one of two being trained to test a monkey-sized prototype of the robotic exoskeleton. Once the animals master the avatar, they’ll take a crack at controlling the exoskeleton.
The monkey version of the exoskeleton looks vaguely insect-like. Color-coded wires hang from the ceiling. When a student switches it on, it sounds like an air gun firefight has suddenly broken out as pneumatic pistons spring to life with clicks and pffft’s and the empty exoskeleton takes a few steps.

The monkey exoskeleton. Photo: Nick Pironio/Wired

It’s suspended over a treadmill and attached to a harness. Nicolelis’ team is currently training the two monkeys to sit in the harness and let their legs go limp so the exoskeleton can do its thing. A few months from now the whole system will be subjected to a sterner test: Researchers will temporarily paralyze the legs of a monkey with an injection, and the primate will then try to transfer what it’s learned from playing with the avatar to control the exoskeleton with its thoughts. If it goes according to plan, the monkey will walk on the treadmill.
The brain of a monkey is about half the size of a person’s fist. A human brain is about 15 times larger. And that’s not the only anatomical difference. “The space between the skull and the brain is different in monkeys, it’s very tight and holds things in place,” said Shenoy. Electrodes in a human brain are more likely to move around and potentially lose the signal, which may be one reason neural prostheses have consistently performed better in monkey experiments than they have so far in people, Shenoy said.
“That translation between monkeys and humans is not a done deal.”

So far only two research teams, the one in Pittsburgh and another started by researchers at Brown University, have published reports on neural prostheses controlled by electrodes implanted in the brains of paralyzed people. Both declined to comment on Nicolelis or his plans.
“He’s a polarizing figure,” said Brendan Allison, a visiting scholar at the University of California, San Diego who researches brain-machine interfaces.
Whether the World Cup demo, if it happens, represents a scientific milestone depends on how much of the work is done by the exoskeleton and how much is done by the patient’s brain, Allison says.
“Getting a signal from the brain to do a task is much easier than people think,” he said. “I could put an electrode cap on your head, in a public place with lots of electrical noise, and within 10 minutes, you could send a reliable signal with thought alone.” If signals from the brain are used to issue simple commands to a super smart exoskeleton – walk, now kick – that’s less of a technological leap, Allison says.
If, on the other hand, signals from the patient’s brain can be used to control exactly when and how each leg of the exoskeleton moves, all while maintaining balance as the patient walks and shifts his or her weight to kick a ball, that would be a phenomenal advance, says Shenoy.
“If he really does what he says he can do, it’s a huge thing,” he said. But Shenoy adds that it will be hard for the public — or even experts — to know exactly what they’re seeing, or more specifically, how much of the exoskeleton’s movement is under neural control. “With a few billion people tuning in, think of the pressure to have something work.”
Gordon Cheng, the roboticist who is developing the physical exoskeleton at the Technical University of Munich in Germany admits that the deadline is tight. “We have bits and pieces of different prototypes being built and tested, we even have a complete mockup built,” he said. “We’re pushing it.”
By design, the exoskeleton will use a mix of signals. “If the signal from the brain is very good, the brain will take control. If the signal from the brain is not so reliable, the robot can take over more of the control,” Cheng said. “This is mainly to guarantee safety.”

A schematic of the human exoskeleton. Image: Gordon Cheng.

Even if the patient’s safety can be guaranteed, some bioethicists see potential red flags.
“I always get nervous with medical breakthroughs that are done partly as showmanship,” said Arthur Caplan, head of medical ethics at New York University’s Langone Medical Center. “They risk exploiting the subject.”
Whether that’s the case depends largely on what happens to the patient after the demo, adds Dan O’Connor of the Berman Institute of Bioethics at Johns Hopkins University. “Will Nicolelis and his lab be the real beneficiaries here, or is it this paraplegic Brazilian kid?” O’Connor asks. “What sort of access will he or she have to the technology [after the demo], and who will pay for it?”
Nicolelis insists the patient selected for the demo, and many others, will benefit from the technology for years to come, thanks to the largesse of the Brazilian government. That’s the goal of the center in São Paulo, he says. “The project doesn’t end with the World Cup, it starts with the World Cup.”
Nicolelis says his colleagues in Brazil are currently combing a database of thousands of patients to identify 10 for initial training. Their ideal profile: a smallish young adult, no more than 70 kilograms (roughly 150 pounds), whose injury isn’t too new or too old. Like the monkeys in the lab at Duke, the trainees will start by learning to control an avatar on a computer screen, but with brain signals recorded by non-invasive EEG electrodes to start. Then, if the plan stays on track, one brave recipient will go under the knife to receive electrode implants in his or her motor cortex.
The clock is running. The outcome is far from certain, but if the demo happens, one thing is clear: The world will be watching. http://www.wired.com/wiredscience/2013/02/robotic-exoskeleton/all/