A dream—or nightmare—is gradually yielding to scientific progress. In their study, “Reconstructing Visual Experiences from Brain Activity Evoked by Natural Movies,” researchers at the University of California, Berkeley, used functional Magnetic Resonance Imaging (fMRI) and quantitative modeling to decode and reconstruct people’s dynamic visual experience. Subjects spent hours inside an MRI scanner, watching Hollywood movie trailers, while the scanner measured blood flow through the brain’s visual cortex. Subsequent quantitative modeling of this data yielded actual videos of what might have been going on in the subjects’ minds (study and video).
The hope is that these efforts will lead to applications that can access the minds of people who cannot communicate verbally, like coma patients. But if the technology ever reaches that stage, there will be no reason to stop at coma patients. Why not read the mind of your spouse? Or of everyone? Google and Facebook, if they still exist in that distant future, will have a field day. And surely, there will be an app for that.
Other significant efforts are underway. At the annual CeBit conference in Hanover, Germany, last March, researchers from the Berlin Institute of Technology showed off a pinball machine controlled by electroencephalography (EEG), which records the brain’s electrical activity via dozens of electrodes placed on the scalp.
In April, Japan announced a big collaborative project by its car makers and research institutions to develop technology beyond thought-controlled games and phones: car navigation systems that would search for restaurants when the driver thinks about food. I mean, come on. We think about food a lot. Will that system take us to the next restaurant instead of our appointment? What if someone with diabetes thinks about a milk shake? What if we think about sex?
Meanwhile, German researchers at the Freie Universität Berlin produced a car that allows drivers to steer left and right as well as accelerate and decelerate—via a neuroheadset developed for video games by Emotiv, a San Francisco company. For this to work, the researchers had to train the headset’s 16 sensors on the scalp to recognize the brain’s electric signals related to specific driving commands.
The latest effort is the collaboration between Nissan and the École Polytechnique Fédérale de Lausanne in Switzerland, which already developed a Brain Machine Interface (BMI) system for wheelchair users (press release). Now Nissan wants to advance the system to the next level. Using profuse amounts of data and quantitative modeling, the device would associate electric activity in the brain with eye movements and data from the surroundings to guess if the driver wants to turn right or stop to pick up a hitchhiker. It would then execute the driver’s thoughts.
So a pedestrian jaywalks. Driver thinks, “You idiot, you.” The BMI captures the thought and activates the horn. Pedestrian holds up his middle finger. Driver thinks, “I’m going to kill you.” The BMI captures the thought and the wild eye movement and….
OK, there would in fact be another application running on Windows 8, possibly, that would determine, based on data from numerous sensors, that this is a human being, the number one item on its list not to hit. And it would shut down the BMI temporarily. We do that in real life all the time. One impulse gets shut down by an application that is trained to do just that. Call it our moral sense, good financial decision making, survival instinct, or whatever.
As we know from the flash crash and countless other incidents, quantitative modeling can lead to compelling but wrong decisions. Combining quantitative modeling with activity in the brain is a prescription for disaster; the brain, our most unreliable organ, uses its profuse creativity on a routine basis to make up for shortcomings in data and knowledge. And the application that is supposed to shut down the BMI at certain critical junctures doesn’t always work. We see that all the time. People make truly crazy decisions and execute them to the utter astonishment of everyone else around.
Enjoy reading WOLF STREET and want to support it? You can donate. I appreciate it immensely. Click on the beer and iced-tea mug to find out how:
Would you like to be notified via email when WOLF STREET publishes a new article? Sign up here.
