An advance in virtual reality gaming gives users haptic feedback on their lips, teeth, and tongue.
Consider the way a cool drop of water flows over your lips. Or, think about the wet splatter of mud that sometimes globs onto your face while you stomp around in a mud puddle. Finally, imagine an ant crawling over your mouth. You could feel all of this and more in virtual reality (VR) if you pop on a gaming headset from researchers at Carnegie Mellon University (CMU) in Pittsburgh, Pennsylvania.
A team at CMU’s Human-Computer Interaction Institute (HCII), looking for novel ways to create in-game haptic feedback, took advantage of the fact that lips have more than one million nerve endings. Other than our fingertips, they are the most sensitive part of the body—perfect for receiving haptic feedback, a gaming feature that converts electrical signals into touch sensations during play.
Just one problem: gamers don’t want to cover their mouths or faces with anything, according to the Future Interfaces Group (FIG) in the HCII. Most haptic feedback in today’s gaming devices is designed for gloves and handheld controllers that use vibration motors. Developing a practical device that utilizes the sensitivity of the mouth (and is still comfortable enough to wear) has been a longtime challenge in the VR industry.
The university team has circumvented that concern by using sound waves in an ultra-high frequency that humans can’t hear. Ultrasound may be a familiar concept for people who need medical imaging of their insides, like observing a fetus moving within a pregnant belly. For the VR application, researchers combined an array of transducers to produce ultrasound waves that are concentrated enough to mimic sensations.
Let’s say your in-game character drinks from a water fountain, for example. “Every time you lean down and think you should be feeling the water, all of a sudden you feel a stream of water across your lips,” Vivian Shen, one of the researchers at the HCII involved in the work, says in a news release. “It’s cool. It makes the experience much more immersive.”
Shen’s team decided to concentrate ultrasound waves to create feelings on the skin. By putting together an array of 64 tiny transducers—modules that convert the ultrasound energy into electrical energy—the team could amplify the ultrasound waves.
“If you time the firing of the transducers just right you can get them all to constructively interfere at one point in space,” Shen explains. Each transducer emits its own ultrasound waves, and the waves of different transducers can interfere with each other to boost the sensation. Or, they can reduce the sensation, since the interfering waves can also cancel each other out.
The transducer array is attached to the bottom of a set of VR goggles, pointed at the mouth. Sixteen volunteers tried out the prototype device and reported that it enhanced the experience of being touched on the lips, teeth, and tongue. When combined with in-game visuals, the users reported experiencing realistic sensations like swipes, vibrations, and pressure.
CMU researchers noted that all effects didn’t work equally well. Mouth-specific activities—like brushing teeth, feeling water droplets from rain, and the creepy-crawly feeling of a bug walking across the lip—felt most authentic. However, when experimenters walked through virtual cobwebs, they didn’t buy it; they expected to feel cobweb sensations over larger parts of their bodies, as well as on the face. That makes sense: ultrasound sensations work best on the mouth or the hands, Shen says, because other parts of our bodies, like the forearms and torso, don’t have enough nerve receptors for the device to stimulate.
Even though the water fountain experiment was mouth-specific, it was not quite realistic, either, and even proved a bit disorienting for the volunteers. But maybe that’s not surprising. “It’s weird, because you feel the water but it’s not wet,” Shen says.
The team plans to keep working on the device to make it even smaller and lighter. Maybe they’ll figure out how to make water fountains more believable for thirsty gamers, too.