Every Body Part Video Games Want To Access

Wearables are changing the ways in which technology interacts with our bodies, and that tech trend is making its way into gaming. Getting players immersed in a game is the goal of any developer, and the latest technology has the potential to create a seismic shift in how people get invested in their games. By utilizing different types of biofeedback, games of the future could adapt to how players are feeling to offer a more customized experience.

Stepping into the future of gaming might mean getting up close and personal with tech companies in ways that may not make everyone feel comfortable. Regardless of the benefits, some gamers aren't going to want to send pictures of their inner ear to Sony or place their memories on a chip owned by Elon Musk. As gaming tech gets more involved in the human body, the best and worst possibilities of sci-fi movies like "Ready Player One" could become realities. With an eye towards the future, here is every body part that video games are planning to access.

Back in March 2020, Sony was showing off some of the features that would come along with the then-upcoming PlayStation 5, and lead console architect Mark Cerny made a shocking comment about its high tech audio capabilities (via Inverse). Cerny explained that the PS5's Tempest 3D AudioTech represented a massive leap forward for gaming audio because it could create customized audio profiles by mapping a user's cranium to something called a head-related transfer function, or HRTF. Cerny said it could eventually be possible for users to submit photos and videos of their ears so Sony could "choose a neural network to pick the closest HRTF in our library."

The idea might have seemed bizarre to some gamers, but this actually wasn't the first time that Sony had asked users to submit ear pictures. In 2019, Sony announced 360 Reality Audio, a technology that uses 24 audio channels to create a fully immersive listening experience by using imaging of ear canals to map out its sound. While the technology is genuinely impressive, Sony should have expected some snarky reception from gamers. Some started uploading pictures of their ears on social media, tagging Sony, PlayStation, and even Cerny himself in the posts.

In the future, gaming controllers will likely be quite aware of how your body is responding to the games you're playing. Sony gave the world a peek at how controllers can gather and incorporate "biofeedback" when it filed a patent for a new controller in February 2020, just ahead of the PS5 release (via Respawn First). The patent revealed that Sony had plans for a controller that could monitor a player's heart rate and analyze how sweaty their hands got as they played. In theory, the controller would gather that data and allow games to actively respond to how players are feeling.

The PS5's DualSense controller is packed with advanced features, but sweat-sensing isn't one of them. It's not all that surprising Sony ultimately held back on the idea. As cool as the technology may be, its usefulness would be entirely dependent on game developers, and some have complained that current games aren't yet doing enough to use the features that did come with DualSense. Don't expect Sony to give up the fight to get into your sweat glands, though. Sony console architect Mark Cerny told The Verge that the idea actually dates back to the planning stages of the DualShock 4 controller.

As wearable technology becomes more prevalent, it's inevitable that some games will find imaginative ways to incorporate biofeedback into their gameplay. Some developers are already ahead of the curve. At Boston Children's Hospital, researchers built a game that monitors a player's heart rate to help children learn to better regulate feelings of anger, stress, and anxiety (via Neuroscience News). The game is called RAGE (Regular and Gain Emotional) Control, and in it players shoot asteroids from a small spaceship, and their performance is tied to how calm they can remain while navigating through the obstacles. By gamifying the biological response to stress, RAGE Control helps kids practice calmness in a way that wouldn't be possible otherwise.

Gamers aren't going to wait for more developers to catch up to biofeedback technology. Some are already finding ways to incorporate biofeedback like heart monitors into their games, and they're discovering the tech can appeal to more than just players. Twitch streamers have started displaying their heart rate while playing to give audiences a more connected experience. The trend is more common for streamers playing pulse-pounding horror games (via Reddit). There's even an entire website dedicated to ranking horror games according to the average heart rate of their players. Eventually biofeedback will be a much more common part of gaming culture at every level.

Millions of gamers had their first motion control experience with the Nintendo Wii. Though it was groundbreaking at the time, the Wii required awkward movements and even then it had problems accurately translating many gestures. Today motion controllers are much more sensitive and accurate, and they are of the verge of going through a massive transformation. Myo, a company that Google acquired in June 2020, pushed the envelope of motion controls by developing an armband that could accurately detect hand gestures. The armband uses Bluetooth to communicate with a set of electrodes to analyze what gestures users are making, and it's so accurate it might have a future as part of prosthetic limbs (via ResearchGate).

Over in the VR sphere, companies are working on developing gesture controls that don't require any additional wearables at all. The Quest 2 has a hand tracking feature that, when it works, allows players to interact with objects in VR with no controllers at all (via UploadVR). The feature uses the Quest's onboard cameras with what the developers call "deep learning" to both track how a player's hands are moving and predict what they are likely doing when the cameras don't have a good view of them. The hand tracking works well enough that playing air guitar is a real possibility, and it's only going to improve from here.

For the most part, video games engage three different senses: sight, sound, and, to varying degrees, touch. Most game companies are hard at work finding ways to make improvements in those three areas with better graphics, higher audio quality, and improved haptic feedback. However, that doesn't mean that companies aren't eyeing ways to get more senses involved. Sony recently filed a patent for technology that could create synthetic smells based on what's happening in a particular game (via GameRant). It's unclear how, or when, Sony would hope to incorporate the tech into new games, but some day while wandering in an open world game like "Elden Ring," you might be able to literally stop and smell the flowers.

Luckily for anyone who's craving that next level of immersion, there's already a way to play games with scents. Feelreal has made a mask that can create hundreds of scents that correspond to various in-game situations. It's designed to be incorporated with VR experiences, and there are models already built for a handful of the more popular VR headsets. Feelreal currently works with only a limited selection of games, but the device proves that scent technology isn't just a hypothetical idea for the future.

Because they can only roughly simulate actions like running or shooting, most game controllers only represent a small barrier to immersion. There are, of course, specialized controllers that can help players get more immersed in what they're doing. HOTAS controllers more accurately simulate the experience of flying, for example. Some VR games use hand-tracking for controls, and most others use some type of motion controller that's much more natural feeling than, say, the Wiimote. In the not-too-distant future, it might be possible to forgo controllers altogether and literally get your head in the game.

Neurable is a developer that's already created a game fully controlled by a player's mind (via OpenMind). In "Awakening," players take control of a telekinetic boy who must use his powers to escape a science lab. Players use their brain to control every action in the game, from walking to throwing objects with telekinesis. The game uses a combination of six electrodes and Neurable's advanced algorithm to translate thoughts into digital actions. The technology is currently way too pricey to bring to store shelves just yet, but now that the tech has been proven to work, it's only a matter of time until costs are lowered enough for general consumers.

Leave it to Elon Musk to try and push tech to its extremes. While Neurable is paving the way for your mind to control a digital environment, Musk's latest side project is trying to turn your mind into a digital environment. According to TheGamer, Musk recently launched a company called Neuralink in 2022, which is working on exactly what you think it is. "In the future, you'll be able to save and replay memories," Musk said, adding, "You could potentially download them into a new body or into a robot body."

For decades, science fiction has imagined a future in which people combine their minds with computers. Neuralink could potentially bring that future into the present. Musk says that the company's brain chip is "working well in monkeys," and it may not be long before Neuralink runs human trials of its tech. Being able to perfectly preserve and even share memories sounds like an amazing prospect, but anyone who's ever had a hard drive get corrupted might be a little hesitant to store their memories on silicon. Time will tell if Neuralink is the future of digital technology or a step towards the dystopian world of "Cyberpunk 2077" (or both).

The Oculus Quest 2 may have its shady elements, but it's also helped push VR into the mainstream by giving people an affordable all-in-one VR platform. Now that more people are experiencing the incredible immersion that's possible with VR, they're getting hungry for more. Playing in VR can seem so real, but there are companies developing new peripherals that will actually let players feel things for real. Haptic gloves and vests are commercially available to gamers who want to become as immersed as possible in their VR games. But as amazing as those peripherals are, they pale in comparison to what's just around the corner.

Full body haptic suits, like the one being developed by Teslasuit, can simulate a stunning range of physical sensations (via ABC News). These suits can simulate everything from the feeling of raindrops to a punch in the gut. For now, these suits are prohibitively expensive and mostly exist in labs, but that might not stay the case for long. 

"The Teslasuit technology can be brought into everyday clothing," said Dr. Ross Smith, director of Wearable Computer Laboratory at the University of South Australia. Haptic feedback suits won't just revolutionize gaming, they'll also be useful as training tools for a wide range of professions, which should help drive costs down faster. Before you know it, you'll be able to truly step inside your favorite game world.