The reviews are in, and the tech press is lauding the Apple Vision Pro headset for delivering on the company’s promises. It’s well-designed, the video and sound are startlingly precise, the “Minority Report”-style gestural interface is future-tastic. Nobody’s exactly sure what it’s for, or whether even the Readiest Players One will spend $3,500 on it, but hey — that’s gadgets for you.
Still, this is a new gadget frontier. The Vision Pro, like the similarly kitted-out Quest 3 and Quest Pro headsets from Meta, uses what’s known as “passthrough” video — cameras and other sensors that capture imagery of the outside world and reproduce it inside the device. They feed you a synthetic environment made to look like the real one, with Apple apps and other non-real elements floating in front of it. Apple and Meta are hoping that this virtual world will be so compelling that you won’t just visit. They’re hoping you’ll live there.
That, unfortunately, could have some very weird and very messy consequences for the human brain. Researchers have found that widespread, long-term immersion in VR headsets could literally change the way we perceive the world — and each other. “We now have companies who are advocating that you spend many hours each day in them,” says Jeremy Bailenson, director of the Virtual Human Interaction Lab at Stanford. “You’ve got many, many people, and they’re wearing it for many, many hours. And everything magnifies at scale.”
Meaning: Our brains are about to undergo a massive, society-wide experiment that could rewire our sense of the world around us, and make it even harder to agree on what constitutes reality.
The short-term side effects of virtual reality are well established. People in synthetic environments tend to misjudge distance, both at a distance and close up. That’s no surprise: Even in the real, three-dimensional universe, our ability to determine how close or far away something is is subject to all kinds of external factors. Virtual environments, with their lower resolution and synthetic 3D, make all that worse — which is especially bad if you’re one of those users posting videos of yourself doing things like skateboarding and driving while wearing a mixed-reality headset. You think your hands are in one place, they’re actually in another, and pretty soon you’re driving your Honda Civic through a supermarket.
Objects in a headset can also get funhoused. That’s called object distortion — things get warped, and change size or shape or color, especially when you move your head. A video render can’t compete with the processing speed and fidelity of your eyes and brain.
These are all, as the IT people say, known issues. For a few minutes or an hour, long enough to play a game or watch a movie, they’re minor annoyances. But wear perception-shifting glasses for days at a time — as Bailenson’s team of researchers did — and the problems get worse. Way worse.
The team wore Vision Pros and Quests around college campuses for a couple of weeks, trying to do all the things they would have done without them (with a minder nearby in case they tripped or walked into a wall). They experienced “simulator sickness” — nausea, headaches, dizziness. That was weird, given how experienced they all were with headsets of all kinds. And they felt all the distance and distortion effects: thinking elevator buttons were farther from their fingers, or experiencing difficulty bringing food to their mouths. But as any of us would, they adapted — their brains and muscles learned to compensate for their new view of the world.
That seems like a solution, but it ain’t. When people adapt to a perceptual change for long enough, the real world starts to look wrong in the opposite direction. If you wore glasses that turned your vision upside down, let’s say, you’d have to adapt again when the glasses came off. The longer you’re inside a funhouse world, the longer the weird perceptual aftereffects last. So people who spend their workday inside a Vision Pro might go home at night with a miscalibrated targeting system and what feels like a shroom hangover.
Here’s where the passthrough video gets uniquely important. Old-school cyberpunk envisioned virtual reality as an all-encompassing synthetic environment. New-school techies, meanwhile, proposed an augmented reality of digital pop-ups floating on see-through lenses, Google Glass-style. But both of those approaches have limits. Full, sense-isolating VR hasn’t progressed much further than niche entertainment, while AR tends to make both its apps and the real world look bad. From a visual standpoint, passthrough is the least-worst solution — but its social consequences are scarier.
Because passthrough captures and then re-renders reality, it can have an unnerving, distancing effect over time. When Bailenson’s colleagues actually tried to talk to people, the world turned into a giant, confusing Zoom. Video chats, as we’ve all experienced, are plagued by delays in responses and missed social cues. Conversations lose subtlety, but it’s good enough for a meeting. But passthrough magnifies the effect — the people you talk to start to seem unreal. Up close, they look like avatars. Farther away, they become just part of the background.
Bailenson describes the feeling as one of social absence. Other people just aren’t fully there. He doesn’t put it this way, but I’ll wave the warning flag: Long-term use of passthrough headsets could make it easier to think of other people as unhumans — non-player characters in a gamified, uncanny valley.