Virtual reality is magical. Showing your latest gadget or game to the uninitiated often impresses. But what comes after the “wow”?

Virtual reality is in that way rather disturbing. You stay disconnected from others, in a personal space, no one else can see. It seems like the antithesis of family. But it’s also something that everyone loved. They laughed. They were astonished. It was magic.

Here are some of the major challenges.

1. Hardware

Headset

The VR experience, in its current form, is imperfect, to say the least. The headsets are bulky, heavy and require either proprietary cameras or base stations to do proper head and body tracking. Two models withstanding, the devices are wired, meaning your movement is restricted to a confined space and limits the immersive experience.

Cost

The technology is out of reach for many. And it will be so for a long time. Less than one percent of the 1.43 billion computers in the world have the graphical capabilities needed for VR, according to the research company Gartner. There are definitely high-end computers that are optimized for it, but they’re costly. This cost shrinks the potential audience for VR content.

Fragmentation

When your application is finished, you might want to run it on a different hardware: You might have created your application for the Oculus Rift and want to try with another HMD, or maybe a stereoscopic wall or a Cave.

You then have to modify your application to take into account the new trackers, the new screens (by managing the virtual cameras and viewports), the cluster synchronization. You will also probably need to modify the interactions because you don’t necessarily have a perfectly equivalent hardware or tracking volume.

Deploying your application on different VR systems can prove to be a very difficult and time-consuming task.

Other platforms (Mobile, PC, Console) have fragmentation as well, but with VR the standards and practices are so nascent and changes are coming fast. Things will diverge much more.

2. User Experience

Input

While the visual experience of VR is such well-defined paradigm shift, the way we interact leaves much to be desired.

Because VR can be easily overwhelming for users (see ‘3. VR MOTION SICKNESS’ below), simply moving through the world needs a new solution. Instead of using a game controller to freely move through the world, many first generation VR games prefer a ‘point and click’ style of teleportation. It works, but I feel it takes me out of the immersion of the experience. Here is a great overview of VR approaches to locomotion.

Feedback

Rendering

VR requires high CPU/GPU requirements. Today’s non-VR game running on an HD monitor; this has a resolution of 1920 by 1080 pixels and runs at least 30 frames-per-second (FPS). But for VR, that screen would be cut into two halves, each brought up close to an eye, with a combined resolution of 2160 by 1200 pixels. This picture occupies most of your peripheral vision, which means it shows more virtual objects than a normal game.

Latency

The first enemy of VR is latency. If you move your head in the real world and the resulting image takes one second to appear, your brain will not accept that this image is related to the head movement. Moreover as a result, you will probably get sick. John Carmack reports that “something magical happens when your lag is less than 20 milliseconds: the world looks stable!

Plus, instead of an FPS of 30, VR demands a refresh rate of 90 to 120 times per second. So more powerful hardware is needed to power the higher resolution and higher FPS.  Software solutions such as reprojection (See 22:30 mins into video) can greatly reduce the hardware needs.

3. VR Motion Sickness

Some users experience a special motion sickness from VR experiences.

The Oculus Rift’s health and safety documentation lists many potential symptoms including, eye strain, nausea, and impairment to balance and coordination.

Research into this area is very new, but already many factors have been identified.

• Age: Most common for ages 2-12.
• Experience with the system: Users who use VR often MAY develop a tolerance which prevents sickness.
• Gender: Women are more likely than men to feel virtual reality sickness.
• Health: Ill people are more likely to be motion sick.
• Motion Sickness Sensitivity: People who are more sensitive to motion sickness tend to be more sensitive to virtual reality sickness as well.
• Latency: Current Virtual Reality just can’t match our natural experiences.  Real life doesn’t have much lag. 🙂

4. VR’s Unsocial Side

There’s also the matter of blindness. While wearing VR goggles, you’re effectively blind to reality and can’t see or hear anything apart from the experience itself. You must completely surrender yourself to the process, which leaves many participants feeling intimidated or socially awkward.

Augmented Reality (AR) solves this by allowing the users to see their real environment either directly or indirectly.

The majority of all VR experiences are not multiplayer. Certainly multiplayer is possible, but the first generation of software we are seeing are relatively simple in scope compared to today’s mass market AAA games. One welcome exception is Survios’ Raw Data — a multiplayer wave-based shooter game.

5. Narrative Challenges

Avatars

When you are using a head-mounted display (HMD), you are completely immersed in the virtual world, and you don’t see your own body anymore. It is very important to display a virtual representation of yourself and others, called avatars. They can be realistic, look like yourself, or be completely different.

If your VR system offers full body tracking and if you want an avatar that has exactly the same dimensions as you, this can be a simple task. But if your VR system only has a few trackers and you want an avatar that is taller or smaller than you, it can be difficult to extrapolate the position of the limbs that are not tracked and to adapt your body posture to a different virtual body.

Past vs. Present

And this brings us to the elephant in the room with the current notion of VR Storytelling. Storytelling is a retrospective thing. It always has been. People didn’t sit around the campfire telling stories in the timeframes that they actually occurred. And I’m not aware of real-time books. Linear narrative mechanisms have evolved to break down the constraints of time and emotive viewpoint.

So how do we tell about the past (linear narrative) while being in the present (non-linear experience)?

More About Narrative

• Why VR “Storytelling” does not currently work. And can it ever work?
• The Art of Storytelling and Narrative in VR
• Narrative Design in VR (Unity Technologies, Inc)

In The End…

Virtual reality is magical. But there are many real challenges and obstacles in the nascent industry. My VR experiences have been full of wonder and inspiration. I am hopeful and willing to shape the long-term future of the technology.