Virtual Reality: The Potential and the Challenges

Articles
vr-image-3

Imagine being in a movie – not watching a movie, but wandering around inside the movie, looking at and listening to what you want. That’s the potential of virtual reality, and it’s getting close. In this article, I’ll look at where virtual reality may take us in the next few years, and I’ll also discuss some of the challenges to this vision.

If you want to learn more about virtual and augmented reality (VR and AR), you can read my two previous articles, covering the technologies that make VR and AR possible and the current state of the art of VR and AR.

The Potential…

Virtual reality will continue to have greatest impact on media, with gaming leading the way. Yet all types of media will be consumed via VR. Consider the potential for movies. According to The Wall Street Journal(Schwartzel, 2016), “Over the next few years, virtual reality will completely reboot your relationship to the moving image. That’s because virtual reality… is becoming shockingly good at making you feel as though you’re in the midst of the action.” Short movies are already being produced in VR, and directors including Ridley Scott, Steven Spielberg, and Jerry Zucker are working on upcoming projects. Sports are available in VR as well. A company called NextVR has broadcast the US Open (golf) and NASCAR in VR, and plans to broadcast more sporting events.

VR will also become more important on social networks. Facebook obviously believes this to be the case, and I am very interested to see what they do with Oculus Rift. Mark Zuckerberg has outlined his vision:

Immersive gaming will be the first, and Oculus already has big plans here that won’t be changing and we hope to accelerate… But this is just the start. After games, we’re going to make Oculus a platform for many other experiences. Imagine enjoying a court side seat at a game, studying in a classroom of students and teachers all over the world or consulting with a doctor face-to-face — just by putting on goggles in your home. This is really a new communication platform. By feeling truly present, you can share unbounded spaces and experiences with the people in your life. Imagine sharing not just moments with your friends online, but entire experiences and adventures.

There is also great potential for VR and AR in industry. Dexter Lily, Executive Vice President of Index AR Solutions, predicts that:

AR will be ubiquitous in industrial environments in 10 years. Everyone will have access to data about their environment, their work or any other situation in which digital on-demand data is required to improve efficiency and effectiveness…. It will leverage 3D assets and other digital information that most companies have already invested in and put that information in the hands of workers and technicians where they need it, when they need it and in the form that’s intuitive and easy to use. AR will enhance workers and technicians to a point that some robots might be replaced by the augmented worker(Mohr, 2016).

Virtual architectural renditions are already difficult to distinguish from photographs, and 3D walkthroughs are improving in quality as well, allowing you to take a stroll through a building as it is being designed.

We will see similar impact in medicine. Google Glass has been used by doctors to record surgeries (O’Connor, 2014). Augmented reality has great potential to put information in front of medical professionals in real time; for example, an AR application currently helps nurses locate veins for IVs. Patients will soon benefit as well; for example, augmented reality could assist patients with memory loss.

…and the Challenges

There are, of course, challenges that must be overcome. I believe these challenges will limit rather than prevent widespread adoption of VR and AR.

I’ll begin with one challenge that has already claimed a victim – Google Glass. The challenge is social acceptance. David Pogue (2013), writing in Scientific American, sums the issue up in the title of his article: “Why Google Glass is Creepy.”

For a year now Google employees and celebrities have been allowed to wear Glass. But when I ran into a Google employee wearing it in public and had a conversation with her, the interaction was screamingly uncomfortable. There she was, wearing this creepy-looking, faux-futuristic forehead band—with a built-in video camera pointed at my face. For all I knew, it was recording me… This puts Glass wearers in a position of control. They can take pictures and videos, post things online and even possibly use face-recognition apps to identify strangers in a crowd.

Reactions such as this were one of several factors that led Google to retreat and retrench, leaving Glass a work in progress. It would be a mistake to generalize from Glass to all VR and AR systems but, in addition to the privacy implications identified by Pogue, a core issue should be considered: eye contact is critical to how we interact with each other, and VR and AR applications interfere with eye contact. Virtual reality eliminates the possibility of eye contact in the real world, and augmented reality can also interfere with eye contact. If someone wearing Google Glass or a HoloLens is looking at me, I don’t know what information is being overlaid on me. Is the person looking at me, or reading a dossier on me?

For some users, such as gamers, social acceptance is less likely to be an issue, as it is the norm to be immersed in a virtual world. Further, Google Glass was intended to be used in social situations in the real world, while Oculus Rift and HoloLens are not. You’re likely not going to be chatting with someone at a cocktail party wearing a HoloLens. That said, social conventions for virtual reality have not been established, and the Google Glass experience indicates what can happen when social acceptance does not occur.

The current state of the hardware is another issue. This is something that Google got right with Glass – it’s lightweight and unobtrusive, especially when compared to headsets such as Rift and HoloLens, which are bulkier and take more of a commitment to wear. Other VR paraphernalia, such as that required for motion tracking, are also encumbering. Gamers, again, may not mind suiting up for their VR experience, but how far beyond will acceptance go?

Another issue is that virtual reality can cause motion sickness. There are various theories for why this occurs, many having to do with sensory conflict – your eyes tell you you’re moving in one manner in the virtual world while your vestibular system tells you’re moving in a different manner in the physical world. The quality of the visual image also seems to have an effect, with poor-quality animations, slow video refresh rates, or excessive system latency all likely to induce motion sickness. The user’s posture may also have an effect. To address this problem, VR developers are focused on increasing image quality, reducing tracking latency, and fine tuning how the user moves through the virtual environment. Research is ongoing.

No Limits for User Interface

Despite these concerns, VR and AR are coming, and they are going to open up so many possibilities for user interface design.

The Martian VR Experience is based on the movie of the same name and will be viewable on Oculus Rift and HTC Vive. I’m looking forward to experiencing it, and to driving Matt Damien’s Martian rover. A writer on Road to VR describes the experience (James, 2016):

You find yourself in an airlock. Reaching out with your motion controller, holding the trigger on either allows you to wrench down with one naturalistic motion and presto! You’re then transitioned to the driving seat of the rover vehicle, so prominent in the movie, and this is where the fun really begins… As you look down, you see your spacesuit clad arms and hands are rendered to mimic your movements with the motion controllers. Holding the trigger on your controller activates a contextual action. What’s immediately rewarding, and indicative of the development team’s attention to detail, is that reaching out with my left hand, I’m able to use that contextual action to flip various switches on the dashboard. These moments may seem trivial, but they can be so very important to sell presence in a virtual space, it’s always rewarding to find experimentation like this paying off.

What’s most fascinating to me is that VR and AR remove the limits on the user interface – the UI can be anything. In this case, the UI is the Martian rover control panel. It is just as easy to imagine any other analogy with a real or imagined physical world, such as Leonardo da Vinci’s workshop or an orchestra conductor’s podium. Yet, designers are not constrained to mimicking the real world — other UI paradigms can be overlaid, allowing users to snatch menus or windows out of the air. The potential is enormous, and it is already revolutionizing human computer interaction.

References

  • James, P. (2016, January 17). Hands On: The Martian VR Experience is a Triumph in Motion. Retrieved March 7, 2016, from Road to VR: http://www.roadtovr.com/hands-on-the-martian-vr-experience-is-a-triumph-in-motion/
  • Mohr, K. (2016, February). 2016 Predictions: Augmented Reality Ready To Tackle Industrial Challenges. Retrieved March 7, 2016, from Manufacturing Net: http://www.manufacturing.net/blog/2016/01/2016-predictions-augmented-reality-ready-tackle-industrial-challenges
  • O’Connor, A. (2014, June 1). Google Glass Enters the Operating Room. Retrieved March 7, 2016, from New York Times: http://well.blogs.nytimes.com/2014/06/01/google-glass-enters-the-operating-room/?_r=1
  • Pogue, D. (2013, June 1). Why Google Glass Is Creepy. Retrieved March 4, 2016, from Scientific American: http://www.scientificamerican.com/article/why-google-glass-is-creepy/
  • Schwartzel, E. (2016, March 4). Virtual-Reality Movies: Get Ready for the VR Revolution. Retrieved March 7, 2016, from The Wall Street Journal: http://www.wsj.com/articles/virtual-reality-movies-get-ready-for-the-vr-revolution-1457030357

 

 

Dr. Craig Rosenberg is an entrepreneur, human factors engineer, computer scientist, and expert witness. You can learn more about Dr. Rosenberg and his expert witness consulting business at www.ui.expert