Virtual and Augmented Reality Today – Part 2

Articles
vr2

In an earlier article I shared my enthusiasm for virtual and augmented reality (VR and AR). I believe these technologies offer tremendous potential, and they are particularly exciting to me as a user interface designer, as they allow new ways to interact with the digital and physical world around.

In the previous article, I introduced the technologies that enable modern VR and AR hardware. Now let’s take a look at some examples ofvirtual and augmented reality implementations that show these technologies at work.

Specialized Applications

Virtual reality applications developed for specialized use have been around for many years, and can be quite impressive. Military vehicle simulations are among the most sophisticated; flight simulators are the best known of these applications. Flying a plane is expensive and can be dangerous, especially if the pilot is untrained. Ground vehicle simulators are in widespread use as well. The military is linking simulators to each other, to simulate what is known as a Future Combat System – think of a networked battlefield (Strickland, 2016).

As important as these simulators are for training purposes, more interesting to me from a user interface perspective are infantry simulators. The U.S. Army’s Dismounted Soldier Training System (DSTS) equips an infantry squad with motion sensors and VR headsets, and then each solider becomes an avatar in a virtual world. Soldiers interactwith each other and simulated enemies in the virtual world by behaving as they would in the real world – looking around, ducking, jumping, aiming and firing a virtual weapon. Granted, this requires a support staff,the soldier must wear a lot of gear to make this happen, and there certainly are limitations, but the idea is that there is no interface per se layered between the user and the application. The idea of a separate “interface” disappears.

But commercial applications are where the excitement is right now. Many believe that commercial virtual reality is about to break through. Let’s take a closer look at two of the most anticipated hardware products that are enabling this shift: Microsoft’s HoloLens and Facebook’s Oculus Rift.

HoloLens

A promotional video for Microsoft’s HoloLens shows “a digital world… blended with your real world,” with holograms projected into the viewer’s reality. It is very impressive, with big-screen television images appearing on empty walls, Skype chats appearing in thin air, and virtual instructions guiding a plumbing repair. HoloLens accomplishes this by projecting holograms into our world, just like R2D2 projected a holographic Princess Leia – although the quality is better, at least in Microsoft’s promotional material, and the image is visible only to the person wearing the HoloLens.

It takes a lot of technology to accomplish this. When you wear HoloLens you basically have a Windows 10 computer on your head, as well as “one 2 MP video camera, an inertial measurement unit (which means, a set of accelerometers, gyroscopes and maybe even a magnetometer), one depth camera, four environment understanding cameras, four microphones, an ambient light sensor as well as mixed reality capture IC. The HoloLens can track gaze, understand gestures and voice commands.” (Shilov, 2016).

The HoloLens user interface appears to be built around many Windows 10 paradigms. Users “air tap” on interface elements; imagine using hand gestures to interact with a virtual menu projected into your field of vision.

Microsoft HoloLens is not yet available to consumers, but Microsoft has released HoloLens to software developers. Users do report limitations, among them a limited field of view where graphics can be projected (Metz, 2015) and limited supported interactive gestures.

Oculus Rift

Palmer Luckey was 17 years old when he began work on a prototype virtual reality headset. He raised $2.4 million on Kickstarter and, within four years, had the attention of Mark Zuckerberg, who believed the immersive 3D, such as that created by Luckey’s headset, was the future of computing, the Next Big Thing after the smartphone. Zuckerberg bought Luckey’s company for $2 billion.

That’s the short story of Oculus Rift. Oculus differs from HoloLens in that it is a VR device. Stereoscopic screens provide a view of the virtual environment. Headphones provide a 3D audio effect. The Rift is not a standalone device; it must be connected to a Windows computer with a powerful graphics processor.

The Rift headset works with other hardware:

  • A separate position tracking system, known as Constellation, uses infrared sensors to track the position of the user’s head).
  • Oculus is also selling a pair of wireless controllers, known as Oculus Touch, providing joystick, button, and trigger controls for gaming. The Constellation tracker tracks these controllers, so the user’s hands can appear in the virtual world.
  • Rift integrates with Microsoft’s Xbox One Wireless Gamepad, because many virtual reality games do not work with motion controllers such as Oculus Touch.

Oculus considers Rift to be primarily a gaming device, at least initially, but in the long run they believe that social applications will be most popular, allowing us to interact with others in virtual worlds. Media applications are also expected to be popular (imagine watching a movie in a virtual theater, or being immersed in a 3D movie). Specialized applications are also anticipated; for example, and application could allow and architect to wander through a virtual building before anything is built.

Some tantalizing interfaces have been previewed, such as Netflix’s UI, with movies and television shows floating in virtual space. Another concept shows how a Windows-based interface might work in an immersive environment, with the user grabbing and arranging windows in a three-dimensional space.

The social applications are also exciting to me. Facebook anticipates that, within a few years, customers will be able to shoot and share immersive 3D videos that will be viewable on phone or other traditional screen, but will become truly immersive when viewed on a headset such as Rift (Metz, 2015).

The Potential

Obviously there is immense potential to these technologies. But commercial virtual reality has failed to meet the hype surrounding it in the early 1990s, and early reviews of Rift, for example, are mixed. Will virtual and augmented reality be successful now? I can’t claim to know how quickly VR and AR will become pervasive, but I have to believe that, given enough time, they will become very popular and ubiquitous.

In the next article I’ll look at the potential of VR and AR, as well as some of the challenges that will need to be overcome.

References

  • Metz, C. (2015, March 27). How Facebook’s Chief Geek Will Meld Reality With the Oculus. Retrieved March 4, 2016, from Wired: http://www.wired.com/2015/03/facebooks-chief-geek-will-meld-reality-oculus/
  • Metz, C. (2015, May 1). The HoloLens isn’t as Great as You Think–at Least Not Yet. Retrieved March 4, 2016, from Wired: http://www.wired.com/2015/05/microsoft-hololens-narrower-than-you-think/
  • Shilov, A. (2016, March 3). Microsoft Reveals Additional Details About HoloLens and Begins to Take Pre-Orders. Retrieved March 4, 2016, from AnandTech: http://www.anandtech.com/show/10115/microsoft-reveals-additional-details-about-hololens-and-begins-to-take-preorders
  • Strickland, J. (2016, March 4). How Virtual Reality Military Applications Work. Retrieved March 4, 2016, from How Stuff Works: Science: http://science.howstuffworks.com/virtual-military1.htm

 

Dr. Craig Rosenberg is an entrepreneur, human factors engineer, computer scientist, and expert witness. You can learn more about Dr. Rosenberg and his expert witness consulting business at www.ui.expert