Confidence is a prerequisite to enjoy VR’s magic. There are so many things that can go wrong wearing a VR headset and most of us know this before the goggles come down over our eyes. “Discomfort” doesn’t quite encapsulate the risk of wearing something that blocks your view of the real world. Just trying to lean on something that isn’t really there is potentially deadly. Headset manufacturers must overcome this problem to give people confidence to move freely.

And there are so many reasons to stay in the real world too — your family, friends, and TV. Some of the world’s largest technology companies have learned over the last three years that VR headsets will never be mainstream as long as they a.) limit your movement too much and b.) block you off from your friends and family.

The company which solves these problems in an affordable way — with zero setup required — will have a strong position in immersive computing. This article is focused on showing why Google might be best positioned to be that company.

‘Organize The World’s Information’

In September 2015 I asked John Carmack, chief technology officer at Oculus, about the prospect of inside-out tracking technology (the idea that a gadget tracks its own location) for a Facebook VR headset. He said:

“We have like 30 computer vision experts at Oculus from the different companies we’ve acquired and none of them want to just go solve this problem. They’re all working on their esoteric, kind of researchy things while this is a problem that I want solved right now. I wish somebody had spent all of this last year on it.”

For some context, Google had acquired the creator of Word Lens more than a year before that comment was made. The developer had created an AR app which recognized text in another language on signage in the real world and replaced it with a translation in your own language.

While the two technologies aren’t the same — and Facebook has made great progress on inside-out tracking — the acquisition of Word Lens by Google is a key example of how the company makes early investments in people and projects that will pay off in half a decade or more. Google Lens now does what Word Lens did for translation and it is available as a core part of Google’s services and apps across millions of devices. You can search a vast Google Photos library using terms like “paper” to find a document you photographed, or “swimming” for a photo of a pool party. Combine that work with other long-term multi-year efforts at Google like Johnny Lee’s Tango tracking project, and you can begin to see how the company will apply its core goal to VR — “Organize the world’s information and make it universally accessible and useful.”

Room Simulator

For the game Job Simulator, Devin Reimer at Owlchemy Labs spent hundreds of hours figuring out how to build a believable cup of coffee for VR. Layer by layer he discovered aspects of reality he needed to recreate so that when people interacted with coffee in his world, it behaved as expected. Put it in the refrigerator? Steam should dissipate faster. Mix it with milk? The coffee’s color should lighten.

Owlchemy is a leader in creating highly engaging VR software that can also resize itself depending on the available area, or the height of the player. Last year, Google acquired Owlchemy Labs. Job Simulator’s sequel, which we tried recently, appears to be larger in scope with more “zones” for players to visit and play around inside.

It seems like a question of when, not if, Google will combine its visual processing technologies with the kind of work being done at Owlchemy Labs to essentially overlay a fun digital world on top of the real one in just the right way.

Daydream’s Promise

While Google’s Cardboard project helped turn millions of phones into relatively low quality VR headsets, Daydream is a more polished and higher quality platform featuring interesting pieces competitors are unable or unwilling to ship just yet.

Daydream’s Chromecast integration is a joy to use as part of a demo party. Wirelessly streaming what the spectator in VR sees to a nearby screen helps keep everyone spending time together. Google’s WorldSense inside-out tracking technology does a pretty good job of understanding your head movement most of the time. But there is nothing in the way of collision detection and object avoidance, and the system is in no way ready to understand situations like going outside and looking up at a clear blue sky.

So the first Daydream standalone — the Lenovo Mirage Solo — restricts movement to less than a meter. That restriction appears to be set in place because Google’s leaders lack confidence in their technology’s ability to keep people safe if used beyond that small space. Nonetheless, Google is pursuing a suite of technologies that could actually solve these incredibly complex problems. Others are working on this too, of course, but Google’s focus on enabling technologies that make information “universally accessible and useful” lets us see the steady improvement and roll-out of these fundamental features across different services.

Cloud Anchors For Multiplayer

There is a reason indie VR app Beat Saber was able to sell 100,000 copies in a month at $20 each while being available only on Rift, Vive and Windows-based PC headsets. It’s an extraordinarily fun game enabled by the best tracking technology available in 2018. These headsets give people the greatest comfort and freedom to move around and interact. If you lean around in Gear VR things don’t feel right, and if you turn around is PSVR your hands might disappear. So despite PSVR and Gear VR selling millions of units, it seems that when people have more freedom to move around naturally they are more willing to pay more money to experience a virtual world.

Consider too that Rift and Vive are a pain to set up and even Windows-based VR headsets have a giant cord tethering them to a computer. This limits those headsets in terms of convenience, causing friction and keeping people from putting on their headset. It is all the more reason to be impressed with some developers finding decent sales and enthusiastic player communities despite those drawbacks.

The first self-contained standalone headsets like Oculus Go, Vive Focus and Mirage Solo certainly cut the cord and drop the all-in price to as low as $200. That’s a big deal, but these headsets still lack the appeal drawing people to experiences like Beat Saber, Lone Echo or Tilt Brush.

Tracking is the key here. Lots of developers and close watchers of this industry believe mainstream acceptance of this technology is as simple as delivering a standalone headset with inside-out tracking capable of playing a game like Beat Saber. It will certainly help but, having felt the wide gaps in the user experience between standalone headsets with varying degrees of movement freedom and interactivity, standalone VR will still be missing a critical piece of what’s needed to make the technology mainstream.

You still need to bring your family and friends into the virtual world with you. Google recently started rolling out a feature it calls “Cloud Anchors” which could one day make it easier for your friends and family to see into your virtual world and play a game with you.

Powered By Google

Google’s multi-year efforts in computer vision and its global scale in making the world’s information useful could be the edge the company needs to supply cornerstone services for immersive computing. Google’s Blocks app and Poly service — which is initially focused on free content — is one early example of this strategy. “Cloud Anchors” could be another.

Right now, you could download the Just A Line experimental app on Google Play or Apple’s App Store and create an AR drawing with someone else. The app works by accessing “nearby” information and then pairing up with another device. The idea is that both phones point at the same object and, after a short wait, the two devices should sync up their understanding of the world around them. This lets two people create a shared 3D drawing together.

The feature from Google is shipping as an experiment, but with improvement it could be the key to unlocking incredibly important features for AR and VR. Imagine a VR headset that could share its location (and world) to nearby phones? Your friends and family could use their phones as mixed reality cameras to see your virtual world. While player 1 uses a VR headset, players 2, 3 and 4 could join in via AR.

“We think VR and AR are going to be a big space,” Google’s head of VR and AR, Clay Bavor, recently told me. “There’s room and there are roles for both Google devices and also for working with partners.”

That’s a predictable answer to the question I asked Bavor, given Google’s history building both its own devices and partnering with others. However, another comment he made is more enlightening about the company’s plans: