Google’s big jump into virtual reality

There's a lot riding on this piece of cardboard

Clay Bavor has a glimmer in his eye and a knowing half-smile. If he has the look of a man with a secret, that’s because he does.

Bavor is Google’s VP of product management for "Cardboard and our efforts in VR," he says, as we sit down in Google’s offices to discuss just how ambitious those efforts have become. Until a couple weeks ago, Cardboard was regarded as an experiment in virtual reality for Android, a 20-percent time project crafted by a small group of engineers because it was, for lack of a better word, neat.

When it was introduced at last year’s I/O conference, Cardboard felt like an afterthought When it was introduced at last year’s I/O conference, Cardboard felt like an afterthought. At the very end of his keynote, Sundar Pichai gave the project an unceremonious introduction: "A set of engineers in their 20-percent time surprised all of us with what you can do just with an off-the-shelf cardboard and your smartphone," he said. "And the combination takes you into a very, very immersive experience." Then he got to the "real" gadget giveaway: a couple of Android Wear smartwatches. "Certainly inside Google, [Cardboard] is not viewed as an experiment," Bavor tells me. He hints that Google has a much larger team than I thought working on VR, and he recounts Cardboard’s surprising and under-reported successes: shipping a million units without a huge marketing budget, while using exclusively third-party manufacturers. As a result, Bavor says, Cardboard has exposed the public to VR experiences at a scale far grander than Oculus, Sony, HTC, or Valve. That kind of reach matters to Sundar Pichai, too: "at our core we want to build products for everyone." Cardboard may seem like the most basic of all the competing VR platforms, but it's also the most widespread. "Cardboard is just the tip of the iceberg of our VR investments," he says, again with that glimmer in his eye. Cardboard, Bavor repeats, is "well beyond an experiment — and [has] many things going on behind the curtains."

In May 2012, The Verge got its first taste of VR when we tried out a prototype Oculus Rift — a jerry-rigged contraption held together by duct tape. Since then, we, along with the rest of the tech industry, have been tracking the resurgence of VR. Almost exactly three years to the day since our Oculus test run, the VR landscape has been transformed: Facebook has purchased Oculus for $2 billion, Sony has jumped into the game with Project Morpheus, and just this year HTC and Valve unveiled their solutions for VR gaming. But the promise of consumer-oriented VR has consistently remained just beyond the horizon. Even now, we are still waiting for each of these companies to put their products on store shelves. The only company to do so is Samsung with its Gear VR, but it still tags it as an "innovator edition," a signal that it’s not the final form of what Samsung intends to offer. Oculus promises to hit shelves early next year. The promise of consumer-oriented VR has consistently remained just beyond the horizon In short, virtual reality is perpetually virtually here. Up to now, Google has been one of many offering something that felt more like a tech demo than a real thing. But one year after its introduction, Bavor insists that Cardboard is finally here — that it’s a viable, stand-alone product. The app that powers Cardboard on Android has surpassed a million downloads. Right now, you can use it to roam Google Earth, take a virtual tour of Versailles, watch YouTube videos, and experience ATAP’s "Windy Day" VR movie. The Google Play store has more Cardboard-compatible apps like a Paul McCartney concert and Seene, which lets you browse through 3D photos. This year, Google is introducing three updates to its Cardboard program. The first is a new design of the headset itself. The updated version works with larger phones, and instead of the magnet switch of the old version, it uses a funny little lever with conductive foam that lets you tap on the screen. Compared to other VR headsets, it still offers a very basic level of interactivity: you look around, you occasionally hit a button.

Those limitations are by design, in some ways. Android isn’t nearly as powerful as the proper computers that drive the Oculus Rift. The result is a pared-down experience, calculated to meet, but not exceed, a phone’s capabilities. "Cardboard doesn’t have a head strap. That was a deliberate decision," Bavor notes, "because you can rotate your torso about one-fifth as fast as you can [rotate] your head." Since you can’t whip your head around, you’re less likely to experience nausea-inducing lag. Cardboard is designed to meet, but not exceed, a phone's capabilities Alongside the new headset spec, Google is unveiling an education-focused package called "Expeditions." It’s a giant cardboard (naturally) box filled with Cardboard visors, phones, a router, a speaker, and an Android tablet. The set-up allows teachers to lead a classroom of up to 30 students on a virtual field trip. The instructor can send a VR scene to an entire classroom, and uses the tablet to direct their attention. I tried it out, and it worked fine — though I imagine the scene of 30 kids all staring into VR headsets would have been more impressive (and hilarious). Both of these new efforts are the sort of thing I expected from Google’s Cardboard: smallish experiences, executed competently. The third project is in the same vein, but it’s actually more significant than it seems. It’s the clearest sign yet that Google is planning something bigger. Most VR experiences today are created using multi-camera rigs — platforms with cameras fanning out from a central point to capture 360 degrees of footage. Shooting that video is as easy as hitting record, but the real challenge is in stitching the footage together into a seamless environment that allows you to look around freely. This year, Google is announcing the launch of Jump, a camera rig that, like Cardboard, is offered as an open-sourced specification. You download the design from Google and 3D print up a carousel that fits 16 GoPro cameras. The set-up isn’t actually all that unique, and though GoPro Hero 4’s can shoot at a wide angle, there will still be gaps at the top and bottom. Jump creates an experience that isn’t just seamless, but fully stereoscopic The magic here is in the stitching process. Bavor says that Google uses a huge array of servers to combine those 16 videos into an experience that isn’t just seamless, but fully stereoscopic, which requires a slightly different camera angle for each eye. "In essence," Bavor says, Google’s algorithms "take the 16 viewpoints from the cameras and turn those into thousands of viewpoints, as if you had taken a video from every place, every point around the circumference of the circle." I tried out a shot made with the Jump, of penguins swimming around at a zoo. I was looking at it through a Nexus 5, so I experienced the "screen door" effect of seeing the grid of pixels. Despite the jarring black lines, what I saw was incredible: no matter how quickly I turned my head, the water waved without glitches, bubbles wafted off the birds, and I never once saw the weird stretching you often get with badly stitched panoramic shots. It was stunning to realize that everything I saw was actually created by software. I couldn’t say which angle was "real" and which one was computationally rendered by Google’s servers. It was just VR.

For now, Jump is still in development — the processing power necessary to deliver its seamless effect for even one video in a reasonable amount of time requires massive computing power, Bavor says. But with enough power, he insists, it could be done in real time. "To get that kind of performance, you’re talking thousands or tens of thousands of computers," he says. "We’re working on optimizing it, but it’s pretty heavy-duty stuff." Over-promising and under-delivering was a big reason that the first VR wave of the ‘80s and ‘90s crashed Right now, there’s a huge backlog of Jump videos. So, although you will be able to download the spec for the Jump and build your own rig, you’ll have to wait for server time to process your video. "We’ll be working really on an invitation basis with real enthusiasts and creators who have something special they want to capture," Bavor says. And as their computing power increases, they’ll open that invitation to more and more users. Over-promising and under-delivering was a big reason that the first VR wave of the '80s and '90s crashed. It’s clear that the new generation has learned from their mistakes. "One of the things I’ve really appreciated about others in this space — Oculus, Valve, HTC, Sony — is they are waiting to get it right," Bavor says. "I think it’s really important that the VR industry wait to get it right." And if you listen closely, you can hear Bavor temper public expectations. He calls Cardboard "a taste of VR"; a "bite-sized VR experience." But the company’s recent investments signal that Google is taking VR very seriously — that Cardboard has graduated from a hobby project to a legitimate initiative. In other words, it’s increasingly obvious that Google isn’t leaving VR to Facebook. Toward the end of our meeting, I wondered whether or not Google was working on something more substantial than just 16 GoPros mounted to a 3D-printed carousel. "Why not create a new kind of super camera that could just do it without having to do all this insane stitching power on the back end?" I asked him. "That’s a great idea," he smiled staring right at me. He said no more, but there was that glimmer in his eye.