Microsoft HoloLens Hands-On: Augmented Reality Done Right

Last week, we were given our first personal glimpse of Microsoft’s augmented-reality device for Xbox One; the HoloLens. This wireless transparent visor creates the illusion of tangible objects in the real world by adding graphical elements to the player’s surroundings — imagine being inside Who Framed Roger Rabbit? and you’ve got a pretty good idea of how it all works. Over the course of an hour, we built Minecraft worlds on a table, shot at rampaging robots crawling through the walls and even stepped inside the Halo universe. So is this stuff truly game-changing? It just might be.

Our hands-on HoloLens session was conducted behind closed doors at Microsoft’s E3 Xbox booth. It was all super-confidential with no photography permitted and only a handful of journalists allowed to enter. (There appeared to be a grand total of two Australians there, including me.)

The first demo we were shown was an interactive intro to Halo 5’s Warzone multiplayer mode. Attendees were each fitted with a HoloLens and ushered into a physically built set that resembled a Spartan stronghold complete with gunmetal corridors and a command deck. The environment already looked pretty impressive but the HoleLens turned it into something truly spectacular.

At first, the only addition to the real-world environment was an interactive HUD display on the visor — this directed the wearer around the set and let them know when and where to stop walking. It was a subtle addition, but it definitely helped to make me feel like I was inside a Halo game.

Things then kicked up a notch — after being prompted to stand next to a wall, a blast hatch opened up to reveal a huge war base teeming with UNSC soldiers and weaponry. In reality, we were still just looking at a wall with the HoloLens “painting in” the graphical elements. This was much more than a simple 2D projection: by leaning forward, it was possible to peer around the sides, just like a physical window. Impressive stuff.

We were next moved to the command deck which was dominated by a holographic display that showed nifty projections of our mission — this included battleground maps, enemy models and myriad spinning 3D effects. Again, the hologram’s base was the only physical element in the room with the HoloLens providing the rest. And with that, it was back to reality.

The Halo demo provided a great example of how this tech can be utilised in a commercial setting. Obviously most people aren’t going to build a Halo set in their homes — but inside an arcade-style environment it could be phenomenal. If nothing else, it should make the wait while queuing for theme park rides far more tolerable. Plus, it will make laser tag balls-out awesome.

The next HoloLens demonstration was for an FPS action game dubbed “Project X-Ray”. This started out life as a tech demo but is now being considered for official release; much like Wii Sports on the Nintendo Wii. It involves blasting various robotic beasties which burst through the walls of your house and using an X-ray power to find where they’re hiding. Your weapons are controlled with a standard Xbox controller with the visor taking care of movement.

The enemies are spatially aware which allows them to navigate the real-world environment as if they are actually in the room. This mainly involved them ducking in and out of virtual holes in the wall and flying towards the player’s face to launch attacks. We were particularly impressed by the way you could duck under enemy lasers — a true AR experience.

Project X-Ray showed off HoloLens’ intriguing potential as an in-house gaming peripheral. While the actual game was pretty one-note, it’s not hard to imagine deeper and more complex gameplay possibilities using this technology. Colour us excited.

The final demo happened to be the most impressive of the bunch: a dedicated version of Minecraft for HoloLens. By now, you’ve doubtlessly seen the onstage demonstration of this game from Microsoft’s E3 press conference. But you really need to experience it first-hand to properly appreciate how special it is.

The game uses a combination of finger gestures and voice commands to navigate the world. Players can throw their map onto a wall and choose between various screen sizes; a handy way to get your Minecrack fix when your roommate is hogging the TV. The HoloLens essentially tricks the wearer into seeing a screen on their wall. In fact, it’s more like a window — like the Halo demonstration, it was possible to peer around the sides to see more of the game world beyond. You can also leap inside the game world and control your character directly with an Xbox One controller.

Of course, this is just one way to play the game: the chief draw card is the ability to create fully interactive 3D maps on any flat surface. This was truly amazing. Pinching your finger allows the world to be dragged around in any direction while a downwards tapping motion selects menu items.

You can also use voice commands in conjunction with the HoloLens to trigger specific game actions. For example, focusing the cursor on a dynamite block and saying “lightning” achieves the desired effect. Similarly, saying “zoom in” brings you nice and close to the action.

I’m not sure what was powering the game’s voice-recognition but it was ridiculously impressive: even with my broad Aussie accent to contend with, the game didn’t mishear me once. That’s bloody first-grade alchemy that is.

There are a range of different game modes on offer. We were particularly enamored with the “follow character” option which automatically tracks the selected avatar as they potter around the game world — it’s like having a flying drone for eyes. Amazing stuff.

Once you get used to “pointing” the cursor with your eyes, it all works pretty seamlessly. There was certainly none of the extreme trial-and-error associated with most Kinect games (other than the setup, which we’ll get to in a moment). It’s become abundantly clear why Microsoft shelled out so much money for the Minecraft licence. This thing is going to make fans of the franchise lose their minds.

So was our HoloLens experience faultless? No. We encountered some significant issues during our hands-on session which we hope are ironed out before launch. First and foremost, the calibration process is currently a bit ridiculous: you need to have your eyes measured with a separate focusing gizmo. No, really.

Presumably the final HoloLens product won’t require additional calibration hardware to run, but we’re extremely leery about how Microsoft aims to solve this problem. Let’s not forget that the setup process for the Kinect was an absolute nightmare.

The visor’s functional area is also a little on the small side — your peripheral vision frequently disrupts the illusion with objects zipping in and out of existence. We also aren’t completely sold on the design of the current visor: it seemed alarmingly fragile for a consumer device (we were told not to touch the glass or sensors under any circumstances) and the overall fit was a bit off. While it’s possible to adjust the straps at the back, it’s extremely easy to end up with an uncomfortably pinched nose. Some more time is needed at the design stage, methinks.

All in all, HoloLens for Xbox One is a mighty intriguing proposition that is guaranteed to deliver unique game experiences. Like the Kinect, we imagine its potential outside of gaming is going to be huge. As with any video game peripheral, it now comes down to crazy and brilliant developers to unleash the platform’s full potential. We’re waiting with bated breath.

Kotaku attended E3 in LA as a guest of Microsoft.