I thought that it must be nicer to spend 30 minutes in Rome than in the NY subway car. There are about 7M units of GearVR sold globally, over 250K Google Daydream headsets, and over 10M Cardboards, so I expected to see at least one appearance in a NYC subway car, as I spend there over an hour each day. But, apart from one scene in Mr Robot, I have not seen a single person using any of these.

So, I borrowed an HP Mixed Reality headset from work, and I wore it during my commute home and back the next morning to see how convenient it is, and if it even works in this setting because of the inside-out tracking.

This headset runs on Windows Mixed Reality platform that is part of the Fall Creators Update released yesterday — 17.10.2017. It runs UWP Store Apps and games, and lets you position them in a 3D virtual environment. Mixed Reality Portal is just a Windows shell in 3D, and it is convenient that once you shut it down, the apps stay in place, frozen in the last state, so you can find them the next time you open the portal in the state you left them. Since it is spatial, you always know where something is; it is something between reality and a Memory Palace.

The headset has a single cable that is split into HDMI and USB port. It is rigid and long, so that part of the setup was a bit more complicated than I expected, as I had to roll the middle section and put it in my backpack between my knees, so nobody trips and breaks my neck. Then, I had to zip the bag as at that moment I have realized that a subway full of immersed people would be a paradise for thieves.

The laptop I have is an older HP Spectre x360, and even though is it not officially supported for MR, it works after a simple registry tweak. With a default Windows 10 driver I only had about 1 fps, until I installed a newer HPs graphic driver which gave me nice 60 fps (did not check, but it was extremely smooth). Mixed Reality requires DX12 GPU with WDDM 2.2, but the Intel HD Graphics 5500 in that model of the Spectre is WDDM 2.0. The HDMI on laptop is also only 1.4a so it only supports 60Hz output (HDMI 2.0 supports 90Hz at this resolution). I can see the screen “pumping” at 60Hz on this headset (and on GearVR and other that run at 60Hz) so I wouldn’t use it long, but it is good that once I get home, I can connect the same headset to the desktop PC and get 90Hz.

The thing I was mostly concerned about was how the inside-out tracking would behave in the moving subway car with the bumpy MTA ride, people moving in front of the camera, and reflective surfaces. I thought that MR headsets used ToF cameras like Microsoft HoloLens, and that it would behave the same way with glass, reflective metal surfaces, and moving objects (HoloLens gets confused), but it seems to me that is uses accelerometers in this case. The tracking was very responsive and steady; only a few times when the car was making sudden breaks I would move in VR laterally with the train, while the VR world would darken and stay affixed in world coordinates, and show it like it was a different dimension. It was a nice effect and since it was spatially aligned with reality, it wasn’t nauseating. Since some part of the track is bad, the bumps made the headset jump on my head, as this headset does not fit well on my oval head, and basically just stands on the 3 cm of my forehead, and 3 cm on the back, and is able to rotate left and right. If you have wider head, it should fit better. I previously made a video about that hinge and the overall ergonomics of the headset (I was not pleased), but once I replaced the original 1 cm foam with the HTC Vive’s face cushion, it was much better (both are velcro):

I had issue with MR portal that whenever I started it, it would ask me to look left, right, and to the floor, but no matter how long I did that, even while train was on the station, nothing would happen. I am guessing it was looking for the calibrated and saved space. The fix was to run the setup every time, and just select seated experience. Since the only calibration there is to look straight for a second, it is not too inconvenient, but it could be simplified (and the Setup button is actually a Hyperlink so it is hard to spot).

I did not want to setup a wired mouse then and there, so I used a touchpad, but it is easy to lose the cursor that way. At one point I was swiping in all directions for at least 30 seconds until I spotted the cursor (maybe after timeout of using a mouse/touchpad, the cursor should default to projected center of the screen). The navigation was a bit hard as I could not physically turn around, so I just ended up at the cliff. That was fine as I did not have internet connection, so my “social media room” would not be much fun anyway. I wanted to see if I could watch a TV show, but my Movies app did not have sound. Viewing video in 360 degrees is a great experience (especially the transition to 360), but the mouse cursor in that mode is gone I have to use the “neckterface” to select buttons.

I tried using the Desktop app, that shows desktop, but it was not smooth enough that I could use it. Remember that this is a low-end laptop, so performance is probably better on laptops with dedicated GPU. It is still good that I can use non-UWP programs in there. I wish that it would stream individual windows as the Envelop VR (first part of my video), so that we could make this shell as useful as the desktop one. That would make this 3D shell truly useful, and have it replace multi-monitor setup sometime in the future. All of those issues are probably getting fixed as you read this.

The looping bird sound was unpleasant, and there is no way to turn it off. You can hear on all VR conferences that the sound is a vital part of the VR experiences (mostly advertised by the sound guys, so…), but here it is completely unnecessary. I have misophonia, which is condition where certain sounds trigger my amygdala resulting in painful responses, and some of the Portal sounds are light triggers. This condition affects 0.1–5% people, depending on who makes stats, so I hope I will be able to turn all sounds off, including UI sounds in the next version, as turning users away at this stage is not good. Maybe MS will hire someone with misophonia and fix that.

I was ready to get away from my cliff house and to check HoloTour App in this commuting settings. It IS much nicer to stand again in the Piazza Navona rather than in the subway, learn something, just watch people, or stand in front of Pantheon watching a dog disappearing in the seams of 360 video stitch :) . Balloon ride scene with the train de-/acceleration was especially different in this setting.

As the train approached my station and I removed the headset, I have noticed that most people were in the other half of the car, which reminded me of a scene from Seinfeld with a naked dude.

It was unexpected as I tried not to move at all, and was only turning my head +-30 degrees, but headset has cameras, so maybe people are just camera shy. At least nobody could sneak upon and take my phone with so much empty space around me (The photo above was from the ride home when I was in the corner, but in the morning commute I had to sit on the 6-seater). I jammed everything in the backpack and left with a headset imprint on my face.

There is a lot to be said on MR portal but I just want to confirm that it works even in the New York subway. If you are in any other place with a subway from this century it would be even better experience, especially on the modern European railroads. Not having a laptop in my lap would simplify things. I think a phone-like device would be perfect here, and you can read an older post on my vision of Surface Phone with Mixed Reality. I will not be using this headset in the subway again, since it is not mine, setup is not the most convenient, and I use this time to catch up with my software development projects during these 30 minutes, but I am glad that it is an option. Just being somewhere nicer, watching TV series in my virtual theater, enjoying meditative scenery, or other passive experiences would make this commuting time more pleasant… until a time when we all are working remotely in a VR office from home, and don’t even need to commute.