Last month, Facebook-owned Oculus released the Oculus Quest, their first all-in-one virtual reality headset that doesn't need external sensors or a PC. When VR developer Mohen Leo brought home an Oculus Quest, he took "virtual reality" in a bit of a more literal sense: he recreated his own apartment, then took a walk around it.

Quick experiment on the Oculus Quest:

Lined up a VR environment with my real flat. On the left that's me, fully in VR (can't see the real world) walking around and sitting on furniture. Right side is simultaneous video from my cell phone held under my nose. #vr #gamedev #ue4 pic.twitter.com/NE3lrxFZxp — mohen (@mohenleo) June 3, 2019

If, over the past few years, you've been able to play around with any room-scale VR headsets like the Oculus Rift or HTC Vive, you already know how impressive Leo's creation is. Before you use a conventional wired VR headset, you've got to set up sensors and clear a sufficiently large space to play in. For many folks with small or otherwise densely arranged living spaces, the need for a large empty area in the home has been more of a barrier to using VR than the high price of headsets and high-end gaming PCs. Thankfully, VR tech is advancing away from this emptiness-or-bust scenario. Leo's apartment experiment shows that location-based experiences in VR are now a lot closer to being widely available thanks to free-roaming tracking tech like what's in the Oculus Quest.

"The first steps date back two years, when I moved into an apartment in San Francisco," Leo explained over email. "My rooms were completely empty, so I thought, 'What if I used VR to visualize what different furniture options would look like before I actually buy the pieces?'" Leo previously worked in visual effects for films and started combining that craft with VR before moving over to ILMxLAB, where he most recently worked on "Vader Immortal," a "Star Wars" launch title for the Oculus Quest — not a bad background to have for recreating a real space in VR. But the process Leo used to render his apartment should be accessible even to curious VR hobbyists.

First, Leo started with a photogrammetry scan of the apartment, which he noted is possible with any digital camera but yields better results if you use better equipment. "You basically just shoot a bunch of photos of a space from different angles and then feed them into photogrammetry software that builds a 3D model from the photos," Leo explained. In his case, he used a Canon Rebel SL1 and Agisoft Photoscan (now Metascape, standard edition $179 USD), but you could also use a free alternative like AliceVision Meshroom. Next, Leo took the "noisy" output model from the photogrammetry scan and built a cleaned-up version in the 3D modeling software of his choice, reprojected the photos onto the model as textures (you could use Blender for free to do both steps) and scaled the entire model to match the size of his real apartment by matching up a few digital lengths/widths to the real-world measurements. For his furniture visualization a couple years ago, he then made some basic 3D models based off of pieces from the IKEA catalog and looked around at the different apartment layouts with his tethered Oculus Rift headset. "I ended up buying the furniture, putting it where I had planned, and pretty much forgot about it."

Quick step by step in images:

– did an Agisoft photogrammetry scan a couple years back

– mapped photos on clean geo

– "previs'ed" furniture dimensions before I bought it back then

– yesterday, matched up a low-poly asset kit I bought to the flat layout in Unreal pic.twitter.com/13kvG4dRrC — mohen (@mohenleo) June 3, 2019

When Leo received his Oculus Quest the other week, he remembered he already had a detailed model of his apartment that he'd now be able to walk freely in. The rest of the furniture and object models he added for the Oculus Quest version of his apartment demo came from an Assetsville pack he bought on the Unreal Marketplace. The apartment simulation itself is running on Unreal Engine 4 (UE4), which anyone can start building with for free. For the purposes of this experimental demo, Leo didn't get too fancy with how he lined up the VR and real spaces: using a UE4 Blueprint setup he can stand in a door frame and fine-tune the position of the VR space around him until it matches.

There's a lot more you can do in VR if you have an accurate model of the space you're in and a headset that can go anywhere than just sitting on a virtual recreation of your couch. Armed with those apartment dimensions, the next demo Leo whipped up is more in-line with what's mainly been the domain of expensive location-based experiences (LBEs). A simple stroll from the edge of his bed to the couch is easily transformed into a vertigo-inducing walk on a narrow platform:

Follow-up to yesterday's Oculus Quest test:

This technique is often used in "location-based VR". The objects you are supposed to touch (in this case the seats) match the real world, while a gap/drop keeps you away from things that don't match (e.g. door frame). #vr #gamedev #ue4 pic.twitter.com/OqcdiYYMek — mohen (@mohenleo) June 4, 2019

While dedicated VR experience companies like The Void have been using mobile headsets and experiences mapped to specially-designed spaces for a while now, as Leo put it "there's no excuse for VR LBEs to just be an empty room anymore" now that there are consumer-grade headsets that don't need cables and external sensors. "I think one of the really exciting opportunities with devices like the Quest is doing 'DYI' VR LBE," said Leo. "All the software you need to create a basic experience is free or open source. You can turn your couch into a captain's chair on a spaceship bridge or stack some coolers in the backyard to make a VR cover shooter." These same principles could be extended to multiplayer VR experiences. You wouldn't even need to be in the same place as the other players so long as every individual's play area was laid out the same way.

As it stands right now, the move to headsets without cables and external sensors might only seem like a "nice to have" improvement since so many commercial VR experiences are designed around being relatively stationary in an empty space, but the limits on VR possibilities have been blown wide-open, both in terms of experience design and immersion. "Your brain buys into the reality of VR so much more when you can walk around and touch, or sit, or lean on matching physical objects," said Leo. That's an incredibly exciting prospect — even when it's just a faithful replica of two rooms in an apartment.​