Finding a Better Solution to Scale

It was clear that using one button to iterate blindly through an array of objects would not scale well. Mike’s collection has over 400 models and I’d love to eventually allow people to upload their own voxel creations. So, I began brainstorming a more effective interface starting with some my own expectations:

As a kid building a VR city with voxel objects…

I want to focus on my primary object that I’m placing. I want that front and center.

I want to always be able to switch objects easily, but that’s not as important as placing the currently selected object so don’t get in my way!

I want to be able to switch objects very quickly without having to move both hands. I want an interface that can use one-hand movement and not require a 2-hand gesture.

I want to see many objects quickly. I want an interface that shows multiple objects at a time.

I don’t want to “get stuck” at the end of a list. I want an interface that loops at the end of a list.

I want to know something about each object, at least a name.

I’d like to be able to figure out what button maps to the UI without having to read instructions.

I started sketching out some ideas on paper:

Ugly sketches but they start the conversation.

I tried using Tilt Brush and found it helpful for experimenting with 3D placement and sizing, but only after having started a creative direction. My initial ideas were most easily brainstormed and iterated with paper and pencil first.

However, after having a basic ideas on paper I found the A-Frame inspector was an excellent tool for rapid prototyping and iteration. I would very often tab back and forth between my prototype project, A-Frame inspector mode, and my codebase. A-Frame inspector was a tremendous time saver to make quick tweaks to positioning and rotation without needing additional round-trips of code changes and browser reloading.

My fine voxel art skills at work.

I next tried building out a metaphor of a “watch” which is out of the way but still accessible with a twist of the hand — even going so far as to build a quick watch voxel model. (I had an idealistic image of a futuristic sci-fi hand like a portal gun mixed with Buzz Lightyear’s arm controls.)

The watch concept didn’t work very well. I found myself wanting to be able to see both the currently selected object and the menu at the same time. Moving my head back and forth to see the watch and the object was not a great experience.

So then I went simpler and literally placed object preview images directly adjacent to the controller and the currently selected object:

The first cut of what became the final horizontal “select bar” design.

Once I threw out the need to match a real life metaphor, like a watch, it became much easier to directly address the design challenges.

Adding borders and “selected” state.

I started refining the design to add some borders to the preview images and added styling to clearly indicate a “selected” state. The controls were really coming together!

One of my user goals was to be able to quickly and easily be able to see upcoming objects, and if I was really in a hurry to make a cool city scene it’d be nice to see ahead.

Adding a total of five menu options and a semi-transparent background frame.

So I added 2 next and 2 previous object previews sandwiched around the currently selected object.

A 50% opaque background frame provided nice separation from other elements in my field of view, and at certain angles it has a nice specular sheen.

If I wanted to return to a skeuomorphic design concept I suppose at this point I could have experimented with designs that resembled a tablet (old style or the iPad variety), but I didn’t see any value in trying to cram that in.

In order to support hundreds and eventually thousands of objects, I wanted to introduce the concept of grouping objects together. You could browse objects of the same group by going forward and back within a group, or switch groups by going up and down.

I added some arrows to experiment with this concept:

I really liked this — seeing the arrows gave me some additional inspiration, wouldn’t it be great to be able to see where your trackpad is hovering before selecting an option? It turns out that the Gamepad API provides the raw axis position which can be interpreted into hover position, allowing me to put that together into a nice hover state:

Another fun thing about this design is that it can lend itself to at least 2 different methods of use to accomplish the same thing — one-handed quick via touch pad, or a “laser pointer” style like that used in Tilt Brush and other palette UI concepts.

I continued to add polish including quick animation when switching objects within the same group. I added labels for each object and the group name using the A-Frame bmfont text component (now part of A-Frame core).

Technical Challenges of VR User Interfaces

I have kept this article focused on the design decisions and steered clear of the technical side of implementation, but I must point out that by this point the vast majority of the development complexity of A-Frame City Builder was in fact building the user interface, not building of the logic of the game itself. An entirely separate blog post could be dedicated to the process of separating this select menu UI out of the base game code and into a standalone A-Frame component.

Another fun topic worthy of a separate thread was the process for making this component compatible with both the HTC Vive and the Oculus Touch controllers. Componentization was a key catalyst for being able to support both control schemes as it forced separation between gamepad events interpretation, user interface state management, and core game logic.

Once I had finished the separation of components, I was stunned to see the size of the select menu UI component— 40kb vs 11kb for the City Builder game logic!

As a result it’s absolutely clear that in order to enable developers to create the content we imagine necessary for VR to be successful, we need many more of these basic UI components available out-of-the-box.

The Finished Product

I have shared a repo for the select-bar component including a bit of documentation and an example project. The API is inspired by the HTML5 select element: https://github.com/kfarr/aframe-select-bar-component

Here is the finished product in shiny YouTube glory:

You can take a look at the current A-Frame City Builder repo, including a link to a live online demo if you have a WebVR compatible browser with the HTC Vive or Oculus Rift: https://github.com/kfarr/aframe-city-builder