Immersive Design: The Next 10 Years of Interfaces

A look into what happens when we design beyond a screen

Like many designers, I started my career as a Graphic Designer. I dealt in picas, carried Pantone books, and swore to measure twice and cut once. Then the web came along and with it came Web Designers. We had to become acquainted with HTML, CSS, Javascript and we’re still trying to keep up with the right way to build for it.

These websites quickly demanded more interaction from us when Flash entered the scene and conquered our hearts. We turned our attention to animation to convey expressive user flows through interaction design. Then, the iPhone showed up and forced us to think smaller. We got excited about skeuomorphism, learned about pixel density, and made a vow to design mobile first.

After a while, we tried to combine all of the above into a holistic practice that would buy us “a seat at the table,” where we could think not just about aesthetics, interactions, and user needs, but also business needs. And so, the modern Product Designer was born.

I’m willing to bet that, like many designers before it, the Product Designer is approaching extinction, and setting the stage for the Immersive Designer.

Virtual Reality (still) matters

Over the last decade, we’ve seen content move from newsstands, to desks, to our laps, and then into our hands. It seems clear that the next step is to remove the device altogether and place the content in the world itself, eliminating the abstraction between the content and its audience. We call the process of designing for this Immersive Design, which includes VR/AR/MR/XR — basically all the Rs.

We are seeing this realized today in phones through Augmented Reality. Tech giants like Apple, Google, and Samsung are rushing to conquer the AR space like a modern Christopher Columbus in search of spices. We’re seeing identity transfer setting a trend in animojis and virtual characters walk around our videos like a Roger Rabbit fever dream. However, designing for mobile Augmented Reality today feels like developing for the Commodore 64 in 1982; investing in a platform that’s novel but filled with practices that will be rendered obsolete before they’re relevant. I’ve found that Augmented Reality in 2018 has two major limitations when it comes to Immersive Design: field of view and input.

Field of view

So far, Augmented Reality is still restricted to a rectangle in your hands. Content can only aspire to be a window into another world; it hasn’t quite inhabited our own yet. Users feel trapped outside in the mundane world while all the fun is happening inside the phone in their hands as if A-ha’s Take on Me never made it to the first chorus.

Input

In 2018, we’ve developed a language for using facial gestures like opening our mouth or raising our eyebrows to control Augmented Reality masks, other experiences rely on the now primitive touch interaction, while the ambitious ones rely on voice commands to interact with the world inside the screen. The input available on the market limits these interactions. However, once we inevitably obviate the phone and achieve immersion through AR glasses, we’ll have to go back to the drawing board and try to answer the billion-dollar question: how do you interact with content in space?

This is where Virtual Reality comes in.

The jury is still out on whether Virtual Reality belongs in your living room or as a museum-like destination that you plan for. We’re still experimenting with the medium to find the adequate cadence for virtual experiences and navigating worlds in the much-adored metaverse. If you think about it, it took film quite a bit of time to arrive at the standard 90-minute duration, which is about as long as it takes your bladder to digest a liter of Coke at the cinema.

Today, Virtual Reality has found a fit as the best way to explore Immersive Design problems present in the Augmented Reality future we crave by taking advantage of a more immersive field of view and using natural gestural interactions. Challenges like thinking in 3D and using volumetric UI that reacts to the environment and the people in it.

Not only can Virtual Reality improve our quality of life by providing great escapism, but it can also open up a bunch of questions around how people could interact with technology if it were all around us. Among other things, I’ve found that Immersive Design invites designers to question the line between content and UI and rethink the process for creating digital products.

Content is changing

As designers we’re often told to get out of the way. To be “content first” and make room for the reason people are using your product in the first place. However, Immersive Design poses an interesting question: where does the line between content and UI start and end?

Game designers have been asking themselves this question for decades. As they envision a world to be inhabited by players, the interface to navigate it can often be abstracted into menus that live outside the world’s logic. For example, the interface to start a game often lives in this weird in-between software that acknowledges the existence of the world inside the game by using the game’s characters and aesthetics, but operates based on the rules of the player’s world.

And so, video game companies draw a line between UI and Game Designers. There’s logic to this decision: Game Designers are often proficient in 3D tools while UI designers generally work in 2D. This decision can sometimes lead to immersion-breaking solutions that require players to suspend their disbelief when the game reminds them they’re in a video game with video game systems.

The explicit line between UI and content is tolerable in a video game, but as we step into the world of Immersive Design, we won’t necessarily have the luxury of flat menu trees that exist outside our reality. We are tasked with finding solutions for UI that follows the rules of our augmented world; where do the menus come from and how do we interact with them?

As they’ve matured, video games have given us examples of how design can be woven into the environment and blur the line between content and interface.