HAptic DEsign iNterface for Expressive Spatial Design in Virtual Reality

Thesis Advisors: Daniel Cardoso Llach, Eddy Man Kim

Background

Recent developments in Virtual Reality have shown a lot of promise in using VR as a design medium. Notable examples include Google Tilt Brush and Oculus Medium. For designers, traditional CAD software tend to be cumbersome and causing high cognition load. Whereas, this new form of design medium can provide much greater intuition and immersion for designers, unleashing their creativity and imagination.

This project aims to explore the use of Virtual Reality as an intuitive and expressive spatial design medium through the physical embodiment of design interfaces.

Design Challenges

One of the biggest design challenges of building VR experiences is how to establish meaningful spatial interactions. While the design language for VR is still in its infancy, several promising design methods have emerged.

Menu system in Tilt Brush (Image Source)

One method is employing a "spatial" interface system. Layers of interface and menu are placed in a three-dimensional context. Although the interactions between the user and the interface are still inherently two dimensional. Users are often required to use a laser pointer to select items, which proves to be tedious and inaccurate. This method also limits the scalability of the interface, making it difficult to accommodate the large amount of functionalities a typical professional software requires.

Game interface of Job Simulator (Image Source)

Another method is applying real world metaphors and physics so that the user would interact with the virtual world in the same way as in the real world. Because of the affordances imbedded in everyday objects, users can quickly get used to the environment and interactions. However, the experience is especially lacking in terms of haptic feedback. While in real world, people rely heavily on haptic feedback to perform precise actions. Even though certain haptic responses can be simulated using different vibration patterns on the controller, it cannot match the fidelity and richness of the real-world haptic feedback.

Design Inspirations

Moving or rotating the “brick” to manipulate the virtual shape

In 1995, George Fitzmaurice, Hiroshi Ishii and William Buxton explored the concept of “Graspable User Interface” in their “Bricks” project. Users can use a physical object called “brick” to manipulate a virtual 2-D object on the surface screen (x, y and yaw). Multiple “bricks” can be used together to accomplish more complex interactions such as generating a spline curve. With the current VR technology, it would be worth exploring on how to extend these interactions into 3-D space. Will 6-DOF tracking of physical objects enable new forms of interactions?

Prototype Design

With the intention of reducing the cognitive load for users, the system spatializes the controls into two cubes and one button, adopting the metaphor of a real-world workspace with interfaces of high affordance, which facilitates the use of users’ spatial memory.

Project Setup

Passive haptic feedback is provided by overlapping real and virtual objects using OptiTrack tracking system. Retroreflective markers are attached to real-world objects so that the tracking system can calculate their positions and orientations. Then the virtual objects in VR are mapped to corresponding real-world objects with a 1:1 scale.

Different iterations of the controller

Pressing the button on the controller

The interactions are implemented in Unity using VRUX framework. Users can control the position, scale and rotation of the generated cube at the same time. This design resembles the pan, pinch, zoom and rotate gestures on the touch screen. In here, these gestures were generalized and extended into 3-D space. This coupled interaction makes it intuitive and efficient for users to generate designs.

Creating and erasing a cube by clicking the button

Changing position and scale at the same time

Changing position and rotation at the same time

Spatial haptic user interface

Discussion

The system was installed in a gallery and demonstrated to the public. Most users quickly learned the controls and could freely generate cubes with different scales and rotations even though there is no explicit instruction telling them how to do so. This proves the interaction design is intuitive and effective.

Live demonstration at the gallery

Some users could also come up with unique designs such as a character or a temple. This shows the system is expressive enough for designers to be creative and imaginative.

Complex and creative designs from simple interactions

Next Steps

A more formal and rigorous user study is needed to test and evaluate the effectiveness of the design decisions. Tests should be carried out on both normal VR system without haptic feedback and system that integrates the tangible user interface in order to gain further insights on the implications of using this system and the effects it has on designers’ creativity and design thinking.