Video: Rubbery objects can control a computer

Consumers are now familiar with prodding at hard glass or plastic screens to interact with computers, phones and other gadgets. But a new Japanese prototype has a touch interface with a squeezy, rubbery feel.

The system is based on a large LCD panel built into a tabletop. Users interact with it by moving objects sculpted in transparent rubber over its surface, or by poking or deforming them to produce particular effects.

The system was built by Hideki Koike and colleagues at the University of Electro-Communications in Tokyo, Japan, working with Kentaro Fukuchi at the Japan Science and Technology Agency.

The LCD panel emits light polarised in a particular way, but a camera positioned overhead can only see light polarised in the opposite direction and so usually sees nothing. Because the transparent rubber objects diffract light differently depending on how they are stretched or compressed, deforming them changes the light polarisation enough to let the camera track what is happening.


Rubber face

The team has demonstrated the technology through some simple applications. In one of these, a rubber face sits above an image of a face on the LCD; squeezing the rubber features alters the expression. A second application involves rubber shapes that act as virtual paint sponges; the user has to squeeze them to “drip” paint onto the LCD below (see video).

“You can easily imagine applying this to many fields,” says Koike. For instance, he suggests that a 3D model of the brain built from the material could give trainee surgeons feedback on their technique as they “operate” on it.

“One disadvantage of our system is the overhead camera,” he concedes. As well as making the system less-than-compact, the camera can sometimes fail to register a squeeze because the user’s hand gets in the way. “In future, it will be possible to embed the camera inside the LCD,” Koike says.

Tangible future

Fukuchi says that providing tangible 3D objects to interact with is a “major trend of display-based interaction”. He thinks they will become increasingly common, as touchscreens have in recent years.

Patrick Baudisch at the Hasso Plattner Institute in Potsdam, Germany, agrees. He recently developed smart building blocks that a Microsoft Surface interactive tabletop can recognise and track in 3D.

Baudisch is impressed with the rubbery interface developed by Koike and Fukuchi’s team: “Other systems had sensed pressure against a table – [this one] senses pressure anywhere on the camera’s field of view on any silicone object being squeezed.”

The Japanese team presented their work at the ACM Symposium on User Interface Software and Technology in Victoria, Canada, this week.