Video: Touchscreen merges worlds

For all the advances in table-top and tablet computing, some design professionals will always prefer the feel of pen on paper to stylus on glass. A new device could provide them with the best of both the digital and the real worlds.

Andy Wilson and colleagues at Microsoft Research in Redmond, Washington, teamed up with Björn Hartmann at the University of California, Berkeley, to design Pictionaire, a touchscreen table 1.8 metres long. The device is positioned directly beneath a ceiling-mounted camera and projector, which can “read” and respond to items placed on the table.

When a user places a sketchbook on the table, the ceiling mounted equipment recognises it by its size and shape, and projects virtual “drag-off” handle onto the corner of the page. If the user swipes over the handle, the camera takes a digital snapshot of the sketchbook page and sends the information to the touchscreen so that a digital version of the page appears on the table.

A similar process works in reverse – a user can drag an image on the touchscreen onto their sketchpad. The ceiling-mounted hardware then projects the image onto the pad and the user can trace key components onto their sketchbook page.


Style issues

“Contrast this with a tablet PC,” says Wilson. “I don’t care how well made it is – it won’t replicate the true feeling of pencil on paper. And designers are very particular about the type of paper they use, the style of pens.” It’s important, he says, to find sympathetic ways to incorporate the existing desires of users with the new technology on offer.

That meant also allowing people to use a physical keyboard. “If you want to enter text it’s nice to have a wireless keyboard,” he says. Pictionaire can even augment the keyboard experience. When the user places the keyboard onto the touchscreen, the overhead camera recognises it. As the user types, images or words conceptually related to those they type appear on the touchscreen around the keyboard to help in the brainstorming process.

“We’re playing with the concept of moving back and forth between the virtual and the real,” Wilson says.

Best of both

Raimund Dachselt at the University of Magdeburg in Germany is working on a related device, called PaperLens. It uses a ceiling-mounted camera to help users interact with graphics in three dimensions. “I think Pictionaire is one of several important projects aimed at blurring the borders between the real and digital world by taking advantage of the best of each,” he says.

“What I do not find so promising is the use of physical keyboards for particular tasks,” he says. “Keyboards might not play such a dominant role in future because many interactions will be made by clever gestural interactions.”

Pictionaire will be presented at the Computer Supported Cooperative Work conference in Savannah, Georgia, next month.