Researchers are working on a new type of virtual reality—no headset required.

MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) developed a way to "touch" objects you see in a video, which it calls Interactive Dynamic Video (IDV).

"One of the most important ways that we experience our environment is by manipulating it: we push, pull, poke, and prod to test hypotheses about our surroundings," the IDV site said. "By observing how objects respond to forces that we control, we learn about their dynamics.

"Unfortunately, regular video does not afford this type of manipulation—it limits us to observing what was recorded," the page continued. "The goal of our work is to record objects in a way that captures not only their appearance, but their physical behavior as well."

Led by CSAIL PhD student Abe Davis, the research team examined tiny, almost invisible tremors they created on camera by banging the surface near an object—a small wire statue, for instance.

Each vibration pattern, or mode, represents different ways the object moves—up, down, side to side. By identifying those modes, the team can predict how an object might act in new situations, allowing someone to virtually interact with the on-screen statue, tugging at it with a mouse and watching it bounce back into place.

This and other simulations, Davis admitts, are not perfect. But they are a "very useful starting point," he says.

IDV technology would most likely be used in engineering applications, such as structural health monitoring (SHM)—the process of predicting the safety of an old bridge or building. Aspiring filmmakers should also keep their eyes on the emerging tech, which—as the video above highlights—is ideal for making low-cost special effects for movies.

Further Reading

Photo & Design Reviews