In order to see this embed, you must give consent to Social Media cookies. Open my cookie preferences.

Microsoft researchers have developed a technique for converting first-person videos, such as those captured with a helmet camera while cycling, into wonderfully-smooth "hyperlapse" videos.

Hyperlapse videos are timelapse videos that appear to be shot with a smoothly moving video camera. An algorithm eliminates the erratic camera shake that tends to be present in casually captured first-person videos.


The technique is particularly useful in an age when we can shoot hours and hours of footage as we go about a particular activity -- whether it's skiing, walking, surfing or climbing. Always-on cameras such as those made by GoPro and Hero are very simple to use but can suffer from camera shake (given that they are generally positioned on helmets or similar) and changing lighting. The videos can also be long and monotonous, so they are boring to watch and difficult to navigate.

While it's easy to create hyperlapse videos from very stable images, such as those generated by a camera attached to a car -- as documented in this story about stitching Google Street View journeys together -- it's much harder to do the same with cameras attached to wobbly bipeds.

Traditional stabilisation methods and simple frame sub-sampling techniques don't work with first-person videos as the shakiness gets exacerbated as the footage is sped up. The Microsoft Research team worked on a system that reconstructs the journey and develops a new, virtual camera path for the output video that is rendered from the input footage.

There are three key parts to the process. The first is scene reconstruction, which involves developing a 3D model of an environment based on the captured frames using "structure-from-motion" algorithms. Once the model has been built, the system will plan an optimised path for the camera that is smooth in location and orientation and makes the most of the supplied input footage. Finally, the image is rendered at ten times the original speed using stitching and blending of carefully selected frames from the original footage. The result is a pretty cool fly-through of the journey with a very fluid motion.


In order to see this embed, you must give consent to Social Media cookies. Open my cookie preferences.

The authors of the paper say: "As the prevalence of first-person video grows, we expect to see a greater demand for creating informative summaries from the typically long video captures. Our hyperlapse work is just one step forward. As better semantic understanding of the scene becomes available, either through improved recognition algorithms or through user input, we hope to incorporate such information, both to adjust the speed along the smoothed path, the camera orientation, or perhaps to simply jump over uninformative sections of the input."

You can read more about the technique in this research paper.