Cliplets is a research project from Microsoft Research. This project explores a form of visual media that juxtaposes still image and video segments, both spatially and temporally, to expressively abstract a moment. The tension between static and dynamic elements in a cliplet reinforces both aspects, strongly focusing the viewer’s attention or conveying a narrative. We develop a set of idioms, essentially spatiotemporal mappings, that characterize these cliplet elements, and use these idioms in an interactive system to quickly compose a cliplet from ordinary handheld video. A key challenge is to avoid seam artifacts by maintaining spatiotemporal continuity in the cliplet composition. We address this using several algorithms from computer graphics and vision.

The publically available app is a result of this research project. Here is a Techinical Report describing the underlying research:

Neel Joshi, Sisil Mehta, Steven Drucker, Eric Stollnitz, Hugues Hoppe, Matt Uyttendaele, and Michael Cohen. Cliplets: Juxtaposing Still and Dynamic Imagery. No. MSR-TR-2012-52. 15 May 2012.