SIGGRAPH 2012

Eulerian Video Magnification for Revealing Subtle Changes in the World

Abstract

Our goal is to reveal temporal variations in videos that are difficult or impossible to see with the naked eye and display them in an indicative manner. Our method, which we call Eulerian Video Magnification, takes a standard video sequence as input, and applies spatial decomposition, followed by temporal filtering to the frames. The resulting signal is then amplified to reveal hidden information. Using our method, we are able to visualize the flow of blood as it fills the face and also to amplify and reveal small motions. Our technique can run in real time to show phenomena occurring at temporal frequencies selected by the user.





@article{Wu12Eulerian, author = {Hao-Yu Wu and Michael Rubinstein and Eugene Shih and John Guttag and Fr\'{e}do Durand and William T. Freeman}, title = {Eulerian Video Magnification for Revealing Subtle Changes in the World}, journal = {ACM Transactions on Graphics (Proc. SIGGRAPH 2012)}, year = {2012}, volume = {31}, number = {4}, }



Paper: PDF, Errata

Supplemental: PDF (the derivation in Appendix A in the paper given in more detail)

SIGGRAPH 2012 Presentation: PPT with videos (150MB)

Visualization of Eulerian motion magnification (Courtesy of Lili Sun)

Videoscope by Quanta Research - upload your videos and have them magnified!

Michael's PhD thesis: Analysis and Visualization of Temporal Variations in Video

SIGGRAPH 2012 Supplemental Video:

Our NSF SciVis 2012 video "Revealing Invisible Changes In The World":

NSF International Science & Engineering Visualization Challenge 2012



Science Vol. 339 No. 6119 pp. 518-519, February 1 2013





Related Publications

Analysis and Visualization of Temporal Variations in Video, Michael Rubinstein, PhD Thesis, MIT Feb 2014

Riesz Pyramids for Fast Phase-Based Video Magnification, ICCP 2014, to appear

Phase-Based Video Motion Processing, SIGGRAPH 2013

Press

Code and Binaries

- Matlab source code (v1.1, 2013-03-02)

Reproduces all the results in the paper. See the README file for details.

- Executables for 64-bit Windows, 64-bit Linux and 64-bit Mac (v1.1, 2013-09-05)

This is a compiled version of the MATLAB code the can be run from the command line. It doesn't require any programming or for MATLAB to be installed. Instead, these binaries use the MATLAB Compiler Runtime (MCR), which is free and only takes a couple of minutes to install. See the README file for details.

The code and executables are provided for non-commercial research purposes only. By downloading and using the code, you are consenting to be bound by all terms of this software release agreement. Contact the authors if you wish to use the code commercially.

* This work is patent pending

Please cite our paper if you use any part of the code or videos supplied on this web page.



Tips for recording and processing videos:

At capture time:

- Minimize extraneous motion. Put the camera on a tripod. If appropriate, provide support for your subject (e.g. hand on a table, stable chair).

- Minimize image noise. Use a camera with a good sensor, make sure there is enough light.

- Record in the highest spatial resolution possible and have the subject occupy most of the frame. The more pixels covering the object of interest - the better the signal you would be able to extract.

- If possible, record/store your video uncompressed. Codecs that compress frames independently (e.g. Motion JPEG) are usually preferable over codecs exploiting inter-frame redundancy (e.g. H.264) that, under some settings, can introduce compression-related temporal signals to the video.

When Processing:

- To amplify motion, we recommend our new phase-based pipeline.

- To amplify color, use the linear pipeline (the paper and code in this page).

- Choose the correct time scale that you want to amplify. For example, heart beats tend to occur around once per second for adults, corresponding to 1Hz, and you can amplify content between 0.5Hz and 3Hz to be safe. The narrower the interval, the more focused the amplification is and the less noise gets amplified, but at the risk of missing physical phenomena.

- Don't forget to account for the video frame rate when specifying the temporal passband! See our code for examples.

Data

All videos are in MPEG-4 format and encoded using H.264.

Acknowledgements

We would like to thank Guha Balakrishnan, Steve Lewin-Berlin and Neal Wadhwa for their helpful feedback, and the SIGGRAPH reviewers for their comments. We thank Ce Liu and Deqing Sun for helpful discussions on the Eulerian vs. Lagrangian analysis. We also thank Dr. Donna Brezinski, Dr. Karen McAlmon, and the Winchester Hospital staff for helping us collect videos of newborn babies. This work was partially supported by DARPA SCENICC program, NSF CGV-1111415, and Quanta Computer. Michael Rubinstein was partially supported by an NVIDIA Graduate Fellowship.