A conventional camera captures blurred versions of scene information away from the plane of focus. Camera systems have been proposed that allow for recording all-focus images, or for extracting depth, but to record both simultaneously has required more extensive hardware and reduced spatial resolution. We propose a simple modi_cation to a conventional camera that allows for the simultaneous recovery of both (a) high resolution image information and (b) depth information adequate for semi-automatic extraction of a layered depth representation of the image.

Our modification is to insert a patterned occluder within the aperture of the camera lens, creating a coded aperture. We introduce a criterion for depth discriminability which we use to design the preferred aperture pattern. Using a statistical model of images, we can recover both depth information and an all-focus image from single photographs taken with the modified camera. A layered depth map is then extracted, requiring user-drawn strokes to clarify layer assignments in some cases. The resulting sharp image and layered depth map can be combined for various photographic applications, including automatic scene segmentation, post-exposure refocussing, or re-rendering of the scene from an alternate viewpoint.



The Paper (PDF) , appeared in SIGGRAPH 2007.

PowerPoint slides from our SIGGRAPH talk.

Supplementary file with hi-resolution results (PDF)

Video demonstrating novel view synthesis produced using the estimated depth map.

Deconvolution using natural image prior- this package (last updated: Sep 27, 2007) contains an implementation of the (non blind) debluring algorithm used in the paper, relying on a natural image prior. This deblurring algorithm significantly outperforms the traditional Richardson-Lucy deconvolution algorithm. A document describing the algorithm in detail can be download here.

Data - this package contains 16bit images captured by our coded aperture camera and our calibrated kernel (at different scales). It also includes our all focused images at full resolution.