Imaging scientist Ren Ng's years of research into capturing "light fields" using increasingly high-resolution digital imaging sensors have finally come to fruition. Ng's company, Lytro, unveiled its first consumer product on Wednesday—a digital camera capable of capturing "living images" that can be infinitely refocused after capture. While the new camera is designed to change the way we capture and share snapshots, the technology has the potential to radically alter how all photographs are made.

The new Lytro camera is a small rectangular tube of aluminum, with an f/2 lens on one end and a small 2" touchscreen on the other. The only controls are a power button, shutter button, and a slider to control the 8x zoom range of its lens. There are no controls for aperture, shutter speed, or focus—because the Lytro doesn't need them. The Lytro is probably the closest thing to "point-and-shoot" photography that has ever existed in the digital era.

Capturing light fields

Ng first began his efforts to capture living images as a researcher at Stanford. There, his cohorts, including Lytro engineer Bennett Wilburn, developed what's know as the Stanford Multi-Camera Array. The array consists of a hundred or more CMOS sensors and matching lenses and it was used to capture, among other research oddities, what is known as a light field. Ng used a similar array in his own research into capturing and processing light field data.

Traditional cameras capture flat images of three-dimensional space by focusing light rays directed toward a camera's lens on to a flat piece of film or digital sensor. Effectively, a camera records a two-dimensional array of colored dots (pixels in one case, film grains in the other) corresponding to light rays directed towards the camera from objects in front of the lens. The Stanford Multi-Camera Array, however, records the color, intensity, and direction of the light rays in a scene. These light rays collectively make up a scene's light field.

If you are familiar with 3D rendering and the concept of ray tracing, that technique is essentially the generation of a light field for a synthetic scene made up of 3D models. A ray tracing renderer follows rays of light emitted from synthesized sources as they bounce from object to object, reflecting at varying angles depending on the surface qualities of an object.

Without knowing anything about the objects in a scene, however, the entire scene and the objects in it can be visually recreated if its unique light field is known. Ng wrote his PhD thesis on the computational concepts and algorithms necessary to do so, as well as methods to capture a light field using digital CMOS sensors.

Ng then set about shrinking the SMCA—its hundred sensors and lenses, four PCs, and striped RAID array wasn't very mobile—into something even a casual photographer could use. Lytro is keeping most of the details of its camera under wraps, but instead of using multiple sensors and lenses, the specially designed Lytro sensor uses a sort of micro lens array on top of a high-megapixel CMOS sensor to record 11 "megarays" of light field information.

These 11 megarays are what allows a Lytro user to shoot first and focus later. "The majority of pictures are shared online casually," Ng told Ars. "10, 12, 14 megapixels doesn't help when 95 percent of them aren't ever going to be viewed. We think it's better to harness those pixels to collect richer data."

That richer data, the light field, actually makes it possible to selectively refocus an image after it's captured. No more worrying about fast-moving subjects moving out of focus, or other objects or people in a scene confusing a camera's autofocus system. In fact, the Lytro camera doesn't have a focus system at all.

"The architecture allows us to do the work of the physical hardware of a lens with computation," Ng explained. "It allows us to increase the performance of the system while simplifying the hardware."

For instance, the Lytro always shoots every image at its maximum f/2.0 aperture, allowing it to capture images in very low light. Images can be captured with very little lag, since the aperture and focus do not need to be adjusted before taking a picture. Though the Lytro includes an 8X optical zoom, it's even possible to do zooming computationally. And despite capturing an image from a single location, light field data even makes it possible to perform some limited perspective shifting, as well—enough to generate 3D images from a single image capture.

Future of photography

Lytro is now taking orders for its camera, which comes in an 8GB version for $399 or a 16GB version for $499. The hardware is set to ship early next year.

Using the light field data captured with a Lytro, users can view living images on the camera's tiny 2" screen, can upload and view them using a desktop utility, or can post them online to Lytro's website. The desktop app is currently Mac-only, but a Windows version is planned for next year. Images posted to the Web require Flash to view on a desktop, and can be embedded on Facebook or other sites. Mobile devices can view the images using an HTML5-based interface.

As far as prints are concerned, though, Lytro's living images aren't quite up to the standards that photographers have come to expect. The 11 megarays it can capture are rendered in "HD resolution," which means at best 2 megapixels. "Light field photography lets you create interactive, living pictures, unlike print, so that's our focus right now," Ng told Ars.

But that doesn't mean the technology couldn't be incorporated into higher-end cameras in the future. "Current CMOS sensor resolutions have already blown past the ability to benefit the consumer," Ng said. "But, we can go so much higher than what's in the market. For instance Canon has 120 megapixel sensor prototype—we could be using that to collect a very high resolution light field."

The tech could even conceivably be shrunk down to fit into, say, a smartphone. A simplified lens meant to capture light fields might be more rugged and less expensive to design than the higher-end lenses seen in current-generation smartphones. Images could be captured with almost no lag, and light field data could be processed into images with adjustable focus and zoom after the fact.

We were only given a short demonstration of what the Lytro can do, but the possibilities left us wanting more. We'll reserve ultimate judgement until we can get our hands on an actual camera—which won't be available for a few more months—but the technology looks extremely promising.