A Preview of the Lytro “Light Field” Camera

Remember that photo – the once in a lifetime shot that you thought you nailed.  But when you scanned the LCD screen, you noticed that the image was just a little out of focus.  Before you knew it, the moment was lost forever.  A new company is promising to solve that problem by doing the impossible – allowing you to correct the focus after you have already taken the image. The concept is being marketed by a company called Lytro and they have boldly promised to release a camera by the end of the year that will revolutionize the way we take photos.  How do they plan to achieve this miracle?  Read on to find out.

Although we don’t know the exact details of what Lytro is up to, we can get some hints through the PhD dissertation of their founder, Ren Ng.  The dissertation explains in plain English the concept for what he calls a “light field” camera.  Just in case you don’t feel like wading through 200 pages, we’ve summarized the idea below.

Most of us are used to thinking in terms of a lens system which focuses a complete image onto a piece of film or a sensor.  Such a configuration is shown below.  We focus our lens onto an object of interest, snap the shutter, and an image is projected onto the sensor to be saved for posterity.  This is the ageless concept which we have come to accept.  It couldn’t possibly get any simpler than that.  Wrong.

The problem with the typical configuration is that although the sensor does a good job counting photons and recording them with deadly accuracy, digital sensors don’t record any directional information. Thus, we are largely stuck with the focal plane and depth of field that were set in the camera during capture. The light field concept aims to record not only the number of photons hitting a given pixel, but directional information, as well.  To do this, an array of micro lenses is placed between the normal lens and the image sensor.  These micro lenses are focused on the aperture of the main lens and project tiny images of the aperture down onto the small arrays of pixels beneath them.  The end result is that instead of one composite image which fills the whole sensor, you now have an array of hundreds of smaller images filling the sensor. Each sub-image contains a wealth of information concerning exactly which rays of light passed through which part of the aperture.

In a conventional camera, each pixel dutifully records every photon that hits it, regardless of where the photon came from. In a light field camera, the pixels on the sensor are partitioned into an array of mini-maps that show the color and quantity of photons that passed through each location on the aperture.  Each x,y pixel under a given micro lens only receives photons from the corresponding x,y portion of the aperture.

 

 

Once the light field image is recorded, the real magic can happen through software manipulation. By stacking, shifting and otherwise manipulating the dozens of sub-images on the sensor, virtual images can be derived which either shift the focus point or extend the depth of field, entirely.  You can play around with this concept on Lytro’s website, but be warned that it is fairly addictive.

The main downside to this technique is loss of overall resolution. By partitioning the pixels on the sensor to give many redundant views of the same thing, the compromise is a loss in final resolution. In order to get adequate resolution for the PhD project, a medium format 16 megapixel sensor was required. Presumably, one of Lytro’s challenges has  been making this concept work on a smaller sized sensor.

Other questions to be addressed?

  • Can the light field option be disabled for “conventional” photography? This would require the micro-lens array to be movable, such that it either is positioned close enough to the sensor to be transparent or removed from the optical axis completely.  Unfortunately, a movable array might impact performance and would certainly add engineering complexity.
  • Within light field mode, what are the available aperture ranges? Changing aperture sizes is not a trivial task with the addition of the micro-lens array.  Since the micro-lenses are focused on the main lens aperture, any change in aperture size could ruin  the image.
  • What is the target market? Professional photographers are used to controlling focus and depth of field as main artistic choices.  They may not be comfortable turning control over to a software.  Amateurs have become complacent with camera phones and don’t normally worry about focus since their small sensors already offer almost infinite depth of field.  Is there room in the middle of these markets for a new brand of camera – regardless of its novelty?

To answer these questions, we’ll have to wait for Lytro’s product launch later this year.  From a technology standpoint, this idea holds exciting possibilities.  Lytro has certainly gotten our attention with this concept – now let’s see if they can deliver on it.

 

Keep on Rolling with These Similar Articles:

Technology Demystified: Backside Illuminated Sensors

Camera Laser Vibration Detector Revisited

The Science of Polarizing Filters

 

Or Get More Camera Technica Greatness:

Subscribe to this Feed ◊   Follow us on Twitter ◊   Email Us

 

Comments

  1. What’s the point of adjusting the aperture if you can get any focal point and depth of field you want in post processing?

    • Preston Scott says:

      Good clarification. Perhaps a better question would have been: what range of digital refocusing is possible with a fixed aperture size?
      Although the depth of field range is greatly increased by digital refocusing, it may not cover the full range from nearfield to infinity. Therefore, some depth of field tweaking may be desirable using the conventional method of changing aperture size. Also, some leeway may be desirable to optimize exposure. Maybe if the lightfield option can be disabled, light field pics would use a single wide aperture size – then further depth of field artistry could then be tackled with a variable aperture in conventional mode. All questions to ask once we see an actual product.

  2. Raytrix came out with a Plenoptics camera last year (http://raytrix.de/). Also, Adobe has been working on their own solution for at lest this long.

    Also, Lytro has been working on this since at least 2008 under the name of “Refocus Imaging”. It will be interesting to see how this technology progresses in the near future.