Motion Invariant Photography

Technology #13075

Questions about this technology? Ask a Technology Manager

Download Printable PDF

Image Gallery
 Left: Blurred motion captured by a static camera. Center: The same scene captured by a camera with a specially designed motion that causes both the static and dynamic regions to blur identically. Right: The blur from the center image can be removed independently of motion via deconvolution of the entire image with a single known point spread function.
Categories
Inventors
Professor William Freeman
CSAIL & Department of EECS, MIT
External Link (billf.mit.edu)
Professor Fredo Durand
CSAIL & Department of EECS, MIT
External Link (graphics.csail.mit.edu)
Taeg Sang Cho
CSAIL & Department of EECS, MIT
Anat Levin
CSAIL, MIT
Peter Sand
CSAIL, MIT
Managed By
Daniel Dardani
MIT Technology Licensing Officer
Patent Protection

Method and apparatus for motion invariant imaging

US Patent 8,451,338
Publications
Motion-invariant photography
ACM Transactions on Graphics (TOG), Vol. 27, No. 3, p. 71

Applications

This technology is camera hardware designed to remove motion blur from photographs.

Problem Addressed

Motion blur often limits the quality of photographs and can be caused by either the shaking of the camera or the movement of objects in the scene. Modern cameras use image stabilization methods to address camera motion; however, these techniques do not help with moving objects.  Eliminating blur is challenging because the blur kernel is unknown, varies over the image as a function of object velocity, and destroys high frequencies. The Inventors observed that these challenges can all be addressed if motion is restricted to a 1D (e.g. horizontal) set of velocities. Using  standardized camera hardware,  they make an image's point-spread function (PSF) invariant to motion and therefore easy to invert.

Technology

To achieve this, the Inventors introduce a specific camera movement during exposure. This movement is designed so that the compound motion of the camera and any object velocity along the selected orientation and at any depth results in the same easy-to-invert PSF. Since the entire scene is blurred with an identical PSF, a single deconvolution kernel can be used to remove blur and create sharp images of scenes with objects moving at different speeds, without requiring any segmentation, and without knowledge of the object speeds. In practice, motions close to the selected 1D orientation can also be deblurred.

This approach is inspired by wavefront coding, where depth of field is improved by modifying a lens to make the defocus blur invariant to depth and easy to invert. When motion blur is analyzed as the integration over curves resulting from camera and object motion, the only integration curve that results in a motion-invariant PSF is a parabola. This corresponds to constant 1D acceleration of the camera, which first moves quickly in one direction, progressively slows down until it stops, then picks up speed in the other direction. The observation that for any object velocity within a range, there is always one moment during exposure where the camera is perfectly tracking the object, motivates the camera movement in this technology.

Advantages

  • PSF is invariant to object speed and preserves more high frequencies for moving objects than a normal exposure
  • Proposed design demonstrates ideal distribution of fixed bandwidth budget for imaging objects at different velocities, resulting in optimal tradeoff between improving reconstruction of motion velocities at the price of reconstruction of static objects