High-resolution light-field imaging

Patent metadata

Field	Value
Publication number	US-9679360-B2
Application number	US-201414274600-A
Country	US
Kind code	B2
Filing date	May 9, 2014
Priority date	May 10, 2013
Publication date	Jun 13, 2017
Grant date	Jun 13, 2017

Abstract

A relatively high-resolution image from a conventional camera can be computationally combined with a relatively low-resolution wavefront measurement from, for example, a Shack-Hartmann sensor in order to construct a relatively high-resolution light-field image.

First claim

What is claimed is:

1. A camera system comprising:
a wavefront sensor configured to spatially sample wavefronts of light from at least a portion of an object space to obtain the intensity of the wavefronts and associated angular propagation directions of the wavefronts in order to obtain a first light-field image;
an image sensor that is not associated with the wavefront sensor, the image sensor being configured to spatially sample light from the object space to obtain a first intensity reference image, wherein the first intensity reference image is not a light-field image; and
a processor configured to compute a second light-field image from the first light-field image using information from the first intensity reference image, the second light-field image having an enhanced resolution that is greater than that of the first light-field image.

2. The camera system of claim 1 , wherein the spatial resolution of the first light-field image is less than that of the first intensity reference image.

3. The camera system of claim 1 , wherein the enhanced resolution comprises enhanced spatial resolution.

4. The camera system of claim 1 , wherein the enhanced resolution comprises enhanced angular resolution.

5. The camera system of claim 1 , wherein the processor is configured to up-sample the first light-field image to correspond to the spatial resolution of the first intensity reference image, and to perform interpolation on the up-sampled first light-field image.

6. The camera system of claim 5 , wherein the processor is configured to combine angular propagation directions for one or more spatial points in order to obtain a two-dimensional projection of the first light-field image.

7. The camera system of claim 6 , wherein the processor is configured to determine one or more constraints from the first intensity reference image, the one or more constraints being used to determine the second light-field image.

8. The camera system of claim 7 , wherein the processor is configured to transform the first light-field image into the frequency domain prior to determining the one or more constraints from the first intensity reference image.

9. A camera system comprising:
an image sensor configured to spatially sample light from an object space to obtain a first intensity reference image;
a wavefront sensor configured to spatially sample wavefronts of light from at least a portion of the object space to obtain the intensity of the wavefronts and associated angular propagation directions of the wavefronts in order to obtain a first light-field image; and
a processor configured to compute a second light-field image using information from the first intensity reference image, the second light-field image having an enhanced resolution that is greater than that of the first light-field image,
wherein the processor is further configured to up-sample the first light-field image to correspond to the spatial resolution of the first intensity reference image, and to perform interpolation on the up-sampled first light-field image,
wherein the processor is further configured to combine angular propagation directions for one or more spatial points in order to obtain a two-dimensional projection of the first light-field image,
wherein the processor is further configured to determine one or more constraints from the first intensity reference image, the one or more constraints being used to determine the second light-field image, and
wherein the one or more constraints comprise one or more coefficients that relate the two-dimensional projection of the first light-field image to the first intensity reference image.

10. The camera system of claim 9 , wherein the processor is configured to apply the one or more constraints to the up-sampled and interpolated first light-field image.

11. The camera system of claim 1 , wherein the processor is configured to perform a registration operation to reduce any difference between respective frames of reference of the first intensity reference image and the first light-field image.

12. The camera system of claim 1 , wherein the processor is configured to determine one or more image characteristics from the first intensity reference image and the first light-field image; to determine an objective cost function using the one or more image characteristics; and to iteratively adjust an estimate of the second light-field image based on the objective cost function in order to obtain the second light-field image.

13. The camera system of claim 12 , wherein the image characteristics comprise differences between the first light-field image and an estimate of the second light-field image; or differences between the intensity reference image and a two-dimensional projection of the estimate of the second light-field image; or the smoothness of the estimate of the second light-field image.

14. The camera system of claim 1 , wherein the image sensor comprises one or more image sensors and the one or more image sensors are configured to obtain a set of intensity reference images, the set of intensity reference images comprising at least the first intensity reference image and a second intensity reference image, the second intensity reference image differing in at least one respect from the first intensity reference image; and wherein the processor is configured to compute the second light-field image using information from the set of intensity reference images.

15. A camera system comprising:
an image sensor configured to spatially sample light from an object space to obtain a first intensity reference image;
a wavefront sensor configured to spatially sample wavefronts of light from at least a portion of the object space to obtain the intensity of the wavefronts and associated angular propagation directions of the wavefronts in order to obtain a first light-field image; and
a processor configured to compute a second light-field image using information from the first intensity reference image, the second light-field image having an enhanced resolution that is greater than that of the first light-field image,
wherein the image sensor comprises one or more image sensors and the one or more image sensors are configured to obtain a set of intensity reference images, the set of intensity reference images comprising at least the first intensity reference image and a second intensity reference image, the second intensity reference image differing in at least one respect from the first intensity reference image,
wherein the processor is further configured to compute the second light-field image using information from the set of intensity reference images, and
wherein the first intensity reference image and the second intensity reference image comprise different respective focal planes.

16. The camera system of claim 1 , wherein the first intensity reference image satisfies an imaging condition for a selected object within the object space, or wherein the first intensity reference image comprises a Fourier transform.

17. The camera system of claim 1 , wherein the processor is configured to iteratively compute the second light-field image.

18. The camera system of claim 1 , wherein the first intensity reference image and the first light-field image are obtained simultaneously.

19. The camera system of claim 1 , wherein the first intensity reference image and the first light-field image are obtained at different times.

20. The camera system of claim 1 , wherein the image sensor and the wavefront sensor comprise separate devices.

Assignees

• Univ Princeton

Inventors

• Fleischer Jason W
• Lu Chien-Hung
• Muenzel Stefan

Classifications

• G01J9/00

  Measuring optical phase difference (devices or arrangements for controlling the phase of light beams G02F1/01); Determining degree of coherence; Measuring optical wavelength (spectrometry G01J3/00) · CPC title

• G01J2009/002

  Wavefront phase distribution · CPC title

• G02B27/0075

  with means for altering, e.g. increasing, the depth of field or depth of focus · CPC title

• G06T3/4061 Primary

  by injecting details from different spectral ranges · CPC title