Automatic adjustment of light field imaging systems for resolution enhancement

Patent metadata

Field	Value
Publication number	US-9544514-B2
Application number	US-201514640309-A
Country	US
Kind code	B2
Filing date	Mar 6, 2015
Priority date	Mar 6, 2015
Publication date	Jan 10, 2017
Grant date	Jan 10, 2017

Abstract

A light field imaging system captures different images provide views that are shifted by sub-pixel amounts relative to one another. These views can be combined to produce a higher resolution digital image of the object.

First claim

What is claimed is:

1. A light field imaging system for producing an image of an object, the light field imaging system comprising:
a primary optical subsystem that forms an optical image of an object, the optical image located at an image plane of the primary optical subsystem;
a light field sensor module comprising a secondary imaging array and a sensor array, the secondary imaging array imaging a pupil of the primary optical subsystem onto the sensor array, the secondary imaging array displaced from the image plane so that the sensor array is configured to capture an array of multiplexed views of the object;
a processing module configured to determine one or more optical parameters to achieve a desired resolution enhancement value, wherein the primary optical subsystem and/or light field sensor module are adjustable according to the one or more optical parameters; and
an auto defocus module to automatically adjust the primary optical subsystem and/or light field sensor module according to the one or more optical parameters determined by the processing module; and
wherein the processing module is further configured to combine the captured array of multiplexed views into a digital image of the object having a resolution higher than a resolution of any of the multiplexed views, and wherein the array of multiplexed views were captured after automatic adjustment by the auto defocus module.

2. The light field imaging system of claim 1 , wherein the one or more optical parameters include an effective focal length f of the primary optical subsystem, and the primary optical subsystem is adjustable in effective focal length.

3. The light field imaging system of claim 1 , wherein the one or more optical parameters include a distance d i from the primary optical subsystem to the secondary imaging array, and the light field imaging system is adjustable in distance d i .

4. The light field imaging system of claim 1 , wherein the desired resolution enhancement value is an integer value greater than 1.

5. The light field imaging system of claim 1 , wherein the processing module is further configured to:
determine a region of interest within the captured array of multiplexed views;
calculate an initial disparity for the region of interest;
determine a depth range of the region of interest using the initial disparity; and
determine the desired resolution enhancement value subject to the determined depth range.

6. The light field imaging system of claim 1 , wherein the processing module is further configured to determine the one or more optical parameters consistent with

Disparity

in
⁢

⁢
units
⁢

⁢
of
⁢

⁢
pixels

=

∝

*

w
p

φ
*

f
μ

*

(

d
o

-

d
f

)

*

(

d
i

-
f

)

(

d
o

-
f

)

wherein f is an effective focal length of the primary optical subsystem,
d o is a distance from the primary optical subsystem to an object plane,
d f is a distance from the primary optical subsystem to an object plane when disparity is zero,
f μ , is a focal length of the secondary imaging array,
α is a factor that is added to account for image processing,
w p is a size of a sensor in the sensor array, and
φ is distance between neighboring pixels in a view.

7. The light field imaging system of claim 1 , wherein the auto defocus module receives feedback from the processing module and uses the feedback to adjust the one or more optical parameters to achieve the desired resolution enhancement value.

8. The light field imaging system of claim 1 , wherein the primary optical subsystem includes an adjustable lens with a resolution ring, wherein the resolution ring is rotatable to adjust the lens and indicates to a user the desired resolution enhancement value of the light field imaging system.

9. The light field imaging system of claim 8 , wherein the adjustable lens also includes a focus ring that is rotatable to adjust the lens and indicates to a user an effective focal length of the primary optical subsystem.

10. The light field imaging system of claim 8 , wherein the adjustable lens also includes a focus ring that is rotatable to adjust the lens and indicates to a user a focus conjugate of the primary optical subsystem.

11. The light field imaging system of claim 1 , further comprising an electronic display configured to:
present a resolution indicator to a user of the light field imaging system, wherein the resolution indicator indicates to a user the desired resolution enhancement value of the light field imaging system.

12. A method for producing an image of an object using a light field imaging system, the light field imaging system comprising a primary optical subsystem and a light field sensor module having a secondary imaging array and a sensor array, the method comprising:
receiving a desired resolution enhancement value;
determining one or more optical parameters to achieve the desired resolution enhancement value, wherein the primary optical subsystem and/or light field sensor module are adjustable according to the one or more optical parameters;
automatically adjusting the light field imaging system according to the one or more determined optical parameters;
capturing an array of multiplexed views of the object after automatic adjustment of the light field imaging system according to the one or more determined optical parameters; wherein the primary optical subsystem forms an optical image of the object, the optical image located at an image plane of the primary optical subsystem; and the secondary imaging array images a pupil of the primary optical subsystem o

Assignees

• Namboodiri Vipin
• Prasad Agara Venkatesha Rao Krishna
• Maruyama Go
• Ricoh Co Ltd

Inventors

• Namboodiri Vipin
• Prasad Agara Venkatesha Rao Krishna
• Maruyama Go

Classifications

• H04N23/957

  Light-field or plenoptic cameras or camera modules · CPC title

• H04N23/951

  by using two or more images to influence resolution, frame rate or aspect ratio · CPC title

• H04N23/633

  for displaying additional information relating to control or operation of the camera · CPC title

• H04N25/48 Primary

  Increasing resolution by shifting the sensor relative to the scene · CPC title

• G06T7/557 Primary

  from light fields, e.g. from plenoptic cameras · CPC title