Generating dolly zoom effect using light field image data

Patent metadata

Field	Value
Publication number	US-10594945-B2
Application number	US-201815944551-A
Country	US
Kind code	B2
Filing date	Apr 3, 2018
Priority date	Apr 3, 2017
Publication date	Mar 17, 2020
Grant date	Mar 17, 2020

Abstract

A sensor is configured to acquire a light field by imaging a scene. A processor is configured to determine four-dimensional (4D) coordinates of points in a light field and generate dollied coordinates from the 4D coordinates based on a dolly transform and a dolly parameter. The processor is also configured to project rays associated with the dollied coordinates from the light field onto corresponding points in an output raster. In some cases, the processor applies an aperture function to filter the rays in the coordinate system of the dollied coordinates. The aperture function has a first value in a first region of an aperture plane and the aperture value has a second value in a second region of the aperture plane. Rays passing through the first region are accepted by the aperture function and rays passing through the second region are rejected.

First claim

What is claimed is:

1. A method comprising:
determining four-dimensional (4D) coordinates of points in a light field;
generating dollied coordinates from the 4D coordinates based on a dolly transform and a dolly parameter;
applying an aperture function to filter rays in the coordinate system of the dollied coordinates, wherein the aperture function has a first value in a first region of an aperture plane and the aperture value has a second value in a second region of the aperture plane such that rays passing through the first region are accepted by the aperture function and rays passing through the second region are rejected; and
projecting rays associated with the dollied coordinates from the light field onto corresponding points in an output raster.

2. The method of claim 1 , wherein applying the aperture function to filter the rays in the coordinate system of the dollied coordinates comprises dollying a virtual aperture by a distance relative to the aperture plane.

3. The method of claim 2 , wherein generating the dollied coordinates comprises applying a depth dependent dolly transformation to the 4D coordinates.

4. The method of claim 3 , wherein dollying the virtual aperture produces a depth-dependent scaling of a two dimensional (2D) reconstructed image that is reconstructed from the light field.

5. The method of claim 1 , further comprising:
refocusing portions of the light field using a linear refocusing transform and a refocusing parameter.

6. The method of claim 1 , wherein generating the dollied coordinates comprises generating the dollied coordinates by applying a quadratic dolly transform.

7. The method of claim 1 , wherein generating the dollied coordinates comprises producing tilt in a lens plane and an aperture plane.

8. An apparatus comprising:
a sensor configured to acquire a light field by imaging a scene; and
a processor configured to:
determine four-dimensional (4D) coordinates of points in a light field;
generate dollied coordinates from the 4D coordinates based on a dolly transform and a dolly parameter;
apply an aperture function to filter rays in the coordinate system of the dollied coordinates, wherein the aperture function has a first value in a first region of an aperture plane and the aperture value has a second value in a second region of the aperture plane such that rays passing through the first region are accepted by the aperture function and rays passing through the second region are rejected; and
project rays associated with the dollied coordinates from the light field onto corresponding points in an output raster.

9. The apparatus of claim 8 , wherein the processor is configured to dolly a virtual aperture by a distance relative to the aperture plane.

10. The apparatus of claim 9 , wherein the processor is configured to apply a depth dependent dolly transformation to the 4D coordinates.

11. The apparatus of claim 10 , wherein dollying the virtual aperture produces a depth-dependent scaling of a two dimensional (2D) reconstructed image that is reconstructed from the light field.

12. The apparatus of claim 8 , wherein the processor is configured to refocus portions of the light field using a linear refocusing transform and a refocusing parameter.

13. The apparatus of claim 8 , wherein the processor is configured to generate the dollied coordinates by applying a quadratic dolly transform.

14. The apparatus of claim 8 , wherein the processor is configured to tilt an image in a lens plane and an aperture plane applying the dolly transform.

Assignees

• Google Llc

Inventors

• Wang Zejing
• Merritt Joel

Classifications

• G06T3/40 Primary

  Scaling of whole images or parts thereof, e.g. expanding or contracting · CPC title

• G06T5/50

  using two or more images, e.g. averaging or subtraction · CPC title

• H04N5/2628

  Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation · CPC title

• G06T2207/10052

  Images from lightfield camera · CPC title

• H04N5/23212

  Electricity · mapped topic