Wedge-based light-field video capture

Patent metadata

Field	Value
Publication number	US-10275898-B1
Application number	US-201715590951-A
Country	US
Kind code	B1
Filing date	May 9, 2017
Priority date	Apr 15, 2015
Publication date	Apr 30, 2019
Grant date	Apr 30, 2019

Abstract

A combined video of a scene may be generated for applications such as virtual reality or augmented reality. In one method, a camera system may be oriented at a first orientation and used to capture first video of a first portion of the scene. The camera system may then be rotated to a second orientation and used to capture second video of a second portion of the scene that is offset from the first portion such that the first video and the second video each have an overlapping video portion depicting an overlapping portion of the scene in which the first portion and the second portion of the scene overlap with each other. The first and second portions may be combined together to generate the combined video, which may depict the first and second portions substantially without duplicative inclusion of the overlapping video portion.

First claim

What is claimed is:

1. A method for generating a combined video of a scene, the method comprising:
orienting a camera system at a first orientation;
with the camera system at the first orientation, capturing first video of a first portion of a scene, wherein capturing the first video includes applying a lighting pattern to the scene, the lighting pattern comprising at least a first lighting change occurring at a first time within a first timeline in which the first video is captured;
rotating the camera system from the first orientation to a second orientation;
with the camera system at the second orientation, capturing second video of a second portion, offset from the first portion, of the scene such that the first video and the second video each comprise an overlapping video portion depicting an overlapping portion of the scene in which the first portion and the second portion of the scene overlap with each other, wherein capturing the second video includes applying the lighting pattern to the scene such that the first lighting change occurs at the first time within a second timeline in which the second video is captured; and
at a processor, combining the first video and the second video to generate a combined video depicting the first portion and the second portion of the scene substantially without duplicative inclusion of the overlapping video portion.

2. The method of claim 1 , further comprising:
rotating the camera system from the second orientation to a third orientation;
with the camera system at the third orientation, capturing third video of a third portion, offset from the first portion and the second portion, of the scene;
rotating the camera system from the third orientation to a fourth orientation;
with the camera system at the fourth orientation, capturing fourth video of a fourth portion, offset from the first portion, the second portion, and the third portion, of the scene; and
rotating the camera system from the fourth orientation to a fifth orientation;
with the camera system at the fifth orientation, capturing fifth video of a fifth portion, offset from the first portion, the second portion, the third portion, and the fourth portion of the scene;
wherein generating the combined video further comprises combining the third video, the fourth video, and the fifth video with the first video and the second video to generate the combined video depicting the first portion, the second portion, the third portion, the fourth portion, and the fifth portion of the scene.

3. The method of claim 1 , wherein generating the combined video further comprises adding at least a first computer-generated element into at least one of the first video and the second video.

4. The method of claim 3 , further comprising generating a depth map indicating a relative depth of objects in the first portion and the second portion of the scene;
wherein adding at least the first computer-generated element into at least one of the first video and the second video comprises positioning the first computer-generated element at a first depth within the depth map.

5. The method of claim 1 , further comprising:
positioning a calibration chart within the first portion of the scene; and
calibrating the camera system by capturing first calibration video depicting the first portion of the scene with the calibration chart positioned within the first portion.

6. The method of claim 5 , further comprising:
positioning the calibration chart within the overlapping portion of the scene;
wherein calibrating the camera system further comprises capturing overlapping calibration video depicting the overlapping portion with the calibration chart positioned within the overlapping portion.

7. A method for generating a combined video of a scene, the method comprising:
calibrating a camera system comprising a camera array having a plurality of cameras arranged in a regular pattern to generate a virtual camera system comprising a plurality of virtual cameras, each of which has virtual characteristics that mimic operating characteristics of a corresponding one of the plurality of cameras;
orienting the camera system at a first orientation;
with the camera system at the first orientation, capturing first video of a first portion of a scene;
rotating the camera system from the first orientation to a second orientation;
with the camera system at the second orientation, capturing second video of a second portion, offset from the first portion, of the scene such that the first video and the second video each comprise an overlapping video portion depicting an overlapping portion of the scene in which the first portion and the second portion of the scene overlap with each other; and
at a processor, combining the first video and the second video to generate a combined video depicting the first portion and the second portion of the scene substantially without duplicative inclusion of the overlapping video portion.

8. The method of claim 7 , wherein the regular pattern comprises a planar shape.

9. The method of claim 7 , wherein each of the cameras comprises a light-field camera.

10. The method of claim 7 , wherein:
the plurality of cameras comprise a central camera positioned at a center of the regular pattern; and
rotating the camera system from the first orientation to the second orientation comprises rotating the camera system about an axis passing through a nodal point of the central camera.

11. The method of claim 7 , wherein combining the first video with the second video comprises using the virtual camera system.

12. The method of claim 7 , wherein generating the combined video further comprises adding at least a first computer-generated element into at least one of the first video and the second video.

13. The method of claim 12 , further comprising:
generating a depth map indicating a relative depth of objects in the first portion and the second portion of the scene; and
wherein adding at least the first computer-generated element into at least one of the first video and the second video comprises positioning the first computer-generated element at a first depth within the depth map.

14. A method for generating a combined video of a scene, the method comprising:
orienting a camera system at a first orientation;
with the camera system at the first orientation, capturing first video of a first portion of a scene;
rotating the camera system from the first orientation to a second orientation;
with the camera system at the second orientation, capturing second video of a second portion, offset from the first portion, of the scene such that the first video and the second video each comprise an overlapping video portion depicting an overlapping portion of the scene in which the first portion and the second portion of the scene overlap with each other;
at a processor, combining the first video and the second video to generate a combined video depicting the first portion and the second portion of the scene substantially without duplicative inclusion of the overlapping video portion; and
prior to capturing the first video, designating a subset of the first portion of the scene as a safe zone that is out-side the overlapping portion.

15. The method of claim 14 , wherein designating the subset of the first portion of the scene as the safe zone comprises, with a laser, projecting a beam into the first portion of the scene, proximate a boundary of the safe zone.

16. A system for generating a combined video of a scene, the system comprising:
a camera system that is rotatable between at least a first orientation

Assignees

• Google Llc

Inventors

• Song Alex
• Frank Jonathan
• Hernandez Zaragoza Julio C
• Green Orin
• Cooper Steve
• Braunstein Ariel
• Milliron Tim
• Pitts Colvin
• Yasui Yusuke
• Shahhosseini Saeid
• Zhang Bipeng

Classifications

• G06T1/0007 Primary

  Image acquisition · CPC title

• H04N23/698

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• H04N5/2224

  related to virtual studio applications · CPC title

• G06T2207/30204

  Marker · CPC title

• H04N13/246

  Calibration of cameras · CPC title