Depth-assisted focus in multi-camera systems

Patent metadata

Field	Value
Publication number	US-9565416-B1
Application number	US-201414167950-A
Country	US
Kind code	B1
Filing date	Jan 29, 2014
Priority date	Sep 30, 2013
Publication date	Feb 7, 2017
Grant date	Feb 7, 2017

Abstract

Example embodiments may help multi-camera devices determine disparity information scene, and use the disparity information in an autofocus process. An example method involves: (a) receiving image data of a scene that comprises at least one image of the scene captured by each of two or more image-capture systems of a computing device that includes a plurality of image-capture systems; (b) using the image data captured by the two or more image-capture systems as a basis for determining disparity information for the scene; and (c) performing, by the computing system, an autofocus process based at least in part on the disparity information, wherein the autofocus process provides a focus setting for at least one of the image-capture systems of the computing device.

First claim

We claim:

1. A method comprising:
receiving, by a computing system, image data of a scene, wherein the image data comprises image data captured by each of three or more image-capture systems of a computing device that includes a plurality of image-capture systems, wherein the three or more image-capture systems collectively provide two or more pairs of image-capture systems having two or more different baselines;
using, by the computing system, the image data captured by the three or more image-capture systems as a basis for determining disparity information for the scene, wherein determining the disparity information comprises:
determining a variation in magnitude of the disparity across the two or more pairs of image-capture systems; and
comparing (a) the variation in the magnitude of the disparity to (b) corresponding variation in baseline length between the two or more pairs of image-capture systems; and

performing, by the computing system, an autofocus process based at least in part on the disparity information, wherein the autofocus process provides a focus setting for at least one of the image-capture systems of the computing device.

2. The method of claim 1 , wherein determining the disparity information for the scene comprises determining an overall disparity for the scene.

3. The method of claim 1 , wherein the scene is divided into a plurality of segments, and wherein determining the disparity information for the scene comprises determining a local disparity for each of two or more segments of the scene.

4. The method of claim 1 , wherein the image data of the scene captured by each of the two or more image-capture systems is captured simultaneously, and wherein determining the disparity information for the scene further comprises:
determining correspondence data indicating correspondence between the image data of the scene captured by a first of the three or more image-capture systems and the image data of the scene captured by a second of the three or more image-capture systems; and
determining, based on the correspondence data, disparity between at least a portion of the image data of the scene captured by the first of the three or more image-capture systems and at a corresponding portion of the image data of the scene captured by the second of the three or more image-capture systems.

5. The method of claim 4 , wherein determining the correspondence data comprises determining a spatial optical flow between the image data of the scene captured by the first of the three or more image-capture systems and the image data of the scene captured by the second of the three or more image-capture systems.

6. The method of claim 1 , wherein the disparity information indicates disparity due to parallax, between image data of the scene captured by first and second image-capture systems.

7. The method of claim 1 , wherein performing an autofocus process based at least in part on the disparity information comprises:
determining, based on the disparity information, depth information for the scene; and
using the depth information as a basis for performing the autofocus process.

8. The method of claim 7 , wherein determining depth information for the scene comprises generating a depth map for the scene.

9. The method of claim 1 , wherein performing the autofocus process comprises:
determining, by the computing system, at least one focus point in the scene;
using the disparity as a basis for determining, by the computing system, depth information for the at least one focus point;
determining, by the computing system, at least one focus setting that corresponds to an indication of depth at the least one focus point, wherein the indication of depth is provided by the disparity; and
configuring the at least one of the image-capture systems according to the at least one determined focus setting.

10. The system of claim 1 , wherein the system is implemented in or takes the form of a mobile device.

11. A method comprising:
receiving, by a computing system, image data of a scene, wherein the image data comprises image data captured by each of two or more image-capture systems of a computing device that includes a plurality of image-capture systems, and wherein the scene is divided into a plurality of segments;
using, by the computing system, the image data captured by the two or more image-capture systems as a basis for determining disparity information for the scene, wherein determining the disparity information for the scene comprises:
determining one or more focus segments in the scene, wherein each focus segment corresponds to at least one of the one or more focus points;
determining local disparity information for each focus segment;
determining local disparity information for one or more other segments of the scene that do not correspond to at least one of the one or more focus points; and
determining a disparity for the scene based at least in part on both (a) the local disparity for each of the one or more focus segments and (b) the local disparity for each of the one or more other segments, wherein greater weight is placed on the local disparity for each of the one or more focus segments than is placed on the local disparity for each of the one or more other segments; and

performing, by the computing system, an autofocus process based at least in part on the disparity information, wherein the autofocus process provides a focus setting for at least one of the image-capture systems of the computing device.

12. The method of claim 11 , further comprising:
displaying, on a touchpad display of the computing system, image data of the scene;
wherein determining the at least one focus point in the scene comprises:
receiving data input corresponding to a touch gesture at a location on the touchpad display that corresponds to first image-frame location in image data of the scene; and
setting the first location as the at least one focus point.

13. The method of claim 11 , wherein determining the at least one focus point in the scene comprises:
detecting at least one face in image data of the scene;
determining a location of the at least one face in image data of the scene; and
using the location of the at least one face as a focus point.

14. A method comprising:
receiving, by a computing system, image data of a scene, wherein the image data comprises image data captured by each of two or more image-capture systems of a computing device that includes a plurality of image-capture systems, and wherein the scene is divided into a plurality of segments, and wherein the image data of the scene comprises a first image captured by a first image-capture system and a second image captured by a second image-capture system;
using, by the computing system, the image data captured by the two or more image-capture systems as a basis for determining disparity information for the scene, wherein determining the disparity information comprises:
determining a first difference between the first image and the second image; determine a net contrast of the first difference;
generating one or more epipolar translations of the first image;
identifying a given epipolar translation from the one or more epipolar translations for which: (a) a net contrast of a difference between the given epipolar translation and the first image is closest to (b) the net contrast of the first difference; and
determining, based at least in part on the identified epipolar translation, disparity due to parallax between the first image and the second image; and

performing, by the computing system, an autofocus process based at least in part on the disparity

Assignees

• Google Inc

Inventors

• Lewkow Roman
• Geiss Ryan

Classifications

• H04N23/70

  Circuitry for compensating brightness variation in the scene · CPC title

• H04N23/45

  for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images · CPC title

• H04N23/673

  based on contrast or high frequency components of image signals, e.g. hill climbing method · CPC title

• H04N5/23212

  Electricity · mapped topic

• H04N13/0207

  Electricity · mapped topic