Depth determination using camera focus

Patent metadata

Field	Value
Publication number	US-2016248968-A1
Application number	US-201615141710-A
Country	US
Kind code	A1
Filing date	Apr 28, 2016
Priority date	Mar 6, 2013
Publication date	Aug 25, 2016
Grant date	—

Abstract

A computing device can capture a plurality of images using a camera of the device, each image being captured with a different focus setting of the camera. In some embodiments, the capturing the plurality of images can be performed during an autofocus process of the camera. The device can determine depth information, such as a position of relative depth, for each of the plurality of images based on the state of the camera when each image was captured. Depth information for any object(s) in focus in a respective one of the plurality of images can be determined to correspond to the depth information for the respective image.

First claim

What is claimed is:

1 . A computer-implemented method, comprising:
capturing a first image at a first focus setting using a camera of a computing device; capturing a second image at a second focus setting using the camera; identifying, in the first image, a first portion of an object, wherein the first portion has a first sharpness score that exceeds a sharpness threshold, the sharpness threshold indicating an amount of image detail conveyed; identifying, in the second image, a second portion of the object, the second portion having an associated second sharpness score that exceeds the sharpness threshold; determining first depth information for the first image based at least in part on the first focus setting; determining second depth information for the second image based at least in part on the second focus setting; and determining a relative depth for the object based on the first depth information and the second depth information.

2 . The computer-implemented method of claim 1 , wherein the first depth information and the second depth information includes a position of the object within a field of view of the camera.

3 . The computer-implemented method of claim 1 , wherein the first depth information and the second depth information comprises a relative depth of the object within a field of view of the camera.

4 . The computer-implemented method of claim 1 , further comprising:
generating a graphical element indicating the relative depth for the object.

5 . The computer-implemented method of claim 1 , further comprising:
generating a depth map for the object based on the relative depth of the object.

6 . The computer-implemented method of claim 1 , further comprising:
identifying, in the first image, a third portion of a second object; identifying, in the second image, a fourth portion of the second object; determining third depth information for the second object in the first image based at least in part on the first focus setting; determining fourth depth information for the second object in the second image based at least in part on the second focus setting; determining a second relative depth for the second portion of the second object based on the third depth information and the fourth depth information; and generating a depth map for the object and the second object, the depth map indicating the relative depth for the object and the second relative depth for the second object with respect to the camera.

7 . The computer-implemented method of claim 1 , further comprising:
associating the object with a virtual plane, the virtual plane being selected based on a focus distance of a lens of the camera, the virtual plane indicating a depth away from the camera at which the object is located.

8 . The method of claim 1 , wherein determining a relative depth for the object based on the first depth information and the second depth information comprises determining a position specifying relative depth for the object in the first image as sequentially compared to the second image.

9 . A computer-implemented method, comprising:
capturing a first image at a first focus setting using a camera of a computing device; capturing a second image at a second focus setting using the camera; identifying, in a first portion of the first image, a first object, wherein the first portion has a first sharpness score that exceeds a sharpness threshold, the sharpness threshold indicating an amount of detail conveyed; identifying, in a second portion of the second image, a second obj ect, wherein the second portion has a second sharpness score that exceeds the sharpness threshold; determining first depth information for the first image based at least in part on the first focus setting; determining second depth information for the second image based at least in part on the second focus setting; and determining a depth of the first object relative to the second object based on the first depth information and the second depth information.

10 . The computer-implemented method of claim 9 , further comprising:
generating a depth map by comparing the first depth information and the second depth information in sequential order for the first object, and specifying relative depth for the first object in each image.

11 . The computer-implemented method of claim 9 , further comprising:
generating a depth map by indicating a location of the first object in a field of view of the camera and indicating relative positions for other objects in the field of view of the camera.

12 . The computer-implemented method of claim 9 , further comprising:
generating a plurality of depth maps for each of a plurality of fields of view of the camera based on the first depth information and the second depth information; and joining each of the plurality of depth maps together to form a single depth map.

13 . The computer-implemented method of claim 9 , further comprising:
associating the first object with a virtual plane; displaying a graphical representation of the relative depth of the first object on the virtual plane, wherein the first object rendered in the graphical representation is capable of occluding other objects rendered in the graphical representation that are positioned farther away from the camera

14 . A computing device comprising:
a camera; a processor; and a memory including instructions that, when executed by the processor, cause the computing device to:
capture a first image at a first focus setting using the camera;
capture a second image at a second focus setting using the camera;
identify, in the first image, a first portion of an object, wherein the first portion has a first sharpness score that exceeds a sharpness threshold, the sharpness threshold indicating an amount of image detail conveyed;
identify, in the second image, a second portion of the object in the second image, the second portion having an associated second sharpness score that exceeds the sharpness threshold;
determine first depth information for the first image based at least in part on the first focus setting;
determine second depth information for the second image based at least in part on the second focus setting; and
determine a relative depth for the object based on the first and the second depth information.

15 . The computing device of claim 14 , wherein the first depth information and the second depth information includes one of: a position of the object within a field of view of the camera, or a relative depth of the object with respect to the first image and the second image.

16 . The computing device of claim 14 , wherein the instructions further cause the computing device to:
generate a depth map by comparing the first depth information and the second depth information in sequential order for the object, and specifying relative depth for the object in each image.

17 . The computing device of claim 14 , wherein the instructions further cause the computing device to:
generate a depth map by indicating a location of the object in a field of view of the camera and indicating relative positions for other objects in the field of view of the camera.

18 . The computing device of claim 14 , wherein the instructions further cause the computing device to:
generate a plurality of depth maps for each of a plurality of fields of view of the camera based on the first depth information and the second depth information; and joining each of the plurality of depth maps together to form a single depth map.

Assignees

• Amazon Tech Inc

Inventors

• Baldwin Leo Benedict

Classifications

• H04N23/673 Primary

  based on contrast or high frequency components of image signals, e.g. hill climbing method · CPC title

• A63F13/65 Primary

  automatically by game devices or servers from real world data, e.g. measurement in live racing competition · CPC title

• H04N23/63

  by using electronic viewfinders · CPC title

• G06T7/0051

  Physics · mapped topic

• H04N5/23212 Primary

  Electricity · mapped topic