Image optimization techniques using depth planes

Patent metadata

Field	Value
Publication number	US-9704250-B1
Application number	US-201414529009-A
Country	US
Kind code	B1
Filing date	Oct 30, 2014
Priority date	Oct 30, 2014
Publication date	Jul 11, 2017
Grant date	Jul 11, 2017

Abstract

A plurality of depth planes corresponding to a scene within a field of view of a camera of a computing device can be computed based on stereographic techniques, infrared transceivers, ultrasonic transceivers, among other approaches. The depth planes can be utilized to independently meter subject matter of interest (e.g., person, object, region, etc.) corresponding to each depth plane to obtain image statistics for each depth plane. The image statistics can be analyzed to determine optimal settings (e.g., focus, exposure, white balance, etc.) for each depth plane, and a camera can be configured to acquire image data according to the optimal settings for one or more of the depth planes. Further, the image statistics can be leveraged for processing portions of the image data that are not optimized during capture. Thus, a final image can reflect optimizations for multiple subject matter of interest located at various distances from a computing device.

First claim

What is claimed is:

1. A computing device, comprising:
a processor;
a first panchromatic camera;
a second panchromatic camera;
a color camera; and
memory including instructions that, upon being executed by the processor, causes the computing device to:
acquire first image data using the first panchromatic camera and second image data using the second panchromatic camera;
determine a plurality of depth planes by analyzing differences in disparity in feature points of the first image data and corresponding feature points of the second image data;
acquire third image data using the color camera;
determine a prioritized depth plane from among the plurality of depth planes;
determine image statistics for at least the prioritized depth plane using the third image data, the image statistics corresponding to measurements for at least one of focus, or exposure;
determine a setting for the color camera based on the image statistics with respect to at least one of a specified focus value, or a specified exposure value; and
acquire an image using the color camera configured with the setting.

2. The computing device of claim 1 , wherein the instructions, upon being executed, further cause the computing device to:
determine second image statistics for at least one other depth plane of the plurality of planes using the third image data, the second image statistics corresponding to second measurements for the at least one of focus, or exposure; and
determine pixel values of at least a portion of the image corresponding to the at least one other depth plane by evaluating the second image statistics with respect to the at least one of the specified focus value, or the specified exposure value.

3. The computing device of claim 1 , wherein the instructions, upon being executed, further cause the computing device to:
determine second image statistics for at least one other depth plane of the plurality of planes using the third image data, the second image statistics corresponding to second measurements for the at least one of focus, or exposure;
determine a second setting for the color camera by evaluating the second image statistics with respect to the at least one of the specified focus value, or the specified exposure value;
acquire a second image using the color camera with the second setting; and
generate a combined image using at least a portion of the image corresponding to the at least the prioritized depth plane and a portion of the second image corresponding to the at least one other depth plane.

4. A computer-implemented method comprising:
acquiring first image data associated with a first image resolution using a first camera of a computing device;
acquiring second image data using a second camera of the computing device;
determining a plurality of depth planes based on differences in disparity in feature points associated with the first image data and corresponding feature points associated with the second image data;
acquiring third image data, associated with a second image resolution greater than the first image resolution, using at least one camera of the computing device;
determining a prioritized depth plane from among the plurality of depth planes;
determining, from the third image data, image statistics based on pixel values associated with the prioritized depth plane,
determining a first setting for the at least one camera based at least in part on the image statistics; and
acquiring fourth image data using the at least one camera configured with the first setting.

5. The computer-implemented method of claim 4 , further comprising:
determining, from the fourth image data, second image statistics based on second pixel values associated with a second depth plane; determining a second setting for the at least one camera based at least in part on the second image statistics; and
acquiring fifth image data using the at least one camera configured with the second setting.

6. The computer-implemented method of claim 5 , further comprising:
generating combined image data using at least a portion of the fourth image data corresponding to the prioritized depth plane and a portion of the fifth image data corresponding to the second depth plane.

7. The computer-implemented method of claim 4 , further comprising:
acquiring the second image data using a panchromatic camera as the second camera.

8. The computer-implemented method of claim 4 , further comprising:
acquiring the second image data using an infrared camera as the second camera.

9. The computer-implemented method of claim 4 , further comprising: determining the pixel values associated with the prioritized depth plane using a fill algorithm based on similarities in at least one of depth, distance, luminosity, or color among the pixel values.

10. The computer-implemented method of claim 4 , further comprising: determining a second prioritized depth plane from the plurality of depth planes;
determining, from the third image data, second image statistics based on second pixel values associated with the second prioritized depth plane; and
determining the first setting further based at least in part on the second image statistics.

11. The computer-implemented method of claim 4 , further comprising: identifying boundaries of representations of targets of interest associated with the prioritized depth plane;
determining, from the third image data, pixels within the boundaries; and
determining the pixel values associated with the prioritized depth plane from values of the pixels within the boundaries.

12. The computer-implemented method of claim 4 , further comprising:
determining, from the third image data, an average of a red channel corresponding to the prioritized depth plane;
determining, from the third image data, an average of a blue channel corresponding to the prioritized depth plane;
determining, from the third image data, an average of a green channel corresponding to the prioritized depth plane; determining a first color gain based on the average of the green channel and the average of the red channel;
determining a second color gain based on the average of the green channel and the average of the blue channel;
applying the first color gain to the red channel; and
applying the second color gain to the blue channel.

13. The computer-implemented method of claim 4 , further comprising: determining, from the third image data, a maximum pixel value of a red channel corresponding to the prioritized depth plane;
determining a maximum pixel value of a blue channel corresponding to the prioritized depth plane;
determining a maximum pixel value of a green channel corresponding to the prioritized depth plane; determining a first color gain based on the maximum pixel value of the green channel and the maximum pixel value of the red channel;
determining a second color gain based on the maximum pixel value of the green channel and the maximum pixel value of the blue channel;
applying the first color gain to the red channel; and
applying the second color gain to the blue channel.

14. The computer-implemented method of claim 4 , further comprising:
receiving a selection of the prioritized depth plane by a user of the computing device.

15. The computer-implemented method of claim 4 , further comprising:
determining the prioritized depth plane based on at least one of detecting a face represented in a portion of the third image data corresponding to the prioritized depth plane, determining an object represented in the portion of the third image data corresponding the prioritized depth plane is a specified di

Assignees

• Amazon Tech Inc

Inventors

• Shah Vivek
• Hanson Robert James
• Gilmour Nicholas Ryan

Classifications

• H04N23/71

  Circuitry for evaluating the brightness variation · CPC title

• H04N23/632

  for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters · CPC title

• H04N23/741

  by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors · CPC title

• H04N23/676

  Bracketing for image capture at varying focusing conditions · CPC title

• H04N23/12

  with one sensor only · CPC title