Auto-focus control using image statistics data with coarse and fine auto-focus scores

Patent metadata

Field	Value
Publication number	US-9398205-B2
Application number	US-87397810-A
Country	US
Kind code	B2
Filing date	Sep 1, 2010
Priority date	Sep 1, 2010
Publication date	Jul 19, 2016
Grant date	Jul 19, 2016

Abstract

Techniques are provided for determining an optimal focal position using auto-focus statistics. In one embodiment, such techniques may include generating coarse and fine auto-focus scores for determining an optimal focal length at which to position a lens associated with the image sensor. For instance, the statistics logic may determine a coarse position that indicates an optimal focus area which, in one embodiment, may be determined by searching for the first coarse position in which a coarse auto-focus score decreases with respect to a coarse auto-focus score at a previous position. Using this position as a starting point for fine score searching, the optimal focal position may be determined by searching for a peak in fine auto-focus scores. In another embodiment, auto-focus statistics may also be determined based on each color of the Bayer RGB, such that, even in the presence of chromatic aberrations, relative auto-focus scores for each color may be used to determine the direction of focus.

First claim

What is claimed is:

1. An image signal processing system comprising:
a front-end pixel processing unit configured to receive a frame of raw image data comprising pixels acquired using an imaging device having a digital image sensor, wherein the front-end pixel processing unit comprises a statistics collection engine having auto-focus statistics logic configured to process the raw image data to collect coarse auto-focus statistics and fine auto-focus statistics, the coarse auto-focus statistics based at least in part on down-scaled image data relating to the raw image data, wherein the down-scaled image data comprises image data reduced in size by a down-scale ratio, and the fine auto-focus statistics based at least in part on the raw image data; and
control logic configured to determine an optimal focal position of a lens of the imaging device using coarse auto-focus scores based upon the coarse auto-focus statistics and fine auto-focus scores based upon the coarse and fine auto-focus statistics and to adjust the focal position of the lens between a minimum position and a maximum position defining a total focal length to reach the optimal focal position based at least in part upon the coarse auto-focus scores and the fine auto-focus scores.

2. The image signal processing system of claim 1 , wherein the control logic is configured to determine the optimal focal position of the lens by:
stepping the focal position across a plurality of coarse score positions relating to the coarse auto-focus scores along the total focal length in a first direction beginning from the minimum position and ending at the maximum position;
determining a coarse auto-focus score for each of the plurality of coarse score positions;
identifying which of the plurality of coarse score positions has a corresponding coarse auto-focus score that decreased relative to a coarse auto-focus score corresponding to the immediately previous coarse score position;
beginning from the identified coarse score position, stepping the focal position in a second direction opposite the first direction and back towards the minimum position across a plurality of fine score positions relating to the fine auto-focus scores;
determining a fine auto-focus score for each of the plurality of fine score positions; and
identifying which of the plurality of fine score positions corresponds to a peak in the fine auto-focus scores and setting the identified fine score position as the optimal focal position.

3. The image signal processing system of claim 2 , wherein the step size between each of the plurality of coarse score positions is greater than the step size between each of the plurality of fine score positions.

4. The image signal processing system of claim 2 , wherein the step size between each of the coarse score positions is variable based at least partially upon the magnitude of the change in coarse auto-focus scores corresponding to adjacent coarse score positions.

5. The image signal processing system of claim 4 , wherein the step size between coarse score positions decreases as the magnitude of the change in coarse auto-focus scores corresponding to the coarse score positions decreases.

6. The image signal processing system of claim 2 , wherein the control logic is configured to adjust the focal position of the lens using a coil, and wherein the control logic steps across coarse score positions along the total focal length to counter the effects of coil settling times.

7. The image signal processing system of claim 1 , wherein the auto-focus statistics logic is configured to provide the coarse auto-focus statistics by applying first and second filters to least one of camera luma values derived from the raw image data after being decimated or from decimated raw image data and to provide the fine auto-focus statistics by either applying third and fourth filters to luma values obtained by applying a transform to the raw image data or by applying horizontal filtering to the raw image data.

8. The image signal processing system of claim 7 , wherein the coarse auto-focus scores at each coarse position is determined based at least partially upon the sum of the outputs of the first and second filters, and wherein the fine auto-focus scores at each fine position is determined based at least partially upon the sums of the outputs of the third and fourth filters.

9. The image signal processing system of claim 7 , wherein the first and second filters for filtering the camera luma values comprise 3×3 filters based upon Scharr operators.

10. A method comprising:
receiving raw image data acquired using a digital image sensor, the raw image data representing an image scene and having red, blue, and green color components;
determining auto-focus scores comprising coarse auto-focus scores relating to scaled raw image data, wherein the down-scaled image data comprises image data reduced in size by a down-scale ratio, and fine auto-focus scores relating to un-scaled image data, the coarse auto-focus scores and the fine auto-focus scores corresponding to each of the red, blue, and green color components; and
selecting a focal adjustment direction based at least in part upon the relativity of the auto-focus scores corresponding to each of the red, blue, and green color components.

11. The method of claim 10 , wherein an optimal focal position for the image scene corresponds to a position where the green color components has an optimal focal position.

12. The method of claim 11 , comprising selecting the focal adjustment direction such that, if the red or blue color components exhibit a higher auto-focus score than the green color component at a particular focal position, focusing in the selected focal adjustment direction lowers the auto-focus score for the red or blue color components while increases the auto-focus score for the green color component.

13. The method of claim 10 , wherein the raw image data comprises Bayer RGB data.

14. The method of claim 10 , wherein the auto-focus scores corresponding to each of the red, blue, and green color components are obtained based upon statistics acquired by applying a multi-tap horizontal filter to the raw image data.

15. The method of claim 14 , wherein the multi-tap horizontal filter comprises a seven-tap filter.

16. A method comprising:
determining coarse auto-focus scores, relating to scaled image data, at various steps along a focal length of a lens of an image capture device;
identifying a step at which a corresponding auto-focus score of the coarse auto-focus scores decreases relative to a previous step;
identifying a optimal focal area in the vicinity of a focal position based at least in part upon the step at which the corresponding auto-focus score decreases; and
analyzing fine auto-focus scores, relating to un-scaled image data, within the optimal focal area to determine an optimal focal position for the lens;
wherein the down-scaled image data comprises image data reduced in size by a down-scale ratio.

17. The method of claim 16 , wherein determining analyzing fine auto-focus scores within the optimal focal area to determine the optimal focal position comprises searching a focal position that provides a maximum fine auto-focus score within the optimal focal area.

18. The method of claim 16 , wherein the coarse auto-focus scores and the fine auto-focus scores are based at least in part on white balanced luma derived from Bayer RGB data.

19. The method of claim 18 , wherein the white balanced luma values for the coarse auto-focus scores are deri

Assignees

• Côté Guy
• Frederiksen Jeffrey E
• Apple Inc

Inventors

• Côté Guy
• Frederiksen Jeffrey E

Classifications

• H04N23/631

  Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters · CPC title

• H04N23/843

  Demosaicing, e.g. interpolating colour pixel values · CPC title

• H04N25/134

  based on three different wavelength filter elements · CPC title

• H04N25/61

  the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4" · CPC title

• H04N23/673 Primary

  based on contrast or high frequency components of image signals, e.g. hill climbing method · CPC title