Image-focusing method and associated image sensor

Patent metadata

Field	Value
Publication number	US-11539875-B1
Application number	US-202117459181-A
Country	US
Kind code	B1
Filing date	Aug 27, 2021
Priority date	Aug 27, 2021
Publication date	Dec 27, 2022
Grant date	Dec 27, 2022

Abstract

An autofocusing method includes capturing an image of a scene with a camera that includes a pixel array; computing a horizontal-difference image, and a vertical-difference image; and combining the horizontal-difference image and the vertical-difference image to yield a combined image. The method also includes determining, from the combined image and the intensity image, an image distance with respect to a lens of the camera at which the camera forms an in-focus image. The pixel array includes horizontally-adjacent pixel pairs and vertically-adjacent pixel pairs each located beneath a respective microlens. The horizontal-difference image includes, for each horizontally-adjacent pixel pair, a derived pixel value that is an increasing function of a difference between pixel values generated by the horizontally-adjacent pixel pair. The vertical-difference image includes, for each vertically-adjacent pixel pair, a derived pixel value that is an increasing function of a difference between pixel values generated by the vertically-adjacent pixel pair.

First claim

What is claimed is:

1. An autofocusing method comprising:
capturing an image of a scene with a camera that includes a pixel array, the pixel array including (i) a plurality of horizontally-adjacent pixel pairs, each being beneath a respective one of a first plurality of microlenses, and (ii) a plurality of vertically-adjacent pixel pairs, each being beneath either (a) a respective one of the first plurality of microlenses or (b) a respective one of a second plurality of microlenses;
computing a horizontal-difference image that includes, for each of the plurality of horizontally-adjacent pixel pairs, a first derived pixel value mapped to a location of the horizontally-adjacent pixel pair within the pixel array and being an increasing function of a difference between pixel values generated by each pixel of the horizontally-adjacent pixel pair;
computing a vertical-difference image that includes, for each of the plurality of vertically-adjacent pixel pairs, a second derived pixel value mapped to a location of the vertically-adjacent pixel pair within the pixel array and being an increasing function of a difference between pixel values generated by each pixel of the vertically-adjacent pixel pair;
combining the horizontal-difference image and the vertical-difference image to yield a combined image;
determining, from the combined image, an image distance with respect to a lens of the camera at which the camera forms an in-focus image of at least part of the scene; and
adjusting a distance between the pixel array and the lens until the distance equals the image distance.

2. The method of claim 1 , each of the plurality of vertically-adjacent pixel pairs being beneath a respective one of the first plurality of microlenses, each of the first plurality of microlenses being above a respective two-by-two pixel subarray, of a plurality of two-by-two pixel subarrays of the pixel array, each pixel of the two-by-two pixel subarray being both (i) a pixel of one of the plurality of horizontally-adjacent pixel pairs and (ii) a pixel of one of the plurality of vertically-adjacent pixel pairs.

3. The method of claim 2 , for each of the plurality of two-by-two pixel subarrays:
capturing comprising generating, with an image sensor that includes the pixel array, pixel values V 1 , V 2 , V 3 , and V 4 from a top-left pixel, a top-right pixel, a bottom-left pixel, and a bottom-right pixel, respectively, of the two-by-two pixel subarray;
computing the horizontal-difference image comprising determining the first derived pixel value as a first increasing function of (V 1 +V 3 )−(V 2 +V 4 );
computing the vertical-difference image comprising determining the second derived pixel value as a second increasing function of (V 3 +V 4 )−(V 1 +V 2 ); and
combining comprising determining a third derived pixel value, of the combined image, that is a third increasing function of one of (i) a sum of respective squares of the first derived pixel value and the second derived pixel value and (ii) a sum of respective absolute values of the first derived pixel value and the second derived pixel value.

4. The method of claim 3 , the third increasing function being a linear function of the arctangent of the square root of the sum of the respective squares of the first derived pixel value and the second derived pixel value.

5. The method of claim 3 , the increasing function including an arctangent function.

6. The method of claim 2 ,
capturing comprising generating, with an image sensor that includes the pixel array, pixel values V 1 , V 2 , V 3 , and V 4 from a top-left pixel, a top-right pixel, a bottom-left pixel, and a bottom-right pixel, respectively, of the two-by-two pixel subarray;
computing the vertical-difference image comprising determining the second derived pixel value as a first increasing function of (V 1 +V 3 )−V 2 +V 4 ))/(V 1 +V 2 +V 3 +V 4 );
computing the horizontal-difference image comprising determining the first derived pixel value as a second increasing function of ((V 3 +V 4 )−(V 1 +V 2 ))/(V 1 +V 2 +V 3 +V 4 ); and
combining comprising determining a third derived pixel value, of the combined image, that is a third increasing function of one of (i) a sum of respective squares of the first derived pixel value and the second derived pixel value and (ii) a sum of respective absolute values of the first derived pixel value and the second derived pixel value.

7. The method of claim 1 , determining comprising:
processing at least part of the combined image with a neural network to determine the image distance, the neural network having been trained via supervised learning based on training images captured by a training camera having a training-camera lens and a training-camera pixel array that are identical to the lens and the pixel array respectively.

8. The method of claim 1 , further comprising:
computing an intensity image that includes, for each of the plurality of horizontally-adjacent pixel pairs and a most proximate vertically-adjacent pixel pair of the plurality of vertically-adjacent pixel pairs thereto, a summation pixel value being (i) mapped to a location within the pixel array adjacent to one of the horizontally-adjacent pixel pair and the most proximate vertically-adjacent pixel pair and (ii) an increasing function of a sum of pixel values generated by each pixel of the horizontally-adjacent pixel pair and the most proximate vertically-adjacent pixel pair; and
determining the image distance comprising determining the image distance from the combined image and the intensity image.

9. The method of claim 8 , each of the plurality of vertically-adjacent pixel pairs being beneath a respective one of the first plurality of microlenses, each of the first plurality of microlenses being above a respective two-by-two pixel subarray, of a plurality of two-by-two pixel subarrays of the pixel array, each pixel of the two-by-two pixel subarray being both (i) a pixel of one of the plurality of horizontally-adjacent pixel pairs and (ii) a pixel of one of the plurality of vertically-adjacent pixel pairs.

10. The method of claim 9 , for each of the plurality of two-by-two pixel subarrays:
capturing comprising generating, with an image sensor that includes the pixel array, pixel values V 1 , V 2 , V 3 , and V 4 from a top-left pixel, a top-right pixel, a bottom-left pixel, and a bottom-right pixel, respectively, of the two-by-two pixel subarray;
computing the horizontal-difference image comprising determining the first derived pixel value as a first increasing function of (V 1 +V 3 )−(V 2 +V 4 );
computing the vertical-difference image comprising determining the second derived pixel value as a second increasing function of (V 3 +V 4 )−(V 1 +V 2 );
computing the intensity image comprising determining the summation pixel value as a third increasing function of (V 1 +V 2 +V 3 +V 4 ); and
the combining comprising determining a third derived pixel value, of the combined image, that is a third increasing function of one of (i) a sum of respective squares of the first derived pixel value and the second derived pixel value and (ii) a sum of respective absolute values of the first derived pixel value and the second derived pixel value.

11. The method of claim 10 , the third increasing function being a linear function of the arctangent of the square root of the sum of the respective squares of the first derived pixel value and the second derived pixel value.

12. The method of claim 10 , the increasing function including an arctangent function.

13. The method of claim 9 ,
capturing comprising generating, with an image sensor that includes the pixel array, pixel values V 1 , V 2

Assignees

• Omnivision Tech Inc

Inventors

• Fowler Boyd

Classifications

• H04N23/672 Primary

  based on the phase difference signals · CPC title

• G02B7/346

  using horizontal and vertical areas in the pupil plane, i.e. wide area autofocusing · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• H04N25/704 Primary

  Pixels specially adapted for focusing, e.g. phase difference pixel sets · CPC title

• H04N5/36961

  Electricity · mapped topic