Quad color filter array image sensor with aperture simulation and phase detection

Patent metadata

Field	Value
Publication number	US-11245878-B2
Application number	US-201916664005-A
Country	US
Kind code	B2
Filing date	Oct 25, 2019
Priority date	Feb 28, 2019
Publication date	Feb 8, 2022
Grant date	Feb 8, 2022

Abstract

Aspects of the present disclosure relate to quad color filter array image sensors. A quad color filter array (CFA) image sensor is configured to capture image data. The quad CFA image sensor includes a quad CFA and an image sensor coupled to the quad CFA. The image sensor includes a plurality of quads including a plurality of pixels. For each quad, each pixel of the plurality of pixels is coupled to a same color filter of the plurality of color filters, a first pixel of the plurality of pixels is associated with a first exposure region of the quad and corresponds to a first aperture size, a second pixel of the plurality of pixels is associated with a second exposure region of the quad and corresponds to a second aperture size, and at least two pixels of the plurality of pixels are configured as phase detection pixels.

First claim

What is claimed is:

1. A device, comprising:
a quad color filter array (CFA) including a plurality of color filters;
an image sensor configured to capture image data, the image sensor coupled to the quad CFA, wherein the image sensor includes a plurality of quads including a respective plurality of pixels, wherein for first quad of the plurality of quads:
each pixel of a plurality of pixels of the first quad is coupled to a same color filter of the plurality of color filters;
a first pixel of the plurality of pixels of the first quad is associated with a first exposure region of the first quad and corresponds to a first aperture size;
a second pixel of the plurality of pixels of the first quad is associated with a second exposure region of the first quad and corresponds to a second aperture size; and
at least two pixels of the plurality of pixels of the first quad are configured as phase detection (PD) pixels; and

a processor implemented in circuitry and coupled to the image sensor, the processor configured to perform aperture simulation in processing one or more frames of captured image data to simulate capturing image data using at least one of the first aperture size, the second aperture size, or a third aperture size based on the at least two PD pixels.

2. The device of claim 1 , wherein the quad CFA is a quad Bayer CFA including a plurality of 4×4 color filter arrangements, a 4×4 color filter arrangement of the plurality of 4×4 color filter arrangements including:
a 2×2 patch of red color filters coupled to the first quad of the image sensor;
a 2×2 patch of blue color filters coupled to a second quad of the image sensor;
a first 2×2 patch of green color filters coupled to a third quad of the image sensor; and
a second 2×2 patch of green color filters coupled to a fourth quad of the image sensor, wherein the first quad, the second quad, the third quad, and the fourth quad are arranged in a Bayer color filter pattern.

3. The device of claim 1 , wherein each pixel of the image sensor includes one photosensitive surface for capturing image data.

4. The device of claim 1 , wherein the first pixel is coupled to a first mask including the first aperture size, wherein the first mask is configured to direct light to the first exposure region.

5. The device of claim 4 , wherein the second pixel is coupled to a second mask including the second aperture size, wherein the second mask is configured to direct light to the second exposure region.

6. The device of claim 1 , wherein the at least two PD pixels include:
a third pixel configured to capture data from a first perspective; and
a fourth pixel configured to capture data from a second perspective, wherein the third pixel and the fourth pixel are associated with a third exposure region of the first quad corresponding to a third aperture size.

7. The device of claim 6 , wherein the third pixel and the fourth pixel share a microlens configured to direct light to the third exposure region.

8. The device of claim 6 , wherein:
the first perspective and the second perspective associated with a first group of the plurality of quads are oriented in a first direction.

9. The device of claim 8 , wherein:
the first perspective and the second perspective associated with a second group of the plurality of quads are oriented in a second direction.

10. The device of claim 9 , wherein a portion of the first group and a portion of the second group of the plurality of quads are coupled to green color filters.

11. The device of claim 1 , wherein the processor is further configured to detect one or more phases from the captured image data.

12. The device of claim 11 , wherein the processor is configured to perform a depth-based operation based on the one or more phases, wherein the depth-based operation includes at least one of:
phase detection autofocus (PDAF);
determining a depth of an object from the image sensor;
generating a depth map; or
adjusting a depth of field associated with the image sensor.

13. The device of claim 1 , wherein:
the at least two PD pixels include:
a third pixel configured to capture image data from a first perspective; and
a fourth pixel configured to capture image data from a second perspective; and

the processor, in performing aperture simulation, is configured to associate captured image data from the third pixel and the fourth pixel with a third exposure region corresponding to a third aperture size.

14. The device of claim 13 , wherein the processor, in associating captured image data from the third pixel and the fourth pixel with the third exposure region, is configured to:
combine the captured image data from the third pixel and the captured image data from the fourth pixel; and
determine a difference between the combined captured image data and the captured image data from at least one of:
the first pixel; or
the second pixel.

15. The device of claim 14 , wherein the processor, in determining the difference, is configured to:
add the captured image data from the third pixel and the captured image data from the fourth pixel;
average the captured image data from the first pixel and the captured image data from the second pixel; and
determine a difference between the added captured image data and the averaged captured image data.

16. The device of claim 1 , wherein the processor is configured to generate a high dynamic range (HDR) image from at least one frame of captured image data based on the aperture simulation.

17. The device of claim 16 , wherein, to generate the HDR image, the processor is configured to:
associate captured image data from the first pixel associated with the first exposure region of each quad with a first frame of captured image data using the first aperture size;
associate captured image data from the second pixel associated with the second exposure region of each quad with a second frame of captured image data using the second aperture size; and
associate captured image data associated with a third exposure region of each quad with a third frame of captured image data using the third aperture size.

18. A method, comprising:
capturing one or more frames of image data by a quad color filter array (CFA) image sensor, wherein the quad CFA image sensor includes:
a quad CFA including a plurality of color filters; and
an image sensor configured to capture image data, the image sensor coupled to the quad CFA, wherein the image sensor includes a plurality of quads including a plurality of pixels, wherein for a first quad of the plurality of quads:
each pixel of a plurality of pixels of the first quad is coupled to a same color filter of the plurality of color filters;
a first pixel of the plurality of pixels of the first quad is associated with a first exposure region of the first quad and corresponds to a first aperture size;
a second pixel of the plurality of pixels of the first quad is associated with a second exposure region of the first quad and corresponds to a second aperture size; and
at least two pixels of the plurality of pixels of the first quad are configured as phase detection (PD) pixels; and

performing aperture simulation in processing the one or more frames of image data to simulate capturing image data using at least one of the first aperture size, the second aperture size, or a third aperture size based on the at least two PD pixels.

19. The method of claim 18 , wherein capturing the one or more frames of image data includes capturing the one or more frames of image data using a

Assignees

• Qualcomm Inc

Inventors

• Galor Gluskin Micha

Classifications

• H04N23/672

  based on the phase difference signals · CPC title

• G06T5/50 Primary

  using two or more images, e.g. averaging or subtraction · CPC title

• H04N25/134

  based on three different wavelength filter elements · CPC title

• H04N23/741

  by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors · CPC title

• H04N23/12

  with one sensor only · CPC title