Imaging systems with visible light sensitive pixels and infrared light sensitive pixels

Patent metadata

Field	Value
Publication number	US-10136107-B2
Application number	US-201314086705-A
Country	US
Kind code	B2
Filing date	Nov 21, 2013
Priority date	Nov 21, 2013
Publication date	Nov 20, 2018
Grant date	Nov 20, 2018

Abstract

An image sensor may have an array of image pixels arranged in color filter unit cells that each have at least one red image pixel that generates red image signals, at least one blue image pixel that generate blue image signals, at least one clear image pixels that generate clear image signals, at least one infrared image pixel that generates infrared image signals, and optionally at least one green image pixel that generates green image signals. The image sensor may be coupled to processing circuitry that performs chroma demosaicking operations on the image signals. The processing circuitry may generate an infrared image using the infrared image signals and a luminance value using the clear, red, blue, and infrared image signals. The processing circuitry may perform point filter operations on the image signals based on the generated luminance value to produce corrected visible light image signals having improved image quality.

First claim

What is claimed is:

1. An imaging system, comprising:
processing circuitry; and
an array of physical image sensor pixels, wherein the array of physical image sensor pixels comprises:
a first group of physical image sensor pixels configured to generate first image signals in response to light of a first color, wherein the first color is selected from a group consisting of: red, green, and blue;
a second group of physical image sensor pixels configured to generate second image signals in response to light of a second color that is different from the first color, wherein the second color is selected from a group consisting of: red, green, and blue;
a third group of physical image sensor pixels configured to generate third image signals in response to light of at least two colors, wherein the at least two colors are selected from a group consisting of: red, green, and blue; and
a group of infrared physical image sensor pixels configured to generate infrared image signals in response to only infrared light, wherein the array of physical image sensor pixels comprises a plurality of repeating unit cells of physical image sensor pixels, wherein each unit cell in the plurality of unit cells comprises eight physical image sensor pixels arranged in first and second rows, the first row comprises a first pair of physical image sensor pixels from the third group and first and second infrared physical image sensor pixels from the group of infrared physical image sensor pixels, and the second row comprises a second pair of physical image sensor pixels from the third group, a physical image sensor pixel from the first group, and a physical image sensor pixel from the second group, and wherein the processing circuitry is configured to use the infrared image signals to correct the first, second, and third image signals.

2. The imaging system defined in claim 1 , wherein each unit cell in the plurality of unit cells comprises four consecutive columns in the array, the first infrared physical image sensor pixel is formed in the second column, and the second infrared physical image sensor pixel is formed in the fourth column.

3. The imaging system defined in claim 1 , wherein the first group of physical image sensor pixels comprises red physical image sensor pixels, and wherein the second group of physical image sensor pixels comprises blue physical image sensor pixels.

4. The imaging system defined in claim 1 , wherein the processing circuitry is configured to perform filtering operations on the first, second, and third image signals that increase noise correlations associated with first, second, and third image signals.

5. The imaging system defined in claim 1 , wherein the first group of physical image sensor pixels comprises red physical image sensor pixels, wherein the second group of physical image sensor pixels comprises blue physical image sensor pixels, and wherein the third group of physical image sensor pixels comprises clear physical image sensor pixels having clear color filter elements.

6. A method of generating images with an imaging system, wherein the imaging system comprises an array of physical image sensor pixels arranged in a plurality of repeating unit cells of physical image sensor pixels and processing circuitry, wherein each unit cell in the plurality of unit cells comprises first, second, third, and fourth physical image sensor pixels in a first row and fifth, sixth, seventh, and eighth physical image sensor pixels in a second row that is adjacent to the first row, the method comprising:
with the second physical image sensor pixel, capturing first image signals in response to light of a first color;
with the fifth physical image sensor pixel, capturing second image signals in response to light of a second color that is different from the first color;
with the seventh physical image sensor pixel, capturing third image signals in response to light of a third color that is different from the first and second colors;
with the first, third, sixth, and eighth physical image sensor pixels, capturing fourth image signals in response to light of at least two colors selected from the group of colors consisting of the first color, the second color, and the third color;
with the fourth physical image sensor pixel, capturing infrared image signals in response to only infrared light;
with the processing circuitry, performing chroma demosaicking operations on the captured first, second, third, and fourth image signals that increase noise correlations associated with the first, second, third, and fourth image signals such that noise fluctuations in the first, second, third, and fourth image signals increase and decrease together in a correlated manner to form first, second, third, and fourth sets of image data;
with the processing circuitry, generating fourth image signals of a fourth color using the infrared image signals;
with the processing circuitry, combining the first image signals, the second image signals, the third image signals, and the fourth image signals to form a first luminance value;
with the processing circuitry, computing a second luminance value based on the first image signals, the second image signals, the third image signals, and the fourth image signals; and
with the processing circuitry, modifying the first image signals, the second image signals, the third image signals, and the fourth image signals using the first and second luminance values.

7. The method defined in claim 6 , further comprising:
with the processing circuitry, performing infrared light contamination subtraction operations on the first, second, third, and fourth sets of image data.

8. The method defined in claim 6 , further comprising:
with the processing circuitry, applying a color correction matrix to the first, second, third, and fourth sets of image data to generate standard red, standard green, and standard blue image data; and
with the processing circuitry, performing point filter operations on the standard red, standard green, and standard blue image data.

9. The method defined in claim 8 , further comprising:
with the processing circuitry, performing the point filter operations on the standard red, standard green, and standard blue image data based on the generated first and second luminance values.

10. A system, comprising:
a central processing unit;
memory;
input-output circuitry; and
an imaging device, wherein the imaging device comprises:
an array of image pixels arranged in rows and columns, wherein the array of image pixels comprises a first group of image pixels configured to generate first pixel values in response to light of a first color, a second group of image pixels configured to generate second pixel values in response to light of a second color, a group of infrared image pixels configured to generate infrared pixel values in response to only infrared light, and a group of broadband image pixels configured to generate broadband pixel values in response to light of at least two colors, wherein the group of broadband image pixels is arranged in a checkerboard pattern on the array of image pixels such that each row and each column in the array comprises at least one broadband image pixel from the group of broadband image pixels.

11. The system defined in claim 10 , wherein the at least two colors are selected from a group consisting of: the first color, the second color, and a third color that is different from the first and second colors.

12. The imaging system defined in claim 2 , wherein the physical image sensor pixel from the first group is formed in the first column and the image sensor pixel from the second group is formed in the third column.

Assignees

• Semiconductor Components Ind Llc

Inventors

• Keelan Brian
• Mlinar Marko

Classifications

• H04N25/135

  based on four or more different wavelength filter elements · CPC title

• H04N25/131 Primary

  including elements passing infrared wavelengths · CPC title

• H04N25/133

  including elements passing panchromatic light, e.g. filters passing white light · CPC title

• H04N23/843

  Demosaicing, e.g. interpolating colour pixel values · CPC title

• H04N9/64

  Circuits for processing colour signals (H04N9/77 takes precedence; camera processing pipelines for processing colour signals H04N23/84) · CPC title