Universal and adaptive de-mosaicing (CFA) system

Patent metadata

Field	Value
Publication number	US-11949995-B2
Application number	US-202217895191-A
Country	US
Kind code	B2
Filing date	Aug 25, 2022
Priority date	Dec 2, 2015
Publication date	Apr 2, 2024
Grant date	Apr 2, 2024

Abstract

A method of de-mosaicing pixel data from an image processor includes generating a pixel block that includes a plurality of image pixels. The method also includes determining a first image gradient between a first set of pixels of the pixel block and a second image gradient between a second set of pixels of the pixel block. The method also includes determining a first adaptive threshold value based on intensity of a third set of pixels of the pixel block. The pixels of the third set of pixels are adjacent to one another. The method also includes filtering the pixel block in a vertical, horizontal, or neutral direction based on the first and second image gradients and the first adaptive threshold value utilizing a plurality of FIR filters to generate a plurality of component images.

First claim

What is claimed is:

1. A processor comprising:
a de-mosaic circuit configured to:
receive pixel data including visible light data and infrared data;
determine first gradient data of the visible light data;
determine second gradient data of the infrared data;
determine a first adaptive threshold value of the visible light data;
determine a second adaptive threshold value of the infrared data; and
filter the pixel data based on the first gradient data, the second gradient data, the first adaptive threshold value, and the second adaptive threshold value to generate component images; and

a color correction circuit coupled to the de-mosaic circuit, wherein the color correction circuit is configured to:
receive the component images;
correct the component images; and
output a color image.

2. The processor of claim 1 , wherein:
the color correction circuit is configured to correct the component images by combining the component images into the color image.

3. The processor of claim 1 , wherein:
the color correction circuit is configured to correct the component images by white balancing the component images.

4. The processor of claim 1 , wherein:
the first gradient data includes a first vertical gradient and a first horizontal gradient; and
the second gradient data includes a second vertical gradient and a second horizontal gradient.

5. The processor of claim 4 , wherein:
the de-mosaic circuit is configured to determine a horizontal edge, a vertical edge, or a lack of edges in the pixel data.

6. The processor of claim 5 , wherein:
in response to the first horizontal gradient greater than the first vertical gradient plus the first adaptive threshold value, the de-mosaic circuit is configured to determine the horizontal edge is present in the pixel data;
in response to the first vertical gradient greater than the first horizontal gradient plus the first adaptive threshold value, the de-mosaic circuit is configured to determine the vertical edge is present in the pixel data; and
in response to the first horizontal gradient not greater than the first vertical gradient plus the first adaptive threshold value and the first vertical gradient not greater than the first horizontal gradient plus the first adaptive threshold value, the de-mosaic circuit is configured to determine the lack of edges in the pixel data.

7. The processor of claim 5 , wherein:
in response to the second horizontal gradient greater than the second vertical gradient plus the second adaptive threshold value, the de-mosaic circuit is configured to determine the horizontal edge is present in the pixel data;
in response to the second vertical gradient greater than the second horizontal gradient plus the second adaptive threshold value, the de-mosaic circuit is configured to determine the vertical edge is present in the pixel data; and
in response to the second horizontal gradient not greater than the second vertical gradient plus the second adaptive threshold value and the second vertical gradient not greater than the second horizontal gradient plus the second adaptive threshold value, the de-mosaic circuit is configured to determine the lack of edges in the pixel data.

8. The processor of claim 6 , wherein:
in response to the de-mosaic circuit determining the horizontal edge is present in the pixel data, the de-mosaic circuit filters the pixel data using a vertical finite impulse response (FIR) filter;
in response to the de-mosaic circuit determining the vertical edge is present in the pixel data, the de-mosaic circuit filters the pixel data using a horizontal FIR filter; and
in response to the de-mosaic circuit determining the lack of edges in the pixel data, the de-mosaic circuit filters the pixel data using a neutral FIR filter.

9. The processor of claim 5 , wherein:
in response to the first horizontal gradient greater than the first vertical gradient times a weighted value plus the first adaptive threshold value, the de-mosaic circuit is configured to determine the horizontal edge is present in the pixel data;
in response to the first vertical gradient greater than the first horizontal gradient times the weighted value plus the first adaptive threshold value, the de-mosaic circuit is configured to determine the vertical edge is present in the pixel data; and
in response to the first horizontal gradient not greater than the first vertical gradient times the weighted value plus the first adaptive threshold value and the first vertical gradient not greater than the first horizontal gradient times the weighted value plus the first adaptive threshold value, the de-mosaic circuit is configured to determine the lack of edges in the pixel data.

10. The processor of claim 5 , wherein:
in response to the second horizontal gradient greater than the second vertical gradient times a weighted value plus the second adaptive threshold value, the de-mosaic circuit is configured to determine the horizontal edge is present in the pixel data;
in response to the second vertical gradient greater than the second horizontal gradient times the weighted value plus the second adaptive threshold value, the de-mosaic circuit is configured to determine the vertical edge is present in the pixel data; and
in response to the second horizontal gradient not greater than the second vertical gradient times the weighted value plus the second adaptive threshold value and the second vertical gradient not greater than the second horizontal gradient times the weighted value plus the second adaptive threshold value, the de-mosaic circuit is configured to determine the lack of edges in the pixel data.

11. The processor of claim 9 , wherein:
in response to the de-mosaic circuit determining the horizontal edge is present in the pixel data, the de-mosaic circuit filters the pixel data using a vertical finite impulse response (FIR) filter;
in response to the de-mosaic circuit determining the vertical edge is present in the pixel data, the de-mosaic circuit filters the pixel data using a horizontal FIR filter; and
in response to the de-mosaic circuit determining the lack of edges in the pixel data, the de-mosaic circuit filters the pixel data using a neutral FIR filter.

12. The processor of claim 7 , wherein:
in response to the de-mosaic circuit determining the horizontal edge is present in the pixel data, the de-mosaic circuit filters the pixel data using a vertical finite impulse response (FIR) filter;
in response to the de-mosaic circuit determining the vertical edge is present in the pixel data, the de-mosaic circuit filters the pixel data using a horizontal FIR filter; and
in response to the de-mosaic circuit determining the lack of edges in the pixel data, the de-mosaic circuit filters the pixel data using a neutral FIR filter.

13. The processor of claim 10 , wherein:
in response to the de-mosaic circuit determining the horizontal edge is present in the pixel data, the de-mosaic circuit filters the pixel data using a vertical finite impulse response (FIR) filter;
in response to the de-mosaic circuit determining the vertical edge is present in the pixel data, the de-mosaic circuit filters the pixel data using a horizontal FIR filter; and
in response to the de-mosaic circuit determining the lack of edges in the pixel data, the de-mosaic circuit filters the pixel data using a neutral FIR filter.

14. A method comprising:
receiving pixel data including visible light data and infrared data;
determining first gradient data of the visible light data;
determining second gradient data of the infrared data;
determining a first adaptive threshold value of the visib

Assignees

• Texas Instruments Inc

Inventors

• Dabral Shashank
• Mody Mihir Narendra
• Beaudoin Denis
• Nandan Niraj
• Hua Gang

Classifications

• H04N23/843 Primary

  Demosaicing, e.g. interpolating colour pixel values · CPC title

• G06T3/4015 Primary

  Image demosaicing, e.g. colour filter arrays [CFA] or Bayer patterns · CPC title

• G06T3/403

  Edge-driven scaling; Edge-based scaling · CPC title

• G06T5/00

  Image enhancement or restoration · CPC title

• G06T2207/20012

  Locally adaptive · CPC title