Rgb-ir pixel pattern conversion via adaptive filtering

Patent metadata

Field	Value
Publication number	US-2024289920-A1
Application number	US-202318175697-A
Country	US
Kind code	A1
Filing date	Feb 28, 2023
Priority date	Feb 28, 2023
Publication date	Aug 29, 2024
Grant date	—

Abstract

Various embodiments disclosed herein relate to pixel pattern conversion, and more specifically to using an adaptive filter to convert complex pixel data to non-complex pixel formats. An image processing pipeline is provided herein that comprises an upstream component, a pattern conversion component downstream with respect to the upstream component in the image processing pipeline, and a downstream component relative to the pattern conversion component. The pattern conversion component is configured to obtain RGB-IR pixel data produced by the upstream component of the image processing pipeline, convert the RGB-IR pixel data into RGB pixel data and IR pixel data using an adaptive filter, and supply the RGB pixel data and the IR pixel data to the downstream component of the image processing pipeline.

First claim

What is claimed is:

1 . An image processing pipeline, comprising:
an upstream component; a pattern conversion component downstream with respect to the upstream component in the image processing pipeline; and a downstream component relative to the pattern conversion component; wherein the pattern conversion component is configured to:
obtain RGB-IR pixel data produced by the upstream component of the image processing pipeline;
convert the RGB-IR pixel data into RGB pixel data and IR pixel data using an adaptive filter; and
supply the RGB pixel data and IR pixel data to the downstream component of the image processing pipeline.

2 . The image processing pipeline of claim 1 , wherein the adaptive filter is configured to:
for each pixel of the RGB-IR pixel data:
identify features of the pixel based on a context of the pixel;
calculate weights for the pixel based on the features; and
apply a weight to the pixel to calculate an interpolated value for the pixel.

3 . The image processing pipeline of claim 2 , wherein the adaptive filter is further configured to convert the RGB-IR pixel data into the RGB pixel data and the IR pixel data using interpolated values of the pixels of the RGB-IR pixel data.

4 . The image processing pipeline of claim 2 , wherein the context comprises a characteristic of the RGB-IR pixel data at a location associated with a given pixel and a characteristic of neighboring pixels, with respect to the given pixel, of the RGB-IR pixel data.

5 . The image processing pipeline of claim 2 , wherein the applied weight is selected based on a color channel of a given pixel and a desired output pixel array including either the RGB pixel data or the IR pixel data.

6 . The image processing pipeline of claim 5 , wherein the color channel of the given pixel includes one among red, blue, green, or infrared.

7 . The image processing pipeline of claim 2 , wherein the adaptive filter is further configured to:
for each pixel of the RGB-IR pixel data:
identify a contamination value of the pixel; and
subtract the contamination value from the interpolated value of the pixel.

8 . The image processing pipeline of claim 1 , wherein the adaptive filter comprises a convolutional neural network.

9 . A pattern conversion component in an image processing pipeline, comprising:
an interface; and circuitry coupled to the interface; wherein the interface is configured to communicate with upstream components and downstream components of the image processing pipeline relative to the pattern conversion component; and wherein the circuitry is configured to:
obtain, via the interface, RGB-IR pixel data produced by an upstream component of the image processing pipeline;
convert the RGB-IR pixel data into RGB pixel data and IR pixel data using an adaptive filter; and
supply, via the interface, the RGB pixel data and the IR pixel data to a downstream component of the image processing pipeline.

10 . The pattern conversion component of claim 9 , wherein the circuitry is configured to:
for each pixel of the RGB-IR pixel data:
identify features of the pixel based on a context of the pixel;
calculate weights for the pixel based on the features; and
apply a weight to the pixel to calculate an interpolated value for the pixel.

11 . The pattern conversion component of claim 10 , wherein the circuitry is further configured to convert the RGB-IR pixel data into the RGB pixel data and the IR pixel data using interpolated values of the pixels of the RGB-IR pixel data.

12 . The pattern conversion component of claim 10 , wherein the context comprises a characteristic of the RGB-IR pixel data at a location associated with a given pixel and a characteristic of neighboring pixels of the RGB-IR pixel data with respect to the given pixel.

13 . The pattern conversion component of claim 10 , wherein the applied weight is selected based on a color channel of a given pixel and a desired output pixel array including either the RGB pixel data or the IR pixel data.

14 . The pattern conversion component of claim 13 , wherein the color channel of the given pixel includes one among red, blue, green, or infrared.

15 . The pattern conversion component of claim 10 , wherein the circuitry is further configured to:
for each pixel of the RGB-IR pixel data:
identify a contamination value of the pixel; and
subtract the contamination value from the interpolated value of the pixel.

16 . The pattern conversion component of claim 9 , wherein the circuitry comprises a convolutional neural network.

17 . A method of operating a pattern conversion component of an image processing pipeline, the method comprising:
obtaining RGB-IR pixel data produced by a component of the image processing pipeline upstream relative to the pattern conversion component; converting the RGB-IR pixel data into RGB pixel data and IR pixel data by:
identifying features of pixels of the RGB-IR pixel data based on context of the pixels;
calculating weights for the pixels based on the features; and
applying a weight to each of the pixels; and
supplying the RGB pixel data and IR pixel data to a downstream component of the image processing pipeline relative to the pattern conversion component.

18 . The method of claim 17 , wherein the context comprises a characteristic of the RGB-IR pixel data at a location associated with a given pixel and a characteristic of neighboring pixels of the RGB-IR pixel data with respect to the given pixel.

19 . The method of claim 17 , wherein the applied weight is selected based on a color channel of a given pixel and a desired output pixel array including either the RGB pixel data or the IR pixel data.

20 . The method of claim 18 , wherein the method further comprises:
identifying contamination values of the pixels of the RGB-IR pixel data; and subtracting the contamination values from the pixels of the RGB pixel data.

Assignees

• Texas Instruments Inc

Inventors

• Garud Hrushikesh
• Mody Mihir
• Allu Rajasekhar
• Ren Jing-Fei
• Nandan Niraj

Classifications

• H04N23/843

  Demosaicing, e.g. interpolating colour pixel values · CPC title

• H04N25/135

  based on four or more different wavelength filter elements · CPC title

• H04N25/13 Primary

  characterised by the spectral characteristics of the filter elements · CPC title

• G06T1/20

  Processor architectures; Processor configuration, e.g. pipelining · CPC title

• G06T3/4015 Primary

  Image demosaicing, e.g. colour filter arrays [CFA] or Bayer patterns · CPC title