Generation of artificial color images from narrow spectral band data aboard a camera-equipped vehicle

Patent metadata

Field	Value
Publication number	US-12223741-B2
Application number	US-202117557448-A
Country	US
Kind code	B2
Filing date	Dec 21, 2021
Priority date	Dec 21, 2021
Publication date	Feb 11, 2025
Grant date	Feb 11, 2025

Abstract

A visual perception system includes a scanning camera, color sensor with color filter array (CFA), and a classifier node. The camera captures full color pixel images of a target object, e.g., a traffic light, and processes the pixel images through a narrow band pass filter (BPF), such that the narrow BPF outputs monochromatic images of the target object. The color sensor and CFA receive the monochromatic images. The color sensor has at least three color channels each corresponding to different colors of spectral data in the monochromatic images. The classifier node uses a predetermined classification decision tree to classify constituent pixels of the monochromatic images into different color bins as a corresponding color of interest. The color of interest may be used to perform a control action, e.g., via an automated driver assist system (ADAS) control unit or an indicator device.

First claim

What is claimed is:

1. A visual perception system comprising:
a liquid crystal polarized gratings (LCPG) scanning camera configured to capture full color pixel images of a target object, and to process the full color pixel images through a narrow band pass filter (BPF), such that the narrow BPF outputs monochromatic pixel images of the target object, wherein the narrow BPF is centered on about 600 nanometers (nm), has a bandwidth of about 40 nm, and a full width half maximum (FWHM) of about 30 nm;
a color sensor equipped with a color filter array (CFA), wherein the color sensor and the CFA are configured to receive the monochromatic pixel images from the narrow BPF, and wherein the color sensor has at least three color channels each corresponding to different colors of spectral data in the monochromatic pixel images; and
a classifier node configured, using a predetermined classification decision tree, to classify constituent pixels of the monochromatic pixel images into one of a plurality of color bins as a corresponding color of interest, including calculating respective probabilities of the constituent pixels being red, yellow, or green as the corresponding color of interest, and thereafter outputting a color estimate having a maximum of the probabilities.

2. The visual perception system of claim 1 , further comprising an image processing node configured to artificially color the constituent pixels of the monochromatic pixel images with the corresponding color of interest to thereby generate a colorized image.

3. The visual perception system of claim 1 , wherein the classifier node is configured to determine a normalized digital number value per pixel per RGB color channel, and to classify each of the constituent pixels of the monochromatic pixel images as one of the red, the yellow, or the green by comparing the normalized digital number value to a scaled signal-to-noise ratio.

4. The visual perception system of claim 1 , wherein the color sensor includes an RGB color sensor.

5. The visual perception system of claim 1 , wherein the color sensor includes a Bayer RGGB sensor.

6. The visual perception system of claim 1 , wherein the predetermined classification decision tree includes manipulating two independent gain settings for each color channel of a single exposure, the two independent gain settings having different color balance parameters.

7. The visual perception system of claim 1 , wherein the predetermined classification decision tree includes processing multiple exposures of the full color pixel images, each of the multiple exposures having different time durations.

8. The visual perception system of claim 1 , wherein the visual perception system is configured to transmit an electronic signal indicative of the color of interest to an automated driver assistance system (ADAS) control unit of a motor vehicle.

9. The visual perception system of claim 1 , wherein the visual perception system is configured to transmit an electronic signal indicative of the color of interest to an indicator device to thereby cause to the indicator device to illuminate and/or broadcast audible sound.

10. The visual perception system of claim 1 , wherein the color sensor incudes a YCCB sensor.

11. A vehicle comprising:
a vehicle body;
an indicator device connected to the vehicle body; and
a visual perception system connected to the vehicle body and comprising:
a liquid crystal polarized gratings (LCPG) scanning camera configured to capture full color pixel images of a multi-color traffic light, and to process the full color pixel images through a narrow band pass filter (BPF), such that the narrow BPF outputs monochromatic pixel images of the multi-color traffic light, wherein the narrow BPF filter is centered on about 600 nanometers (nm), has a bandwidth of about 40 nm, and a full width half maximum (FWHM) of about 30 nm;
a color sensor equipped with a color filter array (CFA), wherein the color sensor and the CFA are configured to receive the monochromatic pixel images from the LCPG scanning camera, and wherein the color sensor has at least three color channels each corresponding to different colors of spectral data in the monochromatic pixel images; and
a classifier node configured, using a predetermined classification decision tree, to classify constituent pixels of the monochromatic pixel images into one of a plurality of color bins as a corresponding color of interest, including calculating respective probabilities of each of the constituent pixels being red, green, or yellow as the corresponding color of interest, and outputting a color estimate having a maximum of the probabilities, the plurality of color bins including red, green, and yellow, wherein the classifier node is configured to output an electronic signal to the indicator device to activate the indicator device, the output signal being indicative of the color of interest.

12. The vehicle of claim 11 , wherein the indicator device includes a heads-up display.

13. The vehicle of claim 11 , further comprising an automated driver assist system (ADAS) control unit operable for selectively controlling a dynamic state of the vehicle in response to the color of interest.

14. The vehicle of claim 11 , wherein the classifier node is configured to determine a normalized digital number value per pixel per RGB color channel, and to classify each of the constituent pixels of the monochromatic pixel images as one of at least three colors of interest, by comparing the normalized digital number value to a scaled signal-to-noise ratio.

15. The vehicle of claim 11 , wherein the predetermined classification decision tree includes manipulating two independent gain settings for each color channel of a single exposure, the two independent gain settings having different color balance parameters.

16. The vehicle of claim 11 , further comprising:
a set of road wheels connected to the vehicle body.

17. The vehicle of claim 11 , wherein the predetermined classification decision tree includes processing multiple exposures of the full color images, each of the multiple exposures having different time durations.

18. A method for use aboard a host platform having an indicator device and a visual perception system including a liquid crystal polarized gratings (LCPG) scanning camera, the method comprising:
capturing, via the (LCPG) scanning camera, full color pixel images of a multi-color traffic light;
processing the full color pixel images through a narrow band pass filter (BPF), such that the narrow BPF outputs monochromatic pixel images of the multi-color traffic light, wherein the narrow BPF is centered on about 600 nanometers (nm), has a bandwidth of about 40 nm, and a full width half maximum (FWHM) of about 30 nm;
receiving the monochromatic pixel images via a color sensor equipped with a color filter array (CFA), wherein the color sensor has at least three color channels each corresponding to different colors of spectral data in the monochromatic pixel images; and
classifying constituent pixels of the monochromatic pixel images into one of a plurality of color bins as a corresponding color of interest, using a classifier node, the plurality of color bins including red, green, and yellow, including calculating respective probabilities of each of the constituent pixels being red, yellow, or green as the corresponding color of interest; and
outputting an electronic signal to an indicator device to activate the indicator device, the electronic signal being indicative of the color of interest, wherein the color of interest is a col

Assignees

• Gm Global Tech Operations Llc

Inventors

• Philipp Tzvi
• Kishon Eran
• Slutsky Michael

Classifications

• B60W2420/403

  Image sensing, e.g. optical camera · CPC title

• G02B27/01 Primary

  Head-up displays · CPC title

• B60W2050/146

  Display means · CPC title

• G02B2027/0196

  having transparent supporting structure for display mounting, e.g. to a window or a windshield · CPC title

• G06V10/25

  Determination of region of interest [ROI] or a volume of interest [VOI] · CPC title