Automatic high beam control for autonomous machine applications

Patent metadata

Field	Value
Publication number	US-11613201-B2
Application number	US-202016991242-A
Country	US
Kind code	B2
Filing date	Aug 12, 2020
Priority date	Aug 12, 2019
Publication date	Mar 28, 2023
Grant date	Mar 28, 2023

Abstract

In various examples, high beam control for vehicles may be automated using a deep neural network (DNN) that processes sensor data received from vehicle sensors. The DNN may process the sensor data to output pixel-level semantic segmentation masks in order to differentiate actionable objects (e.g., vehicles with front or back lights lit, bicyclists, or pedestrians) from other objects (e.g., parked vehicles). Resulting segmentation masks output by the DNN(s), when combined with one or more post processing steps, may be used to generate masks for automated high beam on/off activation and/or dimming or shading—thereby providing additional illumination of an environment for the driver while controlling downstream effects of high beam glare for active vehicles.

First claim

What is claimed is:

1. A method comprising:
applying, to a neural network (NN), image data representative of an image of an environment;
computing, using the NN and based at least in part on the image data, first data representative of a segmentation mask classifying pixels of the image as corresponding to one or more active objects;
determining, based at least in part on the pixels, one or more regions of the environment where the one or more active objects are located; and
adjusting control parameters of a beam configuration to reduce illumination of the one or more regions.

2. The method of claim 1 , further comprising applying a spatial filter to the first data prior to the determining, wherein the segmentation mask is weighted based on other values corresponding to one or more prior images.

3. The method of claim 2 , wherein the spatial filter is a recursive Bayesian filter that receives, as input, the first data and second data representative of one or more prior segmentation masks classifying respective pixels of the one or more prior images that correspond to the one or more active objects.

4. The method of claim 1 , wherein the control parameters correspond to at least one of turning on or turning off the beam configuration corresponding to the one or more regions of the environment where the one or more active objects are located.

5. The method of claim 1 , wherein the control parameters correspond to at least one of dimming or turning off select light emitting diodes (LEDs) of a plurality of LEDs of the beam configuration, the select LEDs of the plurality of LEDs determined to correspond to the one or more regions of the environment where the one or more active objects are located.

6. The method of claim 1 , wherein the control parameters correspond to actuation settings for one or more motors configured for angling or directing lights of the beam configuration, wherein the actuation settings cause actuation of one or more of the lights of the beam configuration away from a direction corresponding to the one or more regions of the environment where the one or more active objects are located.

7. The method of claim 1 , further comprising applying a temporal filter to the first data prior to the determining.

8. A method comprising:
applying image data representative of a first image and a second image captured sequentially;
computing, based at least in part on the image data, first data representative of first confidence values for a set of pixels of the first image corresponding to an active object and second data representative of second confidence values for the set of pixels of the second image;
weighting the second confidence values based at least in part on the first confidence values to generate updated confidence values for the set of pixels;
determining final pixels of the second image that correspond to the active object based at least in part on the updated confidence values; and
adjusting control parameters of a beam configuration to reduce illumination of one or more regions of an environment determined based at least in part on the final pixels.

9. The method of claim 8 , wherein, prior to the weighting the second confidence values, the first confidence values are weighted based at least in part on other confidence values from one or more prior images.

10. The method of claim 8 , wherein the control parameters correspond to at least one of turning on or turning off of the beam configuration corresponding to the one or more regions of the environment determined based at least in part on the final pixels.

11. The method of claim 8 , wherein the control parameters correspond to at least one of dimming or turning off select light emitting diodes (LEDs) of a plurality of LEDs of the beam configuration, the select LEDs of the plurality of LEDs determined to correspond to the one or more regions of the environment determined based at least in part on the final pixels.

12. The method of claim 8 , wherein the control parameters correspond to actuation settings for one or more motors configured for angling or directing lights of the beam configuration, wherein the actuation settings cause actuation of one or more of the lights of the beam configuration away from a direction corresponding to the one or more regions of the environment determined based at least in part on the final pixels.

13. The method of claim 8 , wherein the weighting of the second confidence values is performed by applying the first data and the second data to a recursive Bayesian filter.

14. The method of claim 8 , further comprising applying the first data and the second data to a temporal filter that adjusts the first data and the second data using hysteresis gating.

15. A system comprising:
an image sensor;
a beam configuration; and
one or more processors to execute operations comprising:
applying, to a neural network (NN), image data representative of an image of an environment generated using the image sensor;
computing, using the NN and based at least in part on the image data, data representative of one or more segmentation masks, each segmentation mask of the one or more segmentation masks corresponding to an individual class;
determining, based at least in part on the one or more segmentation masks, pixels that correspond to one or more active objects;
determining, based at least in part on the pixels, one or more locations of the one or more active objects in the environment; and
adjusting control parameters of the beam configuration to reduce illumination of the one or more locations.

16. The system of claim 15 , wherein the operations are further comprising: applying a recursive Bayesian filter to the one or more segmentation masks to weight the one or more segmentation masks based at least in part on one or more prior segmentation masks computed by the NN based at least in part on one or more prior images.

17. The system of claim 15 , wherein the operations are further comprising: applying a single-frame post-processing filter to the one or more segmentation masks, wherein the single-frame post-processing filter clips low confidence predictions using a maximum likelihood estimation (MLE).

18. The system of claim 15 , wherein the control parameters correspond to at least one of turning on or turning off one or more lights of the beam configuration corresponding to illuminating of one or more regions of the environment where the one or more active objects are located.

19. The system of claim 15 , wherein the control parameters correspond to at least one of dimming or turn off select light emitting diodes (LEDs) of a plurality of LEDs of the beam configuration, wherein the select LEDs of the plurality of LEDs determined to correspond to illuminating of one or more regions of the environment where the one or more active objects are located.

20. The system of claim 15 , wherein the control parameters correspond to actuation settings for one or more motors configured for angling or directing lights of the beam configuration, wherein the actuation settings cause actuation of one or more of the lights of the beam configuration away from a direction corresponding to illuminating of one or more regions of the environment where the one or more active objects are located.

Assignees

• Nvidia Corp

Inventors

• Li Jincheng
• Park Minwoo

Classifications

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• H04N19/70

  characterised by syntax aspects related to video coding, e.g. related to compression standards · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title