Visibility distance estimation using deep learning in autonomous machine applications

Patent metadata

Field	Value
Publication number	US-12570282-B2
Application number	US-202117449306-A
Country	US
Kind code	B2
Filing date	Sep 29, 2021
Priority date	Sep 29, 2021
Publication date	Mar 10, 2026
Grant date	Mar 10, 2026

Abstract

In various examples, systems and methods are disclosed that use one or more machine learning models (MLMs)—such as deep neural networks (DNNs)—to compute outputs indicative of an estimated visibility distance corresponding to sensor data generated using one or more sensors of an autonomous or semi-autonomous machine. Once the visibility distance is computed using the one or more MLMs, a determination of the usability of the sensor data for one or more downstream tasks of the machine may be evaluated. As such, where an estimated visibility distance is low, the corresponding sensor data may be relied upon for less tasks than when the visibility distance is high.

First claim

What is claimed is:

1 . A processor comprising processing circuitry to:
determine one or more predetermined ranges of values corresponding to visibility of a sensor, the one or more predetermined ranges of values including at least a predetermined range of values that is associated with one or more operations of an ego-machine; generate, using a machine learning model and based at least on sensor data obtained using the sensor of the ego-machine, a first output indicating a visibility distance corresponding to the sensor; determine, based at least on the visibility distance, the predetermined range of values that includes at least a minimum value corresponding to the sensor and a maximum value corresponding to the sensor; determine a second output using the sensor data, the second output being associated with a distance value; determine, based at least on whether the distance value is within the predetermined range of values, a usability of the second output for the one or more operations of the ego-machine; and perform at least one operation of the one or more operations based at least on the usability of the second output.

2 . The processor of claim 1 , wherein the processing circuitry is further to determine to refrain from performing at least one autonomous or semi-autonomous operation of the one or more operations based at least on the usability of the second output.

3 . The processor of claim 1 , wherein:
the predetermined range of values is associated with a visibility distance bin from a plurality of visibility distance bins corresponding to the sensor; and the visibility distance bin is further indicative of the usability of the sensor data for the one or more operations of the ego-machine.

4 . The processor of claim 1 , wherein the one or more operations include at least one of object tracking, object detection, path planning, obstacle avoidance, or an advanced driver assistance system (ADAS) operation.

5 . The processor of claim 1 , wherein the machine learning model includes a deep neural network (DNN), the DNN trained using a combination of real-world data, augmented real-world data, and synthetic data.

6 . The processor of claim 1 , wherein the processor is included in at least one of:
a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources.

7 . The processor of claim 1 , wherein the determination of the usability of the second output for the one or more operations of the ego-machine comprises determining, based at least on the distance value being within the predefined range of values, that the second output is usable for the one or more operations of the ego-machine.

8 . The processor of claim 1 , wherein the determination of the usability of the second output for the one or more operations of the ego-machine comprises determining, based at least on the distance value being outside the predetermined range of values, that the second output is unusable for the one or more operations of the ego-machine.

9 . The processor of claim 1 , wherein the determination of the predetermined range of values comprises:
determining that the visibility distance is between the minimum value and the maximum value included in the predetermined range of values from the one or more predetermined ranges of values; and determine, based at least on the visibility distance being between the minimum value and the maximum value, the predetermined range of values that includes at least the minimum value corresponding to the sensor and the maximum value corresponding to the sensor.

10 . A system comprising:
one or more processors to:
determine a plurality of distance value ranges associated with a visibility corresponding to a machine, at least a distance value range of the plurality of distance value ranges being associated with one or more operations of the machine;
determine, using a machine learning model and based at least on sensor data obtained using a sensor of the machine, the distance value range that indicates one or more visibility distances corresponding to the sensor;
determine an output using the sensor data, the output being associated with a distance value;
determine, based at least on whether the distance value is within the distance value range, a usability of the output to perform the one or more operations associated with the distance value range;
and
cause the machine to perform at least one operation from the one or more operations.

11 . The system of claim 10 , wherein the one or more operations are associated with a visibility distance bin corresponding to the distance value range.

12 . The system of claim 10 , wherein the system is included in at least one of:
a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources.

13 . The system of claim 10 , wherein the determination of the distance value range comprises:
determine, using the machine learning model and based at least on the sensor data, a second visibility distance; and determine, based at least on the second visibility distance being within the distance value range, the distance value range.

14 . A method comprising:
determine one or more predetermined ranges of values corresponding to visibility of a sensor, the one or more predetermined ranges of values including at least a predetermined range of values that is associated with one or more operations of an ego-machine; determining using a machine learning model and based at least on sensor data obtained using the sensor of the ego-machine, a first output representing the predetermined range of values that includes at least a minimum value corresponding to the sensor and a maximum value corresponding to the sensor; determining a second output using the sensor data, the second output being associated with a distance value; determining, based at least on whether the distance value is within the predetermined range of values, a usability of the second output for the one or more operations of the ego-machine; and performing at least one operation of the one or more operations based at least on the usability of the second output.

15 . The method of claim 14 , further comprising determining to refrain from performing at least one autonomous or semi-autonomous operation of the one or more operations based at least on the usability of the second output.

16 . The method of claim 14 , wherein:
the predetermined range of values is associated with a visibility distance bin from a plurality of visibility distance bins corresponding to the sensor; and the visibility distance bin is further indicative of the usability of the sensor data for the one or more operations of the ego-machine.

17 . The method of claim

Assignees

• Nvidia Corp

Inventors

• Bajpayee Abhishek
• Gupta Arjun
• Tang George
• Seo Hae-Jong

Classifications

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• B60W40/02

  related to ambient conditions · CPC title

• G06N3/08

  Learning methods · CPC title

• B60W2555/20

  Ambient conditions, e.g. wind or rain · CPC title

• B60W30/09

  Taking automatic action to avoid collision, e.g. braking and steering · CPC title