Sensor data point cloud generation for map creation and localization for autonomous systems and applications

Patent metadata

Field	Value
Publication number	US-12416714-B2
Application number	US-202217655781-A
Country	US
Kind code	B2
Filing date	Mar 21, 2022
Priority date	Mar 18, 2022
Publication date	Sep 16, 2025
Grant date	Sep 16, 2025

Abstract

Embodiments of the present disclosure relate to generating RADAR (RAdio Detection And Ranging) point clouds based on RADAR data obtained from one or more RADAR sensors disposed on one or more ego-machines. In these or other embodiments, the RADAR point clouds may be used to generate map data. Additionally or alternatively, the RADAR point clouds may be used for performing localization.

First claim

What is claimed is:

1. A method comprising:
generating a RADAR point cloud based at least on RADAR data associated with a plurality of RADAR scans performed using one or more RADAR sensors, the generating of the RADAR point cloud including:
transforming the RADAR data from different RADAR scans into a common coordinate system;
removing one or more portions of the RADAR data that correspond to one or more dynamic objects;
filtering the RADAR data based at least on strengths of corresponding RADAR return signals with respect to a signal strength threshold that is one or more of:
a dynamic threshold that varies with respect to generation of different RADAR point clouds; or
based at least on a target number of RADAR data points to include in the RADAR point cloud; and

aggregating RADAR data sets of the RADAR data that remain after the filtering into the RADAR point cloud such that the RADAR point cloud includes a subset of the RADAR data and comprises an aggregation of multiple data sets corresponding to multiple RADAR scans.

2. The method of claim 1 , wherein the target number of RADAR data points is based at least on a target localization precision related to performing localization using the RADAR point cloud.

3. The method of claim 1 , wherein the one or more RADAR sensors is disposed on an ego-machine and wherein the transforming the RADAR data includes compensating for movement of the ego-machine between two or more RADAR scans of the plurality of RADAR scans.

4. The method of claim 1 , wherein the RADAR data is obtained from RADAR scans performed using the one or more RADAR sensors.

5. The method of claim 1 , wherein the removing the one or more portions of the RADAR data that correspond to one or more dynamic objects includes performing object tracking to identify one or more of the one or more dynamic objects.

6. A system comprising:
one or more circuits to:
obtain RADAR data associated with one or more RADAR scans performed using one or more RADAR sensors, the RADAR data including a plurality of individual RADAR data points respectively corresponding to individual RADAR reflections corresponding to individual locations in an environment of an ego-machine;
select RADAR data points of the RADAR data based at least on a target number of RADAR data points to include in a RADAR point cloud; and
generate the RADAR point cloud based at least on the RADAR data points as selected such that the RADAR point cloud comprises an aggregation of one or more RADAR data sets corresponding to the one or more RADAR scans.

7. The system of claim 6 , wherein the one or more circuits are further to determine a signal strength threshold based at least on the target number of RADAR data points, and wherein selecting of the RADAR data points is based at least on the signal strength threshold.

8. The system of claim 7 , wherein the signal strength threshold is dynamically determined based at least on a relationship between a total number of RADAR data points included in the RADAR data and the target number of RADAR data points.

9. The system of claim 6 , wherein the target number of RADAR data points is based at least on a target localization precision related to performing localization using the RADAR point cloud.

10. The system of claim 6 , wherein the generating the RADAR point cloud further includes transforming the RADAR data into a common coordinate system.

11. The system of claim 10 , wherein the one or more RADAR sensors are disposed on the ego-machine and wherein the transforming the RADAR data includes compensating for movement of the ego-machine between two or more scans.

12. The system of claim 6 , wherein the generating the RADAR point cloud further includes removing one or more portions of the RADAR data that correspond to one or more dynamic objects.

13. The system of claim 6 , wherein the one or more RADAR sensors are disposed on the ego-machine, and wherein the RADAR data is used to generate the RADAR point cloud in response to RADAR scans that generate the RADAR data individually being performed using individual RADAR sensors of the one or more RADAR sensors.

14. A system comprising:
one or more processing units to:
obtain RADAR data associated with one or more RADAR scans using one or more RADAR sensors;
select RADAR data points of the RADAR data based at least on a signal strength threshold, the signal strength threshold being determined based at least on a target number of RADAR data points to include in a RADAR point cloud; and
generate the RADAR point cloud based at least on the RADAR data points as selected.

15. The system of claim 14 , wherein the target number of RADAR data points is determined based at least on one or more of: a target resolution of the RADAR point cloud, a target data size, one or more RADAR map parameters, one or more RADAR localization parameters, a target spatial coverage of the RADAR point cloud, or a target angular coverage of the RADAR point cloud.

16. The system of claim 14 , wherein the generating the RADAR point cloud further includes transforming the RADAR data into a common coordinate system.

17. The system of claim 14 , wherein the generating the RADAR point cloud further includes removing one or more portions of the RADAR data that correspond to one or more dynamic objects.

18. The system of claim 14 , wherein the one or more RADAR sensors are disposed on an ego-machine and wherein the RADAR data is used to generate the RADAR point cloud in response to RADAR scans that generate the RADAR data individually being performed using individual RADAR sensors of the one or more RADAR sensors.

19. The system of claim 14 , wherein the system comprises one or more of:
a control system for an autonomous or semi-autonomous machine;
a perception system for an autonomous or semi-autonomous machine;
a system that performs simulation operations;
a system that performs deep learning operations;
a system that generates synthetic data;
a system that generates multi-dimensional assets using a collaborative content platform;
a system implemented using an edge device;
a system implemented using a robot;
a system incorporating one or more virtual machines (VMs);
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources.

Assignees

• Nvidia Corp

Inventors

• Akbarzadeh Amir
• Carley Andrew
• Henke Birgit
• Lu Si
• Stojanovic Ivana
• Agrawal Jugnu
• Kroepfl Michael
• Sheng Yu
• Nister David
• Zheng Enliang

Classifications

• B60W2420/408

  Radar; Laser, e.g. lidar · CPC title

• G01S7/003

  Transmission of data between radar, sonar or lidar systems and remote stations · CPC title

• G01S2013/9316

  combined with communication equipment with other vehicles or with base stations · CPC title

• G01S13/86

  Combinations of radar systems with non-radar systems, e.g. sonar, direction finder · CPC title

• G01S7/40

  Means for monitoring or calibrating · CPC title