Annotation of radar-profiles of objects

Patent metadata

Field	Value
Publication number	US-11726176-B2
Application number	US-202117231685-A
Country	US
Kind code	B2
Filing date	Apr 15, 2021
Priority date	Aug 29, 2018
Publication date	Aug 15, 2023
Grant date	Aug 15, 2023

Abstract

A radar-data collection system a radar, a camera, and a controller-circuit. The radar and the camera are intended for mounting on a host-vehicle. The radar is configured is to indicate a radar-profile of an object detected by the radar. The camera is configured to render an image of the object. The controller-circuit is in communication with the radar and the camera. The controller is configured to determine an identity of the object in accordance with the image, and annotate the radar-profile in accordance with the identity.

First claim

We claim:

1. A system comprising a controller circuit configured to:
receive sensor data that captures an object proximate a vehicle, the sensor data comprising image data, radar data, and lidar data;
determine, based on the lidar data, the radar data, and an image of the object from the image data, a sensor profile including a multi-dimensional representation of the sensor data relative to a coordinate system of the vehicle;
annotate the sensor profile by at least labeling the multi-dimensional representation of the sensor data with the image of the object in addition to an orientation of the object including an orientation angle of the object, the orientation of the object being derived from the image data, and the orientation angle being derived from the radar data;
determining, based at least in part on the orientation angle, a distance and a direction to the object;
responsive to determining the distance and the direction to the object based on the sensor profile, further annotate the sensor profile by further labeling the multi-dimensional representation of the sensor data with the distance and the direction to the object;
receive additional sensor data including additional radar data and additional lidar data that each capture another object proximate the vehicle;
determine, based on the additional radar data and the additional lidar data, another sensor profile including another multi-dimensional representation of the additional sensor data relative to the coordinate system of the vehicle;
annotate the other sensor profile by at least labeling the other multi-dimensional representation of the additional sensor data with an image of the other object and an orientation of the other object derived from the additional sensor data; and
responsive to determining another distance and another direction to the other object based on the other sensor profile, further annotate the other sensor profile by further labeling the other multi-dimensional representation of the additional sensor data with the other distance and the other direction to the other object.

2. The system in accordance with claim 1 , wherein the controller circuit is further configured to:
determine a reflection intensity map in accordance with the sensor profile; and
further annotate the sensor profile in accordance with the reflection intensity map.

3. The system in accordance with claim 1 , wherein the controller circuit is configured to:
determine, based on the radar data, an elevation to the object in accordance with the sensor profile; and
further annotate the sensor profile by labeling the multi-dimensional representation of the sensor data with the elevation to the object.

4. The system in accordance with claim 1 , wherein the controller is further configured to:
determine, based on the sensor data, a pose angle of the vehicle; and
further annotate the sensor profile by labeling the multi-dimensional representation of the sensor data with the pose angle of the vehicle.

5. The system in accordance with claim 1 , wherein the controller circuit is further configured to operate the vehicle in accordance with the sensor profile in response to a determination that a sensor map corresponds to the sensor profile.

6. The system in accordance with claim 1 , wherein the controller circuit is configured to upload the sensor profile to a database for use in operating one or more other vehicles in accordance with the sensor profile.

7. The system in accordance with claim 1 , wherein the controller circuit is further configured to further annotate the sensor profile by at least labeling the multi-dimensional representation of the sensor data with an object classification identifying whether the object comprises another vehicle.

8. A method, comprising:
receiving, by a controller circuit of a vehicle, sensor data that captures an object proximate the vehicle, the sensor data comprising image data, radar data, and lidar data;
determining, based on the lidar data, the radar data, and an image of the object from the image data, a sensor profile including a multi-dimensional representation of the sensor data relative to a coordinate system of the vehicle;
annotating the sensor profile by at least labeling the multi-dimensional representation of the sensor data with the image of the object in addition to an orientation of the object including an orientation angle of the object, the orientation of the object being derived from the image data, and the orientation angle being derived from the radar data;
determine, based at least in part on the orientation angle, a distance and a direction to the object;
responsive to determining the distance and the direction to the object based on the sensor profile, further annotating, by the controller circuit, the sensor profile by further labeling the multi-dimensional representation of the sensor data with the distance and the direction to the object;
receiving additional sensor data including additional radar data and additional lidar data that each capture another object proximate the vehicle;
determining, based on the additional radar data and the additional lidar data, another sensor profile including another multi-dimensional representation of the additional sensor data relative to the coordinate system of the vehicle;
annotating the other sensor profile by at least labeling the other multi-dimensional representation of the additional sensor data with an image of the other object and an orientation of the other object derived from the additional sensor data; and
responsive to determining another distance and another direction to the other object based on the other sensor profile, further annotate the other sensor profile by further labeling the other multi-dimensional representation of the additional sensor data with the other distance and the other direction to the object.

9. The method of claim 8 , further comprising:
further annotating the sensor profile by at least labeling the multi-dimensional representation of the sensor data with an object classification identifying whether the object comprises another vehicle.

10. The method of claim 8 , further comprising:
determining a reflection intensity map in accordance with the sensor profile; and
further annotating the sensor profile in accordance with the reflection intensity map.

11. The method of claim 8 , further comprising:
operating the vehicle in accordance with the sensor profile in response to determining that a sensor map corresponds to the sensor profile.

12. The method of claim 8 , further comprising:
uploading the sensor profile to a database for use in operating one or more other vehicles in accordance with the sensor profile.

13. A system, comprising a controller circuit configured to:
receive lidar data representing detections of an object proximate a host vehicle;
receive image data representing an image captured of the object;
determine an identity of the object based on the image data;
determine, based on the lidar data and the image data, a lidar profile including a two-dimensional representation of the lidar data relative to a host vehicle coordinate system and the image of the object;
determine an orientation of the object from the lidar profile or the image;
annotate the lidar profile in accordance with the identity and the orientation by labeling the lidar profile with the image, the identity, and the orientation of the object;
determine a distance and a direction to the object in accordance with the lidar profile; and
further annotate the lidar profile in accordance with the distance and the direction.

14. The system i

Assignees

• Aptiv Tech Ltd

Inventors

• Laur Michael H
• Vijayan Indu
• Balachandran Divya
• Mangal Nandita

Classifications

• G06F40/117

  Tagging; Marking up (details of markup languages G06F40/143); Designating a block; Setting of attributes (style sheets, e.g. eXtensible Stylesheet Language Transformation [XSLT], G06F40/154) · CPC title

• G01S13/931 Primary

  of land vehicles · CPC title

• G01S13/867

  Combination of radar systems with cameras · CPC title

• G01S13/58 Primary

  Velocity or trajectory determination systems; Sense-of-movement determination systems · CPC title

• G01S7/411 Primary

  Identification of targets based on measurements of radar reflectivity (G01S7/415 takes precedence) · CPC title