System and method for ultrasonic sensor enhancement using lidar point cloud

Patent metadata

Field	Value
Publication number	US-2023141590-A1
Application number	US-202217972820-A
Country	US
Kind code	A1
Filing date	Oct 25, 2022
Priority date	Nov 10, 2021
Publication date	May 11, 2023
Grant date	—

Abstract

A system and method for USS reading enhancement using a lidar point cloud. This provides noise reduction and enables the generation of a 2D environmental map. More specifically, the present disclosure provides a system and method for generating an enhanced environmental map using USSs, and the map is enhanced using a lidar point cloud. Using the lidar point cloud has advantages because the lidar point cloud is accurate and thus can provide accurate labels for training and the like.

First claim

What is claimed is:

1 . A method for ultrasonic sensor reading enhancement using a lidar point cloud, the method comprising:
receiving an ultrasonic sensor temporal feature using an ultrasonic sensor; inputting the ultrasonic sensor temporal feature into an autoencoder system comprising instructions stored in a memory and executed by a processor; wherein the autoencoder system is trained using a prior inputted ultrasonic sensor temporal feature and a corresponding prior inputted lidar feature label received from a lidar system; and using the trained autoencoder system, outputting an enhanced ultrasonic sensor environmental mapping.

2 . The method of claim 1 , wherein the ultrasonic sensor temporal feature comprises a 1D environmental map with relatively more noise and the enhanced ultrasonic sensor environmental mapping comprises a 2D environmental map with relatively less noise.

3 . The method of claim 1 , wherein the prior inputted ultrasonic sensor temporal feature is formed by performing ultrasonic sensor data feature extraction using inertial measurement unit data across N frames and a kinematic bicycle model to generate an ego vehicle trajectory, and, for each position in the ego vehicle trajectory, calculating a reflection point in an environment based on a yaw angle and each ultrasonic sensor reading, thereby providing one environmental mapping across the N frames for the ego vehicle trajectory.

4 . The method of claim 3 , wherein the data feature extraction further comprises, for a trajectory cut based on an ultrasonic sensor's field of view, using the environmental mapping from one ultrasonic sensor, as well as a same mapping from the lidar system.

5 . The method of claim 1 , wherein the prior inputted lidar feature label is formed by performing lidar point cloud feature generation by filtering lidar points by height and by a field of view of an ultrasonic sensor.

6 . The method of claim 5 , wherein the lidar point cloud feature generation further comprises finding closest lidar points to an ego vehicle by splitting the field of view of the ultrasonic sensor into angles centered at the ultrasonic sensor and, within each angle, selecting a constant number of lidar points that are closest to the ego vehicle, wherein a third dimension of the selected points is discarded, thereby providing a the lidar feature with a total number of the selected points that matches the inputted ultrasonic sensor temporal feature.

7 . The method of claim 1 , further comprising, at a vehicle control system, receiving the outputted an enhanced ultrasonic sensor environmental mapping and directing operation of a vehicle based on the outputted an enhanced ultrasonic sensor environmental mapping.

8 . A non-transitory computer-readable medium comprising instructions stored in a memory and executed by a processor to carry out steps for ultrasonic sensor reading enhancement using a lidar point cloud, the steps comprising:
receiving an ultrasonic sensor temporal feature using an ultrasonic sensor; inputting the ultrasonic sensor temporal feature into an autoencoder system comprising instructions stored in a memory and executed by a processor; wherein the autoencoder system is trained using a prior inputted ultrasonic sensor temporal feature and a corresponding prior inputted lidar feature label received from a lidar system; and using the trained autoencoder system, outputting an enhanced ultrasonic sensor environmental mapping.

9 . The non-transitory computer-readable medium of claim 8 , wherein the ultrasonic sensor temporal feature comprises a 1D environmental map with relatively more noise and the enhanced ultrasonic sensor environmental mapping comprises a 2D environmental map with relatively less noise.

10 . The non-transitory computer-readable medium of claim 8 , wherein the prior inputted ultrasonic sensor temporal feature is formed by performing ultrasonic sensor data feature extraction using inertial measurement unit data across N frames and a kinematic bicycle model to generate an ego vehicle trajectory, and, for each position in the ego vehicle trajectory, calculating a reflection point in an environment based on a yaw angle and each ultrasonic sensor reading, thereby providing one environmental mapping across the N frames for the ego vehicle trajectory.

11 . The non-transitory computer-readable medium of claim 10 , wherein the data feature extraction further comprises, for a trajectory cut based on an ultrasonic sensor's field of view, using the environmental mapping from one ultrasonic sensor, as well as a same mapping from the lidar system.

12 . The non-transitory computer-readable medium of claim 8 , wherein the prior inputted lidar feature label is formed by performing lidar point cloud feature generation by filtering lidar points by height and by a field of view of an ultrasonic sensor.

13 . The non-transitory computer-readable medium of claim 12 , wherein the lidar point cloud feature generation further comprises finding closest lidar points to an ego vehicle by splitting the field of view of the ultrasonic sensor into angles centered at the ultrasonic sensor and, within each angle, selecting a constant number of lidar points that are closest to the ego vehicle, wherein a third dimension of the selected points is discarded, thereby providing a the lidar feature with a total number of the selected points that matches the inputted ultrasonic sensor temporal feature.

14 . A system for ultrasonic sensor reading enhancement using a lidar point cloud, the system comprising:
an ultrasonic sensor operable for generating an ultrasonic sensor temporal feature; and an autoencoder system comprising instructions stored in a memory and executed by a processor, the autoencoder system operable for receiving the ultrasonic sensor temporal feature from the ultrasonic sensor and outputting an enhanced ultrasonic sensor environmental mapping; wherein the autoencoder system is trained using a prior inputted ultrasonic sensor temporal feature and a corresponding prior inputted lidar feature label generated by a lidar system.

15 . The system of claim 14 , wherein the ultrasonic sensor temporal feature comprises a 1D environmental map with relatively more noise and the enhanced ultrasonic sensor environmental mapping comprises a 2D environmental map with relatively less noise.

16 . The system of claim 14 , wherein the prior inputted ultrasonic sensor temporal feature is formed by performing ultrasonic sensor data feature extraction using inertial measurement unit data across N frames and a kinematic bicycle model to generate an ego vehicle trajectory, and, for each position in the ego vehicle trajectory, calculating a reflection point in an environment based on a yaw angle and each ultrasonic sensor reading, thereby providing one environmental mapping across the N frames for the ego vehicle trajectory.

17 . The system of claim 16 , wherein the data feature extraction further comprises, for a trajectory cut based on an ultrasonic sensor's field of view, using the environmental mapping from one ultrasonic sensor, as well as a same mapping from the lidar system.

18 . The system of claim 14 , wherein the prior inputted lidar feature label is formed by performing lidar point cloud feature generation by filtering lidar points by height and by a field of view of an ultrasonic sensor.

19 . The system of claim 18 , wherein the lidar point cloud feature generation further comprises finding closest lidar points to an ego vehicle by splitting

Assignees

• Volvo Car Corp

Inventors

• Li Ying
• Liu Zhipeng
• Ding Sihao

Classifications

• G01S7/4802

  using analysis of echo signal for target characterisation; Target signature; Target cross-section · CPC title

• G01S15/42

  Simultaneous measurement of distance and other co-ordinates (indirect measurement G01S15/46) · CPC title

• G01S17/931

  of land vehicles · CPC title

• G01S15/931

  of land vehicles · CPC title

• G06T17/05 Primary

  Geographic models · CPC title