Multisensor data fusion method and apparatus to obtain static and dynamic environment features

Patent metadata

Field	Value
Publication number	US-12044776-B2
Application number	US-202217825601-A
Country	US
Kind code	B2
Filing date	May 26, 2022
Priority date	Dec 29, 2018
Publication date	Jul 23, 2024
Grant date	Jul 23, 2024

Abstract

A multisensor data fusion perception method includes receiving feature data from a plurality of types of sensors, obtaining static feature data and dynamic feature data from the feature data, constructing current static environment information based on the static feature data and reference dynamic target information, and constructing current dynamic target information based on the dynamic feature data and reference static environment information such that construction of a dynamic target and construction of a static environment are performed by referring to each other's construction results and the perception capability is for the dynamic target and the static environment that are in an environment in which the moving carrier is located.

First claim

What is claimed is:

1. A method comprising:
receiving, from a plurality of types of sensors, first feature data comprising dynamic millimeter-wave radar feature data, static millimeter-wave radar feature data, and non-millimeter-wave radar feature data;
determining the non-millimeter-wave radar feature data is first dynamic feature data when a distance between the non-millimeter-wave radar feature data and the dynamic millimeter-wave radar feature data is less than a first preset threshold;
determining the non-millimeter-wave radar feature data is first static feature data when a distance between the non-millimeter-wave radar feature data and the static millimeter-wave radar feature data is less than a second preset threshold; and
either:
obtaining first static environment information based on the first static feature data and historical dynamic target information, and obtaining first dynamic target information based on the first dynamic feature data and the first static environment information; or
obtaining the first dynamic target information based on the first dynamic feature data and historical static environment information, and obtaining the first static environment information based on the first static feature data and the first dynamic target information.

2. The method of claim 1 , wherein the first dynamic target information and the historical dynamic target information comprise a moving track of a moving target.

3. The method of claim 1 , wherein the first static environment information and the historical static environment information comprise at least one of a static raster map or road structure information, wherein the static raster map is a data format describing static obstacle distribution in an environment, and wherein the road structure information comprises information of at least one of a road edge or an above-ground object.

4. The method of claim 1 , wherein the first feature data comprises millimeter-wave radar detection data and non-millimeter-wave radar detection data, and wherein obtaining the first static feature data and the first dynamic feature data comprises:
obtaining, based on the millimeter-wave radar detection data, millimeter-wave radar dynamic detection data and millimeter-wave radar static detection data; and
obtaining, based on the millimeter-wave radar dynamic detection data and the millimeter-wave radar static detection data, dynamic non-millimeter-wave feature data and static non-millimeter-wave feature data for the non-millimeter-wave radar detection data,
wherein the first dynamic feature data comprises the millimeter-wave radar dynamic detection data and the dynamic non-millimeter-wave feature data, and
wherein the first static feature data comprises the millimeter-wave radar static detection data and the static non-millimeter-wave feature data.

5. The method of claim 1 , wherein the first static environment information comprises a first static raster map of an environment, wherein the first static raster map is a data format describing static obstacle distribution in the environment, and wherein the method further comprises:
unifying the first static feature data into unified static feature data in a single coordinate system to obtain unified first static feature data; and
performing a local update on a historical static raster map of the environment based on the unified first static feature data and a moving track of a moving target to obtain the first static raster map at least in part by updating a value of a target raster on the historical static raster map,
wherein the target raster is covered by the moving target.

6. The method of claim 5 , wherein the method is performed by a moving carrier, and wherein the method further comprises:
obtaining a motion status of the moving carrier, wherein the motion status comprises a moving speed of the moving carrier; and
performing, based on the moving speed, a global update on a value of each raster on the historical static raster map to obtain the historical static raster map,
wherein the historical static raster map comprises a second static raster map corresponding to a previous time point of a current time point.

7. The method of claim 6 , wherein the historical static raster map is an initial static raster map when the previous time point is a start time point, and wherein a value of each raster on the initial static raster map is a preset value.

8. The method of claim 3 , wherein the road structure information comprises the information of the road edge, and wherein obtaining the first dynamic target information further comprises:
clustering the first dynamic feature data to obtain one or more clustering centers, wherein each of the one or more clustering centers represents a possible dynamic target; and
excluding an invalid clustering center in the one or more clustering centers based on the road structure information, wherein the invalid clustering center is either located outside the road edge or is overlapped with the road edge.

9. The method of claim 3 , wherein the road structure information comprises the information of the above-ground object, and wherein obtaining the first dynamic target information further comprises:
clustering the first dynamic feature data to obtain an immature moving target; and
determining that the immature moving target is the above-ground object based on the information of the above-ground object.

10. The method of claim 1 , further comprising:
obtaining undetermined data based on the first feature data; and
obtaining the first static environment information or the first dynamic target information further based on the undetermined data.

11. An apparatus comprising:
a memory configured to store instructions; and
a processor coupled to the memory and configured to execute the instructions to cause the apparatus to be configured to:
receive, from a plurality of sensors, first feature data comprising dynamic millimeter-wave radar feature data, static millimeter-wave radar feature data, and non-millimeter-wave radar feature data;
determine the non-millimeter-wave radar feature data is first dynamic feature data when a distance between the non-millimeter-wave radar feature data and the dynamic millimeter-wave radar feature data is less than a first preset threshold;
determine the non-millimeter-wave radar feature data is first static feature data when a distance between the non-millimeter-wave radar feature data and the static millimeter-wave radar feature data is less than a second preset threshold; and

either:
obtain first static environment information based on the first static feature data and historical dynamic target information, and obtain first dynamic target information based on the first dynamic feature data and the first static environment information; or
obtain the first dynamic target information based on the first dynamic feature data and historical static environment information, and obtain the first static environment information based on the first static feature data and the first dynamic target information.

12. The apparatus of claim 11 , wherein the first dynamic target information and the historical dynamic target information comprise a moving track of a moving target.

13. The apparatus of claim 11 , wherein the first static environment information and the historical static environment information comprise at least one of a static raster map or road structure information, wherein the static raster map is a data format describing static obstacle distribution in an environment, and wherein the road structure information comprises information of at least one of a road edge or an abo

Assignees

• Huawei Tech Co Ltd

Inventors

• Peng Xueming
• Wei Zhixuan
• Chen Qi

Classifications

• G01S13/52

  Discriminating between fixed and moving objects or between objects moving at different speeds · CPC title

• G01S13/42

  Simultaneous measurement of distance and other co-ordinates (indirect measurement G01S13/46) · CPC title

• G06F18/23

  Clustering techniques · CPC title

• G01S13/86

  Combinations of radar systems with non-radar systems, e.g. sonar, direction finder · CPC title

• G06V20/588

  Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road · CPC title