Multisensor data fusion method and apparatus to obtain static and dynamic environment features

Patent metadata

Field	Value
Publication number	US-11353553-B2
Application number	US-202117353017-A
Country	US
Kind code	B2
Filing date	Jun 21, 2021
Priority date	Dec 29, 2018
Publication date	Jun 7, 2022
Grant date	Jun 7, 2022

Abstract

A multisensor data fusion perception method includes receiving feature data from a plurality of types of sensors, obtaining static feature data and dynamic feature data from the feature data, constructing current static environment information based on the static feature data and reference dynamic target information, and constructing current dynamic target information based on the dynamic feature data and reference static environment information such that construction of a dynamic target and construction of a static environment are performed by referring to each other's construction results and the perception capability is for the dynamic target and the static environment that are in an environment in which the moving carrier is located.

First claim

What is claimed is:

1. A method comprising:
receiving feature data from a plurality of types of sensors;
performing data classification on the feature data to obtain static feature data and dynamic feature data;
obtaining, based on the dynamic feature data, reference dynamic target information;
constructing current static environment information based on the static feature data and the reference dynamic target information;
obtaining, based on the static feature data, reference static environment information;
constructing current dynamic target information based on the dynamic feature data and the reference static environment information; and
outputting the current static environment information and the current dynamic target information,
wherein the feature data comprises millimeter-wave radar detection data and non-millimeter-wave radar detection data, and
wherein performing the data classification on the feature data further comprises:
performing the data classification on the millimeter-wave radar detection data to obtain millimeter-wave radar dynamic detection data and millimeter-wave radar static detection data; and
performing, based on the millimeter-wave radar dynamic detection data and the millimeter-wave radar static detection data, the data classification on the non-millimeter-wave radar detection data to obtain dynamic non-millimeter-wave feature data and static non-millimeter-wave feature data,

wherein the dynamic feature data comprises the millimeter-wave radar dynamic detection data and the dynamic non-millimeter-wave feature data, and
wherein the static feature data comprises the millimeter-wave radar static detection data and the static non-millimeter-wave feature data.

2. The method of claim 1 , wherein obtaining the reference dynamic target information further comprises further obtaining the reference dynamic target information based on the dynamic feature data and historical static environment information.

3. The method of claim 1 , wherein obtaining the reference static environment information further comprises further obtaining the reference static environment information based on the static feature data and historical dynamic target information.

4. The method of claim 1 , wherein the sensors are mounted on a first moving carrier that is located in a first environment, wherein the reference dynamic target information comprises a moving track of a second moving carrier that is located in the first environment, wherein the current static environment information comprises a current static raster map of the first environment, wherein the current static raster map is a data format describing static obstacle distribution in the first environment, and wherein constructing the current static environment information further comprises:
unifying the static feature data into unified static feature data in a single coordinate system; and
performing a local update on a historical static raster map of the first environment based on the unified static feature data and the moving track to obtain the current static raster map, wherein the local update comprises updating a value of a target raster on the historical static raster map, and wherein the target raster is covered by the second moving carrier.

5. The method of claim 4 , wherein before constructing the current static environment information, the method further comprises:
obtaining, from the sensors, a motion status of the first moving carrier, wherein the motion status comprises a moving speed of the first moving carrier;
performing, based on the moving speed, a global update on a value of each raster on the historical static raster map to obtain an updated historical static raster map, wherein the historical static raster map comprises a static raster map at a previous time point of a current time point, wherein the historical static raster map is an initial static raster map when the previous time point is a start time point, and wherein a value of each raster on the initial static raster map is a preset value; wherein
the current static raster map is obtained by performing the local update on the updated historical static raster map based on the unified static feature data and the moving track.

6. The method of claim 1 , wherein the sensors are mounted on a first moving carrier that is located in a first environment, wherein the reference dynamic target information comprises a moving track of a second moving carrier that is located in the first environment, wherein the current static environment information further comprises road structure information in the first environment, and wherein constructing the current static environment information further comprises constructing the road structure information based on the static feature data and the moving track.

7. The method of claim 6 , wherein the reference static environment information comprises the road structure information, and wherein constructing the current dynamic target information further comprises constructing the current dynamic target information based on the dynamic feature data and the road structure information.

8. The method of claim 7 , wherein the road structure information comprises a road edge, and wherein constructing the current dynamic target information further comprises:
clustering the dynamic feature data to obtain one or more clustering centers, wherein each of the one or more clustering centers represents a possible dynamic target; and
excluding an invalid clustering center in the one or more clustering centers based on the road structure information, wherein the invalid clustering center is either located outside the road edge or is overlapped with the road edge.

9. An apparatus comprising:
a memory configured to store programming instructions; and
a processor coupled to the memory, wherein when executed by the processor, the programming instructions cause the processor apparatus to be configured to:
receive feature data from a plurality of types of sensors;
perform data classification on the feature data to obtain static feature data and dynamic feature data;
obtain, based on the dynamic feature data, reference dynamic target information;
construct current static environment information based on the static feature data and the reference dynamic target information;
obtain, based on the static feature data, reference static environment information;
construct current dynamic target information based on the dynamic feature data and the reference static environment information; and
output the current static environment information and the current dynamic target information,

wherein the feature data comprises millimeter-wave radar detection data and non-millimeter-wave radar detection data, and
wherein the programming instructions further cause the apparatus to be configured to:
perform the data classification on the millimeter-wave radar detection data to obtain millimeter-wave radar dynamic detection data and millimeter-wave radar static detection data; and
perform, based on the millimeter-wave radar dynamic detection data and the millimeter-wave radar static detection data, the data classification on the non-millimeter-wave radar detection data to obtain dynamic non-millimeter-wave feature data and static non-millimeter-wave feature data,

wherein the dynamic feature data comprises the millimeter-wave radar dynamic detection data and the dynamic non-millimeter-wave feature data, and
wherein the static feature data comprises the millimeter-wave radar static detection data and the static non-millimeter-wave feature data.

10. The apparatus of claim 9 , wherein when executed by the processor, the programming instructions cause the appara

Assignees

• Huawei Tech Co Ltd

Inventors

• Peng Xueming
• Wei Zhixuan
• Chen Qi

Classifications

• G01S13/931 Primary

  of land vehicles · CPC title

• G06V20/588

  Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road · CPC title

• G06V10/806

  of extracted features · CPC title

• G06V20/58

  Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads · CPC title

• G01S7/415 Primary

  Identification of targets based on measurements of movement associated with the target · CPC title