Systems and methods for unified mapping of an environment

Patent metadata

Field	Value
Publication number	US-10545229-B2
Application number	US-201615262719-A
Country	US
Kind code	B2
Filing date	Sep 12, 2016
Priority date	Apr 22, 2016
Publication date	Jan 28, 2020
Grant date	Jan 28, 2020

Abstract

A method and system for generating a map of an environment based on information acquired by radar combined with information acquired from LIDAR, cameras, or a combination of the LIDAR and camera. The system uses a combination of data from a radar system combined with data from one or both of a camera system and LIDAR system to generate a unified map of the environment.

First claim

What is claimed is:

1. A map data integration system, comprising:
memory;
a processor in communication with the memory and configured to execute computer instructions stored on the memory, wherein execution of the computer instructions causes the processor to:
receive radar map data about an environment acquired by a radar system of a vehicle;
receive LIDAR map data about the environment acquired by a LIDAR of the vehicle; and
receive image map data about the environment acquired by a vision system of the vehicle;
and
generate a reference map of the environment comprising a radar map layer derived from the radar map data, a LIDAR map layer derived from the LIDAR map data, a visual map layer derived from the image map data, and a unified map layer derived from the radar map data, the LIDAR map data, and the image map data.

2. The system of claim 1 , wherein execution of the computer
instructions further causes the processor to detect one or more dynamic objects in the radar map data, and remove data corresponding to each detected dynamic object from the radar map data.

3. The system of claim 1 , wherein execution of the computer instructions further causes the processor:
determine which one of the radar map data, the LIDAR map data and the image map data is the most accurate based on sensed environmental conditions, wherein the sensed environmental conditions is one or more of weather conditions or ambient light level; and
modify the radar map data, the LIDAR map data, and the image map based on the one of the radar map data, the LIDAR map data and the image map data which is determined to be the most accurate.

4. The system of claim 1 , wherein the execution of the computer instructions further causes the processor is configured to detect one or more dynamic objects in the LIDAR map data, and remove data corresponding to each detected dynamic object from the LIDAR map data.

5. The system of claim 1 , wherein the execution of the computer instructions further causes the processor to detect dynamic objects in the image map data, and remove data corresponding to each detected dynamic object from the image map data.

6. The system of claim 1 wherein the processor is further configured to store the radar map layer, the LIDAR map layer, the visual map layer, and the unified map layer as respective distinct data files or a single data file.

7. The system of claim 1 wherein the unified map layer comprises a generic object (GO) map layer and a detailed object map layer.

8. A unified mapping system for mapping an environment of a vehicle, comprising:
a radar system mountable to the vehicle for generating radar map data about;
a LIDAR system mountable to the vehicle for generating LIDAR map data about the environment of the vehicle;
a vision system mountable to the vehicle for generating image map data about the environment of the vehicle;
a map data integration system comprising:
memory;
a processor in communication with the memory and configured to execute computer instructions stored on the memory, wherein execution of the computer instructions causes the processor to:
receive radar map data about the environment from the radar system;
receive LIDAR map data from the LIDAR system;
receive image map data from the vision system;
generate a reference map of the environment including a radar map layer derived from radar map data, a LIDAR map layer derived from the LIDAR data, a visual map layer derived from the image map data, and a unified map layer derived from the radar map data, the LIDAR map data, and the image map data;

wherein the reference map includes static objects in the environment and attributes of the static objects, wherein at least one attribute is determined based on the radar map data and at least one different attribute is determined based on the one of the LIDAR map data and the image map data.

9. The system of claim 8 , wherein execution of the computer instructions causes the processor to determine which one of the radar map data, the LIDAR map data and the image map data has best accuracy based on sensed environmental conditions, to modify the radar map data, the LIDAR map data, and the image map data based on the one of the radar map data, the LIDAR map data, and the image map data which has the best accuracy.

10. The system of claim 9 , wherein the sensed environmental conditions is one or more of weather conditions or ambient light level.

11. The system of claim 8 , further comprising a navigation system configured to receive the unified map from the map data integration and to determine a location of the vehicle within the reference map of the environment.

12. The system of claim 8 , wherein the navigation system is further configured to generate navigation instructions for the vehicle based on the reference map of the environment.

13. A method of generating a reference map of an environment, the method comprising:
receiving radar map data about the environment acquired by a radar system of a vehicle;
receiving LIDAR map data about the environment acquired by a LIDAR system of the vehicle;
receiving image map data about the environment acquired by a vision system of the vehicle; and
generating a reference map of the environment comprising a radar map layer derived from the radar map data, a LIDAR map layer derived from the LIDAR map data, a visual map layer derived from the image map data, and a unified map layer derived from the radar map data, the LIDAR map data, and the image map data.

14. The method of claim 13 , further comprising:
detecting a map data conflict between the radar map data, the LIDAR map data, and the image map data;
determining which of the radar map data, the LIDAR map data and image map data has best accuracy;
modifying the radar map data, the LIDAR map data, and the image map data based on the one of the radar map data, the LIDAR map data and the image map data which is determined to have the best accuracy;
flagging the modified radar map data for review.

15. The method of claim 14 further comprising processing at least one of the radar map data, the LIDAR map data, and the image map data to remove dynamic objects.

16. The method of claim 14 , wherein determining which of the radar map data, the LIDAR map data and the image map data has the best accuracy is based on sensed weather conditions determined based on data collected by one or more environmental sensing systems.

17. The method of claim 13 , further comprising:
detecting a map data ambiguity in the radar map data;
estimating which one of the radar map data, the LIDAR map data and image map data has best accuracy;
modifying the radar map data based on the one of the radar map data, the LIDAR map data and the image map data which has the best accuracy; and
flagging the modified radar map data for review.

18. The method of claim 17 , wherein detecting a map data ambiguity in the radar map data comprises detecting a map data ambiguity when the radar map data for a subject object or region in the environment falls below a predetermined ambiguity threshold.

19. The method of claim 13 further comprising:
storing the radar map layer, the LIDAR map layer, the visual map layer, the unified map layer as respective distinct data files or a single data file.

20. The method of claim 13 wherein the unified map layer comprises a generic object (GO) map layer and a detailed object map layer.

Assignees

• Rohani Mohsen
• Zhang Song
• Huawei Tech Co Ltd

Inventors

• Rohani Mohsen
• Zhang Song

Classifications

• G01S13/931

  of land vehicles · CPC title

• G01S13/865 Primary

  Combination of radar systems with lidar systems · CPC title

• G01S17/89

  for mapping or imaging · CPC title

• G01S13/89 Primary

  for mapping or imaging · CPC title

• G01S13/867

  Combination of radar systems with cameras · CPC title