Techniques for collaborative map construction between an unmanned aerial vehicle and a ground vehicle

Patent metadata

Field	Value
Publication number	US-11709073-B2
Application number	US-202017097393-A
Country	US
Kind code	B2
Filing date	Nov 13, 2020
Priority date	Mar 8, 2019
Publication date	Jul 25, 2023
Grant date	Jul 25, 2023

Abstract

Techniques are disclosed for collaborative map construction using multiple vehicles. Such a system may include a ground vehicle including a first computing device and a first scanning sensor, and an aerial vehicle including a second computing device and a second scanning sensor. The ground vehicle can obtain a first real-time map based on first scanning data using the first scanning sensor, and transmit a first real-time map and position information to the aerial vehicle. The aerial vehicle can receive the first real-time map and the position information from the first computing device, obtain a second real-time map based on second scanning data collected using the second scanning sensor, and obtain a third real-time map based on the first real-time map and the second real-time map.

First claim

What is claimed is:

1. A system for collaborative map construction, comprising:
a ground vehicle including a first computing device;
a first scanning sensor coupled to the ground vehicle;
an aerial vehicle including a second computing device;
a second scanning sensor coupled to the aerial vehicle;
the first computing device including at least one processor and a ground vehicle mapping manager, the ground vehicle mapping manager including first instructions which, when executed by the processor, cause the ground vehicle mapping manager to:
obtain a first real-time map of environment in which the ground vehicle is located from a front-view perspective based on first scanning data using the first scanning sensor; and
transmit the first real-time map and position information to the aerial vehicle;
the second computing device including at least one processor and an aerial vehicle mapping manager, the aerial vehicle mapping manager including second instructions which, when executed by the processor, cause the aerial vehicle mapping manager to:
receive the first real-time map and the position information from the first computing device;
obtain a second real-time map of the environment from an overhead perspective based on second scanning data collected using the second scanning sensor; and
generate a third real-time map of the environment from the overhead perspective based on the first real-time map and the second real-time map,
wherein the third real-time map has precision higher than the second real-time map.

2. The system of claim 1 , wherein the first real-time map is a higher precision map than the second real-time map.

3. The system of claim 1 , wherein to obtain the third real-time map based on the first real-time map and the second real-time map, the second instructions, when executed, further cause the aerial vehicle mapping manager to:
determine an overlapping portion of the first real-time map and the second real-time map; and
merge the first real-time map and the second real-time map using the overlapping portion.

4. The system of claim 1 , wherein the first scanning sensor includes a LiDAR sensor and the second scanning sensor includes a visual sensor.

5. The system of claim 4 , wherein the first real-time map is constructed based on point cloud data obtained from the first scanning sensor, and the second real-time map is constructed based on visual data obtained from the second scanning sensor.

6. The system of claim 5 , wherein to obtain the first real-time map based on the first scanning data collected from a view angle of the ground vehicle using the first scanning sensor, the first instructions, when executed, further cause the ground vehicle mapping manager to:
obtain the second real-time map from the aerial vehicle;
convert coordinates in the second real-time map to a coordinate system to match the first real-time map;
determine an overlapping portion of the first real-time map and the second real-time map in the coordinate system; and
transmit with the overlapping portion to the aerial vehicle.

7. The system of claim 1 , wherein the second instructions, when executed, further cause the aerial vehicle mapping manager to:
determine available resources associated with the second computing device are below a threshold value;
send a request to the first computing device to generate the third real-time map, the request including the second real-time map; and
receive the third real-time map from the first computing device.

8. A method for collaborative map construction, comprising:
receiving, by an aerial vehicle, a first real-time map of environment in which a ground vehicle is located from a front-view perspective and position information from the ground vehicle, wherein the first real-time map is based on first scanning data collected using a first scanning sensor coupled to the ground vehicle;
obtaining a second real-time map of the environment from an overhead perspective based on second scanning data collected using a second scanning sensor coupled to the aerial vehicle; and
generating a third real-time map of the environment from the overhead perspective based on the first real-time map and the second real-time map,
wherein the third real-time map has precision higher than the second real-time map.

9. The method of claim 8 , wherein receiving, by the aerial vehicle, the first real-time map from the ground vehicle, further comprises:
receiving position information of the aerial vehicle from the ground vehicle.

10. The method of claim 8 , wherein the position information includes global navigation satellite system (GLASS) received from the aerial vehicle.

11. The method of claim 8 , wherein the position information includes a real-time relative position determined using the first scanning sensor.

12. The method of claim 8 , wherein the ground vehicle is an autonomous vehicle.

13. The method of claim 8 , wherein the ground vehicle is configured to obtain third scanning data from a third scanning sensor coupled to the ground vehicle, the first scanning sensor including a LiDAR sensor and the third scanning sensor including an imaging sensor, and generate the first real-time map based on the first scanning data and the third scanning data using a Simultaneous Localization and Mapping (SLAM) algorithm.

14. The method of claim 8 , further comprising:
transmitting the second scanning data to the ground vehicle, wherein the ground vehicle is configured to generate the first real-time map based on the first scanning data and the second scanning data using a Simultaneous Localization and Mapping (SLAM) algorithm, wherein the first scanning sensor includes a LiDAR sensor and the second scanning sensor includes an imaging sensor.

15. A non-transitory computer readable storage medium including instructions stored thereon which, when executed by one or more processors, cause the one or more processors to:
receive, by an aerial vehicle, a first real-time map of environment in which a ground vehicle is located from a front-view perspective and position information from the ground vehicle, wherein the first real-time map is based on first scanning data collected using a first scanning sensor coupled to the ground vehicle;
obtain a second real-time map of the environment from an overhead perspective based on second scanning data collected using a second scanning sensor coupled to the aerial vehicle; and
generate a third real-time map of the environment from the overhead perspective based on the first real-time map and the second real-time map,
wherein the third real-time map has precision higher than the second real-time map.

16. The non-transitory computer readable storage medium of claim 15 , wherein the first real-time map is constructed based on point cloud data obtained from the first scanning sensor, and the second real-time map is constructed based on visual data obtained from the second scanning sensor.

17. The non-transitory computer readable storage medium of claim 16 , wherein to receive, by the aerial vehicle, the first real-time map from the ground vehicle, wherein the first real-time map is based on the first scanning data collected using the first scanning sensor coupled to the ground vehicle, the instructions, when executed, further cause the one or more processors to:
transmit the second real-time map to the ground vehicle, wherein the ground vehicle is configured to:
convert coordinates in the second real-time map to a coordinate system to match the first real-time map;
determine an overlapping portion of the first real-

Assignees

• Sz Dji Technology Co Ltd

Inventors

• Wang Mingyu

Classifications

• G01C21/20

  Instruments for performing navigational calculations (G01C21/24, G01C21/26 take precedence) · CPC title

• G01C21/3811

  Point data, e.g. Point of Interest [POI] · CPC title

• G01C21/3807

  characterised by the type of data · CPC title

• G01C21/3804

  Creation or updating of map data · CPC title

• G01C21/38

  Electronic maps specially adapted for navigation; Updating thereof · CPC title