Techniques for collaborative map construction between an unmanned aerial vehicle and a ground vehicle

Patent metadata

Field	Value
Publication number	US-12222218-B2
Application number	US-202318213283-A
Country	US
Kind code	B2
Filing date	Jun 23, 2023
Priority date	Mar 8, 2019
Publication date	Feb 11, 2025
Grant date	Feb 11, 2025

Abstract

Techniques are disclosed for collaborative map construction using multiple vehicles. Such a system may include a ground vehicle including a first computing device and a first scanning sensor, and an aerial vehicle including a second computing device and a second scanning sensor. The ground vehicle can obtain a first real-time map based on first scanning data using the first scanning sensor, and transmit a first real-time map and position information to the aerial vehicle. The aerial vehicle can receive the first real-time map and the position information from the first computing device, obtain a second real-time map based on second scanning data collected using the second scanning sensor, and obtain a third real-time map based on the first real-time map and the second real-time map.

First claim

What is claimed is:

1. A control method, comprising:
detecting a traffic condition around a first vehicle and/or a second vehicle, the first vehicle being a ground vehicle;
in response to the traffic condition affecting the first vehicle and/or the second vehicle, generating a map, wherein generating the map includes:
obtaining first scanning data collected using a first scanning sensor coupled to the first vehicle;
obtaining second scanning data collected using a second scanning sensor coupled to the second vehicle; and
generating the map based on the first scanning data and the second scanning data; and

controlling the first vehicle and/or the second vehicle based on the generated map to navigate an environment around the first vehicle and/or the second vehicle.

2. The control method of claim 1 , wherein the traffic condition around the first vehicle and/or the second vehicle comprises at least one of the following:
roadway information, traffic information, driving conditions, or weather conditions.

3. The control method of claim 2 , wherein
the roadway information comprises roadway objects information, road type, and/or surface information;
the traffic information comprises information related to nearby vehicles and/or traffic density;
the driving conditions comprise current speed and/or direction; and
the weather conditions comprise visibility.

4. The control method of claim 1 , further comprising:
controlling the first vehicle and/or the second vehicle to choose one or more scanning sensor to capture scanning data of an environment at one or more perspective selected from the group consisting of front perspective, rear perspective, side perspective, overhead perspective, and upward perspective,
wherein the scanning data of the environment is configured to detect the traffic condition.

5. The control method of claim 1 , further comprising:
obtaining a first map based on the first scanning data collected using the first scanning sensor coupled to the first vehicle;
obtaining a second map based on the second scanning data collected using the second scanning sensor coupled to the second vehicle.

6. The control method of claim 5 , wherein the generating the map based on the first scanning data and the second scanning data comprises:
generating a third map based on the first map and the second map.

7. The control method of claim 5 , wherein the first map of the environment is obtained from a first perspective; the second map of the environment is obtained from a second perspective, wherein the first perspective is different from the second perspective.

8. The control method of claim 7 , wherein the first perspective is a front perspective, and the second perspective is an overhead perspective.

9. The control method of claim 6 , wherein the third map has precision higher than the second map; and/or the first map has precision higher than the second map.

10. The control method of 6 , further comprising:
obtaining each corresponding map from a plurality of aerial vehicles;
generating a fourth map based on the first map and each corresponding map from the plurality of aerial vehicles; and
transmitting the fourth map to each of the plurality of aerial vehicles.

11. The control method of claim 6 , wherein generating the third map based on the first map and the second map comprises:
determining an overlapping portion of the first map and the second map; and
merging the first map and the second map using the overlapping portion.

12. The control method of claim 1 , wherein the first scanning sensor includes a LiDAR sensor and the second scanning sensor includes a visual sensor.

13. The control method of claim 1 , further comprising:
obtaining position information from the first vehicle and/or the second vehicle and
generating the map based on the first scanning data and the second scanning data and the position information.

14. The control method of claim 13 , wherein the position information includes global navigation satellite system (GNSS) received from the first vehicle and/or the second vehicle, and/or a real-time relative position determined using the first scanning sensor and/or the second scanning sensor.

15. The control method of claim 14 , wherein the position information is position information of the second vehicle and/or the first vehicle.

16. The control method of claim 1 , wherein the first vehicle is an autonomous vehicle and/or the second vehicle is an unmanned aerial vehicle.

17. The control method of claim 5 , wherein the first vehicle is configured to obtain third scanning data from a third scanning sensor coupled to the first vehicle, the first scanning sensor including a LiDAR sensor and the third scanning sensor including an imaging sensor, and generate the first map based on the first scanning data and the third scanning data using a Simultaneous Localization and Mapping (SLAM) algorithm.

18. The control method of claim 1 , further comprising:
controlling the second vehicle to:
launch from the first vehicle or land on the first vehicle; and/or
return to the first vehicle for charging or storage.

19. A control device, comprises a controller configured to:
detect a traffic condition around a first vehicle and/or a second vehicle, the first vehicle being a ground vehicle;
in response to the traffic condition affecting the first vehicle and/or the second vehicle, generate a map, wherein generate the map includes:
obtain first scanning data collected using a first scanning sensor coupled to the first vehicle;
obtain second scanning data collected using a second scanning sensor coupled to the second vehicle;
generate a map based on the first scanning data and the second scanning data; and

control the first vehicle and/or the second vehicle based on the generated map to navigate an environment around the first vehicle and/or the second vehicle.

20. A system for generating a map based on sensor information from multiple vehicles, comprising:
an aerial vehicle including a first computing device;
a first scanning sensor coupled to the aerial vehicle;
a ground vehicle including a second computing device; and
a second scanning sensor coupled to the ground vehicle;
wherein:
the first computing device includes at least one first processor and a scanning manager, the scanning manager including first instructions which, when executed by the at least one first processor, cause the scanning manager to:
obtain first scanning data from the first scanning sensor; and
transmit the first scanning data to the second computing device; and

the second computing device includes at least one second processor and a detection manager, the detection manager including second instructions which, when executed by the at least one second processor, cause the detection manager to:
receive the first scanning data from the first computing device;
obtain second scanning data from the second scanning sensor;
identify one or more roadway objects in the first scanning data;
calibrate the second scanning data based on a reference object depicted in both the second scanning data and the first scanning data; and
map one or more positions associated with the one or more roadway objects in the first scanning data to the calibrated second scanning data;

the aerial vehicle and/or the ground vehicle is controlled based on navigation based on the one or more roadway objects mapped to the calibrated second scanning data.

Assignees

• Sz Dji Technology Co Ltd

Inventors

• Wang Mingyu

Classifications

• B64U2201/20

  Remote controls · CPC title

• B64U2101/20

  for use as communications relays, e.g. high-altitude platforms · CPC title

• B64U2101/32

  for cartography or topography · CPC title

• B64U10/13

  Flying platforms · CPC title

• B64U80/86

  Land vehicles · CPC title