Multi-sensor environmental mapping

Patent metadata

Field	Value
Publication number	US-10429839-B2
Application number	US-201514814159-A
Country	US
Kind code	B2
Filing date	Jul 30, 2015
Priority date	Sep 5, 2014
Publication date	Oct 1, 2019
Grant date	Oct 1, 2019

Abstract

Systems and methods for controlling a movable object within an environment are provided. In one aspect, a method may comprise: determining, using at least one of a plurality of sensors carried by the movable object, an initial location of the movable object; generating a first signal to cause the movable object to navigate within the environment; receiving, using the at least one of the plurality of sensors, sensing data pertaining to the environment; generating, based on the sensing data, an environmental map representative of at least a portion of the environment; receiving an instruction to return to the initial location; and generating a second signal to cause the movable object to return to the initial location, based on the environmental map.

First claim

What is claimed is:

1. A method for controlling an unmanned aerial vehicle within an environment, the method comprising:
determining, using at least one of a plurality of sensors carried by the unmanned aerial vehicle, an initial location of the unmanned aerial vehicle, wherein the plurality of sensors comprise (1) at least one vision sensor configured to capture image data to generate a first sensing signal and (2) at least one proximity sensor configured to obtain distance data to generate a second sensing signal; and
with aid of one or more processors onboard the unmanned aerial vehicle:
(a) assessing suitability of the first and second sensing signals for generating various portions of an environmental map, wherein said assessment is based on distinct characteristics of the various portions of the environment and distinct capabilities of the at least one vision sensor and the at least one proximity sensor;
(b) based on the assessing of (a), (i) selecting the first sensing signal for generating a first portion of the environmental map, and (ii) selecting the second sensing signal for generating a second portion of the environmental map, wherein the first sensing signal is assessed to be more suitable than the second sensing signal for generating the first portion of the environmental map, and wherein the second sensing signal is assessed to be more suitable than the first sensing signal for generating the second portion of the environmental map;
(c) based on the selecting of (b), (i) processing the image data captured by the vision sensor to generate a first set of depth images including depth information for the first portion of the environmental map, and (ii) processing the distance data obtained by the proximity sensor to generate a second set of depth images including depth information for the second portion of the environmental map;
(d) combining the first and second sets of depth images to generate the environmental map; and
(e) effecting the unmanned aerial vehicle to autonomously return to the initial location or fly to a specified location utilizing the environmental map.

2. The method of claim 1 , wherein the effecting step of (e) comprises:
determining, using at least one of the plurality of sensors, a current location of the unmanned aerial vehicle;
determining, based on the environmental map, a path from the current location to the initial location or the specified location; and
effecting the unmanned aerial vehicle to move along the path to return to the initial location or the specified location.

3. The method of claim 1 , wherein the processing and combining steps of (c) and (d) comprise:
identifying a first plurality of feature points in the first set of depth images;
identifying a corresponding second plurality of feature points in the second set of depth images;
determining a correspondence between the first and second pluralities of feature points; and
combining the first and second sets of depth images based on the determined correspondence to generate the environmental map.

4. The method of claim 2 , wherein determining the path comprises determining a path from the current location to the initial location or the specified location that avoids one or more obstacles within the environment.

5. The method of claim 2 , wherein the path includes one or more portions previously traveled by the unmanned aerial vehicle.

6. The method of claim 2 , wherein the path is different from another path previously traveled by the unmanned aerial vehicle, or the path is the shortest path from the current location to the initial location or the specified location.

7. The method of claim 2 , wherein the path is a flight path of the unmanned aerial vehicle including spatial location and orientation of the unmanned aerial vehicle.

8. The method of claim 2 , wherein the path comprises a plurality of waypoints corresponding to previously traveled positions of the unmanned aerial vehicle, the plurality of waypoints being recorded by at least one of the plurality of sensors as the unmanned aerial vehicle travels within the environment.

9. The method of claim 1 , wherein the first portion of the environmental map is different from the second portion of the environmental map.

10. A system for controlling an unmanned aerial vehicle within an environment, the system comprising:
a plurality of sensors carried by the unmanned aerial vehicle, wherein the plurality of sensors comprise (1) at least one vision sensor configured to capture image data to generate a first sensing signal and (2) at least one proximity sensor configured to obtain distance data to generate a second sensing signal; and
one or more processors onboard the unmanned aerial vehicle, individually or collectively configured to:
(a) determine, using at least one of the plurality of sensors, an initial location of the unmanned aerial vehicle;
(b) assess suitability of the first and second sensing signals for generating various portions of an environmental map, wherein said assessment is based on distinct characteristics of the various portions of the environment and distinct capabilities of the at least one vision sensor and the at least one proximity sensor;
(c) based on the assessed suitability of (a), (i) select the first sensing signal for generating a first portion of the environmental map, and (ii) select the second sensing signal for generating a second portion of the environmental map, wherein the first sensing signal is assessed to be more suitable than the second sensing signal for generating the first portion of the environmental map, and wherein the second sensing signal is assessed to be more suitable than the first sensing signal for generating the second portion of the environmental map;
(d) based on the selected sensing signals of (c), (i) process the image data captured by the vision sensor to generate a first set of depth images including depth information for the first portion of the environmental map, and (ii) process the distance data obtained by the proximity sensor to generate a second set of depth images including depth information for the second portion of the environmental map;
(e) combine the first and second sets of depth images to generate the environmental map; and
(f) effect the unmanned aerial vehicle to autonomously return to the initial location or fly to a specified location utilizing the environmental map.

11. The system of claim 10 , wherein the one or more processors are configured to perform the effect step of (f) by:
determining, using at least one of the plurality of sensors, a current location of the unmanned aerial vehicle;
determining, based on the environmental map, a path from the current location to the initial location or the specified location; and
effecting the unmanned aerial vehicle to move along the path to return to the initial location or the specified location.

12. The system of claim 10 , wherein the process and combine steps of (d) and (e) comprise:
identifying a first plurality of feature points in the first set of depth images;
identifying a corresponding second plurality of feature points in the second set of depth images;
determining a correspondence between the first and second pluralities of feature points; and
combining the first and second sets of depth images based on the determined correspondence to generate the environmental map.

13. The system of claim 11 , wherein determining the path comprises determining a path from the current location to the initial location or the specified location that avoids one or more obstacles within the environment.

14. The system of claim 11 , wher

Assignees

• Sz Dji Technology Co Ltd

Inventors

• Liu Ang
• Mo Weiyu
• Ling Yonggen

Classifications

• B64U2201/10

  autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS] · CPC title

• G01S19/42

  Determining position · CPC title

• B64C2201/141

  Operations & Transport · mapped topic

• G05D1/102 Primary

  specially adapted for vertical take-off of aircraft · CPC title

• B64C39/024

  of the remote controlled vehicle type, i.e. RPV · CPC title