Autonomous visual navigation

Patent metadata

Field	Value
Publication number	US-10705528-B2
Application number	US-201615249250-A
Country	US
Kind code	B2
Filing date	Aug 26, 2016
Priority date	Dec 15, 2015
Publication date	Jul 7, 2020
Grant date	Jul 7, 2020

Abstract

A method of visual navigation for a robot includes integrating a depth map with localization information to generate a three-dimensional (3D) map. The method also includes motion planning based on the 3D map, the localization information, and/or a user input. The motion planning overrides the user input when a trajectory and/or a velocity, received via the user input, is predicted to cause a collision.

First claim

What is claimed is:

1. A method of visual navigation for a robot, comprising:
integrating a depth map of a spatial area with a localization information to generate a three-dimensional (3D) map;
motion planning, based on the 3D map, for the robot to reach a target in an unmapped area that is distinct from the spatial area based on a plurality of paths determined from a plurality of first candidate points in unoccupied locations in the 3D map and a plurality of second candidate points in the unmapped area, depth measurements for the unmapped area being unavailable when motion planning;
determining at least one predicted collision-free path between at least one of the plurality of first candidate points in the 3D map and one of the plurality of second candidate points in the unmapped area; and
altering the motion planning based on depth measurements obtained by the robot while navigating on the at least one predicted collision-free path through a portion of the unmapped area.

2. The method of claim 1 , further comprising:
determining the depth map of the spatial area from a plurality of cameras; and
obtaining the localization information from a plurality of sensors.

3. The method of claim 2 , in which the localization information comprises at least one of an image information, an inertial sensor information, or a combination thereof.

4. The method of claim 3 , further comprising obtaining the inertial sensor information from at least one of a gyroscope, an accelerometer, or a combination thereof.

5. The method of claim 1 , further comprising generating the depth map of the spatial area based on measurements obtained from a stereo camera.

6. The method of claim 1 , wherein altering the motion planning includes selecting at least one of a new trajectory, a new velocity, or a combination thereof.

7. The method of claim 1 , further comprising:
receiving a user input to navigate the robot;
motion planning based at least in part on the 3D map and the user input; and
overriding the user input when at least one of a trajectory, a velocity, or a combination thereof, received via the user input, is predicted to cause a collision.

8. The method of claim 1 , further comprising:
randomly selecting the plurality of first candidate points in the 3D map;
determining at least one collision-free path between at least two of the plurality of first candidate points; and
determining a minimum-cost path to the target based on the at least one collision-free path.

9. The method of claim 8 , further comprising altering the minimum-cost path when an obstacle is viewed along the minimum-cost path.

10. The method of claim 1 , further comprising randomly selecting the plurality of second candidate points in the unmapped area.

11. An apparatus, comprising:
a memory; and
at least one processor coupled to the memory, the at least one processor configured to:
integrate a depth map of a spatial area with a localization information to generate a three-dimensional (3D) map;
motion plan, based on the 3D map, for the robot to reach a target in an unmapped area that is distinct from the spatial area based on a plurality of paths determined from a plurality of first candidate points in unoccupied locations in the 3D map and a plurality of second candidate points in the unmapped area, depth measurements for the unmapped area being unavailable when motion planning;
determine at least one predicted collision-free path between at least one of the plurality of first candidate points in the 3D map and one of the plurality of second candidate points in the unmapped area; and
alter the motion planning based on depth measurements obtained by the robot while navigating on the at least one predicted collision-free path through a portion of the unmapped area.

12. The apparatus of claim 11 , in which the at least one processor is further configured to:
determine the depth map of the spatial area from a plurality of cameras; and
obtain the localization information from a plurality of sensors.

13. The apparatus of claim 12 , in which the localization information comprises at least one of an image information, an inertial sensor information, or a combination thereof.

14. The apparatus of claim 13 , in which the at least one processor is further configured to obtain the inertial sensor information from at least one of a gyroscope, an accelerometer, or a combination thereof.

15. The apparatus of claim 11 , in which the at least one processor is further configured to generate the depth map of the spatial area based on measurements obtained from a stereo camera.

16. The apparatus of claim 11 , wherein altering the motion planning includes selecting at least one of a new trajectory, a new velocity, or a combination thereof.

17. The apparatus of claim 11 , in which the at least one processor is further configured to:
receive a user input to navigate the robot;
motion plan based at least in part on the 3D map and the user input; and
override the user input when at least one of a trajectory, a velocity, or a combination thereof, received via the user input, is predicted to cause a collision.

18. The apparatus of claim 11 , in which the at least one processor is further configured to:
randomly select the plurality of first candidate points in the 3D map;
determine at least one collision-free path between at least two of the plurality of first candidate points; and
determine a minimum-cost path to the target based on the at least one collision-free path.

19. The apparatus of claim 18 , in which the at least one processor is further configured to alter the minimum-cost path when an obstacle is viewed along the minimum-cost path.

20. The apparatus of claim 11 , in which the at least one processor is further configured to randomly select the plurality of second candidate points in the unmapped area.

21. An apparatus, comprising:
means for integrating a depth map of a spatial area with a localization information to generate a three-dimensional (3D) map;
means for motion planning, based on the 3D map, for the robot to reach a target in an unmapped area that is distinct from the spatial area based on a plurality of paths determined from a plurality of first candidate points in unoccupied locations in the 3D map and a plurality of second candidate points in the unmapped area, depth measurements for the unmapped area being unavailable when motion planning;
means for determining at least one predicted collision-free path between at least one of the plurality of first candidate points in the 3D map and one of the plurality of second candidate points in the unmapped area; and
means for altering the motion planning based on depth measurements obtained by the robot while navigating on the at least one predicted collision-free path through a portion of the unmapped area.

22. The apparatus of claim 21 , further comprising:
means for determining the depth map of the spatial area from a plurality of cameras; and
means for obtaining the localization information from a plurality of sensors.

23. The apparatus of claim 22 , in which the localization information comprises at least one of an image information, an inertial sensor information, or a combination thereof.

24. The apparatus of claim 23 , further comprising means for obtaining the inertial sensor information from at least one of a gyroscope, an accelerometer, or a combination thereof.

25. The appara

Assignees

• Qualcomm Inc

Inventors

• Wierzynski Casimir Matthew
• Behabadi Bardia Fallah
• Gibson Sarah Paige
• Aghamohammadi Aliakbar
• Agarwal Saurav

Classifications

• G05D2111/50

  Internal signals, i.e. from sensors located in the vehicle, e.g. from compasses or angular sensors · CPC title

• G05D2111/20

  Acoustic signals, e.g. ultrasonic signals · CPC title

• G05D2111/10

  Optical signals · CPC title

• G01C21/3848

  Data obtained from both position sensors and additional sensors · CPC title

• G05D1/81

  Handing over between on-board automatic and on-board manual control · CPC title