Terrain aware step planning system

Patent metadata

Field	Value
Publication number	US-11287826-B2
Application number	US-201916288205-A
Country	US
Kind code	B2
Filing date	Feb 28, 2019
Priority date	Oct 12, 2018
Publication date	Mar 29, 2022
Grant date	Mar 29, 2022

Abstract

A method for terrain and constraint planning a step plan includes receiving, at data processing hardware of a robot, image data of an environment about the robot from at least one image sensor. The robot includes a body and legs. The method also includes generating, by the data processing hardware, a body-obstacle map, a ground height map, and a step-obstacle map based on the image data and generating, by the data processing hardware, a body path for movement of the body of the robot while maneuvering in the environment based on the body-obstacle map. The method also includes generating, by the data processing hardware, a step path for the legs of the robot while maneuvering in the environment based on the body path, the body-obstacle map, the ground height map, and the step-obstacle map.

First claim

What is claimed is:

1. A method comprising:
receiving, at data processing hardware of a robot, image data of an environment about the robot from at least one image sensor, the robot comprising a body and legs coupled to the body;
identifying, by the data processing hardware, from the image data, objects present in the environment;
for each respective object identified from the image data:
determining, by the data processing hardware, whether the respective object is a body obstacle that is not safe for the body of the robot to travel through; and
determining, by the data processing hardware, whether the respective object is a step obstacle where the legs of the robot should not step;

generating, by the data processing hardware, a body-obstacle map including the body obstacles, a ground height map, and a step-obstacle map including the step obstacles based on the image data;
generating, by the data processing hardware, without using the step-obstacle map, a body path for movement of the body of the robot from a body path start point to a body path target point based on the body obstacles of the body-obstacle map; and
generating, by the data processing hardware, a step path for the legs of the robot from the body path start point to the body path target point based on the body path, the body-obstacle map, the ground height map, and the step-obstacle map, the step path avoiding step obstacles encountered by the legs of the robot as the body of the robot travels along the body path.

2. The method of claim 1 , wherein the image data comprises three-dimensional point cloud data captured by a three-dimensional volumetric image sensor.

3. The method of claim 1 , wherein the at least one image sensor comprises one or more of a stereo camera, a scanning light-detection and ranging (LIDAR) sensor, or a scanning laser-detection and ranging (LADAR) sensor.

4. The method of claim 1 , further comprising:
identifying, by the data processing hardware, occupancies of space in the environment based on the image data; and
generating, by the data processing hardware, a three-dimensional space occupancy map based on the identification of occupancies of space in the environment,
wherein generating the body-obstacle map, the ground height map, and the step-obstacle map based on the image data comprises:
generating the body-obstacle map based on the three-dimensional space occupancy map;
generating the ground height map based on the three-dimensional space occupancy map, the ground height map identifying a height of a ground surface at each location near the robot; and
generating the step-obstacle map based on the ground height map, the step-obstacle map identifying no-step regions in the environment where the legs of the robot should not step.

5. The method of claim 4 , wherein generating the body-obstacle map comprises generating a two-dimensional body-obstacle map based on the three-dimensional space occupancy map.

6. The method of claim 4 , wherein the three-dimensional space occupancy map comprises a voxel map having voxels, each voxel representing a three-dimensional space of the environment.

7. The method of claim 6 , wherein each voxel is classified as either a ground surface, an obstacle, or other.

8. The method of claim 4 , further comprising filtering, by the data processing hardware, the three-dimensional space occupancy map to generate the body-obstacle map.

9. The method of claim 1 , wherein generating the step path is based on adjusting a nominal step pattern of a nominal gait for the robot and step constraints.

10. The method of claim 9 , wherein the step constraints comprise at least one of:
a threshold range of a center of pressure offset for each leg in contact with a ground surface, the center of pressure offset indicating an acceptable amount of robot weight distribution for each leg at each step;
whether the step path causes a leg to step into a no-step region of the step-obstacle map;
whether the step path causes the body of the robot to enter a no-body region of the body-obstacle map;
whether the step path causes a self-collision of the robot; or
a margin of space about any no-step region of the step-obstacle map.

11. The method of claim 10 , wherein the step constraints comprise soft constraints or hard constraints.

12. The method of claim 1 , wherein generating the step path for the legs of the robot comprises modifying the generated body path.

13. A robot comprising:
a body;
legs coupled to the body and configured to maneuver the robot about an environment;
data processing hardware in communication with the legs; and
memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations comprising:
receiving image data of an environment about the robot from at least one image sensor;
identifying, from the image data, objects present in the environment;
for each respective object identified from the image data:
determining whether the respective object is a body obstacle that is not safe for the body of the robot to travel through; and
determining whether the respective object is a step obstacle where the legs of the robot should not step;

generating a body-obstacle map including the body obstacles, a ground height map, and a step-obstacle map including the step obstacles based on the image data;
generating, without using the step-obstacle map, a body path for movement of the body of the robot from a body path start point to a body path target point based on the body obstacles of the body-obstacle map; and
generating a step path for the legs of the robot from the body path start point to the body path target point based on the body path, the body-obstacle map, the ground height map, and the step-obstacle map, the step path avoiding step obstacles encountered by the legs of the robot as the body of the robot travels along the body path.

14. The robot of claim 13 , wherein the image data comprises three-dimensional point cloud data captured by a three-dimensional volumetric image sensor.

15. The robot of claim 13 , wherein the at least one image sensor comprises one or more of a stereo camera, a scanning light-detection and ranging (LIDAR) sensor, or a scanning laser-detection and ranging (LADAR) sensor.

16. The robot of claim 13 , wherein the operations further comprise:
identifying occupancies of space in the environment based on the image data; and
generating a three-dimensional space occupancy map based on the identification of occupancies of space in the environment,
wherein generating the body-obstacle map, the ground height map, and the step-obstacle map based on the image data comprises:
generating the body-obstacle map based on the three-dimensional space occupancy map;
generating the ground height map based on the three-dimensional space occupancy map, the ground height map identifying a height of a ground surface at each location near the robot; and
generating the step-obstacle map based on the ground height map, the step-obstacle map identifying no-step regions in the environment where the legs of the robot should not step.

17. The robot of claim 16 , wherein generating the body-obstacle map comprises generating a two-dimensional body-obstacle map based on the three-dimensional space occupancy map.

18. The robot of claim 16 , wherein the three-dimensional space occupancy map comprises a voxel map having voxels, each voxel representing a three-dimensi

Assignees

• Boston Dynamics Inc

Inventors

• Whitman Eric
• Fay Gina Christine

Classifications

• Y10S901/01

  Mobile robot · CPC title

• G05D1/49 Primary

  Control of attitude, i.e. control of roll, pitch or yaw · CPC title

• G05D1/247

  using signals provided by artificial sources external to the vehicle, e.g. navigation beacons · CPC title

• B25J9/1664

  characterised by motion, path, trajectory planning · CPC title

• G05D1/2435

  Extracting 3D information · CPC title