Perception and fitting for a stair tracker

Patent metadata

Field	Value
Publication number	US-11660752-B2
Application number	US-202117305522-A
Country	US
Kind code	B2
Filing date	Jul 9, 2021
Priority date	Apr 12, 2019
Publication date	May 30, 2023
Grant date	May 30, 2023

Abstract

A method for perception and fitting for a stair tracker includes receiving sensor data for a robot adjacent to a staircase. For each stair of the staircase, the method includes detecting, at a first time step, an edge of a respective stair of the staircase based on the sensor data. The method also includes determining whether the detected edge is a most likely step edge candidate by comparing the detected edge from the first time step to an alternative detected edge at a second time step, the second time step occurring after the first time step. When the detected edge is the most likely step edge candidate, the method includes defining, by the data processing hardware, a height of the respective stair based on sensor data height about the detected edge. The method also includes generating a staircase model including stairs with respective edges at the respective defined heights.

First claim

What is claimed is:

1. A computer-implemented method when executed by data processing hardware of a robot causes the data processing hardware to perform operations comprising:
determining that the robot is proximate a descending set of stairs, the robot comprising a front body portion having at least one front leg and a rear body portion having at least one hind leg, the at least one hind leg comprising:
an upper member;
a hip joint coupling the upper member to the rear body portion;
a lower member; and
a knee joint coupling the lower member to the upper member;

instructing the robot to assume an initial stair descent pose by:
orienting the rear body portion to face the descending set of stairs; and
placing the at least one of the hind leg in a stance phase, the stance phase defined by the knee joint extending further outward from the rear body portion toward the descending set of stairs than the hip joint; and

instructing the robot to descend the descending set of stairs starting from the initial stair descent pose.

2. The method of claim 1 , wherein the robot includes more vision sensors on the front body portion than on the rear body portion.

3. The method of claim 2 , wherein:
the vision sensors of the robot comprise stereo cameras; and
the robot includes two stereo cameras at the front body portion and one stereo camera at the rear body portion.

4. The method of claim 1 , wherein determining that the robot is proximate the descending set of stairs comprises determining that a support surface for the robot terminates at a floor edge adjacent a current location of the robot.

5. The method of claim 4 , wherein determining that the support surface for the robot terminates at the floor edge comprises:
determining that sensor data captured by the robot indicates that the support surface disappears in a straight line; and
identifying the location of the straight line as a top stair edge for a top stair of the descending set of stairs.

6. The method of claim 1 , wherein the operations further comprise:
receiving sensor data captured by one or more sensors associated with the robot; and
classifying the sensor data by height with respect to a gravitational axis of the robot, the sensor data classified as a floor classification corresponding to a height of a support surface for the robot and a below-floor classification corresponding to a height below the support surface in a direction away from the robot,
wherein determining that the robot is proximate the descending set of stairs comprises detecting that the robot is proximate the descending set of stairs by determining that the sensor data at a location adjacent the robot transitions from the floor classification to the below-floor classification.

7. The method of claim 6 , wherein the operations further comprise translating the sensor data from a three dimensional image space to a two dimensional image space, at least one dimension of the two dimensional image space corresponding to a height coordinate that indicates a height value for a volumetric point from the sensor data with respect to the gravitation axis of the robot.

8. The method of claim 6 , wherein the one or more sensors are disposed on the rear body portion of the robot and have a field of view extending away from the front body portion and toward the descending set of stairs.

9. The method of claim 1 , wherein the operations further comprise, based on determining that the robot is proximate the descending set of stairs, activating a stair mode to coordinate with one or more controllers dedicated to stair traversal.

10. The method of claim 1 , wherein the operations further comprise, while the robot is descending the descending set of stairs:
receiving sensor data captured by one or more sensors associated with the robot;
classifying the sensor data by height with respect to a gravitational axis of the robot, the sensor data classified as a stair tread classification corresponding to a height of a stair tread supporting the robot and a below-tread classification corresponding to a height below the stair tread support surface in a direction away from the robot; and
determining a foot placement location for the robot based on a cluster of below-tread classifications.

11. A robot comprising:
a front body portion having at least one front leg;
a rear body portion having at least one hind leg, the at least one hind leg comprising:
an upper member;
a hip joint coupling the upper member to the rear body portion;
a lower member; and
a knee joint coupling the lower member to the upper member; and

a stair traversal system configured to coordinate stair traversal for the robot, the stair traversal system comprising data processing hardware and memory hardware, the memory hardware including instructions that when executed by the data processing hardware perform operations comprising;
determining that the robot is proximate a descending set of stairs;
instructing the robot to assume an initial stair descent pose by:
orienting the rear body portion to face the descending set of stairs; and
placing the at least one of the hind leg in a stance phase, the stance phase defined by the knee joint extending further outward from the rear body portion toward the descending set of stairs than the hip joint; and

instructing the robot to descend the descending set of stairs starting from the initial stair descent pose.

12. The robot of claim 11 , wherein the robot includes more vision sensors on the front body portion than on the rear body portion.

13. The robot of claim 12 , wherein:
the vision sensors of the robot comprise stereo cameras; and
the robot includes two stereo cameras at the front body portion and one stereo camera at the rear body portion.

14. The robot of claim 11 , wherein determining that the robot is proximate the descending set of stairs comprises determining that a support surface for the robot terminates at a floor edge adjacent a current location of the robot.

15. The robot of claim 14 , wherein determining that the support surface for the robot terminates at the floor edge comprises:
determining that sensor data captured by the robot indicates that the support surface disappears in a straight line; and
identifying the location of the straight line as a top stair edge for a top stair of the descending set of stairs.

16. The robot of claim 11 , wherein the operations further comprise:
receiving sensor data captured by one or more sensors associated with the robot; and
classifying the sensor data by height with respect to a gravitational axis of the robot, the sensor data classified as a floor classification corresponding to a height of a support surface for the robot and a below-floor classification corresponding to a height below the support surface in a direction away from the robot,
wherein determining that the robot is proximate the descending set of stairs comprises detecting that the robot is proximate the descending set of stairs by determining that the sensor data at a location adjacent the robot transitions from the floor classification to the below-floor classification.

17. The robot of claim 16 , wherein the operations further comprise translating the sensor data from a three dimensional image space to a two dimensional image space, at least one dimension of the two dimensional image space corresponding to a height coordinate that indicates a height value for a volumetric point from the sensor data with respect to the gravitation axis of the robot.

18. The robot of claim 16 , whe

Assignees

• Boston Dynamics Inc

Inventors

• Whitman Eric Cary
• Merewether Gene Brown
• Fay Gina Christine
• Swilling Benjamin

Classifications

• G05D1/49

  Control of attitude, i.e. control of roll, pitch or yaw · CPC title

• G05D1/243

  Means capturing signals occurring naturally from the environment, e.g. ambient optical, acoustic, gravitational or magnetic signals (using passive navigation aids external to the vehicle G05D1/244; using signals from positioning sensors located off-board the vehicle G05D1/249) · CPC title

• B62D57/032

  with alternately or sequentially lifted supporting base and legs; with alternately or sequentially lifted feet or skid (B62D57/024 takes precedence) · CPC title

• B62D57/024 Primary

  specially adapted for moving on inclined or vertical surfaces (endless-track vehicles for ascending or descending stairs B62D55/075; hand-carts with provision for travelling up or down stairs B62B5/02) · CPC title

• B25J9/1666 Primary

  Avoiding collision or forbidden zones · CPC title