Defining and/or applying a planar model for object detection and/or pose estimation

Patent metadata

Field	Value
Publication number	US-10102629-B1
Application number	US-201514850913-A
Country	US
Kind code	B1
Filing date	Sep 10, 2015
Priority date	Sep 10, 2015
Publication date	Oct 16, 2018
Grant date	Oct 16, 2018

Abstract

Methods, apparatus, and computer readable storage media related to defining a planar model that approximates a plurality of surfaces of an object and/or applying the planar model to detect the object and/or to estimate a pose for the object. For example, the planar model may be compared to data points sensed by the a three-dimensional vision sensor of a robot to determine the object is present in the field of view of the sensor and/or to determine a pose for the object relative to the robot. A planar model comprises a plurality of planar shapes modeled relative to one another in a three-dimensional space and approximates an object by approximating one or more surfaces of the object with the planar shapes.

First claim

What is claimed is:

1. A method, comprising:
receiving, from a three-dimensional vision sensor of a robot, a group of data points sensed by the three-dimensional vision sensor;
retrieving, by one or more processors, a three-dimensional planar model that approximates a plurality of surfaces of an object, the planar model defining a plurality of planar shapes modeled relative to one another in a three-dimensional space,
wherein the object is a pallet that includes a plurality of stringers and wherein a first planar shape of the planar model approximates a first stringer surface of a first stringer end of a first stringer of the stringers and a second planar shape of the planar model approximates a second stringer surface of a second stringer end of a second stringer of the stringers, and
wherein the first stringer end and the second stringer end are both on a same side of the pallet;

calculating, by one or more of the processors and for each of a plurality of projections of the model to the data points, differences between the planar model and the data points for a corresponding projection of the projections;
determining, by one or more of the processors, that a projection of the projections satisfies a threshold based on the differences between the planar model and the data points for the projection; and
determining, by one or more of the processors, an object pose for the object relative to the robot based on a planar model pose of the planar model at the projection that satisfies the threshold.

2. The method of claim 1 , further comprising adjusting the robot based on the determined object pose.

3. The method of claim 2 , wherein the planar model further defines an action area relative to one or more of the planar shapes and wherein adjusting the robot is further based on the action area defined by the planar model.

4. The method of claim 3 , wherein the robot includes forks coupled to an elevator, and wherein adjusting the robot includes adjusting a robot pose of the robot to position the forks to encounter the action area.

5. The method of claim 1 , wherein the planar model does not approximate any surfaces of the first stringer and the second stringer that are in addition to the first stringer surface and the second stringer surface.

6. The method of claim 1 , wherein the pallet further includes a plurality of deckboards coupled to the stringers and wherein a third planar shape of the planar model approximates a deckboard of the deckboards, wherein the deckboard is more proximal to the first stringer end and the second stringer end than any other of the deckboards are to the first stringer end and the second stringer end.

7. The method of claim 6 , wherein the planar model does not approximate any surface of at least one of the deckboards.

8. The method of claim 1 , wherein a majority of the surfaces modeled by the planar model are modeled by the planar shapes.

9. The method of claim 8 , wherein a total quantity of the planar shapes defined by the model is less than fifteen.

10. The method of claim 1 , wherein all of the surfaces modeled by the planar model are modeled by the planar shapes.

11. The method of claim 1 , wherein the planar model is not computed from any three-dimensional scan data.

12. The method of claim 11 , wherein the planar model is generated based on human input provided via one or more user interface input devices.

13. The method of claim 1 , wherein the planar shapes are all polygons.

14. The method of claim 13 , wherein the polygons are all four-sided.

15. The method of claim 1 , further comprising:
translating an origin point of the planar model to a data point of the group of data points; and
generating each projection of a group of the projections based on a different rotation of the planar model when the origin point of the planar model is translated to the data point.

16. A robot, comprising:
at least one non-transitory computer readable storage medium storing a three-dimensional planar model that approximates surfaces of an object, the planar model defining a plurality of planar shapes modeled relative to one another in a three-dimensional space,
wherein the object is a pallet that includes a plurality of stringers and wherein a first planar shape of the planar model approximates a first stringer surface of a first stringer end of a first stringer of the stringers and a second planar shape of the planar model approximates a second stringer surface of a second stringer end of a second stringer of the stringers, and
wherein the first stringer end and the second stringer end are both on a same side of the pallet;

a three-dimensional vision sensor; and
one or more processors in communication with the non-transitory computer readable storage medium and receiving data generated by the three-dimensional vision sensor, wherein the one or more processors are configured to:
receive a group of data points sensed by the three-dimensional vision sensor;
calculate, for each of a plurality of projections of the model to the data points, differences between the planar model and the data points for a corresponding projection of the projections;
determine that a projection of the projections satisfies a threshold based on the differences between the planar model and the data points for the projection;
determine an object pose for the object relative to the robot based on a planar model pose of the planar model at the projection that satisfies the threshold; and
provide, to one or more actuators of the robot, control commands that are based on the determined object pose.

17. The robot of claim 16 , wherein the robot includes forks coupled to an elevator, and wherein the control commands adjust a robot pose of the robot to position the forks to encounter fork holes of the pallet.

18. The robot of claim 16 , wherein a majority of the surfaces modeled by the planar model are modeled by the planar shapes.

19. The robot of claim 16 , wherein all of the surfaces modeled by the planar model are modeled by the planar shapes.

20. The robot of claim 16 , wherein the planar model is not computed from any three-dimensional scan data.

21. At least one non-transitory computer-readable medium comprising instructions that, in response to execution of the instructions by one or more processors, cause the one or more processors to perform the following operations:
receiving, from a three-dimensional vision sensor of a robot, a group of data points sensed by the three-dimensional vision sensor;
retrieving a three-dimensional planar model that approximates a plurality of surfaces of an object, the planar model defining a plurality of planar shapes modeled relative to one another in a three-dimensional space,
wherein the object is a pallet that includes a plurality of stringers and wherein a first planar shape of the planar model approximates a first stringer surface of a first stringer end of a first stringer of the stringers and a second planar shape of the planar model approximates a second stringer surface of a second stringer end of a second stringer of the stringers, and
wherein the first stringer end and the second stringer end are both on a same side of the pallet;

calculating, for each of a plurality of projections of the model to the data points, differences between the planar model and the data points for a corresponding projection of the projections;
determining that a projection of the projections satisfies a threshold based on the differences between the planar m

Assignees

• X Dev Llc

Inventors

• Li Wen

Classifications

• G06T7/0046 Primary

  Physics · mapped topic

• G06T2207/30164

  Workpiece; Machine component · CPC title

• Y10S901/47

  Optical · CPC title

• G06K9/00201

  Physics · mapped topic

• Y10S901/01

  Mobile robot · CPC title