Optimization-based spring lattice deformation model for soft materials

Patent metadata

Field	Value
Publication number	US-10766149-B2
Application number	US-201815934498-A
Country	US
Kind code	B2
Filing date	Mar 23, 2018
Priority date	Mar 23, 2018
Publication date	Sep 8, 2020
Grant date	Sep 8, 2020

Abstract

Techniques for controlling a robotic picking arm using estimated seal quality metrics. A plurality of candidate contact points for holding an item using a suction device of the robotic picking arm, based on captured images of the item and an n-dimensional surface model of the item. An expected seal quality metric for a first one of the candidate contact points, by processing the n-dimensional surface model of the item and physical properties of the suction device of the robotic picking arm. Based on the expected seal quality metric, embodiments can determine whether to retrieve the item from the storage container by holding the item at the first candidate contact point using the suction device of the robotic picking arm.

First claim

What is claimed is:

1. A method, comprising:
capturing one or more images of an item within a storage container to be retrieved using a robotic picking arm having a suction device;
retrieving a n-dimensional surface model of the item, based on a unique identifier corresponding to the item;
determining a plurality of candidate contact points for holding the item using the suction device of the robotic picking arm, based on the captured one or more images of the item and the n-dimensional surface model of the item;
determining an expected seal quality metric for a first one of the candidate contact points, by processing the n-dimensional surface model of the item and physical properties of the suction device of the robotic picking arm as inputs, comprising:
modelling deformation and contact between the suction device and the n-dimensional surface model at the first candidate contact point, using at least one n-dimensional spring-lattice to model deformation and contact between the suction device and the n-dimensional surface model at the first candidate contact point, such that potential energy across the n-dimensional spring-lattice is minimized; and
calculating the expected seal quality metric, using the n-dimensional spring-lattice and a fluids model relating fluid force and leak pathways; and

determining, based on the expected seal quality metric, whether to retrieve the item from the storage container by holding the item at the first candidate contact point using the suction device of the robotic picking arm.

2. The method of claim 1 , further comprising:
upon determining to retrieve the item from the storage container by holding the item at the first candidate contact point using the suction device of the robotic picking arm, transmitting an instruction to control logic for the robotic picking arm, the instruction specifying at least location information identifying the first candidate contact point.

3. The method of claim 1 , further comprising:
upon determining not to retrieve the item from the storage container by holding the item at the first candidate contact point using the suction device of the robotic picking arm:
selecting a second one of the plurality of candidate contact points;
determining a second expected seal quality metric for the second candidate contact point; and
determining, based on the second expected seal quality metric, whether to retrieve the item from the storage container by holding the item at the second candidate contact point using the suction device of the robotic picking arm.

4. The method of claim 3 , wherein determining a second expected seal quality metric for the second candidate contact point further comprises:
modelling deformation and contact between the suction device and the second candidate contact point, using at least one n-dimensional spring-lattice to model deformation and contact between the suction device and the second candidate contact point for the item, such that potential energy across the n-dimensional spring-lattice is minimized; and
calculating the expected seal quality metric, using the n-dimensional spring-lattice for the second candidate contact point and the fluids model.

5. The method of claim 1 , further comprising:
determining a pose of the item within the storage container, based on the captured one or more images of the item and the retrieved n-dimensional surface model for the item, wherein determining the plurality of candidate contact points for holding the item using the suction device of the robotic picking arm, is further based on the determined pose of the item within the storage container.

6. The method of claim 5 , wherein determining the plurality of candidate contact points for holding the item using the suction device of the robotic picking arm further comprises:
retrieving an item profile, based on the unique identifier corresponding to the item, wherein the item profile specifies a set of candidate contact points for the item; and
determining that the plurality of candidate contact points are available, based on the determined pose of the item within the storage container, wherein the plurality of candidate contact points is a subset of the set of candidate contact points for the item within the item profile.

7. A system, comprising:
a robotic picking arm having a suction device adapted for retrieving items and reporting the items to a determined location;
one or more computer processors; and
control logic for the robotic picking arm that, when executed by operation of the one or more computer processors, performs an operation, comprising:
determining a first candidate contact point for holding a first item using the suction device of the robotic picking arm, based on a surface geometry of the first item;
estimating a seal quality metric for the first candidate contact point, based on a predicted deformed n-dimensional shape of the suction device and a n-dimensional shape associated with the first item, comprising generating the predicted deformed n-dimensional shape of the suction device by modelling deformation and contact between the suction device and the n-dimensional shape associated with the first item at the first candidate contact point, using at least a first n-dimensional spring-lattice to model deformation and contact between the suction device and the n-dimensional shape associated with the first item at the first candidate contact point, such that potential energy across the first n-dimensional spring-lattice is minimized; and
upon determining that the estimated seal quality metric for the first candidate contact point satisfies one or more predefined criteria, controlling movement of the robotic picking arm to retrieve the first item by holding the first item at the first candidate contact point.

8. The system of claim 7 , wherein a first predefined criteria of the one or more predefined criteria further comprises the estimated seal quality metric exceeding a predefined threshold level of seal quality.

9. The system of claim 7 , wherein estimating the seal quality metric for the first candidate contact point, based on the predicted deformed n-dimensional shape of the suction device and the n-dimensional shape associated with the first item, further comprises calculating the seal quality metric, using the first n-dimensional spring-lattice and a fluids model relating fluid force and leak pathways.

10. The system of claim 9 , wherein estimating the seal quality metric for the first candidate contact point, based on the predicted deformed n-dimensional shape of the suction device and the n-dimensional shape associated with the first item, further comprises:
retrieving an item profile corresponding the first item and specifying data describing physical attributes of the first item; and
generating a second predicted deformed n-dimensional shape for the first item, using at least a second n-dimensional spring-lattice to model deformation and contact between the suction device and the first candidate contact point of the first item, such that potential energy across the first n-dimensional spring-lattice is minimized,
wherein calculating the seal quality metric is further based on the second predicted deformed n-dimensional shape for the first item.

11. The system of claim 9 , wherein estimating the seal quality metric for the first candidate contact point, based on the predicted deformed n-dimensional shape of the suction device and the n-dimensional shape associated with the first item, further comprises:
determining a second n-dimensional shape associated with the first item,
wherein calculating the seal quality metric is further based on the second n-dimensional shape.

Assignees

• Amazon Tech Inc

Inventors

• Marchese Andrew D
• Flannigan William Clay
• Wellman Parris S

Classifications

• G05B2219/39514

  Stability of grasped objects · CPC title

• G05B2219/39484

  Locate, reach and grasp, visual guided grasping · CPC title

• B25J9/1697

  Vision controlled systems · CPC title

• B25J9/1612 Primary

  characterised by the hand, wrist, grip control · CPC title

• B25J9/0009

  Constructional details, e.g. manipulator supports, bases · CPC title