Simulated box placement for algorithm evaluation and refinement

Patent metadata

Field	Value
Publication number	US-12258227-B2
Application number	US-202217837779-A
Country	US
Kind code	B2
Filing date	Jun 10, 2022
Priority date	Jun 16, 2021
Publication date	Mar 25, 2025
Grant date	Mar 25, 2025

Abstract

A robotic system is disclosed. The system includes a memory that stores for each of a plurality of items a set of attribute values. The system includes a processor(s) that uses the attribute values to simulate the placement of items, including by determining, iteratively, for each next item a placement location at which to place the item on a simulated stack of items on the pallet, using the attribute values and a geometric model of where items have been simulated to have been placed to estimate a state of the stack after each of a subset of simulated placements, and using the estimated state to inform a next placement decision. The steps of determining for each next item a placement location and estimating the state of the stack until all of at least a subset of the plurality of items have been simulated as having been placed on the stack.

First claim

The invention claimed is:

1. A robotic system, comprising:
a memory configured to store for each of a plurality of items a set of attribute values representing one or more physical attributes of the item; and
one or more processors coupled to the memory and configured to:
use at least a subset of the set of attribute values to simulate the successive placement of items comprising the plurality of items on a pallet or other receptacle, including by:
determining a search space of potential placements for the plurality of items;
determining, iteratively, for each next item a corresponding placement location at which to place the item on a simulated stack of items on the pallet or other receptacle, wherein the corresponding placement location is selected based at least in part on the search space of potential placements;
using the attribute values and a geometric model of where items have been simulated to have been placed to estimate a state of the stack of items after each of at least a subset of simulated placements;
in response to determining that the estimated state of the stack of items does not satisfy a predefined criteria, pruning the search space to remove the corresponding placement location and placement locations that are downstream of the corresponding placement location;
using the estimated state to inform a next placement decision, including determining a plan for placing a next item; and
causing a robot to be controlled to place the next item according to the plan;

wherein the steps of determining for each next item a corresponding placement location and estimating the state of the stack are performed until all of at least a subset of the plurality of items have been simulated as having been placed on the stack.

2. The robotic system of claim 1 , wherein an item placement comprises a placement of the item in a particular location.

3. The robotic system of claim 1 , wherein the memory is further configured to store one or more state estimation models used to estimate the state of the stack of items.

4. The robotic system of claim 3 , wherein the one or more processors are further configured to run a plurality of simulations to evaluate the one or more state estimation models.

5. The robotic system of claim 3 , wherein the memory is further configured to store a plurality of item pick and place models.

6. The robotic system of claim 5 , wherein the plurality of item pick and place models correspond to palletization models.

7. The robotic system of claim 5 , wherein the one or more processors are further configured to run a plurality of simulations to evaluate the plurality of pick and place models.

8. The robotic system of claim 7 , wherein two or more of the plurality of simulations are executed successively.

9. The robotic system of claim 7 , wherein two or more of the plurality of simulations are executed in parallel.

10. The robotic system of claim 1 , wherein an estimation of the state of the stack of items includes a measure of one or more of density and stability.

11. The robotic system of claim 1 , wherein the subset of the simulated placements are rigid body simulations.

12. The robotic system of claim 1 , wherein one or more of the subset of simulated placements include an addition of noise to simulate a non-idealized result of placing one or more items as simulated.

13. The robotic system of claim 1 , wherein the one or more processors are further configured to select a next placing by using a scoring function value for each of a plurality of candidate item placements.

14. The robotic system of claim 1 , wherein the one or more processors are further configured to:
determine, for each of a plurality of candidate placements for a next item to be placed, a corresponding score with respect to the scoring function;
determine a current state value associated with a current state of the pallet; and
select a selected placement based at least in part on the respective scores for the plurality of candidate placements.

15. The robotic system of claim 14 , wherein the one or more processors are further configured to control a robotic arm to place a next item according to the selected placement.

16. The robotic system of claim 1 , wherein the one or more processors are further configured to simulate an effect of an external force applied to the pallet or the stack of items.

17. The robotic system of claim 16 , wherein the external force for which the effect is simulated comprises a shaking of the pallet.

18. The robotic system of claim 16 , wherein the state of the stack of items is re-evaluated in response to the simulated effect of the external force.

19. The robotic system of claim 18 , wherein re-evaluating the state of the stack of items comprises one or more of:
determining a stability of the stack of items;
determining a density of the stack of items; and
determining whether an item fell off the stack of items.

20. A method to control a robot, comprising:
storing for each of a plurality of items a set of attribute values representing one or more physical attributes of the item;
using at least a subset of the set of attribute values to simulate the successive placement of items comprising the plurality of items on a pallet or other receptacle, including by:
determining a search space of potential placements for the plurality of items;
determining, iteratively, for each next item a corresponding placement location at which to place the item on a simulated stack of items on the pallet or other receptacle, wherein the corresponding placement location is selected based at least in part on the search space of potential placements;
using the attribute values and a geometric model of where items have been simulated to have been placed to estimate a state of the stack of items after each of at least a subset of simulated placements;
in response to determining that the estimated state of the stack of items does not satisfy a predefined criteria, pruning the search space to remove the corresponding placement location and placement locations that are downstream of the corresponding placement location; and
using the estimated state to inform a next placement decision, including determining a plan for placing a next item; and
causing a robot to be controlled to place the next item according to the plan;

wherein the steps of determining for each next item a corresponding placement location and estimating the state of the stack until all of at least a subset of the plurality of items have been simulated as having been placed on the stack.

21. A computer program product to control a robot, the computer program product being embodied in a non-transitory computer readable medium and comprising computer instructions for:
storing for each of a plurality of items a set of attribute values representing one or more physical attributes of the item;
using at least a subset of the set of attribute values to simulate the successive placement of items comprising the plurality of items on a pallet or other receptacle, including by:
determining a search space of potential placements for the plurality of items;
determining, iteratively, for each next item a corresponding placement location at which to place the item on a simulated stack of items on the pallet or other receptacle,

wherein the corresponding placement location is selected based at least in part on the search space of potential placements;
using the attribute values and a geometric model of where items have be

Assignees

• Dexterity Inc

Inventors

• Pidaparthi Rohit Arka
• Clary William Arthur
• Abhyankar Neeraja
• Kuck Jonathan
• Pottayil Ben Varkey Benjamin
• Chavez Kevin Jose
• Kumar Shitij

Classifications

• B25J9/1671

  characterised by simulation, either to verify existing program or to create and verify new program, CAD/CAM oriented, graphic oriented programming systems · CPC title

• B25J9/1664

  characterised by motion, path, trajectory planning · CPC title

• G05B2219/40607

  Fixed camera to observe workspace, object, workpiece, global · CPC title

• G05B2219/45063

  Pick and place manipulator · CPC title

• G05B2219/40006

  Placing, palletize, un palletize, paper roll placing, box stacking · CPC title