System for item placement into non-rigid containers

Patent metadata

Field	Value
Publication number	US-12020451-B2
Application number	US-202218078296-A
Country	US
Kind code	B2
Filing date	Dec 9, 2022
Priority date	Oct 1, 2019
Publication date	Jun 25, 2024
Grant date	Jun 25, 2024

Abstract

Examples provide a system and method for autonomously placing items into non-rigid containers. An image analysis component analyzes image data generated by one or more cameras associated with picked items ready for bagging and/or a non-rigid container, such as, but not limited to, a bag. The image analysis component generates dynamic placement data identifying how much space is available inside the bag, bag tension, and/or contents of the bag. A dynamic placement component generates a per-item assigned placement for a selected item ready for bagging based on a per-bag placement sequence and the dynamic placement data. Instructions, including the per-item assigned placement designating a location within the interior of the non-rigid container to the selected item and an orientation for the selected item after bagging, is sent to at least one robotic device. The robotic device places the selected item into the non-rigid container in accordance with the instructions.

First claim

What is claimed is:

1. A system for item placement into a container, the system comprising:
at least one processor; and
at least one memory comprising computer-readable instructions, the computer-readable instructions configured to cause the at least one processor to:
generate dynamic placement data associated with a container, wherein the dynamic placement data comprises available space data indicating a volume of unoccupied space within an interior of the container and content data describing contents within the interior of the container; and
identify a per-item assigned placement for a selected item ready for placement into the container in accordance with a per-bag placement sequence and the dynamic placement data, the per-item assigned placement comprising an assigned location of the selected item within the interior of the container and an assigned orientation of the selected item within the container, wherein a robotic device places the selected item into the container in accordance with the per-item assigned placement.

2. The system of claim 1 , wherein the computer-readable instructions are further configured to cause the at least one processor to:
generate instructions for placing the selected item into the container in accordance with the per-item assigned placement; and
send the instructions to the robotic device, the instructions including the per-item assigned placement for the selected item, wherein the robotic device places the selected item into the container based on the instructions.

3. The system of claim 1 , wherein the computer-readable instructions are further configured to cause the at least one processor to:
update the dynamic placement data based on a final location and orientation of the selected item after placement of the selected item inside the container by the robotic device.

4. The system of claim 1 , wherein the container is a non-rigid container, and further comprising:
an image capture device generating image data associated with the interior of the non-rigid container, wherein the computer-readable instructions are further configured to cause the at least one processor to:
identify a final location and a final orientation of the selected item inside the interior of the non-rigid container after the selected item settles within the non-rigid container using the image data; and
update the per-bag placement sequence based on the final location and the final orientation of the selected item.

5. The system of claim 1 , further comprising:
a sensor device that generates sensor data associated with the selected item within the interior of the container prior to placement of the selected item into the container.

6. The system of claim 1 , further comprising:
a tote subdivided into at least two compartments, wherein a first compartment supports a first non-rigid container for storing a first set of items and a second compartment supports a second non-rigid container for storing a second set of items.

7. The system of claim 1 , wherein the computer-readable instructions are further configured to cause the at least one processor to:
identify a first per-item placement of a first item and a second per-item placement of a second item in real-time as the first item and the second item are being placed into the container during bagging.

8. The system of claim 1 , further comprising:
a camera mounted on the robotic device generating image data of the interior of the container.

9. The system of claim 1 , further comprising:
a camera mounted above the container generating image data of the interior of the container.

10. A computer-implemented method for placing items into containers, the computer-implemented method comprising:
generating dynamic placement data associated with an interior of a container using image data received from an image capture device, the dynamic placement data including available space data indicating a volume of unoccupied space within the container and content data describing a first item within the interior of the container;
identifying a per-item assigned placement for a second item ready for placement into the container in accordance with a per-bag placement sequence and the dynamic placement data, the per-item assigned placement comprising an assigned location of the second item within the interior of the container and an assigned orientation of the second item, wherein a robotic device places the second item into the interior of the container in accordance with the per-item assigned placement; and
updating the dynamic placement data based on a final location and a final orientation of the second item after placement of the second item inside the container.

11. The computer-implemented method of claim 10 , further comprising:
generating instructions for placement of the second item into the interior of the container by the robotic device in accordance with the per-item assigned placement; and
sending the instructions to the robotic device, wherein the robotic device places the second item into the container in accordance with the instructions.

12. The computer-implemented method of claim 10 , further comprising:
placing, by the robotic device, a third item into the interior of the container in accordance with an updated per-item assigned placement, wherein the updated per-item assigned placement is generated based on a position of the first item and the second item within the container.

13. The computer-implemented method of claim 10 , further comprising:
updating the per-bag placement sequence based on updated image data showing a final location and orientation of the second item received from a set of image capture devices after the second item settles within the container.

14. The computer-implemented method of claim 10 , wherein the container is a non-rigid container, and further comprising:
analyzing image data to identify an amount of tension in the non-rigid container and a degree of fragility of an item already placed inside the non-rigid container prior to placement of the second item into the interior of the non-rigid container.

15. The computer-implemented method of claim 10 , wherein the container is a first non-rigid container, and further comprising:
generating the per-bag placement sequence for a set of items associated with a new order based on dynamic placement data for a next available non-rigid container, wherein the next available non-rigid container is used for bagging the set of items.

16. The computer-implemented method of claim 10 , further comprising:
identifying a first per-item placement of the first item and a second per-item placement of the second item in real-time as the items are being placed into the container during bagging.

17. One or more computer storage devices, having computer-executable instructions for a placing items into containers by a placement manager component that, when executed by a computer cause the computer to perform operations comprising:
generating dynamic placement data associated with a container, wherein the dynamic placement data comprises available space data describing a volume of unoccupied space within an interior of the container and content data describing a first item already placed within an interior of the container;
identifying a per-item assigned placement for a second item ready for placement into the container using a per-bag placement sequence and the dynamic placement data, the per-item assigned placement comprising an assigned location of the second item within the interior of the container and an assigned orientatio

Assignees

• Walmart Apollo Llc

Inventors

• Edwards James Benjamin
• Roth Brian C

Classifications

• H04N23/54

  Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils · CPC title

• G06V20/10

  Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) · CPC title

• G06T7/0004

  Industrial image inspection · CPC title

• B25J9/1697

  Vision controlled systems · CPC title

• B25J9/1669

  characterised by special application, e.g. multi-arm co-operation, assembly, grasping · CPC title