Singulation and sortation defect detection

Patent metadata

Field	Value
Publication number	US-11969760-B1
Application number	US-202117545945-A
Country	US
Kind code	B1
Filing date	Dec 8, 2021
Priority date	Dec 8, 2021
Publication date	Apr 30, 2024
Grant date	Apr 30, 2024

Abstract

Disclosed are systems and methods for detecting defects during the handling and processing of items at a material handling facility. Embodiments of the present disclosure can be implemented at material handling to detect quality defects during processing of items in connection with fulfillment operations that may be designed to receive, store, sort, pick, pack, and ship items. Embodiments of the present disclosure provides a plurality of sensors and a trained machine learning model configured to provide real-time acquisition and analysis of item information to facilitate detection and mitigation of processing quality defects.

First claim

What is claimed is:

1. An automated item handling system, comprising:
an automated item singulation system;
an automated item sortation system;
a conveyance configured to transport items from the automated item singulation system to the automated item sortation system;
a quality defect detection system including:
a first imaging device positioned and oriented at a first position relative to the conveyance to capture a first image of an item being transported on the conveyance from an overhead view of the item;
a second imaging device positioned and oriented at a first position relative to the conveyance to capture a second image of the item being transported on the conveyance from a front side of the item;
a third imaging device positioned and oriented at a second position relative to the conveyance to capture a third image of the item being transported on the conveyance from a rear of the item;
a trained machine learning model executing on a computing resource configured to, at least:
obtain at least one of the first image, the second image, and the third image as inputs; and
process at least one of the first image, the second image, and the third image to determine a quality defect and a quality defect type associated with the item; and

a control system in communication with the quality defect detection system and the automated item sortation system, the control system being configured to, at least:
cause, based at least in part on the determination of the quality defect and the quality defect type associated with the item, the automated item sortation system to sort the item into a container designated for items with quality defects of the quality defect type.

2. The automated item handling system of claim 1 , wherein the quality defect type includes at least one of:
a multiple;
a damaged item;
an unexpected item; or
an incomplete item.

3. The automated item handling system of claim 1 , wherein the trained machine learning model is further configured to, at least:
process at least one of the first image, the second image, and the third image to determine an identification of the item; and
the control system is further configured to, at least:
determine, based at least in part on the identification of the item, that item information associated with the item is stored in a datastore; and
update the item information with at least one of the first image, the second image, and the third image.

4. The automated item handling system of claim 3 , wherein the item information associated with the item includes at least one of a virtual item model or a catalog entry.

5. An automated item processing method, comprising:
ingesting a plurality of items into an automated item singulation system;
singulating, using the automated item singulation system, the plurality of items;
subsequent to singulation of the plurality of items, transporting a first item from the plurality of items, via a conveyance, from the automated item singulation system to an automated item sortation system;
obtaining, via a plurality of sensors, a plurality of item information associated with the first item while the first item is being transported via the conveyance;
processing, using a trained machine learning model, the plurality of item information to determine a quality defect and a quality defect type associated with the first item; and
sorting, using the automated item sortation system and based at least in part on the determination of the quality defect and the quality defect type associated with the first item, the first item into a container designated for items with quality defects of the quality defect type.

6. The automated item processing method of claim 5 , wherein the plurality of sensors includes at least one of:
an imaging device;
a scale;
a time-of-flight sensor;
an infrared sensor;
a depth sensor; or
a scanner.

7. The automated item processing method of claim 5 , wherein the quality defect type includes at least one of:
a multiple;
a damaged item;
an unexpected item; or
an incomplete item.

8. The automated item processing method of claim 7 , wherein the multiple indicates that the first item includes more than one item.

9. The automated item processing method of claim 8 , wherein the more than one items are overlapping such that one of the more than one items is substantially occluded.

10. The automated item processing method of claim 5 , wherein the plurality of item information includes at least one of:
an image of the first item;
a dimension of the first item;
a volume of the first item;
a point cloud of the first item; or
a weight of the first item.

11. The automated item processing method of claim 5 , further comprising:
aggregating, using the trained machine learning model the plurality of item information, prior to determination of the quality defect and the quality defect type.

12. The automated item processing method of claim 5 , further comprising:
subsequent to singulation of the plurality of items, transporting a second item from the plurality of items, via the conveyance, from the automated item singulation system to the automated item sortation system;
obtaining, via a plurality of sensors, a second plurality of item information associated with the second item while the second item is being transported by the conveyance;
processing, using the trained machine learning model, the second plurality of item information to determine an identification of the second item;
comparing the identification of the second item against a plurality of expected items to determine that the second item is not included in the plurality of expected items; and
sorting, using the automated item sortation system and based at least in part on the determination that the second item is not included in the plurality of expected items, the second item into a second container designated for unexpected items.

13. The automated item processing method of claim 5 , further comprising:
processing, using the trained machine learning model, the plurality of item information to determine an identification of the first item;
determining, based at least in part on the identification of the first item, that at least one of a virtual model or catalog information associated with the first item is stored in a datastore; and
updating at least one of the virtual model or the catalog information associated with the first item with the plurality of item information.

14. A defect detection system, comprising:
a plurality of sensors configured to capture a plurality of item information associated with an item being transported via a conveyance from an automated item singulation system to an automated item sortation system;
a trained machine learning model executing on a computing resource configured to, at least:
obtain the plurality of item information; and
process the plurality of item information to determine a quality defect and a quality defect type associated with the item; and

a control system in communication with the quality defect detection system and the automated item sortation system, the control system being configured to, at least, instruct, based at least in part on the determination of the quality defect and the quality defect type associated with the item, the automated item sortation system to sort the item into a container designated for items with quality defects of the quality defect type.

15. The defect detection system of claim 14 , wherein the plurality of sensors includes at least one of:
an imaging device;
a scale;
a time-of-flight sensor;
an infrar

Assignees

• Amazon Tech Inc

Inventors

• De La Rosa Jose Luis
• Xu Ting
• Pratt Duncan

Classifications

• G06N20/00

  Machine learning · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/08

  Learning methods · CPC title

• G07C3/146

  during manufacturing process · CPC title

• G06V20/64

  Three-dimensional [3D] objects · CPC title