Generating an image-based identifier for a stretch wrapped loaded pallet based on images captured in association with application of stretch wrap to the loaded pallet

Patent metadata

Field	Value
Publication number	US-11228751-B1
Application number	US-202016811206-A
Country	US
Kind code	B1
Filing date	Mar 6, 2020
Priority date	Sep 22, 2015
Publication date	Jan 18, 2022
Grant date	Jan 18, 2022

Abstract

Methods, apparatus, systems, and computer-readable media are provided that relate to using one or more vision sensors to capture images of a loaded pallet in association with application of stretch wrap to the loaded pallet by an automated pallet wrapping machine. The images are used to generate an image-based identifier for the loaded pallet that is then used for pallet identification by mobile robots and/or other automated agents in a warehouse or other environment. In some implementations, the images are captured by the vision sensor when the pallet is in the wrapping area of the automated pallet wrapping machine and while the vision sensor and/or the pallet are rotating. In some implementations, the image-based identifier may be assigned to pallet attributes and/or a de-palletizing scheme of the loaded pallet.

First claim

What is claimed is:

1. A method, comprising:
capturing, by at least one vision sensor of a robot, one or more images that each capture a corresponding portion of stretch wrap applied around a plurality of boxes,
wherein the stretch wrap includes a plurality of non-transparent markings;

determining, based on comparing the image to a post-wrap image-based identifier previously stored in association with one or more attributes of the plurality of boxes, that the one or more images match the post-wrap image-based identifier,
wherein the post-wrap image-based identifier captures one or more visible features of the stretch wrap as applied around the plurality of boxes, and wherein the visible features of the stretch wrap as applied around the plurality of boxes include features of the non-transparent markings as they appear on the stretch wrap as applied around the plurality of boxes;

in response to determining that the one or more images match the post-wrap image-based identifier stored in association with the one or more attributes of the plurality of boxes:
performing, by the robot and to the plurality of boxes with the stretch wrap applied, at least one action that is based on at least one of the attributes.

2. The method of claim 1 , wherein the stretch wrap applied around the plurality of boxes is applied from a roll of stretch wrap and wherein the non-transparent markings are a sequence of non-transparent markings that repeat on the roll of the stretch wrap.

3. The method of claim 1 , wherein the non-transparent markings comprise one or more shapes, one or more alphanumeric characters, and/or one or more symbols.

4. The method of claim 3 , wherein the non-transparent markings comprise the one or more shapes.

5. The method of claim 4 , wherein the one or more shapes comprise a triangle, a circle, and/or a square.

6. The method of claim 1 , wherein the post-wrap image-based identifier is a three dimensional model generated based on the images.

7. The method of claim 1 , wherein the one or more attributes include one or more content identifiers of contents of the plurality of boxes.

8. The method of claim 1 , wherein the one or more attributes include a destination identifier of a destination of the plurality of boxes.

9. The method of claim 1 , wherein the at least one action comprises a repositioning action performed by the robot.

10. The method of claim 9 , wherein the at least one of the attributes, based on which the at least one action is performed, comprises a destination identifier of a destination of the plurality of boxes.

11. The method of claim 1 , wherein the robot is a pallet jack robot.

12. A robot, comprising:
a vision sensor;
actuators; and
one or more processors executing stored instructions to:
identify one or more images that are each captured by the vision sensor and that each capture a corresponding portion of stretch wrap applied around a plurality of boxes;
determine, based on comparing the image to a post-wrap image-based identifier previously stored in association with one or more attributes of the plurality of boxes, that the one or more images match the post-wrap image-based identifier,
wherein the post-wrap image-based identifier captures one or more visible features of the stretch wrap as applied around the plurality of boxes, and wherein the visible features of the stretch wrap as applied around the plurality of boxes include features of non-transparent markings, of the stretch wrap, as they appear on the stretch wrap as applied around the plurality of boxes;

in response to determining that the one or more images match the post-wrap image-based identifier stored in association with the one or more attributes of the plurality of boxes:
control one or more of the actuators to cause performance of at least one action that is based on at least one of the attributes, wherein the at least one action is directed to the plurality of boxes with the stretch wrap applied.

13. The robot of claim 12 , wherein the stretch wrap applied around the plurality of boxes is applied from a roll of stretch wrap and wherein the non-transparent markings are a sequence of non-transparent markings that repeat on the roll of the stretch wrap.

14. The robot of claim 12 , wherein the non-transparent markings comprise one or more shapes, one or more alphanumeric characters, and/or one or more symbols.

15. The robot of claim 14 , wherein the non-transparent markings comprise the one or more shapes.

16. The robot of claim 12 , wherein the post-wrap image-based identifier is a three dimensional model generated based on the images.

17. The robot of claim 12 , wherein the one or more attributes include one or more content identifiers of contents of the plurality of boxes.

18. The robot of claim 12 , wherein the one or more attributes include a destination identifier of a destination of the plurality of boxes.

19. The robot of claim 12 wherein the robot further comprises an end effector and wherein the at least one action comprises an action performed by the end effector.

Assignees

• X Dev Llc

Inventors

• Russell Jared Stephen
• Mason Julian Macneille

Classifications

• H04N25/71

  Charge-coupled device [CCD] sensors; Charge-transfer registers specially adapted for CCD sensors · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• H04N7/181

  for receiving images from a plurality of remote sources · CPC title

• G06T7/593

  from stereo images · CPC title

• G06T2207/10021

  Stereoscopic video; Stereoscopic image sequence · CPC title