System and method for detecting a trigger event for identification of an item

Patent metadata

Field	Value
Publication number	US-12524890-B2
Application number	US-202318476425-A
Country	US
Kind code	B2
Filing date	Sep 28, 2023
Priority date	Jun 29, 2021
Publication date	Jan 13, 2026
Grant date	Jan 13, 2026

Abstract

A reference depth image of a platform is captured using a three-dimensional (3D) sensor. A plurality of secondary depth images are captured, wherein for each secondary depth image, a depth difference parameter is determined by comparing the secondary depth image and the reference depth image. In response to determining that the depth difference parameter has stayed constant at a value higher than zero for a duration of a first time interval, it is determined that an item has been placed on the platform.

First claim

The invention claimed is:

1 . An item tracking system, comprising:
a three-dimensional (3D) sensor positioned above a platform, wherein the 3D sensor is configured to capture overhead depth images of the platform, wherein:
each overhead depth image depicts upward-facing surfaces of objects placed on the platform;
each overhead depth image comprises a plurality of pixels distributed across the overhead depth image; and
each of the plurality of pixels is associated with a depth value indicating a distance between the 3D sensor and at least a portion of an upward-facing surface depicted by the pixel; and
one or more processors configured to:
capture a first overhead depth image of the platform using the 3D sensor;
determine a reference average depth value across the plurality of pixels in the first overhead depth image;
capture a plurality of second overhead depth images of the platform using the 3D sensor;
for each second overhead depth image:
determine a second average depth value across the plurality of pixels of the second overhead depth image; and
determine a value of a depth difference parameter by subtracting the second average depth value from the reference average depth value;

determine that the depth difference parameter has stayed constant at a value higher than zero for a duration of a first time interval; and
in response to determining that depth difference parameter has stayed constant at the value higher than zero, determine that an item has been placed on the platform.

2 . The item tracking system of claim 1 , wherein the one or more processors are further configured to determine that the depth difference parameter has stayed constant by:
capturing, at a first time instant within the first time interval, a third overhead depth image of the platform using the 3D sensor; determining a third average depth value across the plurality of pixels of the third overhead depth image; determining a first value of the depth difference parameter by subtracting the third average depth value from the reference average depth value; capturing, at a second time instant within the first time interval after the first time instant, a fourth overhead depth image of the platform using the 3D sensor; determine a fourth average depth value across the plurality of pixels of the fourth overhead depth image; determine a second value of the depth difference parameter by subtracting the fourth average depth value from the reference average depth value; determining a delta difference by:
subtracting the second value from the first value to generate a subtracted value; and
dividing the subtracted value by a time difference between the first time instant and the second time instant;
determining that the delta difference is zero or near zero; and in response to determining that the delta difference is zero or near zero, determining that the depth difference parameter has stayed constant between the first time instant and the second time instant.

3 . The item tracking system of claim 2 , wherein the one or more processors are further configured to:
determine that the first value and the second value of the depth difference parameter equal the value higher than zero; in response to determining that the delta difference is zero or near zero and that the first value and the second value of the depth difference parameter equal the value higher than zero, determine that the depth difference parameter has stayed constant at the value higher than zero between the first time instant and the second time instant; and in response to determining that the depth difference parameter has stayed constant at the value higher than zero between the first time instant and the second time instant, determine that the item has been placed on the platform.

4 . The item tracking system of claim 1 , wherein the one or more processors are further configured to:
detect that the value of the depth difference parameter has changed from zero to above a threshold value, wherein the change in the depth difference parameter from zero to above the threshold value indicates that a user's hand holding the item has moved from a first position that is outside a view of the 3D sensor to a second position on the platform that is within the view of the 3D sensor.

5 . The item tracking system of claim 4 , wherein the one or more processors are further configured to:
after detecting that the value of the depth difference parameter has changed from zero to above the threshold value:
determine that the depth difference parameter has decreased for a duration of a second time interval, wherein the second time interval is before the first time interval; and
in response to determining that the depth difference parameter has decreased for the duration of the second time interval, determine that the user's hand has placed the item on the platform and is moving away from the platform.

6 . The item tracking system of claim 5 , wherein the one or more processors are further configured to determine that the depth difference parameter has decreased for the duration of the second time interval by:
capturing, at a first time instant within the second time interval, a third overhead depth image of the platform using the 3D sensor; determining a third average depth value across the plurality of pixels of the third overhead depth image; determining a first value of the depth difference parameter by subtracting the third average depth value from the reference average depth value; capturing, at a second time instant within the second time interval after the first time instant, a fourth overhead depth image of the platform using the 3D sensor; determine a fourth average depth value across the plurality of pixels of the fourth overhead depth image; determine a second value of the depth difference parameter by subtracting the fourth average depth value from the reference average depth value; determining a delta difference by:
subtracting the second value from the first value to generate a subtracted value; and
dividing the subtracted value by a time difference between the first time instant and the second time instant;
determining that the delta difference is less than zero; and in response to determining that the delta difference is less than zero, determining that the depth difference parameter has decreased between the first time instant and the second time instant.

7 . The item tracking system of claim 5 , wherein the one or more processors are further configured to determine that the item is ready for identification in response to a combination of:
detecting that the value of the depth difference parameter has changed from zero to above a threshold value; determining that the depth difference parameter has decreased for the duration of the second time interval; and determining that depth difference parameter has stayed constant at the value higher than zero.

8 . The item tracking system of claim 7 , wherein the one or more processors are further configured to:
in response to determining that the item is ready for identification, initiate a procedure to identify the item.

9 . The item tracking system of claim 1 , wherein the one or more processors are further configured to:
detect motion near the platform using a proximity sensor; and capture the plurality of second overhead depth images in response to detecting the motion.

10 . The item tracking system of claim 1 , wherein the first overhead depth image is of the platform without any obstructions between the 3D sensor and a surface of the platform.

11 . A method for identifying an item, comprising:
capturing a first overhead depth

Assignees

• 7 Eleven Inc

Inventors

• Wang Xinan
• Dalal Kyle J
• Mirza Fahad
• Crain Jon Andrew

Classifications

• G06T2207/20076

  Probabilistic image processing · CPC title

• G06T2207/20224

  Image subtraction · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06T7/55

  from multiple images · CPC title

• G06T7/80

  Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration · CPC title