Image processing for determining relationships between tracked objects

Patent metadata

Field	Value
Publication number	US-10824868-B2
Application number	US-201816144679-A
Country	US
Kind code	B2
Filing date	Sep 27, 2018
Priority date	Sep 27, 2018
Publication date	Nov 3, 2020
Grant date	Nov 3, 2020

Abstract

Cameras capture time-stamped images of predefined areas. The images are processed to make decisions as to when a person depicted in the images takes possession of or is disposed of an item depicted in the images. Possessed items are added to a shopping cart maintained for the person and dispossessed items are removed from the shopping cart.

First claim

The invention claimed is:

1. A method, comprising:
receiving an image that depicts an individual and an item;
obtaining attributes associated with the individual and the item, wherein obtaining further includes obtaining the attributes as a first bounding box defining a first position of the individual within the image and a second bounding box defining a second position of the item within the image, identifying a limb for the individual based on the first bounding box, wherein identifying further includes passing the first bounding box to a trained machine-learning algorithm and receiving back limb coordinates as output from the trained machine learning algorithm, wherein identifying further includes computing a Euclidean distance represented in the image between the limb coordinates and second bounding box coordinates for the second bounding box, wherein computing further includes determining that the individual is in possession of the item when the Euclidean distance is less than or equal to a predefined threshold distance; and
determining, based on processing the image with the attributes, whether the individual is in possession of the item;
providing an individual identifier for the individual, an item identifier for the item, and an indication that the individual was determined to be in possession of the item to a transaction manager for updating a shopping cart maintained for the individual.

2. The method of claim 1 , wherein obtaining the first bounding box further includes determining that the first bounding box intersects the second bounding box within the image.

3. The method of claim 2 , wherein determining further includes determining that the individual is in possession of the item based on the first bounding box intersecting the second bounding box within the image.

4. The method of claim 1 , wherein determining further includes determining that the individual has dispossessed the item based on a prior determination that the individual was possessed of the item combined with results from processing the image with the attributes.

5. A method, comprising:
training a machine-learning algorithm with first images comprising training bounding boxes within the first images for individuals and limbs of the individuals depicted as touching items to estimate limb coordinates within the first images;
receiving a second image having a first bounding box representing a specific individual within the second image and a second bounding box representing an item within the second image;
providing the first bounding box and the second image to the machine-learning algorithm;
receiving as an output from the machine-learning algorithm specific limb coordinates for the specific individual within the second image; and
determining based on at least the specific limb coordinates, the first bounding box, and the second bounding box whether the specific individual is depicted as being in possession of the item within the second image, wherein determining further includes computing a Euclidean distance between the specific limb coordinates and bounding box coordinates for the second bounding box, wherein computing further includes determining that the specific individual is in possession of the item when the Euclidean distance is less than or equal to a predefined threshold distance.

6. The method of claim 5 further comprising providing an individual tracking identifier for the specific individual, an item tracking identifier for the item, and a possession indication to a transaction manager for updating a shopping cart being maintained for the specific individual with the specific item.

7. The method of claim 5 , wherein determining further includes determining that the specific individual is in possession of the item based on the first bounding box at least partially intersecting the second bounding box within the second image.

8. The method of claim 5 , wherein determining further includes determining that the specific individual is in possession of the item when at least some of the specific limb coordinates intersect at least some of the bounding box coordinates.

9. The method of claim 5 , wherein determining further includes determining that the specific individual is dispossessed of the item based on a prior indication that the specific individual possessed the item combined with the output from the machine-learning algorithm.

10. The method of claim 5 , wherein determining further includes determining that the specific individual is dispossessed of the item when a second computed Euclidean distance between the specific limb coordinates and the bounding box coordinates for the second hounding box are equal to or greater than a predefined threshold distance combined with the prior indication.

Assignees

• Ncr Corp

Inventors

• Zucker Brent Vance
• Lieberman Adam Justin
• Somavarapu Nathan

Classifications

• G06V20/52

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title

• G06V40/107 Primary

  Static hand or arm · CPC title

• G06V20/20

  in augmented reality scenes · CPC title

• G06F18/21355

  nonlinear criteria, e.g. embedding a manifold in a Euclidean space · CPC title

• G06Q20/208

  Input by product or record sensing, e.g. weighing or scanner processing · CPC title