Systems and methods for detecting and tracking humans in captured images

Patent metadata

Field	Value
Publication number	US-12094130-B2
Application number	US-202117389154-A
Country	US
Kind code	B2
Filing date	Jul 29, 2021
Priority date	Jul 30, 2020
Publication date	Sep 17, 2024
Grant date	Sep 17, 2024

Abstract

In some embodiments, apparatuses and methods are provided herein useful to detecting and tracking humans. In some embodiments, there is provided a system for detecting and tracking humans from one image to another image including: a camera; a control circuit configure to: receive a first image; detect a plurality of key body joints of a first human captured on the first image; determine segmentations of the plurality of key body joints to determine one or more body parts of the first human; determine a color distribution map of aggregate pixels associated with each body part of the first human on the first image; and cause a database to store the color distribution map; and a database comprising one or more color distribution map sets each associated with a detected human in a captured image of the camera.

First claim

What is claimed is:

1. A system for detecting and tracking humans from one image to another image captured by a camera at a retail store, the system comprising:
a camera configured to capture, at a first time, a first image of an area at a retail store;
a control circuit coupled to the camera, the control circuit configured to:
receive the first image;
detect a plurality of key body joints of a first human captured on the first image, wherein each of the plurality of key body joints is a point of interest along a skeletal anatomy of the first human;
in response to the detection of the plurality of key body joints, determine segmentations of the plurality of key body joints of the first human to determine one or more body parts of the first human;
determine a color distribution map of aggregate pixels associated with each body part of the one or more body parts of the first human on the first image, wherein each color distribution map is used by the control circuit to differentiate between the first human and another human in the first image;
cause a database to store the color distribution map for each body part of the first human in the first image; wherein the database coupled to the control circuit, the database comprising one or more color distribution map sets each associated with a detected human in a captured image of the camera;

the camera is further configured to capture, at a second time, a second image of the area;
the control circuit is further configured to:
receive the second image;
determine a color distribution map of aggregate pixels associated with each body part of one or more body parts of a second human in the second image;
calculate a correlation value for each color distribution map associated with each body part of the first human with each color distribution map associated with each corresponding body part of the second human;
determine that the first human in the first image is the second human in the second image based on at least one of: a determination that each calculated correlation value is equal to at least a correlation threshold and a determination that a threshold number of calculated correlation values is equal to at least the correlation threshold; and
the control circuit is further configured to cause the database to update a previously stored color distribution map sets associated with the first human by merging the previously stored color distribution map sets with another color distribution map sets of the second human in the second image in response to the determination that the first human in the first image is the second human in the second image.

2. The system of claim 1 , wherein the camera comprises a closed-circuit television camera.

3. The system of claim 1 , wherein the one or more body parts comprise a pair of feet, a head, a hand, a hip, a pair of arms, a torso, a pair of thighs, a pair of entire legs, a neck, a pair of forearms, and a shoulder.

4. The system of claim 1 , wherein the control circuit is further configured to represent the color distribution map as a color histogram.

5. The system of claim 1 , wherein the merging of the previously stored color distribution map sets associated with a first human with another color distribution map sets of a second human in a second image updates a number of pixels associated with each color associated with a first body part in the first image with a number of pixels associated with each color associated with a second body part in the second image, the second body part in the second image being the same as the first body part in the first image.

6. The system of claim 5 , wherein the control circuit is further configured to denormalize the first and second the color distribution maps prior to being merged, perform the merging and/or updating, and then normalize the updated color distribution map prior to causing a database to store the normalized updated color distribution map.

7. The system of claim 5 , wherein the control circuit is further configured to:
determine that the first human in the first image is not the second human in the second image based on a determination that one or more of each calculated correlation value is not equal to at least the correlation threshold; and
cause the database to store the color distribution map for each body part of the second human on the second image, wherein the color distribution map associated with the second human is used to identify whether the second human is detected in a third image of the area captured by the camera.

8. The system of claim 1 , wherein the control circuit is further configured to:
calculate a correlation value for each color distribution map associated with each body part of the first human on the first image with a stored color distribution map for each body part of a reference retail associate to identify whether the first human is a retail associate of the retail store; and
determine that the first human in the first image is the retail associate of the retail store based on at least one of: a determination that each calculated correlation value is equal to at least a correlation threshold and a determination that a threshold number of calculated correlation values is equal to at least the correlation threshold.

9. The system of claim 8 , wherein the control circuit is further configured to:
determine a count of humans that are not identified as one of retail associates of the retail store; and
in response to the count reaching a count threshold, provide an alert message to an electronic device associated with the retail store, wherein the alert message summons another retail associate to the area.

10. The system of claim 1 , wherein the determination of segmentations of the plurality of key body joints of the first human is based on Part Affinity Field (PAF) mapping.

11. A method for detecting and tracking humans from one image to another image captured by a camera at a retail store, the method comprising:
capturing, at a first time by a camera, a first image of an area at a retail store;
receiving, by a control circuit coupled to the camera, the first image;
detecting, by the control circuit, a plurality of key body joints of a first human captured on the first image, wherein each of the plurality of key body joints is a point of interest along a skeletal anatomy of the first human;
in response to the detection of the plurality of key body joints, determining segmentations of the plurality of key body joints of the first human to determine one or more body parts of the first human;
determining, by the control circuit, a color distribution map of aggregate pixels associated with each body part of the one or more body parts of the first human on the first image, wherein each color distribution map is used by the control circuit to differentiate between the first human and another human in the first image;
causing, by the control circuit, a database to store the color distribution map for each body part of the first human in the first image, wherein the database comprises one or more color distribution map sets each associated with a detected human in a captured image of the camera;
capturing, at a second time, by the camera is further configured to capture, at a second time, a second image of the area;
receiving, by the control circuit, the second image;
determining, a color distribution map of aggregate pixels associated with each body part of one or more body parts of a second human on the second image;
calculating, a correlation value for each color distribution map associated with each body part of the first human with each color distribution map associated with each corresponding body part of

Assignees

• Walmart Apollo Llc

Inventors

• Liu Haining
• Reagan Ryan B
• Xiao Zhichun

Classifications

• G06V40/103

  Static body considered as a whole, e.g. static pedestrian or occupant recognition · CPC title

• G06Q30/0201 Primary

  Market modelling; Market analysis; Collecting market data · CPC title

• G06T2207/30242

  Counting objects in image · CPC title

• G06T2207/30196

  Human being; Person · CPC title

• G06F16/5838

  using colour · CPC title