Image/video analysis with activity signatures

Patent metadata

Field	Value
Publication number	US-11475669-B2
Application number	US-202016943140-A
Country	US
Kind code	B2
Filing date	Jul 30, 2020
Priority date	Jul 30, 2020
Publication date	Oct 18, 2022
Grant date	Oct 18, 2022

Abstract

Video frames from a video are compressed into a single image or a single data structure that represents a unique visual flowprint or visual signature for a given activity being modeled from the video frames. The flowprint comprises a computed summary of the original pixel values associated with the video frames within the single image and the flowprint is specific to movements occurring within the video frames that are associated with the given activity. In an embodiment, the flowprint is provided as input to a machine-learning algorithm to allow the algorithm to perform object tracking and monitoring from the flowprint rather than from the video frames of the video, which substantially improves processor load and memory utilization on a device that executes the algorithm, and substantially improved responsiveness of the algorithm.

First claim

The invention claimed is:

1. A method, comprising:
obtaining video frames from a video;
generating a single image from the video frames, wherein generating further includes tracking a region within each video frame associated with a modeled activity, wherein tracking further includes obtaining pixel values for each video frame associated with points along an expected path of movement for the modeled activity within the region, wherein obtaining the expected path further comprises determining an aggregated pixel value for each point along the expected path of movement across the video frames from the corresponding pixel values captured across the video frames for the corresponding point; and
providing the single image as a visual flowprint for an activity that was captured in the video frames.

2. The method of claim 1 , wherein determining further includes calculating the aggregated pixel value as an average of the corresponding pixel values across the video frames.

3. The method of claim 1 , wherein determining further includes selecting the aggregated pixel value as a minimum pixel value or a maximum pixel value associated with the corresponding pixel values across the video frames.

4. The method of claim 1 , wherein determining further includes calculating the aggregated pixel value as a standard deviation associated with the corresponding pixel values across the video frames.

5. The method of claim 1 , wherein determining further includes calculating the aggregated pixel value as an optical flow summary value representing a magnitude of movement along the expected path of movement for the corresponding pixel values across the video frames.

6. The method of claim 1 , wherein generating further includes removing known background pixel values from the video frames before generating the single image.

7. The method of claim 1 , wherein generating further includes removing pixel values from the video frames that are not associated with a region of interest for the modeled activity before generating the single image.

8. The method of claim 1 , wherein generating further includes extracting features from the video frames and generating the single image as a summary of each feature across the video frames.

9. The method of claim 1 , wherein providing further includes determining that a size of the single image is incompatible with a machine-learning algorithm that expects a second size for an input image provided as input to the machine-learning algorithm, normalizing the single image into the second size and providing the single image in the second size as the input image to the machine-learning algorithm.

Assignees

• Ncr Corp

Inventors

• Migdal Joshua
• Srinivasan Vikram

Classifications

• G06F18/24 Primary

  Classification techniques · CPC title

• H04N7/18

  Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast · CPC title

• G06F16/739 Primary

  in form of a video summary, e.g. the video summary being a video sequence, a composite still image or having synthesized frames · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title

• G06V20/47 Primary

  Detecting features for summarising video content · CPC title