Video analytics for retail business process monitoring

Patent metadata

Field	Value
Publication number	US-9158975-B2
Application number	US-82632407-A
Country	US
Kind code	B2
Filing date	Jul 13, 2007
Priority date	May 31, 2005
Publication date	Oct 13, 2015
Grant date	Oct 13, 2015

Abstract

A system for video monitoring a retail business process includes a video analytics engine to process video obtained by a video camera and generate video primitives regarding the video, A user interface is used to define at least one activity of interest regarding an area being viewed, each activity of interest identifying at least one of a rule or a query regarding the area being viewed. An activity inference engine processes the generated video primitives based on each defined activity of interest to determine if an activity of interest occurred in the video.

First claim

What is claimed is:

1. A method for video monitoring a retail business process comprising:
obtaining video from a video camera;
processing, by one or more processors within one or more devices, the video obtained from the video camera;
generating, by one or more processors within one or more devices, video primitives regarding the video, wherein the video primitives comprise a high-value item stealing video primitive;
defining at least one activity of interest regarding an area being viewed, wherein an activity of interest identifies at least one of a user-defined rule or a user-defined query regarding the area being viewed, wherein at least one of the user-defined rule or the user-defined query comprises detection of high-value item stealing based on counting, using only the video, a number of times a person removes high-value items off a shelf;
wherein the number of times the person removes the high-value items off the shelf is counted using only the video by analyzing a motion pattern in the video to detect periodic motion based on an array of motion block information;
wherein the array of motion block information comprises motion blocks associated with a person removing high-value items off the shelf, wherein the motion blocks are generated based on one or more foreground masks and include a predetermined number of foreground pixels, wherein the predetermined number of foreground pixels is a user-defined parameter;
processing, by one or more processors within one or more devices, the generated video primitives based on at least one defined activity of interest to determine that an activity of interest occurred in the video based on determining that the number of times the person removes the high-value items off the shelf exceeds a user-defined threshold; and
generating an alert based on determining that the number of times the person removes the high-value items off the shelf exceeds the user-defined threshold.

2. The method of claim 1 , wherein an array of motion block information associated with the person near the high-value item comprises positions and directions of motion blocks associated with the person near the high-value item.

3. A non-transitory computer-readable medium comprising software for video monitoring a retail business process, which software, when executed by a computer system, causes the computer system to perform operations comprising a method of:
processing video obtained by a video camera;
generating video primitives regarding the video, wherein the video primitives comprise a high-value item stealing video primitive;
defining at least one activity of interest regarding an area being viewed, wherein an activity of interest identifies at least one of a user-defined rule or a user-defined query regarding the area being viewed, wherein at least one of the user-defined rule or the user-defined query comprises detection of high-value item stealing based on counting, using only the video, a number of times a person removes high-value items off a shelf;
wherein the number of times the person removes the high-value items off the shelf is counted using only the video by analyzing a motion pattern in the video to detect periodic motion based on an array of motion block information;
wherein the array of motion block information comprises motion blocks associated with a person removing high-value items off the shelf, wherein the motion blocks are generated based on one or more foreground masks and include a predetermined number of foreground pixels, wherein the predetermined number of foreground pixels is a user-defined parameter;
processing the generated video primitives based on at least one defined activity of interest to determine that an activity of interest occurred in the video based on determining that the number of times the person removes the high-value items off the shelf exceeds a user-defined threshold; and
generating an alert based on determining that the number of times the person removes the high-value items off the shelf exceeds the user-defined threshold.

4. The non-transitory computer-readable medium of claim 3 , wherein the array of motion block information associated with the person near the high-value item comprises at least one of up directions or down directions of motion blocks associated with the person near the high-value item.

5. An apparatus for video monitoring a retail business process comprising:
a video camera configured to obtain video of an area;
a video analytics engine configured to process the obtained video and generate video primitives regarding the video, wherein the video primitives comprise a high-value item stealing video primitive; and
an activity inference engine configured to process the generated video primitives based on at least one activity of interest regarding an area being viewed to determine if an activity of interest occurred in the video, wherein an activity of interest defines at least one of a rule or a query selectively identified by a user regarding the area being viewed, wherein at least one of the rule or the query comprises detection of high-value item stealing based on counting, using only the video, a number of times a person removes high-value items off a shelf, wherein the number of times the person removes the high-value items off the shelf is counted using only the video by analyzing a motion pattern in the video to detect periodic motion based on an array of motion block information;
wherein the array of motion block information comprises motion blocks associated with the person removing the high-value items off the shelf, wherein the motion blocks are generated based on one or more foreground masks and include a predetermined number of foreground pixels, wherein the predetermined number of foreground pixels is a user-defined parameter; and
an alert interface engine coupled to the activity inference engine configured to generate an alert based on determining that the number of times the person removes the high-value items off the shelf exceeds a user-defined threshold.

6. The apparatus as in claim 5 , wherein the video analytics engine is resident within one of a chip, a chip set, or chips.

7. The apparatus as in claim 5 , wherein the video analytics engine and the activity inference engine are resident within one of a chip, a chip set, or chips.

8. The apparatus as in claim 5 , further including:
a plurality of video cameras which at least one obtains video of an associated area, wherein the video analytics engine processes the obtained video of at least one area and generates video primitives regarding the video and wherein the activity inference engine processes the generated video primitives based on at least one activity of interest regarding at least one area being viewed to determine if an activity of interest occurred in an associated video.

9. A system for video monitoring a retail business process comprising:
one or more processors within one or more devices configured to execute:
a video analytics engine configured to process video obtained by a video camera and to generate video primitives regarding the video, wherein the video primitives comprise a high-value item stealing video primitive;
a user interface configured to define at least one activity of interest regarding an area being viewed, wherein an activity of interest identifies at least one of a user-defined rule or a user-defined query regarding the area being viewed, wherein at least one of the user-defined rule or the user-defined query comprises detection of high-value item stealing based on counting, using only the video, a number of times a person removes high-value items off a shelf, wherein the number of times the person removes the high-value items off the shel

Assignees

• Lipton Alan J
• Venetianer Peter L
• Haering Niels
• Brewer Paul C
• Yin Weihong
• Zhang Zhong
• Yu Li
• Hu Yongtong
• Myers Gary W
• Chosak Andrew J
• Cutting Robert A
• Scanlon W Andrew
• Avigilon Fortress Corp

Inventors

• Lipton Alan J
• Venetianer Peter L
• Haering Niels
• Brewer Paul C
• Yin Weihong
• Zhang Zhong
• Yu Li
• Hu Yongtong
• Myers Gary W
• Chosak Andrew J
• Cutting Robert A
• Scanlon W Andrew

Classifications

• G06V20/52

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title

• G08B13/19608 Primary

  Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position · CPC title

• G06F17/3079

  Physics · mapped topic

• G06K9/00369

  Physics · mapped topic

• G06T2207/30232

  Surveillance · CPC title