Detecting primitive events at checkout

Patent metadata

Field	Value
Publication number	US-9299229-B2
Application number	US-32517708-A
Country	US
Kind code	B2
Filing date	Nov 29, 2008
Priority date	Oct 31, 2008
Publication date	Mar 29, 2016
Grant date	Mar 29, 2016

Abstract

Techniques for detecting one or more events are provided. The techniques include identifying one or more segments in a video sequence as one or more candidates for one or more events by a temporal ordering of the one or more candidates, and analyzing one or more motion patterns of the one or more candidates to detect the one or more events.

First claim

What is claimed is:

1. A method for detecting one or more events, comprising:
generating a motion graph from a video sequence captured at a point-of-sale (POS) system, the motion graph representing human motion in the video sequence;
detecting two consecutive scan events at the POS system by identifying a first reference motion pattern in the motion graph;
identifying a segment of the video sequence between the two consecutive scan events; and
upon identifying the video sequence segment, analyzing a portion of the motion graph corresponding to the video sequence segment to detect at least one of a pick-up event and a drop event by identifying a second reference motion pattern in the motion graph.

2. The method of claim 1 ,
wherein the first motion pattern corresponds to one or more motion peaks in the motion graph.

3. The method of claim 1 , wherein analyzing the video sequence segment to detect at least one of a pick-up event and a drop event comprises:
detecting the second motion pattern using a portion of the motion graph corresponding to the video sequence segment, wherein the second motion pattern corresponds to a motion valley flanked by two motion peaks in the motion graph.

4. The method of claim 1 , wherein the motion graph comprises three portions each associated with three regions of interest included within the video sequence, each region of interest corresponding to the scan, pick-up, and drop events, respectively,
wherein monitoring the video sequence to detect two consecutive scan events comprises identifying the two consecutive scan events by monitoring the portion of the motion graph associated with the region of interest of the scan event,
wherein analyzing the video sequence segment comprises analyzing the portion of the motion graph corresponding to at least one of the pick-up event and drop event.

5. The method of claim 1 , wherein the pick-up event comprises a cashier picking up an item from a lead-in portion of the POS system, the scan event comprises the cashier at least one of reading the item via a scanner and weighing the item, and the drop event comprises the cashier placing the item onto a take-away portion of the POS system.

6. The method of claim 1 , wherein generating the motion graph based on the human motion comprises counting one or more motion pixels obtained by frame differencing the video sequence.

7. The method of claim 1 , further comprising, upon detecting the pick-up or drop event, combining one or more of the consecutive scan events and the pick-up or drop event according to one or more temporal ordering constraints to validate a transaction process of the POS system.

8. The method of claim 1 , wherein the two consecutive scan events are detected by determining whether a cashier's motion exceeds a threshold of motion.

9. A computer program product comprising a computer readable device having computer readable program code for detecting one or more events, said computer program product including:
computer readable program code for generating a motion graph from a video sequence captured at a point-of-sale (POS) system, the motion graph representing human motion in the video sequence;
computer readable program code for detecting two consecutive scan events at the POS system by identifying a first reference motion pattern in the motion graph;
computer readable program code for identifying a segment of the video sequence between the two consecutive scan events; and
computer readable program code for, upon identifying the video sequence segment, analyzing a portion of the motion graph corresponding to the video sequence segment to detect at least one of a pick-up event and a drop event at the POS system by identifying a second reference motion pattern in the motion graph.

10. The computer program product of claim 9 ,
wherein the first motion pattern corresponds to one or more motion peaks in the motion graph.

11. The computer program product of claim 9 , wherein analyzing the video sequence segment to detect at least one of a pick-up event and a drop event comprises:
computer readable program code for detecting the second motion pattern using a portion of the motion graph corresponding to the video sequence segment, wherein the second motion pattern corresponds to a motion valley flanked by two motion peaks in the motion graph.

12. The computer program product of claim 9 , wherein the motion graph comprises three portions each associated with three regions of interest included within the video sequence, each region of interest corresponding to the scan, pick-up, and drop events, respectively,
wherein monitoring the video sequence to detect two consecutive scan events comprises identifying the two consecutive scan events by monitoring the portion of the motion graph associated with the region of interest of the scan event,
wherein analyzing the video sequence segment comprises analyzing the portion of the motion graph corresponding to at least one of the pick-up event and drop event.

13. The computer program product of claim 9 , wherein the pick-up event comprises a cashier picking up an item from a lead-in portion of the POS system, the scan event comprises the cashier at least one of reading the item via a scanner and weighing the item, and the drop event comprises the cashier placing the item onto a take-away portion of the POS system.

14. The computer program product of claim 9 , further comprising, computer readable program code for, upon detecting the pick-up or drop event, combining one or more of the consecutive scan events and the pick-up or drop event according to one or more temporal ordering constraints to validate a transaction process of the POS system.

15. A system for detecting one or more events, comprising:
a computer processor; and
a memory containing a program that, when executed on the computer processor, performs an operation for processing data, comprising:
generating a motion graph from a video sequence captured at a point-of-sale (POS) system, the motion graph representing human motion in the video sequence;
detecting two consecutive scan events at the POS system by identifying a first reference motion pattern in the motion graph;
identifying a segment of the video sequence between the two consecutive scan events; and
upon identifying the video sequence segment, analyzing a portion of the motion graph corresponding to the video sequence segment to detect at least one of a pick-up event and a drop event by identifying a second reference motion pattern in the motion graph.

16. The system of claim 15 , wherein
wherein the first motion pattern corresponds to one or more motion peaks in the motion graph.

17. The system of claim 15 , wherein analyzing the video sequence segment to detect at least one of a pick-up event and a drop event comprises:
detecting the second motion pattern using a portion of the motion graph corresponding to the video sequence segment, wherein the second motion pattern corresponds to a motion valley flanked by two motion peaks in the motion graph.

18. The system of claim 15 , wherein the motion graph comprises three portions each associated with three regions of interest included within the video sequence, each region of interest corresponding to the scan, pick-up, and drop events, respectively,
wherein monitoring the video sequence to detect two consecutive scan events comprises identifying the two consecutive scan events by monitoring the portion of the motion graph associated with the region of interest of the scan event,
wherein analyzing th

Assignees

• Bobbitt Russell Patrick
• Fan Quanfu
• Hampapur Arun
• Kjeldsen Frederik
• Pankanti Sharathchandra Umapathirao
• Yanagawa Akira
• Zhai Yun
• Toshiba Global Commerce Solutions Holdings Corp

Inventors

• Bobbitt Russell Patrick
• Fan Quanfu
• Hampapur Arun
• Kjeldsen Frederik
• Pankanti Sharathchandra Umapathirao
• Yanagawa Akira
• Zhai Yun

Classifications

• G07G1/0063

  with means for detecting the geometric dimensions of the article of which the code is read, such as its size or height, for the verification of the registration · CPC title

• G07G3/003

  Anti-theft control (theft alarms G08B13/00) · CPC title

• G07G3/006

  False operation · CPC title

• G07F19/20

  Automatic teller machines [ATMs] · CPC title

• G07G3/00 Primary

  Alarm indicators, e.g. bells · CPC title