System and method for generating an activity summary of a person

Patent metadata

Field	Value
Publication number	US-10271017-B2
Application number	US-201313888941-A
Country	US
Kind code	B2
Filing date	May 7, 2013
Priority date	Sep 13, 2012
Publication date	Apr 23, 2019
Grant date	Apr 23, 2019

Abstract

In accordance with one aspect of the present technique, a method includes receiving one or more videos from one or more image capture devices. The method further includes generating a video-loop of the person from the one or more videos. The video-loop depicts the person in the commercial site. The method also includes generating an action clip from the video-loop. The action clip includes a suspicious action performed by the person in the commercial site. The method further includes generating an activity summary of the person including the video-loop and the action clip.

First claim

What is claimed as new and desired to be protected by Letters Patent of the United States is:

1. A method for generating an activity summary of a person in a commercial site, the method comprising:
receiving one or more videos from one or more image capture devices;
determining metadata associated with the person from the one or more videos, the metadata including location data of the person and an appearance descriptor that represents a spatial distribution of color corresponding to the person;
generating a video-loop of the person from the one or more videos by combining sets of images from the one or more videos based on a similarity of the metadata associated with the person, wherein the video-loop includes images of an entire trip, including all activities, of the person in the commercial site;
generating at least one action clip from the video-loop, wherein the at least one action clip includes a suspicious action performed by the person in the commercial site; and
generating the activity summary of the person including the video-loop and the at least one action clip.

2. The method of claim 1 , wherein the suspicious action includes at least one of grasping an object, removing a component from the object and hiding the object.

3. The method of claim 1 , wherein generating the at least one action clip further comprises:
analyzing one or more images of the video-loop to determine the suspicious action performed by the person;
determining an image analysis score for each of the one or more images based on the analysis of the one or more images;
identifying a suspicious image from the one or more images based on the one or more image analysis scores; and
generating the at least one action clip including the identified suspicious image from the video-loop.

4. The method of claim 3 , wherein the at least one action clip begins with the identified suspicious image.

5. The method of claim 1 , wherein generating the action dip further comprises:
analyzing one or more sequences of images from the video-loop to determine one or more motion features associated with the person;
identifying a suspicious sequence of images from the one or more sequences of images based on the one or more motion features; and
generating the at least one action clip including the suspicious sequence of images from the video-loop.

6. The method of claim 1 , wherein generating the video-loop of the person further comprises:
receiving a first video of the one or more videos from a first image capture device of the one or more image capturing devices
identifying a first set of images including the person from the first video;
receiving a second video of the one or more videos from a second image capture device of the one or more image capturing devices;
identifying a second set of images including the person from the second video; and
generating the video-loop of the person by combining the first set of images and the second set of images.

7. The method of claim 1 , further comprising:
determining whether the person is approaching an exit of the commercial site; and
sending the activity summary for display in response to determining that the person is approaching the exit of the commercial site.

8. A system for generating an activity summary of a person in a commercial site, the system comprising:
at least one processor;
a tracking module stored in a memory and, executable by the at least one processor, the tracking module for receiving one or more videos from one or more image capture devices, determining metadata associated with the person from the one or more videos, the metadata including location data of the person and an appearance descriptor that represents a spatial distribution of color corresponding to the person, and generating a video-loop of the person from the one or more videos by combining sets of images from the one or more videos based on a similarity of the metadata associated with the person, wherein the video-loop includes images of an entire trip, including all activities, of the person in the commercial site;
an analysis module stored in the memory and executable by the at least one processor, the analysis module communicatively coupled to the tracking module for generating at least one action clip from the video-loop, wherein the at least one action clip includes a suspicious action performed by the person in the commercial site; and
a summary generator stored in the memory and executable by the at least one processor, the summary generator communicatively coupled to the analysis module for generating the activity summary of the person including the video-loop and the at least one action clip.

9. The system of claim 8 , wherein the analysis module is further configured to:
analyze one or more images of the video-loop to determine the suspcious action performed by the person;
determine an image analysis score for each of the one or more images based on the analysis of the one or more images;
identify a suspicious image from the one or more images based on the one or more image analysis scores; and
generate the at least one action clip including the identified suspicious image from the video-loop.

10. The system of claim 8 , wherein the analysis module is further configured to:
analyze one or more sequences of images from the video-loop to determine one or more motion features associated with the person;
identify a suspicious sequence of images from the one or more sequences of images based on the one or more motion features; and
generate the at least one action clip including the suspicious sequence of images from the video-loop.

11. The system of claim 8 , wherein the tracking module is further configured to:
receive a first video of the one or more videos from a first image capture device of the one or more image capturing devices;
identify a first set of images including the person from the first video;
receive a second video of the one or more videos from a second image capture device of the one or more image capturing devices;
identify a second set of images including the person from the second video; and
generate the video-loop of the person by combining the first set of images and the second set of images.

12. The system of claim 8 , wherein the summary generator is further configured to determine whether the person is approaching an exit of the commercial site and send the activity summary for display in response to determining that the person is approaching the exit of the commercial site.

13. A computer program product comprising a non-transitory computer readable medium encoding instructions that, in response to execution by at least one processor, cause the processor to perform operations comprising:
receiving one or more videos from one or more image capture devices;
determining metadata associated with the person from the one or more videos, the metadata including location data of the person and an appearance descriptor that represents a spatial distribution of color corresponding to the person:
generating a video-loop of a person from the one or more videos by combining sets of images from the one or more videos based on a similarity of the metadata associated with the person, wherein the video-loop includes images of an entire trip, including all activities, of the person in a commercial site;
generating at least one action clip from the video-loop, wherein the at least one action clip includes a suspicious action performed by the person in the commercial site; and
generating an activity summary of the person including the video-loop and the at least one action clip.

14. The

Assignees

• Gen Electric

Inventors

• Tu Peter Henry
• Yu Ting
• Gao Dashan
• Yao Yi

Classifications

• H04N7/181 Primary

  for receiving images from a plurality of remote sources · CPC title

• H04N21/2187

  Live feed · CPC title

• G08B13/19602

  Image analysis to detect motion of the intruder, e.g. by frame subtraction · CPC title

• G11B27/28

  by using information signals recorded by the same method as the main recording {(G11B27/22 takes precedence)} · CPC title

• G08B13/19613

  Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion · CPC title