Systems and methods for analyzing a video stream

Patent metadata

Field	Value
Publication number	US-9501915-B1
Application number	US-201514737963-A
Country	US
Kind code	B1
Filing date	Jun 12, 2015
Priority date	Jul 7, 2014
Publication date	Nov 22, 2016
Grant date	Nov 22, 2016

Abstract

The various embodiments described herein include methods, devices, and systems for analyzing video streams. In one aspect, a method includes, while receiving a video stream: obtaining motion start information indicating that a portion of the video stream includes a motion event candidate; and segmenting the portion of the video stream into a plurality of segments including an initial segment. The method also includes obtaining a first categorization for the motion event candidate based on the initial segment; and, in accordance with the obtained first categorization, generating a log entry for the motion event candidate including the first categorization. The method further includes: in response to obtaining motion end information, obtaining a second categorization for the motion event based on the plurality of segments; and updating the log entry for the motion event candidate based on the obtained second categorization.

First claim

What is claimed is:

1. A method, comprising:
at a server system having one or more processors and memory:
while receiving video information from one or more cameras, the video information including a video stream:
obtaining motion start information corresponding to a first time in the video stream, the motion start information indicating that a portion of the video stream subsequent to the first time includes a motion event candidate;
while receiving the portion of the video stream that includes the motion event candidate, segmenting the portion of the video stream into a plurality of temporal segments, the plurality of temporal segments including an initial segment;
obtaining a first categorization of a plurality of categorizations for the motion event candidate based on the initial segment;
in accordance with the obtained first categorization, generating an alert indicative of the first categorization for the motion event candidate;
obtaining motion end information corresponding to a second time in the video stream, the motion end information indicating that a portion of the video stream subsequent to the second time does not include the motion event candidate;
in response to obtaining the motion end information, obtaining a second categorization of the plurality of categorizations for the motion event based on the plurality of temporal segments;
determining whether the second categorization is the same as the first categorization; and
in accordance with a determination that the second categorization is not the same as the first categorization, generating an updated alert indicative of the second categorization for the motion event category.

2. The method of claim 1 , wherein the video information is associated with a user; and
the method further comprises sending the alert to the user.

3. The method of claim 1 , further comprising obtaining a confidence score corresponding to the obtained first categorization; and
wherein generating the alert for the motion event candidate includes generating the alert in accordance with a determination that the confidence score meets predefined criteria.

4. The method of claim 1 , further comprising:
obtaining a confidence score corresponding to the obtained first categorization; and
storing the confidence score to a log entry for the motion event candidate.

5. The method of claim 1 , further comprising:
in accordance with a determination that the second categorization is not the same as the first categorization, removing the first categorization from a log entry for the motion event candidate.

6. The method of claim 1 , wherein the second categorization is more descriptive than the first categorization.

7. The method of claim 1 , further comprising:
obtaining a third categorization for the motion event candidate based on at least one segment of the plurality of temporal segments; and
prior to obtaining the second categorization, updating a log entry for the motion event candidate to include the obtained third categorization.

8. The method of claim 1 , wherein segmenting the video stream includes:
identifying a third time in the video stream;
in accordance with a determination that a predefined amount of time has lapsed, identifying a fourth time in the video stream; and
generating a segment corresponding to the portion of the video stream between the third time and the fourth time.

9. The method of claim 1 , wherein each segment of the plurality of temporal segments has a same duration.

10. The method of claim 1 , further comprising:
after obtaining the motion start information, assigning the segmented video stream to a first categorizer; and
storing each segment of the plurality of temporal segments to a particular memory portion, the particular memory portion associated with the first categorizer;
wherein obtaining the first categorization for the motion event candidate comprises:
retrieving, by the first categorizer, the initial segment from the particular memory portion; and
processing, by the first categorizer, the initial segment of the video stream to obtain the first categorization.

11. The method of claim 10 , wherein the particular memory portion corresponds to a queue assigned to the first categorizer.

12. The method of claim 10 , wherein the server system includes a plurality of categorizers; and
wherein assigning the segmented video stream to the first categorizer comprises assigning the segmented video stream to the first categorizer in accordance with a load balancing of the plurality of categorizer.

13. The method of claim 10 , further comprising, checking, by the first categorizer, for additional temporal segments of the video stream until a motion end event occurs.

14. The method of claim 13 , wherein the motion end event includes:
processing, by the first categorizer, a segment denoted as a final segment; or
a time-out event.

15. The method of claim 10 , further comprising:
retrieving a second segment of the plurality of temporal segments from the particular memory portion;
obtaining, by the first categorizer, segment information corresponding to the initial segment; and
processing, by the first categorizer, the second segment of the video stream to obtain a third categorization, wherein the processing of the second segment is based on the segment information corresponding to the initial segment.

16. The method of claim 1 , wherein the first categorization is based on a determination of an amount of motion within the initial segment of the video stream.

17. The method of claim 1 , further comprising:
while receiving the video information:
obtaining second motion start information corresponding to a third time in the video stream, the second motion start information indicating that the video stream subsequent to the third time includes a second motion event candidate; and
while receiving the video stream that includes the second motion event candidate, segmenting the video stream into a second plurality of temporal segments, the second plurality of temporal segments including an initial segment;
obtaining a first categorization for the second motion event candidate based on the initial segment of the second plurality of temporal segments, the first categorization for the second motion event candidate indicating that the second motion event candidate is an unimportant event;
in accordance with the obtained first categorization for the second motion event candidate, forgoing generation of an alert for the second motion event candidate;
after obtaining the first categorization for the second motion event candidate, obtaining a second categorization for the second motion event candidate based on one or more segments of the second plurality of temporal segments, the second categorization for the second motion event candidate indicating that the second motion event candidate is an important event; and
in accordance with the obtained second categorization for the second motion event candidate, generating an alert indicative of the second categorization for the second motion event candidate.

18. A server system comprising:
one or more processors; and
memory coupled to the one or more processors, the memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:
while receiving video information from one or more cameras, the video information including a video stream:
obtaining motion start information corresponding to a first time in the video stream, the motio

Assignees

• Google Inc

Inventors

• Laska Jason N
• Hua Wei
• Reddy Prateek
• Bapat Akshay R
• Neal Lawrence W

Classifications

• G08B13/19682

  Graphic User Interface [GUI] presenting system data to the user, e.g. information on a screen helping a user interacting with an alarm system · CPC title

• G06V10/763

  Non-hierarchical techniques, e.g. based on statistics of modelling distributions · CPC title

• G06F18/2321

  using statistics or function optimisation, e.g. modelling of probability density functions · CPC title

• G06F18/24

  Classification techniques · CPC title

• G06F18/285

  Selection of pattern recognition techniques, e.g. of classifiers in a multi-classifier system · CPC title