Method for identifying events in a motion video

Patent metadata

Field	Value
Publication number	US-10824872-B2
Application number	US-201715851028-A
Country	US
Kind code	B2
Filing date	Dec 21, 2017
Priority date	Dec 21, 2016
Publication date	Nov 3, 2020
Grant date	Nov 3, 2020

Abstract

A method for identifying events in a scene captured by a motion video camera comprises two identification processes, a temporary identification process and a long-term identification process. The temporary process includes: analyzing pixel data from captured image frames and identifying events; registering camera processing data relating to each image frame subjected to the identification of events; and adjusting weights belonging to an event identifying operation, wherein the weights are adjusted for achieving high correlation between the result from the event identifying operation and the result from the identification based on analysis of pixels from captured image frames of the captured scene. The long-term identification process includes: identifying events in the captured scene by inputting registered camera processing data to the event identifying operation. The temporary identification process is then executed during a predetermined time period and the long-term identification process is executed after the predetermined initial time has expired.

First claim

What is claimed is:

1. A method for identifying events in a scene captured by a motion video camera, the method comprising a temporary identification process and a long-term identification process;
the temporary identification process includes:
identifying events in the captured scene by analyzing pixel data from captured image frames;
registering camera processing data relating to the image frames subjected to the identification of events, wherein the camera processing data comprises at least one of: a data rate of an encoded video stream captured by a camera, an auto focus distance, a setting of an auto white balance function, auto exposure settings, shutter time, gain, electronic image stabilization data, a signal to noise ratio, a contrast in a captured frame, a data rate sent to a network, a central processing unit (CPU) usage, a memory usage, data from a gyro or an accelerometer, and position data from a pan-tilt-zoom (PTZ) head connected to the camera; and
training a neural network based event identifying operation using the registered camera processing data relating to the image frames subjected to the identification of events as input and the identified events as a correct classification of an event resulting from the neural network based event identifying operation; and
the long-term identification process includes:
registering camera processing data relating to image frames captured subsequent to the image frames used for the analysis of pixel data; and
identifying events in the captured scene by inputting the registered camera processing data relating to image frames captured subsequent to the image frames used for the analysis of pixel data to the trained neural network based event identifying operation;
wherein the temporary identification process is executed during a predetermined time period and wherein the long-term identification process is executed after the predetermined time period has expired.

2. The method according to claim 1 , wherein the training of the neural network based event identifying operation comprises adjusting weights belonging to nodes of the neural network based event identifying operation, wherein the weights of the nodes of the neural network based event identifying operation are adjusted such that a classification of an event resulting from the neural network based event identifying operation is close to a classification of an event identified by the analysis of pixel data from the captured image frames.

3. The method according to claim 2 , wherein the weights are stored during the temporary identification process in a device connected to the motion video camera via a network.

4. The method according to claim 3 , wherein the weights are transferred to the motion video camera via the network.

5. The method according to claim 1 , wherein the temporary identification process is started upon request.

6. The method according to claim 5 , wherein the request is an instruction inputted to the motion video camera by a user.

7. The method according to claim 1 , wherein the temporary identification process is executed by a device connected to the motion video camera via a network.

8. The method according to claim 1 , wherein weights and a code of the neural network based event identifying operation are executed in the motion video camera in the long-term identification process.

9. The method according to claim 1 , further comprising:
restarting the temporary identification process in response to a confidence value generated by the neural network based event identification operation in the long-term identification process being below a predetermined threshold.

10. A system including a motion video camera and a processing device arranged to communicate via a communication network, the system comprising:
the processing device, configured to use a temporary identification process, including:
a network interface of the processing device operatively coupled to a central processing unit (CPU) of the processing device, the network interface of the processing device and the CPU of the processing device configured to identify events in the captured scene by analyzing pixel data from captured image frames;
the CPU of the processing device configured to register camera processing data relating to the image frames subjected to the identification of events, wherein the camera processing data comprises at least one of: a data rate of an encoded video stream captured by a camera, an auto focus distance, a setting of an auto white balance function, auto exposure settings, shutter time, gain, electronic image stabilization data, a signal to noise ratio, a contrast in a captured frame, a data rate sent to a network, a CPU usage, a memory usage, data from a gyro or an accelerometer, and position data from a pan-tilt-zoom (PTZ) head connected to the camera; and
the CPU of the processing device configured to train a neural network based event identifying operation using the registered camera processing data relating to the image frames subjected to the identification of events as input and the identified events as a correct classification of an event resulting from the neural network based event identifying operation; and
the motion video camera, configured to use a long-term identification process, including:
a CPU of the motion video camera configured to register camera processing data relating to image frames captured subsequent to the image frames used for the analysis of pixel data; and
a network interface of the motion video camera operatively coupled to the CPU of the motion video camera, the network interface of the motion video camera and the CPU of the motion video camera configured to identify events in the captured scene by inputting the registered camera processing data relating to image frames captured subsequent to the image frames used for the analysis of pixel data to the trained neural network based event identifying operation;
wherein the temporary identification process is executed during a predetermined time period and wherein the long-term identification process is executed after the predetermined time period has expired.

11. The system of claim 10 , wherein the training of the neural network based event identifying operation comprises adjusting weights belonging to nodes of the neural network based event identifying operation, wherein the weights of the nodes of the neural network based event identifying operation are adjusted such that a classification of an event resulting from the neural network based event identifying operation is close to a classification of an event identified by the analysis of pixel data from the captured image frames.

12. The system of claim 10 , wherein the temporary identification process is started upon request.

13. The system of claim 12 , wherein the request is an instruction inputted to the motion video camera by a user.

14. The system of claim 10 , further comprising:
the processing device configured to restart the temporary identification process in response to a confidence value generated by the neural network based event identification operation in the long-term identification process being below a predetermined threshold.

15. A motion video camera comprising:
the motion video camera configured to use a temporary identification process, including:
a network interface operatively coupled to a central processing unit (CPU), the network interface and the CPU configured to identify events in the captured scene by analyzing pixel data from captured image frames;
the CPU configured to register camera processing data relating to the image fram

Assignees

• Axis Ab

Inventors

• Edpalm Viktor
• Andersson Erik
• Yuan Song

Classifications

• G06V10/82

  using neural networks · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06V20/41

  Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49) · CPC title

• G06V20/52 Primary

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title