System and method for object and event identification using multiple cameras

Patent metadata

Field	Value
Publication number	US-9665777-B2
Application number	US-201414273653-A
Country	US
Kind code	B2
Filing date	May 9, 2014
Priority date	May 10, 2013
Publication date	May 30, 2017
Grant date	May 30, 2017

Abstract

A system for identifying objects and events of interest uses one or more cameras with image processing capabilities. The system is includes multiple cameras configured to perform image processing of a scene from multiple angles to extract and transmit meta-data corresponding to objects or people in the scene. The cameras transmit the meta-data to a processing station that is configured to process the stream of data over time to detect objects and events of interest to alert monitoring personnel of objects or events in the scene.

First claim

What is claimed:

1. A surveillance system comprising:
a plurality of cameras located in a plurality of positions to record a scene, each camera further comprising:
a sensor configured to generate video data of the scene comprising a series of frames;
a first network device configured to transmit the video data and feature vectors associated with the video data to a processing station; and
a feature extraction processor operatively connected to the sensor and the network device, the feature extraction processor being configured to:
identify a plurality of feature vectors in video data generated by the sensor, the plurality of feature vectors including a first feature vector for a first frame in the video data and a second feature vector for a second frame in the video data, the second frame occurring after the first frame;
transmit only the plurality of feature vectors to the processing station with the first network device in a first operating mode including transmission of the first feature vector and a sparse feature vector corresponding to only a portion of the second feature vector that is different than the first feature vector; and
transmit the video data to the processing station with the first network device in a second operating mode only in response to a request for the video data from the processing station; and

the processing station further comprising:
a second network device;
a video output device; and
a processor operatively connected to the second network device and the video output device, the processor being configured to:
receive the plurality of feature vectors generated by each camera in the plurality of cameras with the second network device;
identify an object and motion of the object in the scene with reference to the plurality of feature vectors received from at least two of the plurality of cameras;
identify a event of interest corresponding to the motion of the object in the scene with reference to a predetermined database of events only in response to;
generate a request for transmission of the video data from at least one camera in the plurality of cameras for only; and
generate a graphical display of the video data from the at least one camera with the video output device to display the object associated with the event.

2. The surveillance system of claim 1 , feature extraction processor in each of the plurality of cameras being further configured to:
identify a portion of one frame in the video data that corresponds to a background of the scene; and
identify the plurality of features in the video data only in portions of the one frame that do not correspond to the background of the scene.

3. The surveillance system of claim 2 , the feature extraction processor in each camera being further configured to:
generate a bounding box corresponding a portion of the one frame that does not correspond to the background of the scene;
resize the portion of the one frame in the bounding box to a generate a fixed resolution image of the portion of the one frame included in the bounding box; and
identify at least one feature vector in the plurality of feature vectors with reference to the fixed resolution image.

4. The surveillance system of claim 3 , the feature extraction processor being further configured to:
identify a plurality of image gradients in the fixed resolution image; and
generate the feature vector including a histogram of gradients corresponding to the plurality of image gradients.

5. The surveillance system of claim 1 , the processor in the processing station being further configured to:
receive another sparse feature vector from a second camera in the plurality of cameras, the other sparse feature vector including only portions of a feature vector generated in the second camera that have changed since generation of an earlier feature vector, the earlier feature vector being generated approximately concurrently to the first feature vector from the first camera; and
generate a full feature vector corresponding to the other sparse feature vector from the second camera with reference to the first feature vector from the first camera.

6. The surveillance system of claim 1 , the processor in the processing station being further configured to:
identify occurrence or non-occurrence of an event in each plurality of feature vectors from each of the plurality of cameras individually with reference to the plurality of feature vectors corresponding to a plurality of key-frames from each of the plurality of cameras corresponding to the motion of the object during a single time period;
identify the event in response to an identification of an occurrence of the event from a majority of the plurality of cameras.

7. The surveillance system of claim 1 , the processor in the processing station being further configured to:
receive a first plurality of feature vectors from a first camera corresponding to a first frame of video data;
identify that the first plurality of feature vectors from the first camera correspond to a first key-frame for an event in the predetermined database of events;
receive a second plurality of feature vectors from a second camera corresponding to a second frame of video data; and
identify that the second plurality of feature vectors from the second camera correspond to a second key-frame from the event; and
generate a score corresponding to a deformation between the first plurality of feature vectors in the first frame and the second plurality of feature vectors in the second frame; and
identify occurrence or non-occurrence of the event with reference to the score.

8. The surveillance system of claim 7 , the processor in the processing station being further configured to:
identify a first location of an object corresponding to the first plurality of feature vectors that are extracted from the first frame of video data;
identify a second location of the object corresponding to the second plurality of feature vectors that are extracted from the second frame of video data;
perform a homographic transformation to identify a distance between the first location of the object and the second location of the object with reference to a ground plane that is present in both the first frame of video data and the second frame of video data; and
generate the score corresponding to the deformation between the first plurality of feature vectors in the first frame and the second plurality of feature vectors in the second frame with reference to the distance between the first location and the second location.

9. A method for surveillance of a scene comprising:
generating with a sensor in a first camera first video data of the scene, the first video data comprising a first series of frames;
identifying with a feature extraction processor in the first camera a first plurality of feature vectors in the first video data including a first feature vector for a first frame in the video data and a second feature vector for a second frame in the video data, the second frame occurring after the first frame;
transmitting with a network device in the first camera only the first plurality of feature vectors to a processing station in a first operating mode, the transmitting including the first feature vector and a sparse feature vector corresponding to only a portion of the second feature vector that is different than the first feature vector;
transmitting with the network device in the first camera the first video data to the processing station in a second operating mode only in response to a request for the first video data from the processing station;
generating with another sensor in a second camera second video data of the scene, the second video data comprising a s

Assignees

• Bosch Gmbh Robert

Inventors

• Naikal Nikhil
• Lajevardi Pedram

Classifications

• H04N21/44008

  involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59) · CPC title

• G06K2009/00738

  Physics · mapped topic

• G06K9/00771 Primary

  Physics · mapped topic

• G06K9/00711

  Physics · mapped topic

• H04N7/181

  for receiving images from a plurality of remote sources · CPC title