Adaptive video end-to-end network with local abstraction

Patent metadata

Field	Value
Publication number	US-9736349-B2
Application number	US-201414582524-A
Country	US
Kind code	B2
Filing date	Dec 24, 2014
Priority date	Dec 24, 2014
Publication date	Aug 15, 2017
Grant date	Aug 15, 2017

Abstract

An adaptive video end-to-end network is described that uses local abstraction. One example includes an image sensor to generate a sequence of images, a processor coupled to the image sensor to analyze the sequence of images to detect an event, to select images related to the event and to generate metadata regarding the event, and a communications interface coupled to the processor to send the metadata information through a network connection to a central node.

First claim

What is claimed is:

1. A networked camera comprising:
an image sensor to generate a sequence of images;
a processor coupled to the image sensor to abstract semantic forms, to compress data from the images in the feature domain using the semantic abstraction, to generate sparse image data describing low level features based on the compressed image data, to generate actionable insights based on the semantic abstraction, and to select the sparse image data and metadata to be sent to a central node based on the insights; and
a communications interface coupled to the processor to send the sparse image data describing low level features and metadata information through a network connection to a central node so that the central node is able to perform detection, tracking and recognition on the received sparse image data and metadata.

2. The apparatus of claim 1 , wherein the communications interface is further to send the sparse image data and metadata to the central node only when objects of predetermined semantic forms are detected.

3. The apparatus of claim 1 , wherein the processor is further to recognize an object in images of the sequences of images to detect an event and to compress a selected image by removing parts of the image not related to the detected event as determined using the recognized object.

4. The apparatus of claim 3 , wherein the processor is further to perform spatial compression on selected images.

5. The apparatus of claim 4 , wherein the spatial compression removes the background from each of the selected images and not the recognized object related to a detected event.

6. The apparatus of claim 1 , wherein the processor is further to detect an event using segmentation of each image and the selected images are a video segment.

7. The apparatus of claim 1 , wherein the metadata includes as confidence value for the detection of an event.

8. The apparatus of claim 7 , wherein the metadata includes a time stamp for the detection of the event.

9. The apparatus of claim 1 , further comprising the central node, the central node comprising a processor and wherein the central node processor performs feature extraction to images of the sequence to extract features from the sequence of images, object detection to the extracted features to detect objects from the sequence of images using the extracted features, and object recognition to the detected objects to recognize detected objects, and wherein the selected images are images which contain recognized objects.

10. The apparatus of claim 1 , wherein the communications interface receives commands from the central node or another node to adjust parameters for analyzing the sequence of images.

11. An apparatus comprising:
a plurality of networked cameras each connected to a network as a network node, each camera having a sensor to receive a sequence of images, a processor to abstract semantic forms, to compress data from the images in the feature domain using the semantic abstraction, to generate sparse image data describing low level features based on the compressed image data, to generate actionable insights based on the semantic abstraction, and to select the sparse image data and metadata to be send to a central node based on the insights, and a communications interface to send the sparse image data describing low level features and metadata to a central node; and
a central node to receive the sparse image data and metadata and to perform detection, tracking and recognition on the received sparse image data and metadata.

12. The apparatus of claim 11 , wherein the camera processors are further to spatially compress a selected image, and wherein the communications interface is further to send the selected image to a central node.

13. The apparatus of claim 12 , wherein the processor is further to detect an event regarding an identified object in the selected image and wherein the communications interface sends metadata to the central node regarding the identified object.

14. The apparatus of claim 13 , wherein the processor is further to spatially compress the selected image removing a background from the selected image and not the identified object.

15. The apparatus of claim 11 , wherein the central node further comprises a processor to perform the object detection on the selected image and wherein the insight comprises an insight of a detected object.

16. The apparatus of claim 11 , wherein the central node is coupled to a user interface and the central node sends the generated insight to the user interface.

17. A method comprising:
receiving a sequence of images from an image sensor of a remote network node;
analyzing images of the sequence of images at the remote node to abstract semantic forms from the images of the sequence, to compress data from the images in the feature domain using the semantic abstraction, to generate sparse image data describing low level features based on the compressed image data, and to generate actionable insights based on the semantic abstraction;
selecting the sparse image data to be sent to a central node based on the actionable insights;
generating metadata regarding the selected sparse image data; and
sending the selected sparse image data and the metadata from the remote node to a central node for detection, tracking, and recognition.

18. The method of claim 17 , wherein the metadata identifies the actionable insight.

19. The method of claim 17 , further comprising recognizing an object in images of the sequences of images to detect an event and wherein compressing comprises compressing by removing parts of the images not containing the recognized object as not related to the detected event.

20. A non-transitory machine-readable medium having instructions that, when operated on by the machine, cause the machine to perform operations comprising:
receiving a sequence of images from an image sensor of a remote network node;
analyzing images of the sequence of images at the remote node to abstract semantic forms from the images of the sequence, to compress data from the images in the feature domain using the semantic abstraction, to generate sparse image data describing low level features based on the compressed image data, and to generate actionable insights based on the semantic abstraction;
selecting the sparse image data to be sent to a central node based on the actionable insights;
generating metadata regarding the selected sparse image data; and
sending the selected sparse image data and the metadata from the remote node to a central node for detection, tracking, and recognition.

Assignees

• Yang shao-wen
• Chen Yen-Kuang
• Intel Corp

Inventors

• Yang shao-wen
• Chen Yen-Kuang

Classifications

• H04N23/90

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title

• H04N23/661 Primary

  Transmitting camera control signals through networks, e.g. control via the Internet · CPC title

• H04N7/181 Primary

  for receiving images from a plurality of remote sources · CPC title

• H04N23/61

  based on recognised objects · CPC title

• H04N5/23206 Primary

  Electricity · mapped topic