Method, Apparatus And Computer Program Product For Generating Semantic Information From Video Content

Patent metadata

Field	Value
Publication number	US-2016112727-A1
Application number	US-201414519492-A
Country	US
Kind code	A1
Filing date	Oct 21, 2014
Priority date	Oct 21, 2014
Publication date	Apr 21, 2016
Grant date	—

Abstract

A method, apparatus and computer program product are provided for generating semantic information from video content. Objects and regions of interest within video content may be identified and monitored for characteristics relating to object detection, motion content, and motion trajectory. Salient events relating to the regions may be detected based on the monitoring. Temporal segments may be identified and used to create summary video content, or highlights. An example embodiment relates to processing video footage of sports. Goals, scored points, unsuccessful scoring attempts, as well as other events may be detected in the video content. Efficiency is gained by monitoring only a relatively small portion of the frame, and by limiting the dependency on tracking moving objects.

First claim

1 . An apparatus comprising at least one processor and at least one memory including computer program code, the at least one memory and the computer program code configured to, with the processor, cause the apparatus to perform at least:
receiving an indication of an object of interest in video content; identifying at least one region of interest based on (a) a position of the at least one region of interest relative to a position of the object of interest and (b) a viewing angle from which the video content is captured; monitoring, with the processor, at least one characteristic in the at least one region of interest in the video content; and in response to the monitoring of the video content, generating semantic information relating to the video content and causing the generated semantic information to be stored in the at least one memory.

2 . The apparatus according to claim 1 , wherein the at least one memory and the computer program code are further configured to, with the processor, cause the apparatus to perform at least:
determining that a salient event relating to the object of interest has occurred; identifying temporal segments relating to the salient event; and generating summary video content comprising the identified temporal segments.

3 . The apparatus according to claim 2 , wherein the at least one memory and the computer program code are further configured to, with the processor, cause the apparatus to perform at least:
generating metadata describing the salient event; storing the metadata in association with the video content; and providing the metadata and video content such that the summary video content is recreated for playback based on the metadata and video content.

4 . The apparatus according to claim 1 , wherein the at least one characteristic comprises at least one of motion detection or object tracking.

5 . The apparatus according to claim 1 , wherein the at least one characteristic comprises at least one of object detection, object recognition or color variation.

6 . The apparatus according to claim 1 , wherein the at least one memory and the computer program code are further configured to, with the processor, cause the apparatus to perform at least:
receiving an indication of a user input identifying the object of interest.

7 . The apparatus according to claim 1 , wherein the at least one memory and the computer program code are further configured to, with the processor, cause the apparatus to perform at least:
in an instance the perspective of the video content changes, tracking the object of interest and the at least one region of interest.

8 . The apparatus according to claim 1 , wherein at least the object of interest or region of interest is identified based on a context of the video content.

9 . A computer program product comprising at least one non-transitory computer-readable storage medium having computer-executable program code instructions stored therein, the computer-executable program code instructions comprising program code instructions for:
receiving an indication of an object of interest in video content; identifying at least one region of interest based on (a) a position of the at least one region of interest relative to a position of the object of interest and (b) a viewing angle from which the video content is captured; monitoring at least one characteristic in the at least one region of interest; and in response to the monitoring, generating semantic information relating to the video content and causing the generated semantic information to be stored in the at least one non-transitory computer-readable storage medium.

10 . The computer program product according to claim 9 , wherein the computer-executable program code instructions further comprise program code instructions for:
determining that a salient event relating to the object of interest has occurred; identifying temporal segments relating to the salient event; and generating summary video content comprising the identified temporal segments.

11 . The computer program product according to claim 10 , wherein the computer-executable program code instructions further comprise program code instructions for:
generating metadata describing the salient event; storing the metadata in association with the video content; and providing the metadata and video content such that the summary video content is recreated for playback based on the metadata and video content.

12 . The computer program product according to claim 9 , wherein the at least one characteristic comprises at least one of motion detection or object tracking.

13 . The computer program product according to claim 9 , wherein the at least one characteristics comprise s at least one of object detection, object recognition or color variation.

14 . The computer program product according to claim 9 , wherein the computer-executable program code instructions further comprise program code instructions for:
receiving an indication of a user input identifying the object of interest.

15 . The computer program product according to claim 9 , wherein the computer-executable program code instructions further comprise program code instructions for:
in an instance the perspective of the video content changes, tracking the object of interest and the at least one region of interest.

16 . The computer program product according to claim 9 , wherein at least the object of interest or region of interest is identified based on a context of the video content.

17 . A method comprising:
receiving an indication of an object of interest in video content; identifying at least one region of interest based on (a) a position of the at least one region of interest relative to a position of the object of interest and (b) a viewing angle from which the video content is captured; monitoring at least one characteristic in the at least one region of interest; and in response to the monitoring, generating semantic information relating to the video content, and causing the generated semantic information to be stored in a memory device.

18 . The method according to claim 17 , further comprising:
determining that a salient event relating to the object of interest has occurred; identifying temporal segments relating to the salient event; and generating summary video content comprising the identified temporal segments.

19 . The method according to claim 17 , further comprising:
generating metadata describing the salient event; storing the metadata in association with the video content; and providing the metadata and video content such that the summary video content is recreated for playback based on the metadata and video content.

20 . (canceled)

Assignees

• Nokia Technologies Oy

Inventors

• Mate Sujeet Shyamsundar
• Shetty Ranjeeth Ravindra

Classifications

• H04N21/2353 Primary

  specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata · CPC title

• H04N21/2187

  Live feed · CPC title

• H04N21/23418

  involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59) · CPC title

• G06F16/739 Primary

  in form of a video summary, e.g. the video summary being a video sequence, a composite still image or having synthesized frames · CPC title

• G06F16/7335

  Graphical querying, e.g. query-by-region, query-by-sketch, query-by-trajectory, GUIs for designating a person/face/object as a query predicate (end-user interface involving hot spots associated with the video H04N21/4725; end-user interface for selecting a Region of Interest H04N21/4728) · CPC title