Method, apparatus and system for applying an annotation to a portion of a video sequence

Patent metadata

Field	Value
Publication number	US-10459976-B2
Application number	US-201615192795-A
Country	US
Kind code	B2
Filing date	Jun 24, 2016
Priority date	Jun 30, 2015
Publication date	Oct 29, 2019
Grant date	Oct 29, 2019

Abstract

A method, system and apparatus for applying an annotation to a portion of a video sequence. The method comprises the steps of receiving the video sequence in real-time during capture of the video sequence, monitoring in real-time a plurality of signals associated with the video sequence, and receiving an indication associated with a spatial area of interest of at least one frame during capture of the video sequence. The method further comprises selecting, from the plurality of monitored signals, a temporal portion of one of the plurality of monitored signals for annotation, said selection being based upon at least the spatial area of interest and a temporal variation measure in at least one of the plurality of monitored signals, applying an annotation to a portion of the video sequence corresponding to the selected temporal portion; and storing the annotation in an annotation record associated with the video sequence.

First claim

The invention claimed is:

1. A method implemented by a processor to apply an annotation to a portion of a video sequence, the method comprising:
receiving the video sequence in real-time during capture of the video sequence;
monitoring, in real-time, a plurality of signals associated with the video sequence, wherein the plurality of monitored signals include at least two monitored signals of the following types of signals: an image capture apparatus motion signal, an image capture apparatus zoom signal, an image capture apparatus frame rate signal, a video image lighting signal, a video image colour signal, a video image blur signal, a video image edge density signal, a video image corner density signal, a video image face appearance signal, a video image character motion signal, a video image object motion signal, a video image ambient noise signal, and a video image dialog signal, wherein one of the at least two monitored signals is a type of signal that is different from a type of signal of the remainder of the at least two monitored signals;
receiving an indication associated with a spatial area of interest of at least one frame during capture of the video sequence;
selecting, from the at least two monitored signals, a temporal portion of one of the at least two monitored signals for annotation,
wherein selecting the temporal portion for annotation is based upon each of the following: (i) the spatial area of interest, (ii) a temporal variation measure in one of the at least two monitored signals, and (iii) the one of the at least two monitored signals having a signal change nearest to a time of subsequently receiving the indication, wherein the signal change is a most recent signal change of one of the at least two monitored signals relative to a different signal change of the remaining of the at least two monitored signals;
applying an annotation to a portion of the video sequence corresponding to the selected temporal portion; and
storing the annotation in an annotation record associated with the video sequence.

2. The method according to claim 1 , further comprising determining a region of interest of the at least one frame of the video sequence using the spatial area of interest,
wherein selecting the temporal portion for annotation further is based on (iv) the region of interest.

3. The method according to claim 1 , further comprising determining a region of interest of the at least one frame of the video sequence using the spatial area of interest,
wherein selecting the temporal portion for annotation further is based on (iv) the region of interest, and
wherein the region of interest includes the spatial area of interest and a portion of the at least one frame having content associated with content of the spatial area of interest.

4. The method according to claim 1 , wherein each of the at least two monitored signals is associated with a spatial region of the at least one frame.

5. The method according to claim 1 , wherein each of the at least two monitored signals is associated with a spatial region of the at least one frame, and the spatial region of the at least one frame is a portion of the video frame.

6. The method according to claim 1 , wherein each of the at least two monitored signals is associated with a spatial region of the at least one frame, and the spatial region of the at least one frame is the entire video frame.

7. The method according to claim 1 , wherein the indication is a touch gesture received by a touch screen displaying the video sequence.

8. The method according to claim 1 wherein the selected temporal portion starts at a transition time of the selected monitored signal.

9. The method according to claim 1 , wherein the selected temporal portion starts at a transition time of the selected monitored signal, and the selected temporal portion ends at a further transition of the selected monitored signal.

10. The method according to claim 1 , further comprising determining a category of the annotation from the selected temporal portion.

11. The method according to claim 1 , wherein a subject of the annotation is identified in the at least one frame by matching a type of the indication to the selected temporal portion.

12. The method according to claim 1 , wherein an area of the annotation includes the spatial area of interest.

13. The method according to claim 1 , wherein an area of the annotation includes the spatial area of interest and a region of the at least one frame having similar texture content to the spatial area of interest.

14. The method according to claim 1 , wherein an area of the annotation includes the spatial area of interest and a region of the at least one frame having a similar motion signature to the spatial area of interest.

15. A non-transitory computer-readable medium having computer program stored thereon to perform a method implemented by a processor to apply an annotation to a portion of a video sequence, the method comprising:
receiving the video sequence in real-time during capture of the video sequence;
monitoring, in real-time, a plurality of signals associated with the video sequence, wherein the plurality of monitored signals include at least two monitored signals of the following types of signals: an image capture apparatus motion signal, an image capture apparatus zoom signal, an image capture apparatus frame rate signal, a video image lighting signal, a video image colour signal, a video image blur signal, a video image edge density signal, a video image corner density signal, a video image face appearance signal, a video image character motion signal, a video image object motion signal, a video image ambient noise signal, and a video image dialog signal, wherein one of the at least two monitored signals is a type of signal that is different from a type of signal of the remainder of the at least two monitored signals;
receiving an indication associated with a spatial area of interest of at least one frame during capture of the video sequence;
selecting, from the at least two monitored signals, a temporal portion of one of the at least two monitored signals for annotation,
wherein selecting the temporal portion for annotation is based upon each of the following: (i) the spatial area of interest, (ii) a temporal variation measure in one of the at least two monitored signals, and (iii) the one of the at least two monitored signals having a signal change nearest to a time of subsequently receiving the indication, wherein the signal change is a most recent signal change of one of the at least two monitored signals relative to a different signal change of the remaining of the at least two monitored signals;
applying an annotation to a portion of the video sequence corresponding to the selected temporal portion; and
storing the annotation in an annotation record associated with the video sequence.

16. An apparatus to apply an annotation to a portion of a video sequence, the apparatus comprising:
at least one processor coupled to memory storing instructions that, when executed by the at least processors, cause the apparatus to perform operations including:
receiving the video sequence in real-time during capture of the video sequence,
monitoring, in real-time, a plurality of signals associated with the video sequence, wherein the plurality of monitored signals include at least two monitored signals of the following types of signals: an image capture apparatus motion signal, an image capture apparatus zoom signal, an image capture apparatus frame rate signal, a video image lighting signal, a video image colour signal, a video

Assignees

• Canon Kk

Inventors

• Wang Ij Eric
• Xu Jie

Classifications

• G06F3/0488

  using a touch-screen or digitiser, e.g. input of commands through traced gestures · CPC title

• G06F16/7867 Primary

  using information manually generated, e.g. tags, keywords, comments, title and artist information, manually generated time, location and usage information, user ratings · CPC title