Apparatus and method for generating motion effects by analyzing motions of objects

Patent metadata

Field	Value
Publication number	US-2016182769-A1
Application number	US-201514969757-A
Country	US
Kind code	A1
Filing date	Dec 15, 2015
Priority date	Dec 19, 2014
Publication date	Jun 23, 2016
Grant date	—

Abstract

Disclosed are apparatuses and methods for generating motion effects in real time by analyzing motions of interest objects in a video. The motion effect generation apparatus may comprise an extraction part extracting motions between sequential frames by calculating relations of respective pixels of the sequential frames in a video signal; a clustering part generating clusters of similar motions by grouping the motions; a computation part calculating a representative motion of each of the clusters; and a selection part selecting a cluster suitable for generating a motion effect from the clusters by comparing the representative motions of the clusters, and output the representative motion of the cluster selected by the selection part as the motion effect or motion information for the motion effect.

First claim

What is claimed is:

1 . A motion effect generation apparatus, the apparatus comprising:
an extraction part extracting motions between sequential frames by calculating relations of respective pixels of the sequential frames in a first video signal; a clustering part generating clusters of similar motions by grouping the motions; a computation part calculating representative motions of respective clusters; and a selection part selecting a cluster suitable for generating a motion effect among the clusters by comparing the representative motions of the respective clusters, wherein the apparatus outputs motion information of the motion effect based on the representative motion of the cluster selected by the selection part.

2 . The apparatus according to claim 1 , further comprising a generation part generating the motion effect based on the representative motion of the cluster selected by the selection part.

3 . The apparatus according to claim 2 , wherein the generation part uses a washout filter or a trajectory planning method.

4 . The apparatus according to claim 2 , further comprising a synchronization part outputs a second video signal delayed for a predetermined time as compared to the first video signal inputted to the extraction part, wherein the second video signal is synchronized with the motion effect outputted by the generation part.

5 . The apparatus according to claim 1 , wherein the extraction part uses an optical flow method or a feature point matching method.

6 . The apparatus according to claim 1 , wherein the clustering part uses a K-means clustering method, a single linkage clustering method, or a spectral clustering method.

7 . The apparatus according to claim 1 , wherein the computation part selects arithmetic means or median values of all motions of the respective clusters as the representative motions for respective clusters.

8 . The apparatus according to claim 1 , wherein the selection part select a cluster whose representative motion has the largest absolute value or a cluster having the largest visual saliency as the cluster suitable for generating the motion effect.

9 . A motion effect generation method, the method comprising:
extracting motions between sequential frames by calculating relations of respective pixels of the sequential frames in a first video signal; generating clusters of similar motions by grouping the motions; calculating representative motions of respective clusters; and selecting a cluster suitable for generating a motion effect among the clusters by comparing the representative motions of the clusters.

10 . The method according to claim 9 , further comprising generating the motion effect based on the representative motion of the selected cluster.

11 . The method according to claim 9 , wherein the generating clusters uses a washout filter or a trajectory planning method.

12 . The method according to claim 9 , further comprising outputting a second video signal delayed for a predetermined time as compared to the first video signal wherein the second video signal is synchronized with the motion effect.

13 . The method according to claim 9 , wherein the extracting uses an optical flow method or a feature point matching method.

14 . The method according to claim 9 , wherein the generating clusters uses a K-means clustering method, a single linkage clustering method, or a spectral clustering method.

15 . The method according to claim 9 , wherein, in the calculating the representative motions, arithmetic means or median values of all motions of the respective clusters as the representative motions for the respective clusters.

16 . The method according to claim 9 , wherein, in the selecting the cluster, a cluster whose representative motion has the largest absolute value or a cluster having the largest visual saliency as the cluster suitable for generating the motion effect is selected.

17 . A motion effect generation apparatus, the apparatus comprising:
a video signal synchronization module outputting a first video signal based on an input video signal and outputting a second video signal delayed from the first video signal; a motion information generation module outputting motion information based on a representative motion of a cluster selected from the first video signal; and a motion effect generation module generating a motion effect based on the motion information and outputting the motion effect synchronized with the second video signal, wherein the motion information generation module extracts motions between two frames by calculating relations of respective pixels of the two frames in the first video signal, generates clusters of similar motions by aggregating the motions, calculates representative motions of respective clusters, selects a cluster suitable for generating a motion effect from the clusters by comparing the representative motions of the clusters, and outputs the representative motion of the selected cluster as the motion effect.

18 . The apparatus according to claim 17 , wherein at least one of the video signal synchronization module, the motion information generation module, and the motion effect generation module is executed by a processor.

19 . The apparatus according to claim 17 , further comprising at least one of a memory system, an input/output device, and a communication device for providing the input video signal.

Assignees

• Postech Acad Ind Found

Inventors

• Choi Seung Moon
• Lee Jae Bong

Classifications

• G06V10/762

  using clustering, e.g. of similar faces in social networks · CPC title

• G06V10/62

  relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking · CPC title

• G06V10/462

  Salient features, e.g. scale invariant feature transforms [SIFT] · CPC title

• G06F18/23

  Clustering techniques · CPC title

• G06T2207/30241

  Trajectory · CPC title