Comment information generating apparatus and comment information generating method

Patent metadata

Field	Value
Publication number	US-2016309239-A1
Application number	US-201615192301-A
Country	US
Kind code	A1
Filing date	Jun 24, 2016
Priority date	Jul 17, 2012
Publication date	Oct 20, 2016
Grant date	—

Abstract

A comment information generating apparatus includes: a video obtaining unit which receives a video; a comment input receiving unit which receives position information of an object in the video and a comment that is displayed with the object and is caused to track the object starting from a predetermined timing; a target time determining unit which determines, to be a target value indicating a time length, a target time during which the comment is displayed, based on the comment; an object motion determining unit which determines, for the object, a motion trajectory having a time length substantially equal to the target time and along which the comment is caused to track the object; a comment information generating unit which generates comment information including the comment and the motion trajectory of the object; and an output unit which outputs the comment information.

First claim

1 . A comment information generating apparatus which (i) receives a video, position information of an object in the video, and a comment that is displayed with the object and is caused to track the object starting from a predetermined timing, (ii) generates a motion trajectory of the object along which the comment is displayed with the object and is caused to track the object, and (iii) outputs comment information including the comment and the motion trajectory of the object, the comment information generating apparatus comprising:
a video obtaining unit configured to receive the video; a comment input receiving unit configured to receive inputs of the position information of the object in the video received by the video obtaining unit and the comment that is displayed with the object and is caused to track the object starting from the predetermined timing; and an object motion generating unit configured to generate the motion trajectory of the object based on a length of the comment received by the comment input receiving unit.

2 . The comment information generating apparatus according to claim 1 , further comprising
a target time determining unit configured to determine a target time during which the comment is displayed, based on the length of the comment received by the comment input receiving unit, the target time being a target value indicating a time length; a comment information generating unit configured to generate comment information to be output including the comment and the motion trajectory of the object generated by the object motion generating unit; and an output unit configured to output the comment information generated by the comment information generating unit.

3 . The comment information generating apparatus according to claim 2 ,
wherein the target time determining unit is configured to calculate the target time which lengthens as the comment received by the comment input receiving unit lengthens, and the output unit is configured to output the motion trajectory having the time length which lengthens as the comment input to the comment input receiving unit lengthens.

4 . The comment information generating apparatus according to claim 3 ,
wherein the target time determining unit is configured to calculate, as the target time, a value obtained by multiplying a unit display time which is a predetermined display time per character and the number of characters in the comment received by the comment input receiving unit, and the output unit is configured to output the motion trajectory having, as the time length, the value obtained by multiplying the unit display time and the number of characters, when the comment is input to the comment input receiving unit.

5 . The comment information generating apparatus according to claim 3 ,
wherein the target time determining unit is further configured to calculate, as the target time, a predetermined visual recognition time required to visually recognize the characters, when the calculated target time is shorter than the predetermined visual recognition time, and the output unit is configured to output, as the motion trajectory, a motion trajectory which has a time length larger than or equal to the visual recognition time even if the comment is shorter than the visual recognition time, by outputting the motion trajectory which corresponds to a longer one of the visual recognition time and a value obtained by multiplying a unit display time which is a predetermined display time per character and the number of characters in the comment when the comment is input to the comment input receiving unit.

6 . The comment information generating apparatus according to claim 2 ,
wherein the output unit is configured to output a plurality of the motion trajectories which are different from each other for a plurality of the comments received by the comment input receiving unit, when the plurality of the comments have different numbers of characters and are provided at a same position in a same frame.

7 . The comment information generating apparatus according to claim 2 ,
wherein the object motion generating unit is configured to calculate a plurality of the motion trajectories for the object which is in the video and shown by the position information received by the comment input receiving unit, using a plurality of motion estimation methods or motion estimation parameters, and determine, for the object, the motion trajectory which is selected as having a time length closest to the target time from among the plurality of the motion trajectories calculated for the object.

8 . The comment information generating apparatus according to claim 7 ,
wherein the object motion generating unit is configured to calculate a plurality of the motion trajectories for the object which is in the video and shown by the position information received by the comment input receiving unit, using, as the plurality of motion estimation parameters, one kind of (i) allowance parameters having different values which affect trackability of the object, (ii) search window regions having different sizes, and (iii) feature values having different values, and determine, for the object, the motion trajectory which is selected as having a time length closest to the target time from among the plurality of the motion trajectories calculated for the object.

9 . The comment information generating apparatus according to claim 7 ,
wherein, in a case of occurrence of a “motion estimation impossible state” in which the motion trajectory closest to the target time cannot be determined for the object using the plurality of motion estimation methods or motion estimation parameters, the object motion generating unit is further configured to determine whether the “motion estimation impossible state” is caused by occlusion or a scene change, and switch between object motion determining methods based on a result of the determination.

10 . The comment information generating apparatus according to claim 9 ,
wherein when determining that the “motion estimation impossible state” is caused by the occlusion, the object motion generating unit is configured to determine the motion trajectory having the time length closest to the target time for the object by extrapolating a motion trajectory of the object from a frame at which the occlusion occurred to a following frame, based on a motion trajectory of the object to the frame at which the occlusion occurred.

11 . The comment information generating apparatus according to claim 9 ,
wherein when determining that the “motion estimation impossible state” is caused by the scene change, the object motion generating unit is configured to determine, for the object, a motion trajectory to a frame at which the scene change occurred as the motion trajectory to be output.

12 . The comment information generating apparatus according to claim 9 ,
wherein the object motion generating unit is configured to determine that the “motion estimation impossible state” is caused by the scene change, when a difference between luminance histograms of frames composing the video is larger than or equal to a predetermined threshold value, and that the “motion estimation impossible state” is caused by the occlusion, when a difference between the luminance histograms of the frames composing the video is smaller than the predetermined threshold value.

13 . The comment information generating apparatus according to claim 7 ,
wherein when a motion trajectory among the plurality of the motion trajectories of the object calculated using the plurality of motion estimation methods or motion estimation parameters is shorter than the ti

Assignees

• Panasonic Ip Man Co Ltd

Inventors

• Maruyama Ayako
• Nobori Kunio
• Hotta Koichi
• Sasakura Shuhei

Classifications

• G06F16/7867

  using information manually generated, e.g. tags, keywords, comments, title and artist information, manually generated time, location and usage information, user ratings · CPC title

• H04N21/44008

  involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59) · CPC title

• G09G5/38

  with means for controlling the display position · CPC title

• H04N21/488

  Data services, e.g. news ticker {(systems specially adapted for using meteorological information in broadcast systems H04H60/71)} · CPC title

• G06T2207/30241

  Trajectory · CPC title