Content retrieval to facilitate recognizing content details of a moving image

Patent metadata

Field	Value
Publication number	US-8971633-B2
Application number	US-201113639012-A
Country	US
Kind code	B2
Filing date	Mar 30, 2011
Priority date	Apr 9, 2010
Publication date	Mar 3, 2015
Grant date	Mar 3, 2015

Abstract

An apparatus, a content retrieval method and a program facilitate recognizing content details of a moving image by displaying a retrieval result for each of a plurality of images. A metadata extracting unit extracts metadata of a representative image from content supplied from a content input unit and stores the metadata in a metadata database. An image feature quantity extracting unit extracts image feature quantity from content and stores it in an image feature quantity database while extracting image feature quantity from a retrieval image. A retrieving unit compares the image feature quantity of the retrieval image with the image feature quantity stored in the image feature quantity database and specifies a still image which is similar to the retrieval image. An output unit reads metadata of the content, generates display data necessary for performing timeline displaying, and generates display data for displaying a retrieval result on a timeline.

First claim

The invention claimed is:

1. A content retrieval apparatus, comprising:
retrieval image input means which receives input of a plurality of retrieval images being still images;
image feature quantity extracting means which extracts image feature quantity of each of the plurality of retrieval images with a predetermined method;
similarity degree calculation means which calculates a similarity degree of content of a moving image between the image feature quantity previously extracted from a still image of a frame structuring the content and the image feature quantity of each of the retrieval images;
specifying means which specifies a frame of the still image being similar to each of the retrieval images by comparing the calculated similarity degree with a predetermined threshold value; and
timeline displaying means which generates display data for performing timeline displaying by displaying the content on a time axis along with information indicating a temporal position of the frame of the still image being similar to each of the retrieval images based on the specified frame and metadata which is previously extracted from the content,
wherein the timeline displaying means generates display data for displaying a screen having a retrieval image display unit which displays a list of the plurality of retrieval images and a timeline display unit which displays the content on a timeline,
the plurality of retrieval images are displayed at the retrieval image display unit while being surrounded respectively by display frames in mutually different colors, and
information indicating a temporal position of the frame of the still image being similar to each of the retrieval images is displayed at the timeline display unit as a color bar in a same color as a color of the display frame.

2. The content retrieval apparatus according to claim 1 ,
wherein the timeline displaying means displays each of a plurality of pieces of content with the timeline displaying at the timeline display unit on a single screen, and
information indicating a temporal position of the frame of the still image being similar to each of the retrieval images for each of the plurality of pieces of content is displayed as a color bar in the same color as the color of the display frame.

3. The content retrieval apparatus according to claim 1 ,
wherein the timeline displaying means generates display data for displaying a screen further having a moving image display unit which displays a moving image obtained by reproducing the content in a case that reproduction of the content displayed with the timeline displaying is instructed.

4. The content retrieval apparatus according to claim 1 ,
wherein the timeline displaying means specifies scenes structuring the content based on information of scene change points of the content included in the metadata, and displays the content on a timeline in such a manner that figures indicating the respective scenes are displayed while being associated with temporal length of the scenes, with the horizontal or vertical direction of a screen being the time axis.

5. The content retrieval apparatus according to claim 4 ,
wherein the timeline displaying means displays the content on a timeline while the figures indicating the scenes are displayed respectively in representative color of the scenes specified with a predetermined method.

6. A content retrieval method, comprising the steps of:
receiving input of a plurality of retrieval images being still images with retrieval image input means;
extracting image feature quantity of each of the plurality of retrieval images with a predetermined method with image feature quantity extracting means;
calculating a similarity degree of content of a moving image between the image feature quantity previously extracted from a still image of a frame structuring the content and the image feature quantity of each of the retrieval images with similarity degree calculation means;
specifying a frame of the still image being similar to each of the retrieval images by comparing the calculated similarity degree with a predetermined threshold value with specifying means; and
generating display data for performing timeline displaying by displaying the content on a time axis along with information indicating a temporal position of the frame of the still image being similar to each of the retrieval images based on the specified frame and metadata which is previously extracted from the content with timeline displaying means,
wherein the generated display data displaying a screen having a retrieval image display unit which displays a list of the plurality of retrieval images and a timeline display unit which displays the content on a timeline,
the plurality of retrieval images are displayed at the retrieval image display unit while being surrounded respectively by display frames in mutually different colors, and
information indicating a temporal position of the frame of the still image being similar to each of the retrieval images is displayed at the timeline display unit as a color bar in a same color as a color of the display frame.

7. A computer program embodied on a non-transitory computer readable medium for causing a computer to function as a content retrieval apparatus, the apparatus comprising:
retrieval image input means which receives input of a plurality of retrieval images being still images;
image feature quantity extracting means which extracts image feature quantity of each of the plurality of retrieval images with a predetermined method;
similarity degree calculation means which calculates a similarity degree of content of a moving image between the image feature quantity previously extracted from a still image of a frame structuring the content and the image feature quantity of each of the retrieval images;
specifying means which specifies a frame of the still image being similar to each of the retrieval images by comparing the calculated similarity degree with a predetermined threshold value; and
timeline displaying means which generates display data for performing timeline displaying by displaying the content on a time axis along with information indicating a temporal position of the frame of the still image being similar to each of the retrieval images based on the specified frame and metadata which is previously extracted from the content,
wherein the timeline displaying means generates display data for displaying a screen having a retrieval image display unit which displays a list of the plurality of retrieval images and a timeline display unit which displays the content on a timeline,
the plurality of retrieval images are displayed at the retrieval image display unit while being surrounded respectively by display frames in mutually different colors, and
information indicating a temporal position of the frame of the still image being similar to each of the retrieval images is displayed at the timeline display unit as a color bar in a same color as a color of the display frame.

Assignees

• Tanaka Kazumasa
• Tanaka Kenji
• Takahashi Yoshihiro
• Sony Corp

Inventors

• Tanaka Kazumasa
• Tanaka Kenji
• Takahashi Yoshihiro

Classifications

• G11B27/105 Primary

  of operating discs · CPC title

• G06F17/30256

  Physics · mapped topic

• H04N21/47217

  for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks (specific graphical features in visual interfaces H04N21/4312) · CPC title

• H04N21/440263

  by altering the spatial resolution, e.g. for displaying on a connected PDA · CPC title

• H04N21/84

  Generation or processing of descriptive data, e.g. content descriptors {(systems specially adapted for using meta-information in broadcast systems H04H60/73)} · CPC title