Apparatus and method for automatically generating visual annotation based on visual language

Patent metadata

Field	Value
Publication number	US-9606975-B2
Application number	US-201414320362-A
Country	US
Kind code	B2
Filing date	Jun 30, 2014
Priority date	Dec 10, 2013
Publication date	Mar 28, 2017
Grant date	Mar 28, 2017

Abstract

Provided are an apparatus and method for automatically generating a visual annotation with respect to a massive image based on a visual language. The apparatus for automatically generating a visual annotation based on a visual language includes an image input unit configured to receive an input image, an image analyzing unit configured to extract feature information of the input image received by the image input unit, a searching unit configured to search a similar image with respect to the input Image and text information included in the similar image by using the feature information extracted by the image analyzing unit, and a visual annotation configuring unit configured to configure a visual annotation with respect to the input image by using the text information searched by the searching unit.

First claim

What is claimed is:

1. An apparatus including a memory and computer processor for automatically generating a visual annotation based on a visual language, the apparatus comprising:
image input instructions configured to, using the processor, receive an input image;
image analyzing instructions configured to extract feature information of the input image received by the image input instructions;
searching instructions configured to, using a processor, receive feature information of images included in an image and metadata database (DB), compare the feature information of the input image with the feature information of the images included in the image and metadata DB to select a similar image with respect to the input image, set a candidate group by using text information of the similar image, and select text information having frequency equal to or higher than pre-set frequency among a plurality of text information included in the candidate group, as text information for the input image; and
visual annotation instructions configured to, using a processor, generate a keyword list by analyzing the text information for the input image selected by the searching instructions, and generate the visual annotation as at least one sentence expressing contents included in the input image with respect to at least one of a plurality of objects included in the input image by semantically analyzing the text information in the keyword list.

2. The apparatus of claim 1 , wherein the image analyzing instructions analyze the input image to extract at least one of a color, a name, a shape, and a position of an object included in the input image, and an interrelation between objects, as the feature information of the input image.

3. The apparatus of claim 1 , wherein the searching instructions calculate similarity between the feature information of the input image and the feature information of the images included in the image and meat data DB, and selects an image, included in the image and metadata DB, having calculated similarity equal to or greater than a pre-set value, as the similar image with respect to the input image.

4. A method for automatically generating a visual annotation based on a visual language, the method comprising:
receiving an input image;
extracting feature information of the input image;
receiving feature information of images included in an image and metadata database (DB); and
comparing the feature information of the input image with the feature information of the images included in the image and metadata DB to select a similar image with respect to the input image;
analyzing the feature information of the input image and text information of the similar image to select text information for the input image; and
generating a keyword list by using the selected text information and configuring the visual annotation as at least one sentence expressing contents included in the input image with respect to the input image by semantically analyzing the text information in the keyword list,
wherein the analyzing comprises:
setting a candidate group by using the text information of the similar image; and
selecting text information having frequency equal to or higher than pre-set frequency among a plurality of text information included in the candidate group, as the text information for the input image.

5. The method of claim 4 , wherein the extracting feature information of the input image comprises analyzing the input image to extract at least one of a color, a name, a shape, and a position of an object included in the input image, and an interrelation between objects, as the feature information of the input image.

6. The method of claim 4 , wherein the comparing comprises calculating similarity between the feature information of the input image and the feature information of the images included in the image and metadata DB, and selecting an image having high calculated similarity as the similar image of the input image.

7. The method of claim 4 , wherein the analyzing the feature information of the input image and the text information further comprises selecting the text information of the input image by receiving selection information with respect to at least one text information of the candidate group.

8. The method of claim 7 , wherein the generating a keyword list by using the selected text information comprises generating the keyword list by using the text information of the input image, and
wherein the configuring the visual annotation comprises configuring the visual annotation with respect to at least one of a plurality of objects included in the input image by using the keyword list.

9. A system for automatically generating a visual annotation based on a visual language, the system comprising:
a terminal configured to receive an input image and transmit the input image and a visual annotation request signal with respect to the input image; and
a server including a processor,
wherein the processor is configured to:
receive the input image and the visual annotation request signal with respect to the input image from the terminal,
extract feature information of the input image, search images stored in an image and metadata database (DB) using the feature information of the input image,
compare the feature information of the input image with feature information of the images stored in the image and metadata DB to select a similar image with respect to the input image,
set a candidate group by using text information of the similar image, select text information having frequency equal to or greater than pre-set frequency among a plurality of text information in the candidate group, as text information for the input image,
generate a keyword list by analyzing the text information of the input image, and
configure the visual annotation as at least one sentence expressing contents included in the input image with respect to at least one of a plurality of objects included in the input image by semantically analyzing the text information in the keyword list.

10. The system of claim 9 , wherein the processor analyzes the input image received from the terminal to extract any one of a color, a name, a shape, and a position of an object included in the input image, and an interrelation between objects, as the feature information of the input image.

11. The system of claim 9 , wherein the processor searches images previously collected from cloud computing, a Web, or a different server, and the image and metadata DB including feature information and text with respect to the images to select the similar image with respect to the input image.

12. The system of claim 11 , wherein the processor calculates similarity between the feature information of the input image and the feature information of the images stored in the image and metadata DB, and selects an image having high similarity as the similar image.

13. The system of claim 9 , wherein the processor receives selection information with respect to at least one text information in the candidate group, and selects the text information for the input image according to the selection information.

Assignees

• Electronics & Telecommunications Res Inst

Inventors

• Oh Weon Geun
• Na Sang Il

Classifications

• G06F40/169 Primary

  Annotation, e.g. comment data or footnotes · CPC title

• G06F16/5862

  using texture · CPC title

• G06F16/5854

  using shape and object relationship · CPC title

• G06F16/5838 Primary

  using colour · CPC title

• G06F17/30256

  Physics · mapped topic