Image retrieval system and image retrieval method

Patent metadata

Field	Value
Publication number	US-12174882-B2
Application number	US-201917291196-A
Country	US
Kind code	B2
Filing date	Nov 6, 2019
Priority date	Nov 13, 2018
Publication date	Dec 24, 2024
Grant date	Dec 24, 2024

Abstract

An image retrieval system that enables high-accuracy image retrieval in a short time is provided. The image retrieval system includes a processing portion provided with a neural network. The neural network includes a layer provided with a neuron. The processing portion has a function of comparing query image data with a plurality of pieces of database image data, and extracting the database image data including an area with a high degree of correspondence to the query image data as extracted image data. The processing portion has a function of extracting data of the area with a high degree of correspondence to the query image data from the extracted image data, as partial image data. The layer has a function of outputting an output value corresponding to the features of the image data input to the neural network. The processing portion has a function of comparing the above output values in the case where the respective pieces of partial image data are input with the above output value in the case where the query image data is input.

First claim

The invention claimed is:

1. An image retrieval system comprising a processing circuitry,
wherein the processing circuitry comprises a neural network,
wherein the neural network comprises a convolutional layer and a pooling layer,
wherein the processing circuitry is configured to calculate a degree of correspondence between image data and each area of a plurality of pieces of database image data by area-based matching when the image data and the plurality of pieces of database image data are input to the processing circuitry,
wherein the processing circuitry is configured to extract database image data of the plurality of pieces of database image data based on the degree of correspondence to the image data as extracted image data,
wherein, after extracting the database image data of the plurality of pieces of database image data as extracted image data, the processing circuitry is configured to extract data of an area based on the degree of correspondence to the image data as partial image data from the extracted image data,
wherein, after extracting the partial image data from the extracted image data, the image data and the partial image data are input to a first layer of the neural network,
wherein the pooling layer is configured to output a first output value corresponding to the image data after inputting the image data to the first layer of the neural network,
wherein the pooling layer is configured to output a second output value corresponding to the partial image data after inputting the partial image data to the first layer of the neural network,
wherein the processing circuitry is configured to compare the first output value with the second output value,
wherein, during the area-based matching, a plurality of first pixel data of the image data is compared with a plurality of second pixel data of the plurality of pieces of database image data,
wherein the plurality of first pixel data corresponds to a plurality of pixels of the image data, and
wherein each of the plurality of first pixel data corresponds to a luminance value.

2. The image retrieval system according to claim 1 ,
wherein the number of pieces of pixel data included in the image data is less than or equal to the number of pieces of pixel data included in the plurality of pieces of database image data,
wherein third pixel data of the plurality of first pixel data and fourth pixel data of the plurality of first pixel data are compared with fifth pixel data of the plurality of second pixel data and sixth pixel data of the plurality of second pixel data,
wherein the degree of correspondence between the third pixel data of the plurality of first pixel data and the fourth pixel data of the plurality of first pixel data and an area formed by the fifth pixel data of the plurality of second pixel data and the sixth pixel data of the plurality of second pixel data is calculated,
wherein the plurality of first pixel data compared with the plurality of second pixel data correspond to a compared data area, and
wherein the compared data area slides one column at a time in the pixel data included in the plurality of pieces of database image data.

3. The image retrieval system according to claim 1 , wherein the processing circuitry comprises a transistor.

4. The image retrieval system according to claim 1 ,
wherein the image data and the plurality of pieces of database image data each comprise a symbol.

5. The image retrieval system according to claim 1 ,
wherein the image data and the plurality of pieces of database image data represent a drawing included in intellectual property information.

6. An image retrieval method comprising:
calculating a degree of correspondence between image data and each area of a plurality of pieces of database image data by area-based matching;
extracting database image data of the plurality of pieces of database image data based on the degree of correspondence to the image data as extracted image data;
extracting data of an area based on the degree of correspondence to the image data as partial image data from the extracted image data;
inputting the image data to a neural network comprising a convolutional layer, a fully connected layer, and a plurality of pooling layers so as to acquire a first output value output from one of the plurality of pooling layers;
inputting the partial image data to the neural network so that a second output value output from the one of the plurality of pooling layers is acquired; and
comparing the first output value with the second output value,
wherein an input data value input into the convolutional layer corresponds to a gray level represented by pixel data,
wherein, during the area-based matching, a plurality of first pixel data of the image data is compared with a plurality of second pixel data of the plurality of pieces of database image data,
wherein the plurality of first pixel data corresponds to a plurality of pixels of the image data,
wherein each of the plurality of first pixel data corresponds to a luminance value, and
wherein the luminance value represents a plurality of gray levels.

7. The image retrieval method according to claim 6 ,
wherein the image data comprises a plurality of pieces of pixel data, and
wherein a plurality of pieces of image data that differ in the number of pieces of the pixel data to be provided are generated on the basis of the image data, and then the image data is compared with the plurality of pieces of database image data.

8. The image retrieval method according to claim 6 ,
wherein the image data and the plurality of pieces of database image data each comprise a symbol.

9. The image retrieval method according to claim 6 ,
wherein the image data and the plurality of pieces of database image data represent a drawing included in intellectual property information.

Assignees

• Semiconductor Energy Lab

Inventors

• Akimoto Kengo
• Fukutome Takahiro
• Hayashi Kentaro

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/04

  Architecture, e.g. interconnection topology · CPC title

• G06V10/751

  Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching · CPC title

• G06V10/25

  Determination of region of interest [ROI] or a volume of interest [VOI] · CPC title