Image retrieval method, image retrieval devices, image retrieval system and image display system

Patent metadata

Field	Value
Publication number	US-12026192-B2
Application number	US-202117781093-A
Country	US
Kind code	B2
Filing date	Apr 30, 2021
Priority date	Apr 30, 2020
Publication date	Jul 2, 2024
Grant date	Jul 2, 2024

Abstract

An image retrieval method, an image retrieval device, and an image retrieval system, which are used for image retrieval, are provided. The image retrieval method comprises: receiving a first original image (S100); extracting an image feature of the first original image to obtain a first feature code (S200); obtaining first target information according to the first feature code (S300); according to the first target information, searching for a first target painting set corresponding to the first target information in at least one of a painting library and a knowledge graph library, so as to obtain the first target painting set (S400); and outputting the first target painting set (S500).

First claim

What is claimed is:

1. An image retrieval method, comprising:
receiving a first original image;
extracting an image feature of the first original image to obtain a first feature code, the first feature code being a string of numbers in binary;
obtaining first target information according to the first feature code, the first target information including a name, a serial number, or a store address of at least one image;
searching in at least one of a painting library and a knowledge graph library for a first target painting set corresponding to the first target information according to the first target information, so as to obtain the first target painting set; and
outputting the first target painting set.

2. The image retrieval method according to claim 1 , wherein
extracting the image feature of the first original image to obtain the first feature code, includes:
extracting the image feature of the first original image using a first image feature extraction model to obtain the first feature code; and
the image retrieval method further comprises:
acquiring operation information of a user in response to an operation of the user on the first target painting set;
updating the first image feature extraction model according to the operation information of the user to obtain a second image feature extraction model;
receiving a second original image;
extracting an image feature of the second original image using the second image feature extraction model to obtain a second feature code;
obtaining second target information according to the second feature code;
searching in the painting library and the knowledge graph library for a second target painting set corresponding to the second target information according to the second target information, so as to obtain the second target painting set; and
outputting the second target painting set.

3. The image retrieval method according to claim 2 , wherein
updating the first image feature extraction model according to the operation information of the user to obtain the second image feature extraction model, includes:
classifying the operation information of the user;
calculating a proportion of operation information of the user corresponding to each type of label;
adjusting a weight of an image feature corresponding to each type of label according to the proportion of the operation information of the user corresponding to each type of label;
forming the second image feature extraction model by training according to the first image feature extraction model and the adjusted weight; and
replacing the first image feature extraction model with the second image feature extraction model.

4. The image retrieval method according to claim 2 , wherein
obtaining the second target information according to the second feature code, includes:
calculating a distance between the second feature code and each feature code in a third feature code library and a fourth feature code library;
acquiring feature codes corresponding to distances in a preset range and taking the acquired feature codes as a second target feature code set; and
determining the second target information according to the second target feature code set;
wherein the third feature code library is a feature code library obtained by performing an image feature extraction on a plurality of images in the painting library using the second image feature extraction model; the fourth feature code library is a feature code library obtained by performing the image feature extraction on a plurality of images in the knowledge graph library using the second image feature extraction model.

5. The image retrieval method according to claim 1 , wherein
obtaining the first target information according to the first feature code, includes:
calculating a distance between the first feature code and each feature code in a first feature code library and a second feature code library;
acquiring feature codes corresponding to distances in a preset range and taking the acquired feature codes as a first target feature code set; and
determining the first target information according to the first target feature code set;
wherein the first feature code library is a feature code library obtained by performing an image feature extraction on a plurality of images in the painting library using a first image feature extraction model; and the second feature code library is a feature code library obtained by performing the image feature extraction on a plurality of images in the knowledge graph library using the first image feature extraction model.

6. The image retrieval method according to claim 1 , wherein
searching in the at least one of the painting library and the knowledge graph library for the first target painting set corresponding to the first target information according to the first target information, so as to obtain the first target painting set, includes:
searching in the painting library according to the first target information to obtain a first search result;
searching in the knowledge graph library according to the first target information to obtain a second search result;
if the first search result includes a hit painting, taking the first search result as the first target painting set; the hit painting being a painting with a highest similarity to the first original image among paintings corresponding to the first target information;
if the first search result does not include the hit painting and the second search result includes the hit painting, taking the first search result and the hit painting together as the first target painting set; and
if neither the first search result nor the second search result includes the hit painting, taking the first search result as the first target painting set.

7. The image retrieval method according to claim 1 , wherein
searching in the at least one of the painting library and the knowledge graph library for the first target painting set corresponding to the first target information according to the first target information, so as to obtain the first target painting set, includes:
searching in the painting library according to the first target information to obtain a first search result;
searching in the knowledge graph library according to the first target information to obtain a second search result;
if the first search result and the second search result are same, taking the first search result as the first target painting set; and
if the first search result and the second search result are not completely same, taking a union set of the first search result and the second search result as the first target painting set.

8. An image retrieval device configured to perform the image retrieval method according to claim 1 , the image retrieval device comprising a processor and a memory, wherein
the memory stores program instructions, and the program instructions are executed by the processor.

9. An image retrieval system, comprising:
an image retrieval device configured to perform the image retrieval method according to claim 1 ; and
a terminal device configured to capture the first original image and upload the first original image to the image retrieval device, receive the first target painting set output by the image retrieval device, and display the first target painting set in response to an operation of a user.

10. An image display system, comprising an image retrieval device, a terminal device and a painting display terminal, wherein
the image retrieval device is configured to perform the image retrieval method according to claim 1 ;
the terminal device is configured to capture the first original image and upload the first original image to the

Assignees

• Boe Technology Group Co Ltd

Inventors

• Xu Jingtao
• Zhang Shaonan

Classifications

• G06V10/774

  Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06V10/40

  Extraction of image or video features · CPC title

• G06F16/538

  Presentation of query results · CPC title

• G06F16/583 Primary

  using metadata automatically derived from the content · CPC title

• Y02P90/30

  Computing systems specially adapted for manufacturing · CPC title