Determining visually similar products

Patent metadata

Field	Value
Publication number	US-11907987-B2
Application number	US-202117550245-A
Country	US
Kind code	B2
Filing date	Dec 14, 2021
Priority date	Jan 22, 2020
Publication date	Feb 20, 2024
Grant date	Feb 20, 2024

Abstract

A computer-implemented method for determining image similarity includes determining, by a first neural network, a first feature value associated with a first characteristic of a first product based on an image of the first product. The method also includes determining, by a second neural network, a second feature value associated with a second characteristic of the first product based on the image of the first product. The method further involves calculating a first vector space distance between the first feature value and a third feature value associated with the first characteristic of a second product, and calculating a second vector space distance between the second feature value and a fourth feature value associated with the second characteristic of the second product. Additionally, the method includes determining a similarity value based on the first vector space distance and the second vector space distance.

First claim

What is claimed is:

1. A computer-implemented method for determining image similarity, the method comprising:
determining, by a first neural network, a first feature value associated with a first characteristic of a first image;
determining, by a second neural network, a second feature value associated with a second characteristic of the first image;
calculating a first distance between the first feature value and a third feature value associated with the first characteristic of a second image;
calculating a second distance between the second feature value and a fourth feature value associated with the second characteristic of the second image; and
based on a comparison of the first distance to the second distance, displaying, on a user interface, a representation of the first image in association with the second image.

2. The method of claim 1 , wherein the representation of the first image is displayed in a first location within a list of images, the method further comprising:
receiving a feature weighting value associated with the first characteristic;
determining a first weighted distance based at least in part on the first distance and the feature weighting value;
calculating a weighted similarity value based on the first weighted distance and the second distance; and
determining a second location within the list of images at which to display the first image, based at least in part on the weighted similarity value.

3. The method of claim 1 , wherein the first neural network is independent from the second neural network.

4. The method of claim 1 , wherein the first image comprises a first product, and the second image comprises a second product.

5. The method of claim 1 , wherein the second characteristic is substantially independent from the first characteristic.

6. The method of claim 1 , wherein calculating the first distance comprises at least one of:
calculating, as the first distance, a cosine similarity value between the first feature value and the third feature value;
calculating, as the first distance, a Euclidean distance between the first feature value and the third feature value; or
calculating, as the first distance, a Chebyshev distance between the first feature value and the third feature value.

7. The method of claim 1 , wherein the first neural network is configured to extract one or more feature values representative of the first characteristic from input images.

8. The method of claim 1 , wherein the second neural network is configured to extract one or more feature values representative of the second characteristic from input images.

9. The method of claim 1 , further comprising:
determining, by a third neural network, a fifth feature value associated with a third characteristic of the first image.

10. The method of claim 1 , wherein the first characteristic includes color information determinable based on an input image.

11. The method of claim 1 , wherein the first characteristic includes shape information determinable based on an input image.

12. The method of claim 1 , wherein the first characteristic includes pattern information determinable based on an input image.

13. The method of claim 1 , wherein the first characteristic includes style information determinable based on an input image.

14. The method of claim 1 , wherein the comparison comprises determination of a first similarity value, the method further comprising:
determining, by the first neural network, a fifth feature value associated with the first characteristic of a third image;
determining, by the second neural network, a sixth feature value associated with the second characteristic of the third image;
determining a third distance between the third feature value associated with the second image and the fifth feature value associated with the third image;
determining a fourth distance between the fourth feature value associated with the second image and the sixth feature value associated with the third image;
determining a second similarity value based on the third distance and the fourth distance; and
based on the second similarity value being greater than the first similarity value, displaying, on the user interface, a representation of images that are visually similar to the second image in descending order, in which the third image precedes the first image.

15. A computer-implemented method for determining image similarity, the method comprising:
retrieving a set of weighting values based on a product category for a first product;
determining, by a first feature extractor, a first feature value associated with a first characteristic of an image of the first product;
determining, by a second feature extractor, a second feature value associated with a second characteristic of the first product image;
determining a first weighted distance based on the first feature value, a third feature value associated with the first characteristic of a second product image, and a first weighting value of the set of weighting values;
determining a second weighted distance based on the second feature value, a fourth feature value associated with the second characteristic of the second product image, and a second weighting value of the set of weighting values; and
in response to a comparison of the first weighted distance and the second weighted distance, displaying, on a user interface, a representation of the second product image in association with the first product image.

16. The method of claim 15 , wherein the comparison comprises:
determining a similarity value based on the first weighted distance and the second weighted distance; and
comparing the similarity value to a threshold value.

17. The method of claim 15 , wherein the representation of the first product image is displayed in a first location within a list of products, the method further comprising:
receiving a user-specified feature weighting value associated with the first characteristic based on user input;
determining a third weighted distance based on the first feature value, the user-specified feature weighting value, and the third feature value; and
based on a comparison of the third weighted distance and the second weighted distance, determining a second location within the list of products at which to display the first product image.

18. The method of claim 15 , wherein determining the first weighted distance comprises adjusting a first distance based on the first weighting value, the first distance calculated as at least one of:
a cosine similarity value between the first feature value and the third feature value;
a Euclidean distance between the first feature value and the third feature value; or
a Chebyshev distance between the first feature value and the third feature value.

19. The method of claim 15 , wherein the first feature extractor is a convolutional neural network.

Assignees

• Home Depot Product Authority Llc

Inventors

• Afshar Estelle
• Hagen Matthew
• Qu Huiming

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/0985

  Hyperparameter optimisation; Meta-learning; Learning-to-learn · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06Q30/0623 Primary

  by investigating goods or services · CPC title

• G06F18/22 Primary

  Matching criteria, e.g. proximity measures · CPC title