Determining visually similar products

Patent metadata

Field	Value
Publication number	US-11200445-B2
Application number	US-202016749629-A
Country	US
Kind code	B2
Filing date	Jan 22, 2020
Priority date	Jan 22, 2020
Publication date	Dec 14, 2021
Grant date	Dec 14, 2021

Abstract

A computer-implemented method for determining image similarity includes determining, by a first neural network, a first feature value associated with a first characteristic of a first product based on an image of the first product. The method also includes determining, by a second neural network, a second feature value associated with a second characteristic of the first product based on the image of the first product. The method further involves calculating a first vector space distance between the first feature value and a third feature value associated with the first characteristic of a second product, and calculating a second vector space distance between the second feature value and a fourth feature value associated with the second characteristic of the second product. Additionally, the method includes determining a similarity value based on the first vector space distance and the second vector space distance.

First claim

What is claimed is:

1. A computer-implemented method for determining image similarity, the method comprising:
determining, by a first neural network, a first feature value associated with a first characteristic of a first product based on an image of the first product;
determining, by a second neural network, a second feature value associated with a second characteristic of the first product based on the image of the first product;
calculating a first vector space distance between the first feature value and a third feature value associated with the first characteristic of a second product;
calculating a second vector space distance between the second feature value and a fourth feature value associated with the second characteristic of the second product;
determining a similarity value based on the first vector space distance and the second vector space distance; and
based on the similarity value exceeding a threshold value, displaying, on a user interface, a representation of the first product in association with the second product.

2. The method of claim 1 , wherein the representation of the first product is displayed in a first location within a list of products, the method further comprising:
receiving a feature weighting value associated with the first characteristic;
determining a first weighted vector space distance based at least in part on the first vector space distance and the feature weighting value; and
calculating a weighted similarity value based on the first weighted vector space distance and the second vector space distance; and
determining a second location within the list of products at which to display the first product, based at least in part on the weighted similarity value.

3. The method of claim 1 , wherein the first neural network is independent from the second neural network.

4. The method of claim 1 , further comprising:
determining, by the first neural network, the third feature value associated with the first characteristic of the second product based on an image of the second product; and
determining, by the second neural network, the fourth feature value associated with the second characteristic of the second product based on the image of the second product.

5. The method of claim 1 , wherein the second characteristic is substantially independent from the first characteristic.

6. The method of claim 1 , wherein calculating the first vector space distance comprises:
calculating, as the first vector space distance, a cosine similarity value between the first feature value and the third feature value.

7. The method of claim 1 , wherein calculating the first vector space distance comprises:
calculating, as the first vector space distance, a Euclidean distance between the first feature value and the third feature value.

8. The method of claim 1 , wherein calculating the first vector space distance comprises:
calculating, as the first vector space distance, a Chebyshev distance between the first feature value and the third feature value.

9. The method of claim 1 , wherein the first neural network is configured to extract one or more feature values representative of the first characteristic from input images.

10. The method of claim 1 , wherein the second neural network is configured to extract one or more feature values representative of the second characteristic from input images.

11. The method of claim 1 , further comprising:
determining, by a third neural network, a fifth feature value associated with a third characteristic of the first product based on the image of the first product.

12. The method of claim 1 , wherein the first characteristic includes color information determinable based on an input image.

13. The method of claim 1 , wherein the first characteristic includes shape information determinable based on an input image.

14. The method of claim 1 , wherein the first characteristic includes pattern information determinable based on an input image.

15. The method of claim 1 , wherein the first characteristic includes style information determinable based on an input image.

16. The method of claim 1 , wherein the similarity value is a first similarity value, the method further comprising:
determining, by the first neural network, a fifth feature value associated with the first characteristic of a third product based on an image of the third product;
determining, by the second neural network, a sixth feature value associated with the second characteristic of the third product based on the image of the third product;
determining a third vector space distance between the third feature value associated with the second product and the fifth feature value associated with the third product;
determining a fourth vector space distance between the fourth feature value associated with the second product and the sixth feature value associated with the third product;
determining a second similarity value based on the third vector space distance and the fourth vector space distance; and
based on the second similarity value being greater than the first similarity value, displaying, on the user interface, a representation of products that are visually similar to the second product in descending order, in which the third product precedes the first product.

17. A computer-implemented method for determining image similarity, the method comprising:
determining, by a first feature extractor, a first feature value associated with a first characteristic of a first product based on an image of the first product;
determining, by a second feature extractor, a second feature value associated with a second characteristic of the first product based on the image of the first product;
determining a first weighted vector space distance based on the first feature value, a third feature value associated with the first characteristic of a second product, and a predetermined weighting value;
determining a second vector space distance based on the second feature value and a fourth feature value associated with the second characteristic of the second product;
determining a similarity value based on the first weighted vector space distance and the second vector space distance; and
based on the similarity value exceeding a threshold value, displaying, on a user interface, a representation of the second product in association with the first product.

18. The method of claim 17 , wherein the representation of the first product is displayed in a first location within a list of products, the method further comprising:
receiving a user-specified feature weighting value associated with the first characteristic based on user input;
determining a third weighted vector space distance based at least in part on the first feature value, the user-specified feature weighting value, and the third feature value;
determining a user-specified weighted similarity value based on the third weighted vector space distance and the second feature value vector; and
determining a second location within the list of products at which to display the first product, based at least in part on the user-specified weighted similarity value.

19. The method of claim 17 , wherein the first feature extractor is a convolutional neural network.

20. A system for determining visual similarity between images of products, the system comprising:
a processor; and
a non-transitory, computer-readable memory storing instructions that, when executed by the processor, cause the processor to perform a method comprising:
determ

Assignees

• Home Depot Product Authority Llc

Inventors

• Afshar Estelle
• Hagen Matthew
• Qu Huiming

Classifications

• G06V10/806

  of extracted features · CPC title

• G06V10/761

  Proximity, similarity or dissimilarity measures · CPC title

• G06F18/22 Primary

  Matching criteria, e.g. proximity measures · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06F18/253

  of extracted features · CPC title