Systems and methods for recommending a product based on an image of a scene

Patent metadata

Field	Value
Publication number	US-12086859-B2
Application number	US-202217726723-A
Country	US
Kind code	B2
Filing date	Apr 22, 2022
Priority date	Nov 21, 2019
Publication date	Sep 10, 2024
Grant date	Sep 10, 2024

Abstract

In some embodiments, a computer-implemented system within an e-commerce platform may receive and analyse digital images of scenes in order to generate a recommendation for one or more products offered for sale by a merchant. For example, a user may use their device to capture an image of a room that the user wishes to furnish, and the system may use a trained machine learning algorithm to recommend a product to be placed in the room. The recommended product may be superimposed on the image of the room.

First claim

The invention claimed is:

1. A computer-implemented method comprising:
obtaining a digital image of a scene, the digital image having been captured by a user device;
determining visual property values of the digital image of the scene; and
generating, based on the visual property values of the digital image of the scene and relationships between merchant products and respective visual property values, a recommended merchant product from the merchant products, wherein the recommended merchant product is generated based on a relationship between the visual property values of the digital image and visual property values of at least one scene that includes the recommended merchant product.

2. The computer-implemented method of claim 1 , wherein for each merchant product of the merchant products, the respective visual property values are indicative of visual properties of at least one scene that includes the merchant product.

3. The computer-implemented method of claim 1 , wherein the recommended merchant product is further generated by selecting the recommended merchant product as one of the merchant products that has respective visual property values closest to the visual property values of the digital image.

4. The computer-implemented method of claim 1 , wherein the recommended merchant product is further generated by executing a trained machine learning algorithm using the visual property values of the digital image.

5. The computer-implemented method of claim 1 , further comprising instructing the user device to display a digital image of the recommended merchant product on a display of the user device.

6. The computer-implemented method of claim 5 , comprising instructing the user device to superimpose the digital image of the recommended merchant product on the digital image of the scene.

7. The computer-implemented method of claim 5 , further comprising transmitting to the user device a website link for a website associated with the recommended merchant product.

8. The computer-implemented method of claim 1 , further comprising generating the relationships between the merchant products and the respective visual property values by performing operations including:
receiving a plurality of images of scenes, at least some of the scenes including one or more of the merchant products; and
using machine learning on the plurality of images to identify, for each merchant product of one or more of the merchant products: at least some of the respective visual property values that correspond to the merchant product.

9. The computer-implemented method of claim 1 , comprising:
receiving, from a merchant device, a merchant image that includes a particular merchant product;
determining, using the merchant image, at least some of the respective visual property values that correspond to the particular merchant product.

10. The computer-implemented method of claim 1 , further comprising:
requesting a plurality of images from a social media platform;
associating a particular image of the plurality of images with a particular merchant product of the merchant products;
determining a visual property of the particular image;
modifying a visual property value associated with the particular merchant product based on the visual property of the particular image.

11. A system comprising:
a memory to store a digital image of a scene that was captured by a user device;
at least one processor to:
determine visual property values of the digital image of the scene; and
generate, based on the visual property values of the digital image of the scene and relationships between merchant products and respective visual property values, a recommended merchant product from the merchant products, wherein the recommended merchant product is generated based on a relationship between the visual property values of the digital image and visual property values of at least one scene that includes the recommended merchant product.

12. The system of claim 11 , wherein for each merchant product of the merchant products, the respective visual property values are indicative of visual properties of at least one scene that includes the merchant product.

13. The system of claim 11 , wherein the at least one processor is to generate the recommended merchant product by performing operations including: selecting the recommended merchant product as one of the merchant products that has respective visual property values closest to the visual property values of the digital image.

14. The system of claim 11 , wherein the at least one processor is to generate the recommended merchant product by performing operations including: executing a trained machine learning algorithm using the visual property values of the digital image.

15. The system of claim 11 , wherein the at least one processor is further to instruct the user device to display a digital image of the recommended merchant product on a display of the user device.

16. The system of claim 15 , wherein the at least one processor is to instruct the user device to superimpose the digital image of the recommended merchant product on the digital image of the scene.

17. The system of claim 15 , wherein the at least one processor is further to transmit, to the user device, a website link for a website associated with the recommended merchant product.

18. The system of claim 11 , wherein the at least one processor is to generate the relationships between the merchant products and the respective visual property values by performing operations including:
receiving a plurality of images of scenes, at least some of the scenes including one or more of the merchant products; and
using machine learning on the plurality of images to identify, for each merchant product of one or more of the merchant products: at least some of the respective visual property values that correspond to the merchant product.

19. The system of claim 11 , wherein the at least one processor is to:
receive, from a merchant device, a merchant image that includes a particular merchant product;
determine, using the merchant image, at least some of the respective visual property values that correspond to the particular merchant product.

20. A non-transitory computer-readable medium having stored thereon computer-executable instructions that, when executed, cause a computer to perform operations comprising:
obtaining a digital image of a scene, the digital image having been captured by a user device;
determining visual property values of the digital image of the scene; and
generating, based on the visual property values of the digital image of the scene and relationships between merchant products and respective visual property values, a recommended merchant product from the merchant products, wherein the recommended merchant product is generated based on a relationship between the visual property values of the digital image and visual property values of at least one scene that includes the recommended merchant product.

Assignees

• Shopify Inc

Inventors

• D'Souza Franklyn
• Wade Jonathan
• Haapoja Juho Mikko

Classifications

• G06F16/583

  using metadata automatically derived from the content · CPC title

• G06N20/00

  Machine learning · CPC title

• G06F16/5838

  using colour · CPC title

• G06Q30/0631 Primary

  Recommending goods or services · CPC title