Generating visually-aware item recommendations using a personalized preference ranking network

Patent metadata

Field	Value
Publication number	US-11100400-B2
Application number	US-201815897822-A
Country	US
Kind code	B2
Filing date	Feb 15, 2018
Priority date	Feb 15, 2018
Publication date	Aug 24, 2021
Grant date	Aug 24, 2021

Abstract

The present disclosure relates to a fashion recommendation system that employs a task-guided learning framework to jointly train a visually-aware personalized preference ranking network. In addition, the fashion recommendation system employs implicit feedback and generated user-based triplets to learn variances in the user's fashion preferences for items with which the user has not yet interacted. In particular, the fashion recommendation system uses triplets generated from implicit user data to jointly train a Siamese convolutional neural network and a personalized ranking model, which together produce a user preference predictor that determines personalized fashion recommendations for a user.

First claim

What is claimed is:

1. A non-transitory computer-readable medium storing instructions that, when executed by at least one processor, cause a computer system to:
maintain an item dataset comprising images of each of the items;
determine latent user features for a user identity generated by a personalized ranking machine-learning model for the user identity;
generate latent item features for the user identity utilizing an item personalization neural network that is part of a Siamese convolutional neural network and jointly built with the personalized ranking machine-learning model;
determine preference prediction scores between the user identity and one or more items of the item dataset based on correlating the latent item features and the latent user features; and
provide, based on the preference prediction scores, one or more items from the item dataset to a client device associated with the user identity.

2. The non-transitory computer-readable medium of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the computer system to provide the one or more items from the dataset after ranking the one or more items based on the preference prediction scores.

3. The non-transitory computer-readable medium of claim 1 , wherein the item dataset comprises item with which the user has not yet interacted.

4. The non-transitory computer-readable medium of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the computer system to identify the latent user features outputted from the personalized ranking machine-learning model based on a user identifier of the user.

5. In a digital medium environment for recommending fashion items to a user, a computer-implemented method of determining latent user features based on implicit user feedback, comprising:
maintaining an item dataset comprising images of each of the items;
identifying a user identity;
a step for determining item preferences for the user identity from an unobserved dataset of items; and
providing, based on the preferences for the user identity, one or more items from the dataset to a client device associated with the user identity.

6. The method of claim 5 , further comprising generating triplets where each of the triplets comprises a user identity, a positive item image from an image dataset of items based on implicit feedback from the user identity, and a negative item image from the image dataset of items, wherein the positive item image in each triplet is ranked with a higher user preference for the user identity than the negative image item in the triplet.

7. The method of claim 6 , wherein the unobserved dataset of items corresponds to a subcategory of articles of clothing or fashion accessories.

8. The method of claim 6 , wherein the implicit feedback comprises clicks, views, and purchases of items in the dataset of items.

9. A system for generating high-resolution digital images from edited semantic layouts, the system comprising:
one or more memory devices comprising:
item dataset comprising images of each of the items;
latent user features determined for a user identity by a personalized ranking machine-learning model;
a dual-branch Siamese convolutional neural network that is jointly generated with the personalized ranking machine-learning model; and
a preference predictor model that correlates latent user features with latent item features

at least one processor configured to cause the system to:
generate latent item features for the user identity utilizing an item personalization neural network that is a branch of the dual-branch Siamese convolutional neural network;
determine preference prediction scores between the user identity and one or more items of the item dataset based on correlating the latent item features and the latent user features utilizing the preference predictor model; and
provide, based on the preference prediction scores, one or more items from the dataset to a client device associated with the user identity.

10. The system of claim 9 , wherein the dual-branch Siamese convolutional neural network is jointly utilized with the personalized ranking machine-learning model based on implicit feedback observed for the user identity.

11. The system of claim 9 , wherein the at least one processor is further configured to cause the system to utilize the preference predictor model to determine preference prediction scores between the user identity and one or more items of the item dataset by comparing visual-user item preferences determined from the latent user features with latent item features of the one or more items.

12. The system of claim 9 , wherein the at least one processor is further configured to cause the system to determine the latent user features for the user identity utilizing the personalized ranking machine-learning model for the user identity based on images of items that the user identity interacted with inherently.

13. The system of claim 9 , wherein the at least one processor is further configured to cause the system to determine the latent user features for the user identity utilizing the personalized ranking machine-learning model for the user identity based on a Bayesian personalized ranking algorithm and implicit visual user feedback.

14. The system of claim 9 , wherein the at least one processor is further configured to cause the system to rank the one or more items of the item dataset based on the preference prediction scores.

15. The system of claim 10 , wherein the at least one processor is further configured to cause the system to generate a set of triplets, each triplet comprising the user identity, a positive item image from an image dataset of items based on the implicit feedback from the user identity, and a negative item image from the image dataset of items, wherein the positive item image in each triplet is ranked with a higher user preference for the user identity than the negative image item in the triplet.

16. The system of claim 9 , wherein the item dataset comprises items with which the user identity has not yet viewed or interacted.

17. The system of claim 13 , wherein implicit visual user feedback comprises actions detected by the user identity with images of items for which the user identity does not expressly intend to be shared with other user identities.

18. The system of claim 9 , wherein the dual-branch Siamese convolutional neural network comprises a positive item personalization neural network and a negative item personalization neural network.

19. The system of claim 18 , wherein the item personalization neural network comprises one of the positive item personalization neural network or the negative item personalization neural network.

20. The system of claim 9 , wherein the items correspond to articles of clothing or fashion accessories.

Assignees

• Adobe Inc
• Univ California

Inventors

• Fang Chen
• Wang Zhaowen
• Kang Wangcheng
• Mcauley Julian

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06N3/084 Primary

  Backpropagation, e.g. using gradient descent · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06F16/51

  Indexing; Data structures therefor; Storage structures · CPC title