Deep generation of user-customized items

Patent metadata

Field	Value
Publication number	US-11694248-B2
Application number	US-202117192713-A
Country	US
Kind code	B2
Filing date	Mar 4, 2021
Priority date	Feb 15, 2018
Publication date	Jul 4, 2023
Grant date	Jul 4, 2023

Abstract

The present disclosure relates to a personalized fashion generation system that synthesizes user-customized images using deep learning techniques based on visually-aware user preferences. In particular, the personalized fashion generation system employs an image generative adversarial neural network and a personalized preference network to synthesize new fashion items that are individually customized for a user. Additionally, the personalized fashion generation system can modify existing fashion items to tailor the fashion items to a user's tastes and preferences.

First claim

What is claimed is:

1. A system for synthesizing user-customized images based on latent user preferences comprising:
a memory component comprising:
an image generative adversarial network that generates realistic images of items; and
a preference predictor network that determines preferences of individual users;
one or more processing devices coupled to the memory component, the one or more processing devices to perform operations comprising:
identifying a latent code that causes the image generative adversarial network to generate an image that approximates a query image;
determining an updated latent code optimized for a user based on the identified latent code and the preference predictor network by:
iteratively searching for an additional latent code in an adjacent random latent space that yields a higher preference prediction score by the preference predictor network than the latent code; and
selecting, as the updated latent code, the additional latent code that yields a highest preference prediction score for the user;

generating, using the updated latent code and the image generative adversarial network, a realistic synthesized image of an item customized for the user; and
providing the realistic synthesized image of the item customized for the user to a client device associated with the user.

2. The system of claim 1 , wherein:
the image generative adversarial network is trained using a corpus of images of fashion items corresponding to fashion categories to identify latent representations of fashion characteristics; and
the realistic synthesized image customized for the user comprises a synthesized image of a new fashion item generated for the user.

3. The system of claim 2 , the operations further comprising identifying a plurality of latent codes, wherein the image generative adversarial network employs the plurality of latent codes to generate realistic synthesized images of items.

4. The system of claim 2 , wherein the image generative adversarial network comprises:
a generator neural network trained on a corpus of fashion images to generate synthesized images of fashion items; and
a discriminator neural network trained on the corpus of fashion images to determine when generated synthesized images of new fashion items resemble real images of fashion items.

5. The system of claim 4 , wherein:
the image generative adversarial network trains in an unsupervised manner using the corpus of fashion images;
the image generative adversarial network alternates training the generator neural network and the discriminator neural network using objective functions via back propagation and least squares loss; and
the generator neural network generates synthesized images of fashion items following a same distribution of images from the corpus of images of fashion items.

6. The system of claim 1 , the operations further comprising determining the latent code from a plurality of latent codes that corresponds to latent user features of the user by employing a hyper-parameter that controls a trade-off between user preference score and image quality.

7. The system of claim 1 , wherein the preference predictor network determines, for the user and based on latent user features of the user, a preference predictor score for each image generated by the image generative adversarial network.

8. The system of claim 7 , the operations further comprising determining the latent code from a plurality of latent codes that corresponds to latent user features of the user based on iteratively searching for low-dimensional latent code that maximizes the preference predictor score for the user.

9. The system of claim 7 , the operations further comprising optimizing the determined latent code by employing gradient ascent within a constrained space to identify an optimized latent code that yields a higher preference predictor score from the preference predictor network than the determined latent code.

10. The system of claim 9 , the operations further comprising repeat identifying the optimized latent code by:
randomizing initial positions of latent code within random latent space of the image generative adversarial network for a predetermined number of iterations;
optimizing the latent code based on the latent user features of the user; and
selecting the optimized latent code that yields the higher preference predictor score for the user as the determined latent code.

11. The system of claim 1 , the operations further comprising identifying a plurality of latent codes, the plurality of latent codes comprising latent random noise vectors within random latent space of the image generative adversarial network.

12. The system of claim 1 , the operations further comprising:
generating a plurality of realistic synthesized images customized for the user; and
providing the plurality of realistic synthesized images customized for the user based on employing a probabilistic selection algorithm to increase diversity among the provided plurality of realistic synthesized images customized for the user.

13. A non-transitory computer-readable medium storing executable instructions which, when executed by a processing device, cause the processing device to perform operations comprising:
identifying a latent code that causes an image generative adversarial network to generate an image that approximates a query image;
determining an updated latent code optimized for a user based on the identified latent code, a preference predictor network, and the image generative adversarial network by:
iteratively searching for an additional latent code in an adjacent random latent space that yields a higher preference prediction score by the preference predictor network than the latent code; and
selecting, as the updated latent code, the additional latent code that yields a highest preference prediction score for the user;

generating, using the updated latent code and the image generative adversarial network, a realistic synthesized image of an item customized for the user; and
providing the realistic synthesized image of the item customized for the user to a client device associated with the user.

14. The non-transitory computer-readable medium of claim 13 , wherein the query image comprises an image of a fashion item of a given fashion category.

15. The non-transitory computer-readable medium of claim 13 , wherein the operations further comprise identifying the latent code that approximates the query image by iteratively searching for a latent code having a smallest L 1 distance between a corresponding image generated by the image generative adversarial network and the query image.

16. The non-transitory computer-readable medium of claim 13 , wherein the operations further comprise constraining the latent code by a hyperbolic tangent before determining the updated latent code.

17. The non-transitory computer-readable medium of claim 13 , wherein the operations further comprise:
determining, for the user and based on latent user features of the user, a preference predictor score for each image generated by the image generative adversarial network; and
determining the latent code from a plurality of latent codes that corresponds to latent user features of the user based on iteratively searching for low-dimensional latent code that maximizes the preference predictor score for the user.

18. The non-transitory computer-readable medium of claim 13 , wherein the realistic synthesized image of the item customized for the user yields a higher preference prediction score by the p

Assignees

• Adobe Inc
• Univ California

Inventors

• Fang Chen
• Wang Zhaowen
• Kang Wangcheng
• Mcauley Julian

Classifications

• G06N3/0475

  Generative networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/094

  Adversarial learning · CPC title

• G06N3/045

  Combinations of networks · CPC title