Selecting logo images using machine-learning-logo classifiers

Patent metadata

Field	Value
Publication number	US-11106944-B2
Application number	US-201916557330-A
Country	US
Kind code	B2
Filing date	Aug 30, 2019
Priority date	Aug 30, 2019
Publication date	Aug 31, 2021
Grant date	Aug 31, 2021

Abstract

This disclosure relates to methods, non-transitory computer readable media, and systems that can initially train a machine-learning-logo classifier using synthetic training images and incrementally apply the machine-learning-logo classifier to identify logo images to replace the synthetic training images as training data. By incrementally applying the machine-learning-logo classifier to determine one or both of logo scores and positions for logos within candidate logo images, the disclosed systems can select logo images and corresponding annotations indicating positions for ground-truth logos. In some embodiments, the disclosed systems can further augment the iterative training of a machine-learning-logo classifier to include user curation and removal of incorrectly detected logos from candidate images, thereby avoiding the risk of model drift across training iterations.

First claim

We claim:

1. A non-transitory computer readable medium storing instructions thereon that, when executed by at least one processor, cause a computing device to:
train a machine-learning-logo classifier based on a set of synthetic training images comprising logos corresponding to a logo class;
apply the machine-learning-logo classifier to a set of candidate logo images to select a subset of logo images;
replace a subset of synthetic training images with the subset of logo images to generate a set of mixed training images;
retrain the machine-learning-logo classifier based on the set of mixed training images; and
select a set of logo images comprising ground-truth logos corresponding to the logo class utilizing the retrained machine-learning-logo classifier.

2. The non-transitory computer readable medium of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the computing device to apply the machine-learning-logo classifier to the set of candidate logo images to select the subset of logo images by:
generating logo scores for the set of candidate logo images utilizing the machine-learning-logo classifier, wherein a logo score indicates a likelihood that a candidate logo image portrays a logo corresponding to the logo class; and
selecting the subset of logo images from the set of candidate logo images based in part on the logo scores.

3. The non-transitory computer readable medium of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the computing device to:
apply the machine-learning-logo classifier to generate logo scores for the set of candidate logo images; and
based on the logo scores, provide, for display within a curation user interface, candidate logos from a subset of candidate logo images from the set of candidate logo images and logo-rejection options corresponding to the subset of candidate logo images.

4. The non-transitory computer readable medium of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the computing device to, after retraining the machine-learning-logo classifier based on the set of mixed training images:
apply the machine-learning-logo classifier to an additional set of candidate logo images to select an additional subset of logo images;
replace an additional subset of training images from within the set of synthetic training images with the additional subset of logo images to generate an additional set of mixed training images; and
retrain the machine-learning-logo classifier based on the additional set of mixed training images.

5. The non-transitory computer readable medium of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the computing device to:
select an additional set of logo images comprising additional ground-truth logos corresponding to an additional logo class utilizing an additional machine-learning-logo classifier; and
train a multi-logo-classification model based on the set of logo images and the additional set of logo images.

6. The non-transitory computer readable medium of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the computing device to apply the machine-learning-logo classifier to the set of candidate logo images by generating boundary identifiers indicating positions of candidate logos within the set of candidate logo images utilizing the machine-learning-logo classifier.

7. The non-transitory computer readable medium of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the computing device to:
identify candidate images from among image-search results generated by a search query corresponding to the logo class; and
remove duplicate images from the candidate images based on average-pixel values to generate the set of candidate logo images.

8. The non-transitory computer readable medium of claim 5 , further comprising instructions that, when executed by the at least one processor, cause the computing device to:
select a new set of logo images comprising new ground-truth logos corresponding to a new logo class utilizing a new machine-learning-logo classifier; and
train the multi-logo-classification model based on the set of logo images, the additional set of logo images, and the new set of logo images.

9. A system comprising:
one or more memory devices comprising a set of synthetic training images comprising logos corresponding to a logo class and a set of candidate logo images; and
one or more server devices that cause the system to:
train a machine-learning-logo classifier based on the set of synthetic training images;
generate logo scores for the set of candidate logo images utilizing the machine-learning-logo classifier, wherein a logo score indicates a likelihood that a candidate logo image portrays a logo corresponding to the logo class;
select a subset of logo images from the set of candidate logo images based on the logo scores;
replace a subset of synthetic training images from the set of synthetic training images with the subset of logo images to generate a set of mixed training images;
retrain the machine-learning-logo classifier based on the set of mixed training images; and
determine, from the set of candidate logo images, a set of logo images comprising ground-truth logos corresponding to the logo class utilizing the retrained machine-learning-logo classifier.

10. The system of claim 9 , wherein the one or more server devices further cause the system to, based on the logo scores for the set of candidate logo images, provide, for display within a curation user interface, candidate logos from a subset of candidate logo images from the set of candidate logo images and logo-rejection options corresponding to the subset of candidate logo images.

11. The system of claim 9 , wherein the one or more server devices further cause the system to, after retraining the machine-learning-logo classifier based on the set of mixed training images:
generate additional logo scores for an additional set of candidate logo images utilizing the machine-learning-logo classifier;
select an additional subset of logo images from the additional set of candidate logo images based on the additional logo scores;
replace an additional subset of synthetic training images from the set of synthetic training images with the additional subset of logo images to generate an additional set of mixed training images;
retrain the machine-learning-logo classifier based on the additional set of mixed training images.

12. The system of claim 9 , wherein the one or more server devices further cause the system to:
select an additional set of logo images comprising additional ground-truth logos corresponding to an additional logo class utilizing an additional machine-learning-logo classifier; and
train a multi-logo-classification model based on the set of logo images and the additional set of logo images.

13. The system of claim 9 , wherein the one or more server devices further cause the system to apply the machine-learning-logo classifier to the set of candidate logo images to generate boundary identifiers indicating positions of candidate logos within the set of candidate logo images.

14. The system of claim 9 , wherein the one or more server devices further cause the system to:
identify candidate images from among image-search results generated by a search query corresponding to the logo class; and
remove duplicate images from the candidate images based on average-pixel values to generat

Assignees

• Adobe Inc

Inventors

• Swaminathan Viswanathan
• Mitra Saayan
• Guo Han

Classifications

• G06V10/82

  using neural networks · CPC title

• G06V10/454

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06F18/217

  Validation; Performance evaluation; Active pattern learning techniques · CPC title

• G06N3/08 Primary

  Learning methods · CPC title