Training an Image Classifier

Patent metadata

Field	Value
Publication number	US-2018189597-A1
Application number	US-201615395328-A
Country	US
Kind code	A1
Filing date	Dec 30, 2016
Priority date	Dec 30, 2016
Publication date	Jul 5, 2018
Grant date	—

Abstract

In one embodiment, a method includes accessing for each of a plurality of input images a feature vector corresponding to the input image and metadata indicating a relationship of the input image to a predetermined concept; training with machine learning an image classifier associated with the predetermined concept based on the feature vectors of the input images and the metadata indicating their respective relationships to the predetermined concept; accessing for each of a plurality of evaluation images a feature vector that corresponds to the evaluation image; for each evaluation image calculating with the image classifier as trained a score indicating how closely related the evaluation image is to the predetermined concept, based on the feature vector corresponding to the evaluation image; and providing for display to a user one or more of the evaluation images and their respective scores as calculated by the image classifier.

First claim

What is claimed is:

1 . A method comprising:
by one or more computing devices, accessing for each of a plurality of input images:
a feature vector that corresponds to the input image and maps it to a point in a d-dimensional vector space; and
metadata indicating a relationship of the input image to a predetermined concept;
by one or more computing devices, training with machine learning an image classifier associated with the predetermined concept based on the feature vectors of the input images and the metadata indicating their respective relationships to the predetermined concept; by one or more computing devices, accessing for each of a plurality of evaluation images a feature vector that corresponds to the evaluation image and maps it to a point in the d-dimensional vector space; by one or more computing devices, for each evaluation image calculating with the image classifier as trained a score indicating how closely related the evaluation image is to the predetermined concept, based on the feature vector corresponding to the evaluation image; and by one or more computing devices, providing for display to a user one or more of the evaluation images and their respective scores as calculated by the image classifier.

2 . The method of claim 1 , wherein, for each input image, the metadata comprises a tag indicating whether the image is associated with the predetermined concept.

3 . The method of claim 1 , wherein:
the predetermined concept is associated with inappropriateness; and for each input image, the metadata comprises a tag indicating whether the input image is inappropriate.

4 . The method of claim 1 , wherein, for each input image, the metadata comprises a tag added by a user of an online social networking system.

5 . The method of claim 1 , wherein the plurality of evaluation images are accessed from an online social networking system.

6 . The method of claim 1 furthering comprising, by one or more computing devices, providing a display metric associated with the image classifier for display to the user.

7 . The method of claim 1 , further comprising:
by one or more computing devices, determining whether the image classifier is similar to a different image classifier based on a comparison of the scores calculated with the image classifier as trained for the each image of the plurality of evaluation images and scores calculated with the different image classifier for each image of the plurality of evaluation images, respectively; and by one or more computing devices, providing for display to the user an indication that the image classifier is similar to the different image classifier.

8 . The method of claim 1 , further comprising, by one or more computing devices, providing for display to a user one or more of the input images and their respective scores as calculated by the image classifier.

9 . One or more computer-readable non-transitory storage media embodying software that is operable when executed to:
access for each of a plurality of input images:
a feature vector that corresponds to the input image and maps it to a point in a d-dimensional vector space; and
metadata indicating a relationship of the input image to a predetermined concept;
train with machine learning an image classifier associated with the predetermined concept based on the feature vectors of the input images and the metadata indicating their respective relationships to the predetermined concept; access for each of a plurality of evaluation images a feature vector that corresponds to the evaluation image and maps it to a point in the d-dimensional vector space; for each evaluation image, calculate with the image classifier as trained a score indicating how closely related the evaluation image is to the predetermined concept, based on the feature vector corresponding to the evaluation image; and provide for display to a user one or more of the evaluation images and their respective scores as calculated by the image classifier.

10 . The media of claim 9 , wherein, for each input image, the metadata comprises a tag indicating whether the image is associated with the predetermined concept.

11 . The media of claim 9 , wherein:
the predetermined concept is associated with inappropriateness; and for each input image, the metadata comprises a tag indicating whether the input image is inappropriate.

12 . The media of claim 9 , wherein, for each input image, the metadata comprises a tag added by a user of an online social networking system.

13 . The media of claim 9 , wherein the plurality of evaluation images are accessed from an online social networking system.

14 . The media of claim 9 , wherein the software is further operable when executed to provide a display metric associated with the image classifier for display to the user.

15 . The media of claim 9 , wherein the software is further operable when executed to:
determine whether the image classifier is similar to a different image classifier based on a comparison of the scores calculated with the image classifier as trained for the each image of the plurality of evaluation images and scores calculated with the different image classifier for each image of the plurality of evaluation images, respectively; and provide for display to the user an indication that the image classifier is similar to the different image classifier.

16 . The media of claim 9 , wherein the software is further operable when executed to provide for display to a user one or more of the input images and their respective scores as calculated by the image classifier.

17 . A system comprising:
one or more processors; and a memory coupled to the processors and comprising instructions operable when executed by the processors to cause the processors to:
access for each of a plurality of input images:
a feature vector that corresponds to the input image and maps it to a point in a d-dimensional vector space; and
metadata indicating a relationship of the input image to a predetermined concept;

train with machine learning an image classifier associated with the predetermined concept based on the feature vectors of the input images and the metadata indicating their respective relationships to the predetermined concept;
access for each of a plurality of evaluation images a feature vector that corresponds to the evaluation image and maps it to a point in the d-dimensional vector space; for each evaluation image, calculate with the image classifier as trained a score indicating how closely related the evaluation image is to the predetermined concept, based on the feature vector corresponding to the evaluation image; and provide for display to a user one or more of the evaluation images and their respective scores as calculated by the image classifier.

18 . The system of claim 17 , wherein, for each input image, the metadata comprises a tag indicating whether the image is associated with the predetermined concept.

19 . The system of claim 17 , wherein:
the predetermined concept is associated with inappropriateness; and for each input image, the metadata comprises a tag indicating whether the input image is inappropriate.

20 . The system of claim 17 , wherein, for each input image, the metadata comprises a tag added by a user of an online social networking system.

Assignees

• Facebook Inc

Inventors

• Johri Nikhil
• Chen Kevin Brian
• Garcia Dario Garcia
• Paluri Balmanohar

Classifications

• G06V10/7784

  based on feedback from supervisors · CPC title

• G06V10/945

  User interactive design; Environments; Toolboxes · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• H04L67/535

  Tracking the activity of the user (network monitoring arrangements H04L43/00; recording of computer activity G06F11/34) · CPC title

• G06Q10/40 Primary

  Business processes related to social networking or social networking services · CPC title