Systems and methods for image recognition normalization and calibration

Patent metadata

Field	Value
Publication number	US-9946926-B2
Application number	US-201715643928-A
Country	US
Kind code	B2
Filing date	Jul 7, 2017
Priority date	Dec 29, 2015
Publication date	Apr 17, 2018
Grant date	Apr 17, 2018

Abstract

Systems, methods, and non-transitory computer-readable media can calculate raw scores for a plurality of media items based on a classifier model and a target concept. The plurality of media items are ranked based on the raw scores. A review set of the plurality of media items is determined, the review set comprising a subset of the plurality of media items. Each of the media items of the review set is associated with a content depiction determination. A normalized score formula is calculated based on the raw scores and the content depiction determinations for the media items of the review set.

First claim

What is claimed is:

1. A computer-implemented method comprising:
calculating, by a computing system, raw scores for a plurality of media items based on a classifier model and a target concept;
associating, by the computing system, each media item of the plurality of media items with a content depiction determination indicative of whether the media item depicts the target concept; and
calculating, by the computing system, normalized scores for the plurality of media items based on the raw scores and the content depiction determinations, wherein,
for each media item of the plurality of media items, the normalized score is associated with the target concept and is indicative of a likelihood that the media item depicts the target concept.

2. The computer-implemented method of claim 1 , wherein the calculating normalized scores for the plurality of media items comprises calculating a normalized score formula based on the raw scores and the content depiction determinations.

3. The computer-implemented method of claim 2 , wherein the calculating the normalized score formula comprises calculating a logistic regression formula based on the raw scores and the content depiction determinations.

4. The computer-implemented method of claim 2 , further comprising re-training the classifier model based on the normalized score formula.

5. The computer-implemented method of claim 4 , further comprising repeating the computer-implemented method using the re-trained classifier model.

6. The computer-implemented method of claim 2 , wherein the normalized score formula is configured to convert a raw score calculated by the classifier model into a normalized score.

7. The computer-implemented method of claim 2 , wherein
the plurality of media items defines a review subset of a larger set of media items, and
the method further comprises calculating a normalized score for a first media item based on the normalized score formula, wherein the first media item is contained within the larger set of media items, but not within the review subset.

8. The computer-implemented method of claim 7 , wherein the review subset comprises a fixed number of media items.

9. The computer-implemented method of claim 7 , wherein the review subset is selected from the larger set based on a sampling rate.

10. The computer-implemented method of claim 1 , further comprising presenting a user interface configured to receive content depiction determinations for the plurality of media items.

11. A system comprising:
at least one processor; and
a memory storing instructions that, when executed by the at east one processor, cause the system to perform a method comprising:
calculating raw scores for a plurality of media items based on a classifier model and a target concept;
associating each media item of the plurality of media items with a content depiction determination indicative of whether the media item depicts the target concept; and
calculating normalized scores for the plurality of media items based on the raw scores and the content depiction determinations, wherein,
for each media item of the plurality of media items, the normalized score is associated with the target concept and is indicative of a likelihood that the media item depicts the target concept.

12. The system of claim 11 , wherein the calculating normalized scores for the plurality of media items comprises calculating a normalized score formula based on the raw scores and the content depiction determinations.

13. The system of claim 12 , wherein the calculating the normalized score formula comprises calculating a logistic regression formula based on the raw scores and the content depiction determinations.

14. The system of claim 12 , wherein the method further comprises re-training the classifier model based on the normalized score formula.

15. The system of claim 12 , wherein the normalized score formula is configured to convert a raw score calculated by the classifier model into a normalized score.

16. A non-transitory computer-readable storage medium including instructions that, when executed by at least one processor of a computing system, cause the computing system to perform a method comprising:
calculating raw scores for a plurality of media items based on a classifier model and a target concept;
associating each media item of the plurality of media items with a content depiction determination indicative of whether the media item depicts the target concept; and
calculating normalized scores for the plurality of media items based on the raw scores and the content depiction determinations, wherein,
for each media item of the plurality of media items, the normalized score is associated with the target concept and is indicative of a likelihood that the media item depicts the target concept.

17. The non-transitory computer-readable storage medium of claim 16 , wherein the calculating normalized scores for the plurality of media items comprises calculating a normalized score formula based on the raw scores and the content depiction determinations.

18. The non-transitory computer-readable storage medium of claim 17 , wherein the calculating the normalized score formula comprises calculating a logistic regression formula based on the raw scores and the content depiction determinations.

19. The non-transitory computer-readable storage medium of claim 17 , wherein the method further comprises re-training the classifier model based on the normalized score formula.

20. The non-transitory computer-readable storage medium of claim 17 , wherein the normalized score formula is configured to convert a raw score calculated by the classifier model into a normalized score.

Assignees

• Facebook Inc

Inventors

• Johri Nikhil
• Paluri Balamanohar
• Bourdev Lubomir

Classifications

• G06V20/70

  Labelling scene content, e.g. deriving syntactic or semantic representations · CPC title

• G06V10/945

  User interactive design; Environments; Toolboxes · CPC title

• G06F18/40

  Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor · CPC title

• G06V20/10 Primary

  Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) · CPC title

• G06K9/00536 Primary

  Physics · mapped topic