Systems and methods for image recognition normalization and calibration

Patent metadata

Field	Value
Publication number	US-9767357-B2
Application number	US-201514983385-A
Country	US
Kind code	B2
Filing date	Dec 29, 2015
Priority date	Dec 29, 2015
Publication date	Sep 19, 2017
Grant date	Sep 19, 2017

Abstract

Systems, methods, and non-transitory computer-readable media can calculate raw scores for a plurality of media items based on a classifier model and a target concept. The plurality of media items are ranked based on the raw scores. A review set of the plurality of media items is determined, the review set comprising a subset of the plurality of media items. Each of the media items of the review set is associated with a content depiction determination. A normalized score formula is calculated based on the raw scores and the content depiction determinations for the media items of the review set.

First claim

What is claimed is:

1. A computer-implemented method comprising:
calculating, by a computing system, raw scores for a plurality of media items based on a classifier model and a target concept;
ranking, by the computing system, the plurality of media items based on the raw scores;
determining, by the computing system, a review set of the plurality of media items, the review set comprising a subset of the plurality of media items;
associating, by the computing system, each of the media items of the review set with a content depiction determination, wherein the content depiction determination is indicative of whether a media item depicts the target concept; and
calculating, by the computing system, a normalized score formula based on the raw scores and the content depiction determinations for the media items of the review set.

2. The computer-implemented method of claim 1 , wherein the review set is determined based on a sampling rate.

3. The computer-implemented method of claim 2 , further comprising receiving a sampling rate selection from a user.

4. The computer-implemented method of claim 1 , wherein calculating the normalized score formula comprises calculating a logistic regression formula based on the raw scores and the content depiction determinations for the media items of the review set.

5. The computer-implemented method of claim 1 , further comprising presenting a user interface configured to receive content depiction determinations for the media items of the review set.

6. The computer-implemented method of claim 1 , further comprising re-training the classifier model based on the normalized score formula.

7. The computer-implemented method of claim 6 , further comprising repeating the computer-implemented method with the re-trained classifier model.

8. The computer-implemented method of claim 1 , wherein the normalized score formula is configured to convert a raw score calculated by the classifier model into a normalized score.

9. The computer-implemented method of claim 8 , wherein the normalized score is a probability value.

10. The computer-implemented method of claim 1 , wherein the review set comprises a fixed number of media items.

11. A system comprising:
at least one processor; and
a memory storing instructions that, when executed by the at least one processor, cause the system to perform a method comprising:
calculating raw scores for a plurality of media items based on a classifier model and a target concept;
ranking the plurality of media items based on the raw scores;
determining a review set of the plurality of media items, the review set comprising a subset of the plurality of media items;
associating each of the media items of the review set with a content depiction determination, wherein the content depiction determination is indicative of whether a media item depicts the target concept; and
calculating a normalized score formula based on the raw scores and the content depiction determinations for the media items of the review set.

12. The system of claim 11 , wherein the review set is determined based on a sampling rate.

13. The system of claim 12 , wherein the method further comprises receiving a sampling rate selection from a user.

14. The system of claim 11 , wherein calculating the normalized score formula comprises calculating a logistic regression formula based on the raw scores and the content depiction determinations for the media items of the review set.

15. The system of claim 11 , wherein the method further comprises presenting a user interface configured to receive content depiction determinations for the media items of the review set.

16. A non-transitory computer-readable storage medium including instructions that, when executed by at least one processor of a computing system, cause the computing system to perform a method comprising:
calculating raw scores for a plurality of media items based on a classifier model and a target concept;
ranking the plurality of media items based on the raw scores;
determining a review set of the plurality of media items, the review set comprising a subset of the plurality of media items;
associating each of the media items of the review set with a content depiction determination, wherein the content depiction determination is indicative of whether a media item depicts the target concept; and
calculating a normalized score formula based on the raw scores and the content depiction determinations for the media items of the review set.

17. The non-transitory computer-readable storage medium of claim 16 , wherein the review set is determined based on a sampling rate.

18. The non-transitory computer-readable storage medium of claim 17 , wherein the method further comprises receiving a sampling rate selection from a user.

19. The non-transitory computer-readable storage medium of claim 16 , wherein calculating the normalized score formula comprises calculating a logistic regression formula based on the raw scores and the content depiction determinations for the media items of the review set.

20. The non-transitory computer-readable storage medium of claim 16 , wherein the method further comprises presenting a user interface configured to receive content depiction determinations for the media items of the review set.

Assignees

• Facebook Inc
• Facebook Inc

Inventors

• Johri Nikhil
• Paluri Balamanohar
• Bourdev Lubomir

Classifications

• G06V20/70

  Labelling scene content, e.g. deriving syntactic or semantic representations · CPC title

• G06V10/945

  User interactive design; Environments; Toolboxes · CPC title

• G06F18/40

  Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor · CPC title

• G06V20/10 Primary

  Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) · CPC title

• G06K9/6293

  Physics · mapped topic