Clustering historical images using a convolutional neural net and labeled data bootstrapping

Patent metadata

Field	Value
Publication number	US-10943146-B2
Application number	US-201916397114-A
Country	US
Kind code	B2
Filing date	Apr 29, 2019
Priority date	Dec 28, 2016
Publication date	Mar 9, 2021
Grant date	Mar 9, 2021

Abstract

Systems and methods for classifying historical images. A feature extractor may create feature vectors corresponding to a plurality of images. A first classification of the plurality of images may be performed based on the plurality of feature vectors, which may include assigning a label to each of the plurality of images and assigning a probability for each of the assigned labels. The assigned probability for each of the assigned labels may be related to a statistical confidence that a particular assigned label is correctly assigned to a particular image. A subset of the plurality of images may be displayed to a display device. An input corresponding to replacement of an incorrect label with a corrected label for a certain image may be received from a user. A second classification of the plurality of images based on the input from the user may be performed.

First claim

What is claimed is:

1. A method for classifying a plurality of images comprising:
creating, by a feature extractor, a plurality of feature vectors corresponding to the plurality of images;
performing, by a feature classifier, a first classification of the plurality of images based on the plurality of feature vectors, wherein performing the first classification includes:
for each image of the plurality of images, assigning a label of a plurality of labels to the image as a whole based on a corresponding feature vector of the plurality of feature vectors such that the label corresponds to every pixel of the image; and
assigning a first probability for each of the assigned labels, wherein the assigned first probability for each of the assigned labels is related to a statistical confidence that a particular assigned label is correctly assigned to a particular image;

determining a subset of probabilities of the assigned first probabilities;
determining a subset of the plurality of images corresponding to the subset of probabilities;
receiving a corrected label for replacement of an incorrect label for a certain image of the subset of the plurality of images;
receiving a confidence level associated with the corrected label;
adjusting the feature classifier using the corrected label and the confidence level associated with the corrected label; and
performing, by the adjusted feature classifier, a second classification of the plurality of images based on the plurality of feature vectors, wherein performing the second classification includes:
assigning at least one of the plurality of labels to each of the plurality of images, including assigning the corrected label to the certain image; and
assigning a second probability for each of the assigned labels.

2. The method of claim 1 , wherein the feature extractor is a convolutional neural network (CNN), the CNN having been previously trained and the CNN being compatible with the plurality of images such that the plurality of images are receivable as inputs by the CNN.

3. The method of claim 1 , wherein the plurality of images are historical images.

4. The method of claim 1 , further comprising:
determining a second subset of probabilities of the assigned second probabilities;
determining a second subset of the plurality of images corresponding to the second subset of probabilities;
receiving a second corrected label for replacement of a second incorrect label for a second certain image of the second subset of the plurality of images.

5. The method of claim 1 , wherein each of the plurality of feature vectors comprise 4096 numbers.

6. The method of claim 1 , wherein the subset of probabilities of the assigned first probabilities includes one of the following:
all assigned first probabilities that are less than a probability threshold;
all assigned first probabilities that are between a lower probability threshold and an upper probability threshold;
one or more first probabilities that are below an average probability of the assigned first probabilities; and
one or more first probabilities that are below a median probability of the assigned first probabilities.

7. The method of claim 1 , further comprising:
receiving input for creation of a new label, wherein the new label is added to the plurality of labels.

Assignees

• Ancestry Com Operations Inc

Inventors

• Brown Laryn
• Murdock Michael
• Reese Jack
• Reid Shawn

Classifications

• G06V10/7788

  the supervisor being a human, e.g. interactive learning with a human teacher · CPC title

• G06V10/82 Primary

  using neural networks · CPC title

• G06F18/41

  Interactive pattern learning with a human teacher · CPC title

• G06K9/6254 Primary

  Physics · mapped topic