Methods and Apparatus for Classification

Patent metadata

Field	Value
Publication number	US-2020285897-A1
Application number	US-202016882487-A
Country	US
Kind code	A1
Filing date	May 24, 2020
Priority date	Aug 18, 2017
Publication date	Sep 10, 2020
Grant date	—

Abstract

A human expert may initially label a white light image of teeth, and computer vision may initially label a filtered fluorescent image of the same teeth. Each label may indicate presence or absence of dental plaque at a pixel. The images may be registered. For each pixel of the registered images, a union label may be calculated, which is the union of the expert label and computer vision label. The union labels may be applied to the white light image. This process may be repeated to create a training set of union-labeled white light images. A classifier may be trained on this training set. Once trained, the classifier may classify a previously unseen white light image, by predicting union labels for that image. Alternatively, the items that are initially labeled may comprise images captured by two different imaging modalities, or may comprise different types of sensor measurements.

First claim

What is claimed:

1 . A method comprising:
(a) creating a set of union-labeled modality A images in such a way that each union-labeled image in the set is created by
(i) capturing a modality A image,
(ii) capturing a modality B image,
(iii) applying initial labels to the modality A image on a region-by-region basis,
(iv) applying initial labels to the modality B image on a region-by-region basis,
(v) performing registration of the modality A image and modality B image, and
(vi) after the registration, computing union labels for the modality A image in such a way that, for each specific region in a group of regions of the modality A image, a union label for the specific region is a union of (A) the initial label for the specific region and (B) the initial label for a corresponding region of the modality B image;
(b) training a classifier on at least the set of union-labeled modality A images; and (c) after the training, calculating, with the classifier, labels for a previously unseen modality A image, on a region-by-region basis.

2 . The method of claim 1 , wherein each union-labeled image in the set captures a different physical object or different portion of a physical object than that which is captured in each other union-labeled image in the set.

3 . The method of claim 1 , wherein, for each particular union-labeled modality A image in the set:
(a) the particular union-labeled modality A image is an image of tissue before the tissue has been stained, and (b) a modality B image, which corresponds to the union-labeled modality A image, is an image of the tissue after the tissue has been stained.

4 . The method of claim 1 , wherein, for each particular union-labeled modality A image in the set:
(a) the particular union-labeled modality A image is an image of tissue before the tissue has been stained by hematoxylin and eosin (H&E) stain, and (b) a modality B image, which corresponds to the union-labeled modality A image, is an image of the tissue after the tissue has been stained by H&E stain.

5 . The method of claim 1 , wherein each modality A image and each modality B image is a tomographic image.

6 . The method of claim 1 , wherein:
(a) each modality A image is a positron emission tomography image; and (b) each modality B image is a magnetic resonance imaging image.

7 . The method of claim 1 , wherein:
(a) each modality A image is a positron emission tomography image; and (b) each modality B image is an x-ray computed tomography image.

8 . The method of claim 1 , wherein each modality A image is a white light image and each modality B image is a fluorescent image.

9 . The method of claim 1 , wherein each modality A image and each modality B image is an image of all or a portion of one or more teeth.

10 . The method of claim 1 , wherein each initial label and each union label indicates either presence of, or absence of, dental plaque.

11 . The method of claim 1 , wherein each initial label and each union label indicates either presence of, or absence of, gingivitis.

12 . The method of claim 1 , wherein:
(a) each initial label of a modality A image is inputted by a human being; and (b) each initial label of a modality B image is generated by a computer vision algorithm.

13 . The method of claim 1 , wherein each region mentioned in claim 1 consists of only a single pixel.

14 . The method of claim 1 , wherein each region mentioned in claim 1 consists of multiple pixels.

15 . A system comprising:
(a) a modality A imaging device; (b) a modality B imaging device; and (c) one or more computers;
wherein
 (i) the one or more computers are programmed to acquire a set of union-labeled modality A images, in such a way that for each specific union-labeled image in the set, the one or more computers are programmed
(A) to output an instruction for the modality A imaging device to capture a modality A image,
(B) to output an instruction for the modality B imaging device to capture a modality B image,
(C) to apply initial labels to the modality A image on region-by-region basis,
(D) to apply initial labels to the modality B image on a region-by-region basis,
(E) to perform registration of the modality A image and modality B image, and
(F) to compute, after the registration, union labels for the modality A image in such a way that, for each specific region in a group of regions of the modality A image, a union label for the specific region is a union of (I) the initial label for the specific region and (II) the initial label for a corresponding region of the modality B image, and

 (ii) the one or more computers are also programmed
(A) to train a classifier on the set of union-labeled modality A images; and
(B) after the training, to calculate, with the classifier, labels for a previously unseen modality A image, on a region-by-region basis.

16 . The system of claim 15 , wherein:
(a) each modality A image is a white light image and each modality B image is a fluorescent image; and (b) each initial label and union label indicates either presence of, or absence of, dental plaque.

17 . The system of claim 15 , wherein each union-labeled image in the set captures a different physical object or different region of a physical object than that which is captured in each other union-labeled image in the set.

18 . The system of claim 15 , wherein, for each particular union-labeled modality A image in the set:
(a) the particular union-labeled modality A image is an image of tissue before the tissue has been stained, and (b) a modality B image, which corresponds to the union-labeled modality A image, is an image of the tissue after the tissue has been stained.

19 . The method of claim 15 , wherein the one or more computers are also programmed:
(a) to accept input from one or more human beings, which input specifies the initial labels for the modality A images; (b) to generate, by a computer vision algorithm, the initial labels for the modality B images.

19 . The system of claim 15 wherein the modality A imaging device and the modality B imaging device are each a tomographic imaging device.

20 . The system of claim 15 , wherein:
(a) each modality A image is a white light image and each modality B image is a fluorescent image; and (b) each initial label and union label indicates either presence of, or absence of, gingivitis.

Assignees

• Massachusetts Inst Technology

Inventors

• Shah Pratik
• Yauney Gregory

Classifications

• G06V10/7784

  based on feedback from supervisors · CPC title

• G06V10/764 Primary

  using classification, e.g. of video objects · CPC title

• G06V10/774

  Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06F18/2178

  based on feedback of a supervisor · CPC title