Focus-Weighted, Machine Learning Disease Classifier Error Prediction for Microscope Slide Images

Patent metadata

Field	Value
Publication number	US-2019340468-A1
Application number	US-201815972929-A
Country	US
Kind code	A1
Filing date	May 7, 2018
Priority date	May 7, 2018
Publication date	Nov 7, 2019
Grant date	—

Abstract

A method is described for generating a prediction of a disease classification error for a magnified, digital microscope slide image of a tissue sample. The image is composed of a multitude of patches or tiles of pixel image data. An out-of-focus degree per patch is computed using a machine learning out-of-focus classifier. Data representing expected disease classifier error statistics of a machine learning disease classifier for a plurality of out-of-focus degrees is retrieved. A mapping of the expected disease classifier error statistics to each of the patches of the digital microscope slide image based on the computed out-of-focus degree per patch is computed, thereby generating a disease classifier error prediction for each of the patches. The disease classifier error predictions thus generated are aggregated over all of the patches.

First claim

We claim:

1 . A method for generating a prediction of a disease classification error for a magnified, digital microscope slide image of a tissue sample, comprising the steps of:
(a) scanning with a slide scanner a microscope slide containing the tissue sample and generating the digital microscope slide image, the digital microscope slide image composed of a multitude of patches of pixel image data; (b) computing an out-of-focus degree per patch for the digital microscope slide image; (c) retrieving data representing expected disease classifier error statistics of a machine learning disease classifier for a plurality of out-of-focus degrees; (d) computing a mapping of the expected disease classifier error statistics to each of the patches of the digital microscope slide image based on the computed out-of-focus degree per patch computed in step (b) and thereby generating a disease classifier error prediction for each of the patches; (e) aggregating the disease classifier error predictions generated in step (d) over all of the patches.

2 . The method of claim 1 , wherein the tissue sample comprises a prostate tissue sample.

3 . The method of claim 1 , wherein the tissue sample comprises a lymph node sample.

4 . The method of claim 1 , wherein step b) is performed by a deep convolutional neural network trained to classify patches of tissue images by degree of out-of-focus.

5 . The method of claim 1 , further comprising step (f): generating a focus-weighted disease classifier error prediction for the digital microscope slide image as a whole.

6 . The method of claim 1 , wherein steps b)-e) are performed locally by computing resources in the slide scanner.

7 . A pathology system comprising, in combination:
a) a slide scanner adapted to generate a digital slide image of a microscope slide; b) a memory storing
1) parameters for a deep convolutional neural network trained to compute an out-of-focus degree per patch for a digital microscope slide image generated by the slide scanner;
2) data representing expected disease classifier error statistics of a machine learning disease classifier for a plurality of out-of-focus degrees;
c) a computer configured for computing (1) out-of-focus degree per patch for the digital microscope slide image using the deep convolutional neural network, (2) a mapping of the expected disease classifier error statistics to each of the patches of the digital microscope slide image based on the computed out-of-focus degree per patch and thereby generating a disease classifier error prediction for each of the patches; and (3) an aggregation of the disease classifier error predictions over all of the patches.

8 . The system of claim 7 , wherein the microscope slide contains a prostate tissue sample.

9 . The system of claim 7 , wherein the tissue sample comprises a lymph node sample.

10 . The system of claim 7 , wherein the memory and computer are local to the slide scanner.

11 . The system of claim 7 , wherein the memory and computer are remote to the slide scanner.

12 . A method for generating a prediction of a disease classification error for a magnified, digital microscope slide image of a tissue sample, the digital microscope slide image composed of a multitude of patches of pixel image data, comprising the steps of:
(a) computing an out-of-focus degree per patch for the digital microscope slide image; (b) retrieving data representing expected disease classifier error statistics of a machine learning disease classifier for a plurality of out-of-focus degrees; (c) computing a mapping of the expected disease classifier error statistics to each of the patches of the digital microscope slide image based on the computed out-of-focus degree per patch computed in step (a) and thereby generating a disease classifier error prediction for each of the patches; and (d) aggregating the disease classifier error predictions generated in step (c) over all of the patches.

13 . The method of claim 12 , wherein the tissue sample comprises a prostate tissue sample.

14 . The method of claim 12 , wherein the tissue sample comprises a lymph node sample.

15 . The method of claim 12 , wherein step (a) is performed by a deep convolutional neural network trained to classify patches of tissue images by degree of out-of-focus.

16 . The method of claim 13 , further comprising step (e): generating a focus-weighted disease classifier error prediction for the digital microscope slide image as a whole.

17 . A method for characterizing a disease classifier configured to generate a classification label for digital microscope slide of a tissue sample or portion thereof, comprising the steps of:
a) acquiring a set of slide images, each composed of patches of pixel image data, which are in focus and which are associated with ground truth labels for each image patch; b) defining a set of out-of-focus degrees, and for each degree:
1) applying a corresponding amount of synthetic out-of-focus to each of the patches of an image in the set of slides;
2) computing a disease classification error for each patch in the image;
3) computing a mean error across all of the patches in the image;
c) storing the mean error computed in step b) 3) for all of the degrees defined in step b) as an expected error for the disease classifier for the out-of-focus degrees defined in step b), d) repeating steps b1), b2), b3), and c) for each of the slide images in the set.

18 . The method of claim 17 , further comprising the step of repeating steps b), c) and d) at different magnifications for the digital microscope slide images in the set.

19 . The method of claim 17 , further comprising the step of repeating steps a), b), c) and d) at least once for each of a plurality of different slide scanners of different manufacturers.

20 . The method of claim 17 , wherein the tissue sample comprises a prostate tissue sample.

21 . The method of claim 17 , wherein tissue sample comprises a lymph node sample.

22 . The method of claim 17 , wherein the synthetic out-of-focus applied at step b) 1) is applied using a computational Bokeh filter.

23 . The method of claim 17 , wherein the expected error is presented by 1.0 minus the area under a receiver operating characteristic curve (AUC) for the disease classifier.

24 . The method of claim 17 wherein the mean error stored in step c) is stored in a table format of degrees of out-of-focus and associated expected disease classifier errors.

Assignees

• Google Llc

Inventors

• Stumpe Martin
• Kohlberger Timo

Classifications

• G06T7/0002 Primary

  Inspection of images, e.g. flaw detection · CPC title

• G06F18/217

  Validation; Performance evaluation; Active pattern learning techniques · CPC title

• G06T2207/10056

  Microscopic image · CPC title

• G06T2207/30168

  Image quality inspection · CPC title

• G06T2207/20021

  Dividing image into blocks, subimages or windows · CPC title