Tissue object-based machine learning system for automated scoring of digital whole slides

Patent metadata

Field	Value
Publication number	US-10176579-B2
Application number	US-201414776571-A
Country	US
Kind code	B2
Filing date	Mar 12, 2014
Priority date	Mar 15, 2013
Publication date	Jan 8, 2019
Grant date	Jan 8, 2019

Abstract

A facility includes systems and methods for providing a learning-based image analysis approach for the automated detection, classification, and counting of objects (e.g., cell nuclei) within digitized pathology tissue slides. The facility trains an object classifier using a plurality of reference sample slides. Subsequently, and in response to receiving a scanned image of a slide containing tissue data, the facility separates the whole slide into a background region and a tissue region using image segmentation techniques. The facility identifies dominant color regions within the tissue data and identifies seed points within those regions using, for example, a radial symmetry based approach. Based at least in part on those seed points, the facility generates a tessellation, each distinct area in the tessellation corresponding to a distinct detected object. These objects are then classified using the previously-trained classifier. The facility uses the classified objects to score slides.

First claim

The invention claimed is:

1. A non-transitory computer-readable storage medium storing instructions that, if executed by a computing system having a processor, cause the computing system to perform operations comprising:
receiving a digitized image associated with a slide;
detecting seed points within the digitized image associated with the slide said detecting seed points within the slide comprising:
generating an image gradient for a first identified region, and
generating a voting response matrix for the identified region based at least on the generated image gradient, the voting response matrix comprising, for each pixel within the first identified region, a vote value;

based at least on the detected seed points, extracting objects from the digitized image associated with the slide; and
for each of the extracted objects,
identifying characteristics of the extracted object, and
using a classifier trained based at least on digitized images of tissue samples to classify the extracted object based at least on the identified characteristics of the extracted object.

2. The non-transitory computer-readable storage medium of claim 1 , said detecting seed points within the slide further comprising:
identifying local maxima within the generated voting response matrix.

3. The non-transitory computer-readable storage medium of claim 2 , the operations further comprising:
for each identified local maximum within the generated voting response matrix,
only in response to determining that the local maximum exceeds a threshold value, identifying a corresponding pixel of the first identified region as a seed point.

4. The non-transitory computer-readable storage medium of claim 1 , the operations further comprising:
identifying tissue within the received digitized image of data associated with the slide;
identifying dominant colors within the identified tissue;
for each identified dominant color,
associating pixels of identified tissue with the dominant color.

5. The non-transitory computer-readable storage medium of claim 1 , wherein the classifier is a multi-stage classifier.

6. The non-transitory computer-readable storage medium of claim 5 , wherein the multi-stage classifier classifies at least one extracted object as a positively-stained nuclear object or a negatively-stained nuclear object.

7. The non-transitory computer-readable storage medium of claim 1 , further comprising:
training a tissue-object classifier using a plurality of reference tissue samples prior to using the classifier to classify the extracted objects, wherein the training comprises seed detection, nuclei object extraction, and features computation based at least on a whole slide image context.

8. A method comprising:
receiving a digitized image associated with a slide;
detecting seed points within the digitized image associated with the slide, said detecting seed points within the slide comprising:
generating an image gradient for a first identified region, and
generating a voting response matrix for the identified region based at least on the generated image gradient;

based at least on the detected seed points, extracting objects from the digitized image associated with the slide; and
for each of the extracted objects,
identifying characteristics of the extracted object, and
using a classifier trained based at least on digitized images of tissue samples to classify the extracted object based at least on the identified characteristics of the extracted object.

9. The method of claim 8 , wherein said detecting seed points within the slide further comprises:
identifying local maxima within the generated voting response matrix.

10. The method of claim 9 , further comprising:
for each identified local maximum within the generated voting response matrix,
only in response to determining that the local maximum exceeds a threshold value, identifying a corresponding pixel of the first identified region as a seed point.

11. The method of claim 8 , further comprising:
identifying tissue within the received digitized image of data associated with the slide;
identifying dominant colors within the identified tissue;
for each identified dominant color,
associating pixels of identified tissue with the dominant color.

12. The method of claim 8 , wherein the classifier is a multi-stage classifier.

13. The method of claim 12 , wherein the multi-stage classifier is configured to classify at least one extracted object as a positively-stained nuclear object or a negatively-stained nuclear object.

14. The method of claim 8 , further comprising:
training a tissue-object classifier using a plurality of reference tissue samples prior to using the classifier to classify the extracted objects, wherein the training comprises seed detection, nuclei object extraction, and features computation based at least on a whole slide image context.

15. A computing system comprising:
one or more processors;
at least one memory configured to store instructions which, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
receiving a digitized image associated with a slide;
detecting seed points within the digitized image associated with the slide, said detecting seed points within the slide comprising:
generating an image gradient for a first identified region, and
generating a voting response matrix for the identified region based at least on the generated image gradient;

based at least on the detected seed points, extracting objects from the digitized image associated with the slide; and
for each of the extracted objects,
identifying characteristics of the extracted object, and
using a classifier trained based at least on digitized images of tissue samples to classify the extracted object based at least on the identified characteristics of the extracted object.

16. The computing system of claim 15 , wherein said detecting seed points within the slide further comprises:
identifying local maxima within the generated voting response matrix.

17. The computing system of claim 16 , wherein the operations further comprise:
for each identified local maximum within the generated voting response matrix,
only in response to determining that the local maximum exceeds a threshold value, identifying a corresponding pixel of the first identified region as a seed point.

18. The computing system of claim 15 , wherein the operations further comprise:
identifying tissue within the received digitized image of data associated with the slide;
identifying dominant colors within the identified tissue; and
for each identified dominant color, associating pixels of identified tissue with the dominant color.

19. The computing system of claim 15 , wherein the classifier is a multi-stage classifier.

20. The computing system of claim 19 , wherein the multi-stage classifier is configured to classify at least one extracted object as a positively-stained nuclear object or a negatively-stained nuclear object.

21. The computing system of claim 15 , wherein the operations further comprise:
training a tissue-object classifier using a plurality of reference tissue samples prior to using the classifier to classify the extracted objects, wherein the training comprises seed detection, nuclei object extraction, and features computation based at least on a whole slide image context.

Assignees

• Ventana Med Syst Inc

Inventors

• Chukka Srinivas
• Chivate Sujit Siddheshwar
• Patil Suhas Hanmantrao
• Sabata Bikash
• Sertel Olcay
• Sarkar Anindya

Classifications

• G06T2207/30024

  Cell structures in vitro; Tissue sections in vitro · CPC title

• G06T2207/30068

  Mammography; Breast · CPC title

• G06K9/0014

  Physics · mapped topic

• G06T7/0014 Primary

  using an image reference approach · CPC title

• G06T7/0012 Primary

  Biomedical image inspection · CPC title