Digital histopathology and microdissection

Patent metadata

Field	Value
Publication number	US-10607343-B2
Application number	US-201715791209-A
Country	US
Kind code	B2
Filing date	Oct 23, 2017
Priority date	Oct 21, 2016
Publication date	Mar 31, 2020
Grant date	Mar 31, 2020

Abstract

A computer implemented method of generating at least one shape of a region of interest in a digital image is provided. The method includes obtaining, by an image processing engine, access to a digital tissue image of a biological sample; tiling, by the image processing engine, the digital tissue image into a collection of image patches; identifying, by the image processing engine, a set of target tissue patches from the collection of image patches as a function of pixel content within the collection of image patches; assigning, by the image processing engine, each target tissue patch of the set of target tissue patches an initial class probability score indicating a probability that the target tissue patch falls within a class of interest, the initial class probability score generated by a trained classifier executed on each target tissue patch; generating, by the image processing engine, a first set of tissue region seed patches by identifying target tissue patches having initial class probability scores that satisfy a first seed region criteria, the first set of tissue region seed patches comprising a subset of the set of target tissue patches; generating, by the image processing engine, a second set of tissue region seed patches by identifying target tissue patches having initial class probability scores that satisfy a second seed region criteria, the second set of tissue region seed patches comprising a subset of the set of target tissue patches; calculating, by the image processing engine, a region of interest score for each patch in the second set of tissue region seed patches as a function of initial class probability scores of neighboring patches of the second set of tissue region seed patches and a distance to patches within the first set of issue region seed patches; and generating, by the image processing engine, one or more region of interest shapes by grouping neighboring patches based on their region of interest scores.

First claim

What is claimed is:

1. A computer implemented method of generating at least one shape of a region of interest in a digital image, the method comprising:
obtaining, by an image processing engine, access to a digital tissue image of a biological sample;
tiling, by the image processing engine, the digital tissue image into a collection of image patches;
identifying, by the image processing engine, a set of target tissue patches from the collection of image patches as a function of pixel content within the collection of image patches;
assigning, by the image processing engine, each target tissue patch of the set of target tissue patches an initial class probability score indicating a probability that the target tissue patch falls within a class of interest, the initial class probability score generated by a trained classifier executed on each target tissue patch;
generating, by the image processing engine, a first set of tissue region seed patches by identifying target tissue patches having initial class probability scores that satisfy a first seed region criteria, the first set of tissue region seed patches comprising a subset of the set of target tissue patches;
generating, by the image processing engine, a second set of tissue region seed patches by identifying target tissue patches having initial class probability scores that satisfy a second seed region criteria, the second set of tissue region seed patches comprising a subset of the set of target tissue patches;
calculating, by the image processing engine, a region of interest score for each patch in the second set of tissue region seed patches as a function of initial class probability scores of neighboring patches of the second set of tissue region seed patches and a distance to patches within the first set of issue region seed patches; and
generating, by the image processing engine, one or more region of interest shapes by grouping neighboring patches based on their region of interest scores.

2. The method of claim 1 , wherein the step of tiling the digital tissue image includes creating image patches that are of a uniform size and a uniform shape.

3. The method of claim 2 , wherein the image patches of uniform size and uniform shape include square patches of less than or equal to 1,000 pixels by 1,000 pixels.

4. The method of claim 3 , wherein the square patches are less than or equal to 400 pixels by 400 pixels.

5. The method of claim 4 , wherein the square patches are less than or equal to 256 pixels by 256 pixels.

6. The method of claim 1 , wherein the step of tiling the digital tissue image includes creating image patches that are of non-uniform size and shape.

7. The method of claim 1 , wherein the collection of image patches includes non-overlapping patches.

8. The method of claim 1 , wherein the step of identifying the set of target tissue patches includes filtering the collection of image patches based on color channels of pixels within the image patches.

9. The method of claim 8 , further comprising filtering image patches of the collection of image patches as a function of variance with respect to the color channels.

10. The method of claim 1 , wherein the trained classifier includes a trained neural network.

11. The method of claim 1 , wherein the trained classifier includes a trained implementation of at least one of the following types of classifiers: support vector machine, softmax, decision tree, random forest, k nearest neighbor, Linear and Quadratic Discriminant Analysis, Ridge Regression, Multilayer Perceptron (MLP), Hyper-pipes, Bayes net, k-means clustering and naïve bayes.

12. The method of claim 1 , further comprising causing a computing device to render the region of interest shapes on a display.

13. The method of claim 1 , wherein the region of interest shapes includes at least one tissue mask.

14. The method of claim 13 , wherein the at least one tissue mask comprises a microdissection mask.

15. The method of claim 1 , wherein the class of interest comprises at least one cancer class.

16. The method of claim 15 , wherein the at least one cancer class includes one of the following types of cancer: breast cancer, bladder cancer, brain cancer, lung cancer, pancreatic cancer, skin cancer, colorectal cancer, prostate cancer, stomach cancer, liver cancer, cervical cancer, esophageal cancer, leukemia, non-hodgkin lymphoma, kidney cancer, uterine cancer, bile duct cancer, bone cancer, ovarian cancer, gallbladder cancer, gastrointestinal cancer, oral cancer, throat cancer, ocular cancer, pelvic cancer, spinal cancer, testicular cancer, vaginal cancer, vulvar cancer, and thyroid cancer.

17. The method of claim 1 , wherein the class of interest comprises at least one of the following types of tissue: abnormal tissue, benign tissue, malignant tissue, bone tissue, skin tissue, nerve tissue, interstitial tissue, muscle tissue, connective tissue, scar tissue, lymphoid tissue, fat, epithelial tissue, nervous tissue, and blood vessels.

18. The method of claim 1 , wherein the digital tissue image comprises a slide image of a tissue sample slide.

19. The method of claim 18 , wherein the slide image comprises a digital histopathology image.

20. The method of claim 1 , further comprising obtaining access to a database of a priori trained classifiers.

21. The method of claim 20 , further comprising selecting the trained classifier from the a priori trained classifiers according to classifier selection criteria defined according to biological sample metadata bound to the digital tissue image.

22. The method of claim 21 , wherein the biological sample metadata includes digital information associated with at least one of the following: a tissue type, a tissue donor, a scanner, a stain, a staining technique, an identifier of a preparer, an image size, a sample identifier, a tracking identifier, a version number, a file type, an image date, a symptom, a diagnosis, an identifying information of treating physician, a medical history of the tissue donor, a demographic information of the tissue donor, a medical history of family of the tissue donor, and a species of the tissue donor.

23. The method of claim 1 , wherein calculating a region of interest score for each patch in the second set of tissue region seed patches as a function of neighboring patches includes calculating a conditional random field (CRF) among the neighboring patches.

24. The method of claim 1 , wherein calculating a region of interest score for each patch in the second set of tissue region seed patch as a function of neighboring patches includes evaluating nearest neighbors.

25. The method of claim 1 , wherein the region of interest shapes includes shapes composed of patches.

26. The method of claim 1 , wherein the region of interest shapes comprise shape sub-patch level features.

27. The method of claim 26 , wherein the image data comprises tissue image data and the region of interest shapes comprise sub-patch level shapes at the cell level.

28. The method of claim 26 , wherein the step of generating the region of interest shapes includes classifying cells by cell-type within patches in the second set of tissue region seed patches according to a cell-level trained neural network.

29. The method of claim 27 , wherein the step of generating the region of interest shapes includes delineating a boundary between cells via a

Assignees

• Nantomics Llc

Inventors

• Song Bing
• Chu Gregory

Classifications

• G06T2207/20156

  Automatic seed setting · CPC title

• G06T2207/10056

  Microscopic image · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T2207/20021

  Dividing image into blocks, subimages or windows · CPC title

• G06T7/187

  involving region growing; involving region merging; involving connected component labelling · CPC title