Artificial intelligence segmentation of tissue images

Patent metadata

Field	Value
Publication number	US-10991097-B2
Application number	US-201916732242-A
Country	US
Kind code	B2
Filing date	Dec 31, 2019
Priority date	Dec 31, 2018
Publication date	Apr 27, 2021
Grant date	Apr 27, 2021

Abstract

Techniques for generating an overlay map on a digital medical image of a slide are provided, and include cell detection and tissue classification processes. Techniques include receiving a medical image, separating the image into tiles, and performing tile classifications and tissue classifications based on a multi-tile analysis. Techniques additionally include identifying cell objects in the image, separating the image into and displaying polygons identifying the cell objects and cell classifications. Generated displays may be overlays over the initial digital image.

First claim

What is claimed is:

1. A method for creating an overlay map on a digital image of a slide, the method comprising:
receiving the digital image;
separating the digital image into a plurality of tiles, each tile of the plurality of tiles containing a respective portion of the digital image of the slide; and
for each tile of the plurality of tiles:
identifying features of the tile;
identifying structural tissue features of a second portion of the digital image of the slide including at least part of one or more other tiles of the plurality of tiles, wherein the second portion is larger than the respective portion of the digital image contained in the tile; and
identifying the majority class of tissue visible within the tile based at least in part on the features of the tile and the structural tissue features of the second portion of the digital image of the slide.

2. The method of claim 1 , further comprising:
generating a digital overlay drawing of an outer edge of each cell in the image.

3. The method of claim 2 , wherein the digital overlay drawing is prepared at the resolution level of an individual pixel.

4. The method of claim 1 , wherein the majority class of tissue is selected from the group consisting of epithelium, normal epithelium, immune, stroma, necrosis, blood, and fat.

5. The method of claim 1 , wherein the structural tissue features comprises at least one of glands, ducts, vessels, and immune clusters.

6. A method for tissue classification of a digital image of a slide, the method comprising:
receiving the digital image;
segmenting the digital image into a plurality of tiles, each tile of the plurality of tiles containing a respective portion of the digital image of the slide;
for each tile of the plurality of tiles:
identifying features of the tile;
identifying structural tissue features of a second portion of the digital image of the slide including at least part of one or more other tiles of the plurality of tiles, wherein the second portion is larger than the respective portion of the digital image contained in the tile; and
determining a predicted class for each tile based at least in part on the features of the tile and the structural tissue features of the second portion of the digital image of the slide;

identifying a plurality of cell objects in the digital image;
determining a predicted class for each of the plurality of cell objects; and
for each of the plurality of tiles that corresponds to one of the plurality of cell objects, assigning the tile the predicted class of the corresponding cell object in place of the predicted class of the tile.

7. The method of claim 6 , further comprising:
storing, in a first file, for each tile, a tile position and the predicted class of the tile; and
storing, in a second file, for each cell object, a polygon outlining the cell object and the predicted class of the cell object.

8. The method of claim 6 , further comprising:
generating a digital overlay drawing for the digital image, where the digital overlay drawing is a cell mask displaying a polygon around each cell object.

9. The method of claim 6 , further comprising:
generating a digital overlay drawing for the digital image, where the digital overlay drawing is a histology mask displaying the plurality of tiles and the predict class for each tile.

10. The method of claim 6 , further comprising:
generating a digital overlay drawing for the digital image; and
displaying, in the digital overlay drawing, the plurality of tiles and the predicted class for each tile that does not correspond to a cell object, and displaying, the plurality of cell objects and the predicted class of each cell object.

11. The method of claim 6 , further comprising:
generating a digital overlay drawing for the digital image, wherein the digital overlay drawing includes the digital image; and
displaying the digital overlay drawing.

12. The method of claim 6 , further comprising:
generating a digital overlay drawing for the digital image, wherein the digital overlay drawing includes a generated version of the digital image; and
displaying the digital overlay drawing.

13. The method of claim 6 , further comprising:
generating a plurality of digital overlay drawings for the digital image, where each digital overlay drawing corresponds to a different predicted class; and
selectively displaying one of the plurality of digital overlay drawings.

14. The method of claim 6 , further comprising:
generating a digital overlay drawing for the digital image, where the digital overlay drawing comprises percentages of predict classes corresponding to the digital image.

15. The method of claim 6 , further comprising:
generating a digital overlay drawing for the digital image, where the digital overlay drawing comprises total counts of predict classes corresponding to the digital image.

16. The method of claim 6 , wherein the cell objects comprise CD3, CD8, CD20, pancytokeratin, and smooth muscle actin.

17. The method of claim 6 , wherein the cell objects include lymphocyte cells and not lymphocyte cells.

18. The method of claim 6 , wherein the structural tissue features comprises at least one of glands, ducts, vessels, and immune clusters.

Assignees

• Tempus Labs Inc

Inventors

• Yip Stephen
• Ho Irvin
• Sha Lingdao
• Osinski Boleslaw

Classifications

• G06V20/69

  Microscopic objects, e.g. biological cells or cellular parts · CPC title

• G06T2207/30024

  Cell structures in vitro; Tissue sections in vitro · CPC title

• G06T7/0012 Primary

  Biomedical image inspection · CPC title

• G06T7/11

  Region-based segmentation · CPC title

• G06T2207/30096

  Tumor; Lesion · CPC title