Prediction of recurrence of non-small cell lung cancer with tumor infiltrating lymphocyte (TIL) graphs

Patent metadata

Field	Value
Publication number	US-10078895-B2
Application number	US-201615389872-A
Country	US
Kind code	B2
Filing date	Dec 23, 2016
Priority date	Dec 30, 2015
Publication date	Sep 18, 2018
Grant date	Sep 18, 2018

Abstract

Methods and apparatus predict non-small cell lung cancer (NSCLC) recurrence using radiomic features extracted from digitized hematoxylin and eosin (H&E) stained slides of a region of tissue demonstrating NSCLC. One example apparatus includes an image acquisition circuit that acquires an image of a region of tissue demonstrating NSCLC, a segmentation circuit that segments a cellular nucleus from the image, a feature extraction circuit that extracts a set of features from the image, a tumor infiltrating lymphocyte (TIL) identification circuit that classifies the segmented nucleus as a TIL or non-TIL, a graphing circuit that constructs a TIL graph and computes a set of TIL graph statistical features, and a classification circuit that computes a probability that the region will experience NSCLC recurrence. The classification circuit may compute a quantitative continuous image-based risk score based on the probability or the image. A treatment plan may be provided based on the risk score.

First claim

What is claimed is:

1. A non-transitory computer-readable storage device storing computer executable instructions that when executed by a computer control the computer to perform a method for predicting recurrence of non-small cell lung cancer (NSCLC), the method comprising:
accessing an image of a region of tissue demonstrating NSCLC, where the region of tissue includes a cell;
defining a segmented cell by segmenting a cell represented in the image from the background of the image;
extracting a set of morphological features from the segmented cell, where the set of morphological features includes a shape feature or a color feature;
classifying the segmented cell as a tumor infiltrating lymphocyte (TIL) or as a non-TIL based, at least in part, on the set of morphological features;
constructing a graph of TIL spatial architecture represented in the image;
computing a set of TIL density statistical features based on the graph;
providing the set of TIL density statistical features to an automated classifier;
receiving, from the automated classifier, a probability that the region of tissue will experience NSCLC recurrence; and
controlling a computer aided diagnosis (CADx) system to classify the region of tissue as a non-recurrence region or a recurrence region based, at least in part, on the probability or the graph.

2. The non-transitory computer-readable storage device of claim 1 , where the image is a digital whole slide image (WSI) of a hematoxylin and eosin (H&E) stained slide of a region of tissue demonstrating NSCLC.

3. The non-transitory computer-readable storage device of claim 2 , where defining the segmented cell by segmenting the cell represented in the image from the background of the image includes segmenting the cell using a watershed approach.

4. The non-transitory computer-readable storage device of claim 2 , where defining the segmented cell by segmenting the cell represented in the image from the background of the image includes segmenting the cell using a pixel-level convolutional neural network approach, or a region growing approach.

5. The non-transitory computer-readable storage device of claim 1 , where the set of morphological features includes a texture feature, a shape feature, or an intensity feature.

6. The non-transitory computer-readable storage device of claim 5 , where the set of morphological features includes a median red channel value.

7. The non-transitory computer-readable storage device of claim 6 , where a node of the graph is a TIL, and where an edge of the graph is a distance between a first node and a second, different node.

8. The non-transitory computer-readable storage device of claim 7 , where the set of TIL density statistical features includes a mean TIL density or a maximum TIL density.

9. The non-transitory computer-readable storage device of claim 1 , where the automated classifier is an automated deep learning classifier.

10. The non-transitory computer-readable storage device of claim 9 , where the automated deep learning classifier is a linear discriminant analysis (LDA) classifier, a random forest classifier, or a support vector machine (SVM) classifier.

11. The non-transitory computer-readable storage device of claim 1 , where the automated classifier computes the probability that the region of tissue will experience NSCLC recurrence based, at least in part, on the set of TIL density statistical features.

12. The non-transitory computer-readable storage device of claim 1 , where classifying the region of tissue as a non-recurrence region or a recurrence region includes computing a quantitative continuous image-based risk score based, at least in part, on the probability or the graph.

13. The non-transitory computer-readable storage device of claim 1 , the method further comprising training the automated classifier.

14. The non-transitory computer-readable storage device of claim 13 , where training the automated classifier includes:
accessing a training set of digitized whole slide images of H&E stained slides of a region of tissue demonstrating NSCLC, where a first subset of the training set includes an image of a region of tissue that experienced NSCLC recurrence, and where a second subset of the training set includes an image of a region of tissue that did not experience NSCLC recurrence, where a member of the training set includes a set of morphological features;
segmenting a cell represented in a member of the training set;
classifying the cell as a TIL or a non-TIL based, at least in part, on the set of morphological features;
constructing a graph of TIL spatial architecture represented in the member of the training set;
computing a set of TIL statistical features based on the graph;
training the automated classifier with the set of TIL statistical features;
accessing a testing set of digitized whole slide images of H&E stained slides of a region of tissue demonstrating NSCLC, where a first subset of the testing set includes an image of a region of tissue that experienced NSCLC recurrence, and where a second subset of the testing set includes an image of a region of tissue that did not experience NSCLC recurrence; and
testing the automated classifier with the testing set.

15. An apparatus, comprising:
a processor;
a memory;
an input/output interface;
a set of circuits, where the set of circuits includes an image acquisition circuit, a segmentation circuit, a feature extraction circuit, a tumor infiltrating lymphocyte (TIL) identification circuit, a graphing circuit, and a classification circuit; and
an interface to connect the processor, the memory, the input/output interface and the set of circuits:
where the image acquisition circuit accesses a set of images of a region of tissue demonstrating non-small cell lung cancer (NSCLC), where a member of the set of images is a digitized whole slide image (WSI) of a hematoxylin and eosin (H&E) stained pathology slice of an NSCLC tumor, where the member of the set images includes a set of morphological features;
where the segmentation circuit detects a cellular nucleus in the member of the set of images, and where the segmentation circuit generates a segmented nucleus based on the detected cellular nucleus;
where the feature extraction circuit extracts a set of discriminative features from the segmented nucleus, and where the feature extraction circuit provides the set of discriminative features to the TIL identification circuit;
where the TIL identification circuit classifies the segmented nucleus as a TIL or non-TIL based, at least in part, on the set of discriminative features;
where the graphing circuit constructs a TIL architecture graph, based, at least in part, on the segmented nucleus identified as a TIL, where the graphing circuit computes a set of TIL architecture graph statistical features based on the TIL architecture graph; and
where the classification circuit computes a probability that the region of tissue will experience NSCLC recurrence using a quadratic discriminant analysis (QDA), where the probability is based, at least in part, on the set of TIL architecture graph statistical features, or on the TIL architecture graph.

16. The apparatus of claim 15 , where the segmentation circuit generates the segmented nucleus using a watershed technique, a pixel-level convolutional neural network, or a region growing approach.

17. The apparatus of claim 15 , where the set of morphological features includes a shape feature or a color feature, where the color feature is a median red channel value.

18. The apparatus of clai

Assignees

• Univ Case Western Reserve

Inventors

• Madabhushi Anant
• Wang Xiangxue
• Velcheti Vamsidhar
• Prada German Corredor

Classifications

• G06V30/19173

  Classification techniques · CPC title

• G06V10/82

  using neural networks · CPC title

• G06F18/2453

  non-linear, e.g. polynomial classifier · CPC title

• G06F18/2451

  linear, e.g. hyperplane · CPC title

• G06F18/2411

  based on the proximity to a decision surface, e.g. support vector machines · CPC title