Classifying nuclei in histology images

Patent metadata

Field	Value
Publication number	US-10628658-B2
Application number	US-201715591936-A
Country	US
Kind code	B2
Filing date	May 10, 2017
Priority date	Nov 10, 2014
Publication date	Apr 21, 2020
Grant date	Apr 21, 2020

Abstract

Disclosed, among other things, is a computer device and computer-implemented method of classifying cells within an image of a tissue sample comprising providing the image of the tissue sample as input; computing nuclear feature metrics from features of nuclei within the image; computing contextual information metrics based on nuclei of interest with the image; classifying the cells within the image using a combination of the nuclear feature metrics and contextual information metrics.

First claim

The invention claimed is:

1. A computer system for classifying cells within an image of a tissue sample stained in an immunohistochemistry (IHC) assay, the computer system comprising:
one or more processors; and
at least one non-transitory computer-readable memory, the at least one memory storing instructions for execution by the one or more processors to cause the one or more processors to:
detect cell nuclei in the image of the tissue sample; and
for a cell nucleus of the cell nuclei:
determine a set of nuclear features based on portions of the image associated with the cell nucleus;
compute a set of nuclear feature metrics based on the set of nuclear features;
detect a set of other portions of the image associated with the cell nucleus, each other portion of the set of other portions including a region surrounding the cell nucleus;
identify a set of image texture features based on the set of other portions of the image, wherein at least one image texture feature of the set of image texture features is indicative of a presence of at least one biomarker in a corresponding portion of the set of other portions of the image;
compute a set of contextual information metrics based on the set of image texture features, wherein the set of contextual information metrics is indicative of a tissue classification corresponding to the region surrounding the cell nucleus; and
classify a cell associated with the cell nucleus based on the set of nuclear feature metrics and the set of contextual information metrics.

2. The computer system of claim 1 , wherein a nuclear feature of the set of nuclear features is selected from a group consisting of morphology features, appearance features, and background features.

3. The computing system of claim 1 , wherein the cells are classified as positive immune cells, positive tumor cells, negative immune cells, and/or negative tumor cells.

4. The computing system of claim 3 , wherein the biomarker compenses a Programmed death-Ligand 1 (PD-L1) biomarker.

5. The computer system of claim 4 , wherein the set of nuclear feature metrics is computed on a first image channel that represents a local overall staining intensity, a second image channel that represents an intensity of an IHC label indicative of a presence of a PD-L1 biomarker, and/or a third image channel that represents a local counterstain intensity indicative of a presence of a cell nucleus.

6. The computer system of claim 1 , wherein the set of contextual information metrics is further computed from data describing neighboring nuclei that surround the cell nucleus.

7. The computer system of claim 6 , wherein the set of image texture features are derived from an image patch surrounding the cell nucleus in the image of the tissue sample.

8. The computer system of claim 7 , wherein an image texture feature of the set of image texture features is selected from a group consisting of texton histogram features, Garbor features, Haralick features, histogram of intensity features, and histogram of gradient magnitude and gradient orientation features.

9. The computer system of claim 7 , wherein at least one of the set of image texture features is a texton histogram feature, wherein the texton histogram feature indicates a number of pixels contained in said image patch being respectively assigned to each of a plurality of textons.

10. The computer system of claim 9 , wherein the texton histogram feature is derived by (1) applying a bank of maximum response filters on the image of the tissue sample to obtain a list of filter response images, each filter response image comprising one or more filter responses; (2) clustering the filter responses from the filter response images into textons; (3) assigning each pixel in the image of the tissue sample into one of the textons; and (4) computing the texton histogram feature from all the pixels in the image patch surrounding the cell nucleus.

11. The computer system of claim 8 , wherein at least one of the set of image texture features is a histogram of intensity features, and wherein the histogram of intensity features is derived from image channels selected from the group consisting of a primary stain channel, a counterstain channel, an IHC stain channel, and a luminance channel.

12. The computer system of claim 11 , wherein differences in signals from the image channels are analyzed to compute intensity-based features within the image patch surrounding the cell nucleus, the intensity-based features comprising metric values derived from pixel intensity values within the image patch.

13. The computer system of claim 8 , wherein the set of image texture features includes Haralick features, wherein Haralick features are derived from a co-occurrence matrix based on an angular relationship between a pixel and its specified neighbor in the image patch.

14. The computer system of claim 6 , wherein the data describing neighboring nuclei is derived from a histogram of cluster assignment.

15. The computer system of claim 14 , wherein the histogram of cluster assignment is derived by (1) applying a K-means algorithm on nuclear feature vectors to obtain cluster centers; (2) assigning individual neighboring nuclei of the cell nucleus to a closest cluster center; and (3) computing the histogram of cluster assignment based on the assigning.

16. The computer system of claim 15 , further comprising the steps of (2a) measuring an Euclidean distance from a nuclear feature vector of each individual neighboring nucleus to a center of each cluster; and (2b) assigning an individual neighboring nucleus of the individual neighboring nuclei to a cluster whose center is closest to a nuclear feature vector of the nuclear feature vectors.

17. The computer system of claim 1 , wherein the cells of the image of the tissue sample are classified with a support vector machine.

18. The computer system of claim 1 , wherein the classification of the cells within the image of the tissue sample is performed by a trained classifier, wherein the instructions cause the one or more processors to generate the trained classifier by: automatically performing a computational training of a classifier on a plurality of training nuclei, a training nucleus being a nucleus identified in one of a plurality of training images, by: (a) generating a nuclear feature vector for each training nucleus of each training image by extracting one or more nuclear feature metrics from each training nucleus; (b) obtaining a plurality of C pre-trained clusters by performing a clustering procedure using a K-means algorithm on the nuclear feature vectors; (c) assigning each nucleus that neighbors the training nucleus in one of the training images to one of the plurality of C pre-trained clusters by: (c1) measuring an Euclidean distance from the nuclear feature vector of each individual neighboring nucleus to a center of each cluster; and (c2) assigning each neighbor nucleus of each nucleus of each training image to a cluster whose center has a smallest Euclidian distance to the nuclear feature vector representing the center of that nucleus; (d) determining, for each of the training nuclei in the training images, contextual feature metrics of said training nucleus by calculating a histogram of cluster assignments of all training nuclei being neighbor nuclei of said training nucleus; and (e) for each training nucleus in each of the training images, combining the one or more nuclear feature metrics of said training nucleus and the contextual feature metrics of said training nucleus into a single complete feature vector for

Assignees

• Ventana Med Syst Inc

Inventors

• Bredno Joerg
• Chefd'Hotel Christophe
• Nguyen Kien

Classifications

• G06T7/00

  Image analysis · CPC title

• G06K9/4604

  Physics · mapped topic

• G06K9/0014 Primary

  Physics · mapped topic

• G06K9/6223

  Physics · mapped topic

• G06K9/20

  Physics · mapped topic