Systems and methods for automated cell segmentation and labeling in immunofluorescence microscopy

Patent metadata

Field	Value
Publication number	US-11538261-B2
Application number	US-201916711917-A
Country	US
Kind code	B2
Filing date	Dec 12, 2019
Priority date	Dec 13, 2018
Publication date	Dec 27, 2022
Grant date	Dec 27, 2022

Abstract

Various techniques are provided for performing automated full-cell segmentation and labeling in immunofluorescent microscopy. These techniques perform membrane segmentation and nuclear seed detection separate and independently from each other, then combine their results to identify cell boundaries. Some embodiments use texture- and kernel-based image processing to perform the method. In some embodiments, the method for obtaining membrane features disclosed herein can be used in conjunction with or separate from the nuclear features. The results can be used for a variety of purposes, including whole-area cell segmentation in fluorescence-based tissue imaging.

First claim

What is claimed is:

1. A method comprising:
staining a plurality of cells with a membrane stain to generate a membrane-stained sample;
generating a first fluorescence image of at least a portion of the membrane-stained sample;
converting the first fluorescence image into binary, thereby generating a binary image;
detecting cell boundaries of the plurality of cells in the binary image and generating one or more membrane labels based on the cell boundaries, thereby generating a membrane-segmented label image;
staining the plurality of cells with a nuclear stain to generate a nuclear-stained sample;
generating a second fluorescence image of at least a portion of the nuclear-stained sample;
performing nuclear seed detection on the second fluorescence image by locating nuclei of the plurality of cells using the nuclear stain and generating one or more nuclear labels based on the nuclear seed detection;
segmenting the plurality of cells in at least one of the first fluorescence image or the second fluorescence image based on the membrane labels and the nuclear labels;
responsive to identifying one or more incorrect membrane labels, correcting the membrane labels;
obtaining one or more undersegmented membrane labels and determining one or more additional membrane labels using the undersegmented labels and detected peaks in the second fluorescence image;
merging the corrected membrane labels and the one or more additional membrane labels;
labeling, using the merged membrane labels, the plurality of cells in at least one of the first fluorescence image or the second fluorescence image to generate a labeled fluorescence image; and
displaying the labeled fluorescence image.

2. The method of claim 1 , wherein the membrane stain exhibits a different color than the nuclear stain.

3. The method of claim 1 , wherein the plurality of cells are comprised in a tissue sample.

4. The method of claim 1 , wherein detecting the cell boundaries further comprises performing hysteresis thresholding and skeletonization.

5. The method of claim 1 , wherein detecting the cell boundaries further comprises performing connected component labeling.

6. The method of claim 1 , wherein performing nuclear seed detection further comprises computing a Laplacian of the nuclear stain in the second fluorescence image, wherein peaks of the Laplacian indicate nuclear seeds.

7. The method of claim 1 , wherein performing nuclear seed detection further comprises detecting nuclear seeds by applying a trained machine-learning technique.

8. The method of claim 1 , wherein labeling the plurality of cells further comprises performing masked watershed segmentation.

9. A computer-program product tangibly embodied in a non-transitory machine-readable storage medium, including instructions configured to cause one or more data processors to:
access a first fluorescence image of at a least a portion of membrane-stained sample, wherein the membrane-stained sample includes a biological sample that comprises a plurality of cells and that is stained with a membrane stain;
convert the first fluorescence image into binary, thereby generating a binary image;
detect cell boundaries of the plurality of cells in the binary image and generate one or more membrane labels based on the cell boundaries, thereby generating a membrane-segmented label image;
access a second fluorescence image of at a least a portion of nuclear-stained sample, wherein the nuclear-stained sample includes a biological sample that comprises a plurality of cells and that is stained with a nuclear stain;
perform nuclear seed detection on the second fluorescence image by locating nuclei of the plurality of cells using the nuclear stain and generate one or more nuclear labels based on the nuclear seed detection;
segment the plurality of cells in at least one of the first fluorescence image or the second fluorescence image based on the membrane labels and the nuclear labels;
responsive to an identification of one or more incorrect membrane labels, correct the membrane labels;
obtain one or more undersegmented membrane labels and determine one or more additional membrane labels using the undersegmented labels and detected peaks in the second fluorescence image;
merge the corrected membrane labels and the one or more additional membrane labels;
label, using the merged membrane labels, the plurality of cells in at least one of the first fluorescence image or the second fluorescence image to generate a labeled fluorescence image; and
display the labeled fluorescence image.

10. The computer-program product of claim 9 , wherein the membrane stain exhibits a different color than the nuclear stain.

11. The computer-program product of claim 9 , wherein the plurality of cells are comprised in a tissue sample.

12. The computer-program product of claim 9 , further including instructions configured to cause one or more data processors to perform hysteresis thresholding and skeletonization.

13. The computer-program product of claim 9 , further including instructions configured to cause one or more data processors to perform connected component labeling.

14. The computer-program product of claim 9 , further including instructions configured to cause one or more data processors to compute a Laplacian of the nuclear stain in the second fluorescence image, wherein peaks of the Laplacian indicate nuclear seeds.

15. The computer-program product of claim 9 , further including instructions configured to cause one or more data processors to detect nuclear seeds by applying a trained machine-learning technique.

16. The computer-program product of claim 9 , further including instructions configured to cause one or more data processors to perform masked watershed segmentation.

17. A system comprising:
one or more data processors; and
a non-transitory computer readable storage medium containing instructions which, when executed on the one or more data processors, cause the one or more data processors to:
access a first fluorescence image of at a least a portion of membrane-stained sample, wherein the membrane-stained sample includes a biological sample that comprises a plurality of cells and that is stained with a membrane stain;
convert the first fluorescence image into binary, thereby generating a binary image;
detect cell boundaries of the plurality of cells in the binary image and generate one or more membrane labels based on the cell boundaries, thereby generating a membrane-segmented label image;
access a second fluorescence image of at a least a portion of nuclear-stained sample, wherein the nuclear-stained sample includes a biological sample that comprises a plurality of cells and that is stained with a nuclear stain;
perform nuclear seed detection on the second fluorescence image by locating nuclei of the plurality of cells using the nuclear stain and generate one or more nuclear labels based on the nuclear seed detection;
segment the plurality of cells in at least one of the first fluorescence image or the second fluorescence image based on the membrane labels and the nuclear labels;
responsive to an identification of one or more incorrect membrane labels, correct the membrane labels;
obtain one or more undersegmented membrane labels and determine one or more additional membrane labels using the undersegmented labels and detected peaks in the second fluorescence image;
merge the corrected membrane labels and the one or more additional membrane labels;
label, using the merged membrane labels, the plurality of cells in at least one of the first fluorescenc

Assignees

• Verily Life Sciences Llc

Inventors

• Behrooz Ali
• Santori Charles

Classifications

• G06T2207/10064

  Fluorescence image · CPC title

• G06T5/30

  Erosion or dilatation, e.g. thinning · CPC title

• G06T2207/30024

  Cell structures in vitro; Tissue sections in vitro · CPC title

• G06T2207/20152

  Watershed segmentation · CPC title

• G06T7/11 Primary

  Region-based segmentation · CPC title