Method and system for digital staining of label-free phase images using deep learning

Patent metadata

Field	Value
Publication number	US-12106552-B2
Application number	US-201917261542-A
Country	US
Kind code	B2
Filing date	Mar 29, 2019
Priority date	Jul 19, 2018
Publication date	Oct 1, 2024
Grant date	Oct 1, 2024

Abstract

A deep learning-based digital staining method and system are disclosed that provides a label-free approach to create a virtually-stained microscopic images from quantitative phase images (QPI) of label-free samples. The methods bypass the standard histochemical staining process, saving time and cost. This method is based on deep learning, and uses a convolutional neural network trained using a generative adversarial network model to transform QPI images of an unlabeled sample into an image that is equivalent to the brightfield image of the chemically stained-version of the same sample. This label-free digital staining method eliminates cumbersome and costly histochemical staining procedures, and would significantly simplify tissue preparation in pathology and histology fields.

First claim

What is claimed is:

1. A microscopy method for label-free samples comprising:
providing a trained, deep neural network that is executed by image processing software using one or more processors, wherein the trained, deep neural network is trained with a plurality of matched chemically stained images or image patches and their corresponding quantitative phase microscopy images or image patches of the same sample with quantitative phase information of the sample;
inputting a quantitative phase microscopy image of the label-free sample to the deep neural network trained to process quantitative phase information; and
outputting a digitally stained brightfield output image of the sample that is substantially equivalent to a corresponding brightfield image of the same sample that has been chemically stained.

2. The microscopy method of claim 1 , wherein the trained, deep neural network is trained with one or more ground truth images of chemically stained samples along with a training set of label-free sample images or image patches of the same sample, which are used to set or fine tune parameters for the deep neural network.

3. The microscopy method of any of claim 2 , wherein the ground truth images of the chemically stained sample comprise human, animal or plant tissue samples stained with a preselected stain or a mixture of stains.

4. The microscopy method of claim 1 , wherein the trained, deep neural network is trained using a plurality of training images or image patches of samples of the same type as the label-free sample being imaged.

5. The microscopy method of claim 1 , wherein the trained, deep neural network is trained using a plurality of training images or image patches of samples of a different type compared to the label-free samples being imaged.

6. The microscopy method of claim 1 , wherein the quantitative phase microscopy image comprises a holographic microscopy image.

7. The microscopy method of claim 1 , wherein the quantitative phase microscopy image comprises a lens-free microscopy image.

8. The microscopy method of claim 1 , wherein the quantitative phase microscopy image comprises a lens-based microscopy image.

9. The microscopy method of claim 1 , wherein the quantitative phase microscopy image comprises a computational microscopy image.

10. The microscopy method of claim 2 , wherein the ground truth images of chemically stained sample and the training set of label-free sample images are registered to one another using global registration for rotational or aberration correction followed by local registration for distortion correction.

11. The microscope method of claim 10 , wherein local registration comprises using an elastic image registration algorithm.

12. The microscopy method of claim 1 , wherein the trained, deep neural network comprises a convolutional neural network (CNN).

13. The microscopy method of claim 12 , wherein the CNN is a GAN-trained model or deep neural network.

14. The microscopy method of claim 1 , wherein the sample comprises tissue, cells, pathogens, biological fluid smears, or other micro-objects of interest.

15. A system for outputting digitally stained images of a sample obtained by a quantitative phase microscope device, the system comprising:
a computing device having image processing software executed thereon or thereby, the image processing software comprising a trained, deep neural network that is executed using one or more processors of the computing device, wherein the trained deep neural network is trained with one or more ground truth images or image patches of chemically stained samples along with a training set of label-free images or image patches obtained of the same samples with quantitative phase information obtained from the quantitative phase microscope device that are used to set or fine tune parameters for the deep neural network, the trained, deep neural network configured to receive a quantitative phase microscopy input image of the label-free sample and output a digitally stained brightfield output image of the sample that is substantially equivalent to a corresponding brightfield image of the same sample that has been chemically stained.

16. The system of claim 15 , wherein the quantitative phase microscope device comprises a holographic microscope device, a lens-free microscope device, a lens-based microscope device, or a computational microscope.

17. The system of claim 15 , wherein the computing device comprises a personal computer, mobile computer, laptop, server, or one or more graphics processing units (GPUs).

Assignees

• Univ California

Inventors

• Ozcan Aydogan
• Rivenson Yair
• Wei Zhensong

Classifications

• G06N3/0475

  Generative networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/094

  Adversarial learning · CPC title

• G06V20/698

  Matching; Classification · CPC title