Method and system for digital staining of label-free fluorescence images using deep learning

Patent metadata

Field	Value
Publication number	US-12327362-B2
Application number	US-202318543168-A
Country	US
Kind code	B2
Filing date	Dec 18, 2023
Priority date	Mar 30, 2018
Publication date	Jun 10, 2025
Grant date	Jun 10, 2025

Abstract

A deep learning-based digital staining method and system are disclosed that enables the creation of digitally/virtually-stained microscopic images from label or stain-free samples based on autofluorescence images acquired using a fluorescent microscope. The system and method have particular applicability for the creation of digitally/virtually-stained whole slide images (WSIs) of unlabeled/unstained tissue samples that are analyzes by a histopathologist. The methods bypass the standard histochemical staining process, saving time and cost. This method is based on deep learning, and uses, in one embodiment, a convolutional neural network trained using a generative adversarial network model to transform fluorescence images of an unlabeled sample into an image that is equivalent to the brightfield image of the chemically stained-version of the same sample. This label-free digital staining method eliminates cumbersome and costly histochemical staining procedures and significantly simplifies tissue preparation in pathology and histology fields.

First claim

What is claimed is:

1. A method of generating a digitally stained microscopic image comprising:
providing a neural network using one or more processors of a computing device, wherein the neural network is trained with a plurality of chemically stained images or image patches matched with corresponding label-free images or image patches of training samples;
obtaining one or more label-free images of a label-free test sample using a fluorescence microscope and one or more excitation light sources, wherein the detected light is emitted from at least one endogenous fluorophore or at least one endogenous emitter of the label-free test sample, and wherein each of the one or more label-free images comprises a single image per each of one or more channels;
inputting the one or more label-free images of the label-free test sample to the neural network; and
outputting the digitally stained microscopic image of the label-free test sample via the neural network.

2. The method of claim 1 , wherein the neural network comprises a plurality of neural networks.

3. The method of claim 1 , wherein the neural network is trained using a Generative Adversarial Network (GAN) model.

4. The method of claim 1 , wherein the deep neural network is trained using a generator network configured to learn statistical transformation between the matched label-free and chemically stained images or image patches of the same training sample and a network configured to discriminate between a ground truth chemically stained image of the training sample and the outputted digitally stained microscopic image of the training sample.

5. The method of claim 1 , wherein the label-free test sample comprises animal tissue, plant tissue, cells, pathogens, or biological fluid smears.

6. The method of claim 1 , wherein the neural network outputs a digitally stained microscopic image in less than one second of inputting the one or more label-free image(s).

7. The method of claim 1 , wherein the label-free test sample comprises a non-fixed tissue sample.

8. The method of claim 1 , wherein the label-free test sample comprises a fixed tissue sample.

9. The method of claim 8 , wherein the fixed tissue sample is embedded in paraffin.

10. The method of claim 1 , wherein the label-free test sample comprises a frozen tissue sample.

11. The method of claim 1 , wherein the label-free test sample comprises a fresh tissue sample.

12. The method of claim 1 , wherein the excitation light source emits ultra-violet or near ultra-violet light.

13. The method of claim 1 , wherein the one or more label-free images are obtained using one or more spectral filters of a filter set.

14. The method of claim 13 , wherein a plurality of spectral filters is used to capture a plurality of label-free images.

15. The method of claim 14 , wherein the plurality of images is obtained by multiple excitation light sources emitting light at different wavelengths or wavelength bands.

16. The method of claim 1 , wherein the one or more label-free images are subject to one or more image pre-processing operations prior to being input to the neural network.

17. The method of claim 16 , wherein the one or more image pre-processing operations comprises contrast enhancement, contrast reversal, image filtering, or combinations thereof.

18. The method of claim 1 , wherein the neural network is trained using one or more GPUs or ASICs.

19. The method of claim 1 , wherein the neural network is executed using one or more GPUs or ASICs.

20. The method of claim 1 , wherein obtaining the one or more label-free images of the label-free test sample comprises obtaining at least two fluorescence images of the label-free test sample.

21. The method of claim 20 , wherein the at least two fluorescence images are obtained using different wavelengths.

22. The method of claim 20 , wherein the at least two fluorescence images are obtained using different resolutions.

23. The method of claim 1 , wherein the neural network comprises a convolutional neural network.

24. The method of claim 1 , wherein detected light is emitted from one or both of the at least one endogenous fluorophore and the at least one endogenous emitter of frequency shifted light of the label-free test sample.

Assignees

• Univ California

Inventors

• Ozcan Aydogan
• Rivenson Yair
• Wang Hongda
• Wei Zhensong

Classifications

• G06N3/0475

  Generative networks · CPC title

• G06N3/094

  Adversarial learning · CPC title

• G06N3/096

  Transfer learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title