Systems and methods for deep learning microscopy

Patent metadata

Field	Value
Publication number	US-12190478-B2
Application number	US-202117530471-A
Country	US
Kind code	B2
Filing date	Nov 19, 2021
Priority date	Apr 26, 2018
Publication date	Jan 7, 2025
Grant date	Jan 7, 2025

Abstract

A microscopy method includes a trained deep neural network that is executed by software using one or more processors of a computing device, the trained deep neural network trained with a training set of images comprising co-registered pairs of high-resolution microscopy images or image patches of a sample and their corresponding low-resolution microscopy images or image patches of the same sample. A microscopy input image of a sample to be imaged is input to the trained deep neural network which rapidly outputs an output image of the sample, the output image having improved one or more of spatial resolution, depth-of-field, signal-to-noise ratio, and/or image contrast.

First claim

What is claimed is:

1. A system for outputting microscopy images of a test sample comprising a computing device having image processing software executed thereon, the image processing software comprising a trained deep neural network that is executed using one or more processors of the computing device, wherein the trained deep neural network is trained with a training set of non-fluorescence images comprising co-registered pairs of high-resolution microscopy images or image patches of a training sample and their corresponding low-resolution microscopy images or image patches of the same training sample, the image processing software configured to receive a non-fluorescence microscopy input image of the test sample comprising one of a bright-field microscopy image, a holographic microscopy image, a confocal microscopy image, a dark-field microscopy image, or an optical coherence tomography (OCT) image and output an output image of the test sample having improved one or more of spatial resolution, depth-of-field, signal-to-noise ratio, and/or image contrast.

2. The system of claim 1 , further comprising a microscope configured to generate the non-fluorescence microscopy input image of the test sample.

3. The system of claim 2 , wherein the microscope comprises a confocal microscope.

4. The system of claim 1 , wherein the trained deep neural network comprises a trained convolutional neural network (CNN) that is a GAN-trained network.

5. The system of claim 1 , wherein the computing device comprises a personal computer, laptop, server, mobile computing device, one or more graphics processing units (GPUs), or application specific integrated circuit (ASIC).

6. The system of claim 1 , wherein the trained deep neural network outputs the output image of the test sample within one second of inputting the non-fluorescence microscopy input image to the trained deep neural network.

7. The system of claim 1 , further comprising a monitor or display.

8. The system of claim 1 , wherein the non-fluorescence microscopy input image of the test sample comprises a histopathological slide image.

9. The system of claim 1 , wherein the training sample comprises tissue of a first type and wherein the test sample also comprises a tissue sample of the first type.

10. The system of claim 1 , wherein the training sample comprises tissue of a first type and wherein the test sample comprises a tissue sample of the second type different from the first type.

11. The system of claim 1 , wherein the training sample comprises tissue stained with a stain or dye of a first type and wherein the test sample also comprises a tissue sample stained with a stain or dye of a first type.

12. The system of claim 1 , wherein the training sample comprises tissue stained with a stain or dye of a first type and wherein the test sample comprises a tissue sample stained with a stain or dye of a second type different from the first type.

13. The system of claim 1 , wherein the high-resolution microscopy images or image patches of a training sample are obtained by synthesizing a higher resolution image from multiple, sub-pixel shifted low-resolution images.

Assignees

• Univ California

Inventors

• Ozcan Aydogan
• Rivenson Yair
• Wang Hongda
• Gunaydin Harun
• De Haan Kevin

Classifications

• G06N3/0475

  Generative networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/094

  Adversarial learning · CPC title

• G06N3/096

  Transfer learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title