Processing cell images using neural networks

Patent metadata

Field	Value
Publication number	US-10692001-B2
Application number	US-201815979104-A
Country	US
Kind code	B2
Filing date	May 14, 2018
Priority date	Feb 26, 2016
Publication date	Jun 23, 2020
Grant date	Jun 23, 2020

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing cell images using neural networks. One of the methods includes obtaining data comprising an input image of one or more biological cells illuminated with an optical microscopy technique; processing the data using a stained cell neural network; and processing the one or more stained cell images using a cell characteristic neural network, wherein the cell characteristic neural network has been configured through training to receive the one or more stained cell images and to process the one or more stained cell images to generate a cell characteristic output that characterizes features of the biological cells that are stained in the one or more stained cell images.

First claim

The invention claimed is:

1. A method for generating an output image of one or more biological cells that includes a prediction of one or more fluorescent channels, the method comprising:
obtaining input data comprising one or more images of one or more biological cells, wherein each image is an image of the one or more biological cells illuminated with a respective microscopy technique;
processing the input data comprising the one or more images using a convolutional neural network, wherein the convolutional neural network has been configured through training to receive the input data comprising the one or more images of the one or more biological cells and to process respective pixels of the one or more images of the input data to perform fully convolutional image-to-image regression from the input data to predict one or more fluorescent channels of an output image of the one or more biological cells, wherein each pixel in each of the one or more fluorescent channels generated by the convolutional neural network corresponds to a respective pixel of the one or more images; and
providing the output image with the predicted one or more fluorescent channels.

2. The method of claim 1 , wherein the respective microscopy techniques include one or more of an optical microscopy technique, a hyperspectral microscopy technique, or a fluorescent microscopy technique.

3. The method of claim 1 , wherein different images in the one or more images have different focal points.

4. The method of claim 1 , further comprising:
processing the output image using a cell characteristic neural network, wherein the cell characteristic neural network has been configured through training to generate a cell characteristic output that characterizes features of the biological cells.

5. The method of claim 4 , wherein the cell characteristic output comprises data characterizing one or more of a cell type, a cell health, a cell population, a cell phenotype, or a cell state of differentiation in induced pluripotent stem cell protocols of the one or more biological cells.

6. The method of claim 4 , wherein the cell characteristic output specifies an average cell health or a percentage of cells that have correctly differentiated through induced pluripotent stem cell protocols.

7. The method of claim 4 , wherein the cell characteristic output specifies a cell condition for each biological cell of the one or more biological cells.

8. A system comprising one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to perform operations comprising:
obtaining input data comprising one or more images of one or more biological cells, wherein each image is an image of the one or more biological cells illuminated with a respective microscopy technique;
processing the input data comprising the one or more images using a convolutional neural network, wherein the convolutional neural network has been configured through training to receive the input data comprising the one or more images of the one or more biological cells and to process respective pixels of the one or more images of the input data to perform fully convolutional image-to-image regression from the input data to predict one or more fluorescent channels of an output image of the one or more biological cells, wherein each pixel in each of the one or more fluorescent channels generated by the convolutional neural network corresponds to a respective pixel of the one or more images; and
providing the output image with the predicted one or more fluorescent channels.

9. The system of claim 8 , wherein the respective microscopy techniques include one or more of an optical microscopy technique, a hyperspectral microscopy technique, or a fluorescent microscopy technique.

10. The system of claim 8 , wherein different images in the one or more images have different focal points.

11. The system of claim 8 , wherein the operations further comprising:
processing the output image using a cell characteristic neural network, wherein the cell characteristic neural network has been configured through training to generate a cell characteristic output that characterizes features of the biological cells.

12. The system of claim 11 , wherein the cell characteristic output comprises data characterizing one or more of a cell type, a cell health, a cell population, a cell phenotype, or a cell state of differentiation in induced pluripotent stem cell protocols of the one or more biological cells.

13. The system of claim 11 , wherein the cell characteristic output specifies an average cell health or a percentage of cells that have correctly differentiated through induced pluripotent stem cell protocols.

14. The system of claim 11 , wherein the cell characteristic output specifies a cell condition for each biological cell of the one or more biological cells.

15. One or more non-transitory storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:
obtaining input data comprising one or more images of one or more biological cells, wherein each image is an image of the one or more biological cells illuminated with a respective microscopy technique;
processing the input data comprising the one or more images using a convolutional neural network, wherein the convolutional neural network has been configured through training to receive the input data comprising the one or more images of the one or more biological cells and to process respective pixels of the one or more images of the input data to perform fully convolutional image-to-image regression from the input data to predict one or more fluorescent channels of an output image of the one or more biological cells, wherein each pixel in each of the one or more fluorescent channels generated by the convolutional neural network corresponds to a respective pixel of the one or more images; and
providing the output image with the predicted one or more fluorescent channels.

16. The non-transitory storage media of claim 15 , wherein the respective microscopy techniques include one or more of an optical microscopy technique, a hyperspectral microscopy technique, or a fluorescent microscopy technique.

17. The non-transitory storage media of claim 15 , wherein different images in the one or more images have different focal points.

18. The non-transitory storage media of claim 15 , wherein the operations further comprise:
processing the output image using a cell characteristic neural network, wherein the cell characteristic neural network has been configured through training to generate a cell characteristic output that characterizes features of the biological cells.

19. The non-transitory storage media of claim 18 , wherein the cell characteristic output comprises data characterizing one or more of a cell type, a cell health, a cell population, a cell phenotype, or a cell state of differentiation in induced pluripotent stem cell protocols of the one or more biological cells.

20. The non-transitory storage media of claim 18 , wherein the cell characteristic output specifies an average cell health or a percentage of cells that have correctly differentiated through induced pluripotent stem cell protocols.

Assignees

• Google Llc

Inventors

• Nelson Philip Charles
• Christiansen Eric Martin
• Berndl Marc
• Frumkin Michael

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06T7/0014 Primary

  using an image reference approach · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06V20/69 Primary

  Microscopic objects, e.g. biological cells or cellular parts · CPC title