Processing cell images using neural networks

Patent metadata

Field	Value
Publication number	US-9971966-B2
Application number	US-201615055446-A
Country	US
Kind code	B2
Filing date	Feb 26, 2016
Priority date	Feb 26, 2016
Publication date	May 15, 2018
Grant date	May 15, 2018

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing cell images using neural networks. One of the methods includes obtaining data comprising an input image of one or more biological cells illuminated with an optical microscopy technique; processing the data using a stained cell neural network; and processing the one or more stained cell images using a cell characteristic neural network, wherein the cell characteristic neural network has been configured through training to receive the one or more stained cell images and to process the one or more stained cell images to generate a cell characteristic output that characterizes features of the biological cells that are stained in the one or more stained cell images.

First claim

What is claimed is:

1. A method comprising:
obtaining input data comprising a plurality of images of one or more biological cells, wherein each image is an image of the one or more biological cells illuminated with a respective optical microscopy technique of a plurality of different optical microscopy techniques, wherein the plurality of different optical microscopy techniques include at least one of differential interference contrast microscopy or foveated imaging;
processing the input data using a stained cell neural network, wherein the stained cell neural network has been configured through training to receive the input data comprising the plurality of images of the one or more biological cells and to process the input data to perform fully convolutional image-to-image regression from the input data to one or more stained cell images, wherein each stained cell image depicts a respective stained version of the one or more biological cells in the input data; and
processing the one or more stained cell images generated by the stained cell neural network using a cell characteristic neural network, wherein the cell characteristic neural network has been configured through training to receive the one or more stained cell images and to process the one or more stained cell images to generate a cell characteristic output that characterizes features of the biological cells that are stained in the one or more stained cell images.

2. The method of claim 1 , wherein the plurality of different optical microscopy techniques further include: red light microscopy or ultraviolet light microscopy.

3. The method of claim 1 , wherein the stained cell image has a fluorescent channel.

4. The method of claim 1 , wherein each pixel in each of the one or more stained cell images corresponds to a pixel in the input images of the one or more biological cells.

5. The method of claim 1 , wherein the cell characteristic output comprises data characterizing one or more of a cell type, a cell health, a cell population, a cell phenotype, or a cell state of differentiation in induced pluripotent stem cell protocols of the one or more biological cells that are stained in the one or more stained cell images.

6. The method of claim 1 , wherein the cell characteristic output specifies an average cell health or a percentage of cells that have correctly differentiated through induced pluripotent stem cell protocols.

7. The method of claim 1 , wherein the cell characteristic output specifies a cell condition for each biological cell of the one or more biological cells.

8. The method of claim 1 , wherein different images in the plurality of images have different focal points.

9. A system comprising one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to perform operations comprising:
obtaining input data comprising a plurality of images of one or more biological cells, wherein each image is an image of the one or more biological cells illuminated with a respective optical microscopy technique of a plurality of different optical microscopy techniques, wherein the plurality of different optical microscopy techniques include at least one of differential interference contrast microscopy or foveated imaging;
processing the input data using a stained cell neural network, wherein the stained cell neural network has been configured through training to receive the input data comprising the plurality of images of the one or more biological cells and to process the input data to perform fully convolutional image-to-image regression from the input data to one or more stained cell images, wherein each stained cell image depicts a respective stained version of the one or more biological cells in the input data; and
processing the one or more stained cell images generated by the stained cell neural network using a cell characteristic neural network, wherein the cell characteristic neural network has been configured through training to receive the one or more stained cell images and to process the one or more stained cell images to generate a cell characteristic output that characterizes features of the biological cells that are stained in the one or more stained cell images.

10. The system of claim 9 , wherein the plurality of different optical microscopy techniques further include: red light microscopy or ultraviolet light microscopy.

11. The system of claim 9 , wherein the stained cell image has a fluorescent channel.

12. The system of claim 9 , wherein each pixel in each of the one or more stained cell images corresponds to a pixel in the input images of the one or more biological cells.

13. The system of claim 9 , wherein the cell characteristic output comprises data characterizing one or more of a cell type, a cell health, a cell population, a cell phenotype, or a cell state of differentiation in induced pluripotent stem cell protocols of the one or more biological cells that are stained in the one or more stained cell images.

14. The system of claim 9 , wherein the cell characteristic output specifies an average cell health or a percentage of cells that have correctly differentiated through induced pluripotent stem cell protocols.

15. The system of claim 9 , wherein the cell characteristic output specifies a cell condition for each biological cell of the one or more biological cells.

16. The system of claim 9 , wherein different images in the plurality of images have different focal points.

17. One or more non-transitory storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:
obtaining input data comprising a plurality of images of one or more biological cells, wherein each image is an image of the one or more biological cells illuminated with a respective optical microscopy technique of a plurality of different optical microscopy techniques, wherein the plurality of different optical microscopy techniques include at least one of differential interference contrast microscopy or foveated imaging;
processing the input data using a stained cell neural network, wherein the stained cell neural network has been configured through training to receive the input data comprising the plurality of images of the one or more biological cells and to process the input data to perform fully convolutional image-to-image regression from the input data to one or more stained cell images, wherein each stained cell image depicts a respective stained version of the one or more biological cells in the input data; and
processing the one or more stained cell images generated by the stained cell neural network using a cell characteristic neural network, wherein the cell characteristic neural network has been configured through training to receive the one or more stained cell images and to process the one or more stained cell images to generate a cell characteristic output that characterizes features of the biological cells that are stained in the one or more stained cell images.

18. The non-transitory storage media of claim 17 , wherein the cell characteristic output comprises data characterizing one or more of a cell type, a cell health, a cell population, a cell phenotype, or a cell state of differentiation in induced pluripotent stem cell protocols of the one or more biological cells that are stained in the one or more stained cell images.

Assignees

• Google Llc

Inventors

• Nelson Philip Charles
• Christiansen Eric Martin
• Berndl Marc
• Frumkin Michael

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06T7/60

  Analysis of geometric attributes · CPC title

• G06N3/02

  Neural networks · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T7/0014 Primary

  using an image reference approach · CPC title