Processing cell images using neural networks

Patent metadata

Field	Value
Publication number	US-11915134-B2
Application number	US-202217943089-A
Country	US
Kind code	B2
Filing date	Sep 12, 2022
Priority date	Feb 26, 2016
Publication date	Feb 27, 2024
Grant date	Feb 27, 2024

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing cell images using neural networks. One of the methods includes obtaining data comprising an input image of one or more biological cells illuminated with an optical microscopy technique; processing the data using a stained cell neural network; and processing the one or more stained cell images using a cell characteristic neural network, wherein the cell characteristic neural network has been configured through training to receive the one or more stained cell images and to process the one or more stained cell images to generate a cell characteristic output that characterizes features of the biological cells that are stained in the one or more stained cell images.

First claim

What is claimed is:

1. A method comprising:
training a neural network comprising one or more parameter values and that is configured to receive input data comprising one or more images of one or more biological cells, and to process respective pixels of the one or more images to predict one or more fluorescent channels of an output image of the one or more biological cells,
wherein the training comprises:
obtaining training data comprising one or more images of one or more biological cells and one or more stained images depicting the one or more biological cells, each stained image depicting the one or more biological cells according to a respective fluorescent channel of one or more fluorescent channels;
processing the one or more images through the neural network to obtain one or more fluorescent channels of a predicted output image;
determining an error between the one or more stained images and the one or more fluorescent channels of the predicted output image, comprising comparing a respective error between each pixel of each fluorescent channel of the predicted output image, with each corresponding pixel in a respective stained image corresponding to the fluorescent channel; and
updating the one or more parameter values of the neural network according to the determined error.

2. The method of claim 1 , wherein the one or more images of the training data are each of the one or more biological cells illuminated with a respective microscopy technique of one or more microscopy techniques.

3. The method of claim 2 , wherein the one or more microscopy techniques include one or more of an optical microscopy technique, a hyperspectral microscopy technique, or a fluorescent microscopy technique.

4. The method of claim 2 , wherein the one or more images of the training data have different focal points.

5. The method of claim 1 , further comprising:
processing input data comprising processing one or more second images of one or more second biological cells to predict, through the neural network, one or more fluorescent channels of an output image of the one or more second biological cells.

6. The method of claim 5 , further comprising:
processing the output image of the one or more second biological cells using another neural network trained to generate a cell characteristic output that characterizes features of the one or more second biological cells in the output image.

7. The method of claim 6 , wherein the cell characteristic output comprises data characterizing one or more of a cell type, a cell health, a cell population, a cell phenotype, or a cell state of differentiation in induced pluripotent stem cell protocols of the one or more second biological cells.

8. The method of claim 6 , wherein the cell characteristic output specifies, from the one or more second biological cells, an average cell health or a percentage of cells that have correctly differentiated through induced pluripotent stem cell protocols.

9. The method of claim 6 , wherein the cell characteristic output specifies a cell condition for each biological cell of the one or more second biological cells.

10. A system comprising:
one or more storage devices for storing data representing a neural network, and
one or more processors communicatively coupled to the one or more storage devices and configured to perform operations associated with the neural network, the operations comprising:
obtaining input data comprising one or more images of one or more biological cells, wherein each image is an image of the one or more biological cells illuminated with a respective microscopy technique of one or more microscopy techniques;
processing the input data comprising the one or more images using the neural network, wherein the neural network has been configured through training
to receive the input data comprising the one or more images of the one or more biological cells, and
to process respective pixels of the one or more images of the input data to predict one or more fluorescent channels of an output image of the one or more biological cells,

wherein each pixel in each of the one or more fluorescent channels generated by the neural network corresponds to a respective pixel of the one or more images; and
providing the output image with the one or more fluorescent channels.

11. The system of claim 10 , wherein the system is operatively coupled to a microscopic camera, and wherein the operations further comprise generating the one or more images of the one or more biological cells using the microscopic camera.

12. The system of claim 11 , wherein generating the one or more images of the one or more biological cells comprises generating the one or more images to have different focal points.

13. The system of claim 10 , wherein the one or more microscopy techniques include one or more of an optical microscopy technique, a hyperspectral microscopy technique, or a fluorescent microscopy technique.

14. The system of claim 10 , wherein the system is operatively coupled to a display device, and wherein providing the output image comprises providing the output image for display on the display device.

15. The system of claim 10 , wherein the operations further comprise:
processing the output image using another neural network trained to generate a cell characteristic output that characterizes features of the one or more biological cells in the output image.

16. The system of claim 15 , wherein the cell characteristic output comprises data characterizing one or more of a cell type, a cell health, a cell population, a cell phenotype, or a cell state of differentiation in induced pluripotent stem cell protocols of the one or more biological cells.

17. The system of claim 15 , wherein the cell characteristic output specifies, from the one or more biological cells, an average cell health or a percentage of cells that have correctly differentiated through induced pluripotent stem cell protocols.

18. The system of claim 15 , wherein the cell characteristic output specifies a cell condition for each biological cell of the one or more biological cells.

19. A system comprising:
one or more one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to perform operations comprising:
training a neural network comprising one or more parameter values and that is configured to receive input data comprising one or more images of one or more biological cells, and to process respective pixels of the one or more images to predict one or more fluorescent channels of an output image of the one or more biological cells,
wherein the training comprises:
obtaining training data comprising one or more images of one or more biological cells and one or more stained images depicting the one or more biological cells, each stained image depicting the one or more biological cells according to a respective fluorescent channel of one or more fluorescent channels;
processing the one or more images through the neural network to obtain one or more fluorescent channels of a predicted output image;
determining an error between the one or more stained images and the one or more fluorescent channels of the predicted output image, comprising comparing a respective error between each pixel of each fluorescent channel of the predicted output image, with each corresponding pixel in a respective stained image corresponding to the fluorescent channel; and
updating the one or more parameter values of the neural network ac

Assignees

• Google Llc

Inventors

• Nelson Philip Charles
• Christiansen Eric Martin
• Berndl Marc
• Frumkin Michael

Classifications

• G06T7/0014 Primary

  using an image reference approach · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06V10/82

  using neural networks · CPC title

• G06N3/08 Primary

  Learning methods · CPC title