Image embeddings via deep learning and adaptive batch normalization

Patent metadata

Field	Value
Publication number	US-12572849-B2
Application number	US-202217845508-A
Country	US
Kind code	B2
Filing date	Jun 21, 2022
Priority date	Jun 21, 2022
Publication date	Mar 10, 2026
Grant date	Mar 10, 2026

Abstract

One embodiment of the present invention sets forth a technique for training a machine learning model to generate image embeddings for images captured during multiple experiments. The technique includes inputting a batch of images into a plurality of layers in the machine learning model, wherein the batch of images has been sampled from a plurality of images generated via a first experiment. The technique also includes, for at least one layer included in the plurality of layers, computing a set of statistics associated with a plurality of outputs generated by the layer based on the batch of images and normalizing the plurality of outputs based on the statistics. The technique further includes updating a plurality of parameters for each of the plurality of layers based on a set of predictions generated by the first machine learning model from the batch of images and the normalized plurality of outputs.

First claim

What is claimed is:

1 . A computer-implemented method comprising:
training a machine learning model to generate a trained machine learning model by:
inputting a first batch of images into a first machine learning model that includes a plurality of layers having a set of parameters, wherein the first batch of images has been sampled from a first plurality of images generated via a first perturbation experiment comprising a first set of experimental conditions;
for at least one layer included in the plurality of layers:
computing, by a batch normalization layer of the plurality of layers, a first set of batch normalization statistics to account for the first set of experimental conditions associated with a first plurality of outputs generated by the at least one layer based on the first batch of images of the first perturbation experiment; and
normalizing the first plurality of outputs to account for a first batch effect in the first batch of images from the first perturbation experiment based on the first set of batch normalization statistics to generate a first plurality of batch normalized outputs;
generating an updated set of parameters by updating the set of parameters of the plurality of layers based on a first set of predictions generated by the first machine learning model from the first batch of images of the first perturbation experiment and from the first plurality of batch normalized outputs;

inputting a second batch of images into the first machine learning model, wherein the second batch of images is sampled from a second plurality of images generated via a second perturbation experiment comprising a second set of experimental conditions;
for the at least one layer:
computing a second set of batch normalization statistics to account for the second set of experimental conditions associated with a second plurality of outputs generated by the at least one layer based on the second batch of images of the second perturbation experiment; and
normalizing the second plurality of outputs to account for a second batch effect in the second batch of images from the second perturbation experiment based on the second set of batch normalization statistics to generate a second plurality of batch normalized outputs;

updating the updated set of parameters in the plurality of layers based on a second set of predictions generated by the first machine learning model from the second batch of images of the second perturbation experiment and from the second plurality of batch normalized outputs; and
generating, using the trained machine learning model, a batch normalized embedding from an image corresponding to a new perturbation experiment.

2 . The computer-implemented method of claim 1 , further comprising:
generating, by an input layer of the plurality of layers of the first machine learning model, an initial set of output values based on image data associated with the first batch of images; generating, by a plurality of hidden layers of the plurality of layers of the first machine learning model, a plurality of hidden layer outputs from the initial set of output values; and selecting the at least one layer for batch normalization from the input layer or the plurality of hidden layers.

3 . The computer-implemented method of claim 1 , further comprising:
updating the updated set of parameters based on a set of labels associated with the second batch of images, wherein the set of labels corresponds to a set of cell perturbations that occurred during the first perturbation experiment and the second perturbation experiment.

4 . The computer-implemented method of claim 1 , further comprising:
generating a first plurality of embeddings for the second batch of images based on a third plurality of outputs generated by a hidden layer included in the plurality of layers.

5 . The computer-implemented method of claim 1 , further comprising:
computing a first set of metrics associated with a first plurality of embeddings generated by the first machine learning model and a second set of metrics associated with a second plurality of embeddings generated by a second machine learning model; and selecting the trained machine learning model from the first machine learning model and the second machine learning model based on a comparison of the first set of metrics and the second set of metrics.

6 . The computer-implemented method of claim 1 , further comprising:
generating a first distribution of similarities between a first plurality of embeddings generated by the first machine learning model from the second batch of images and a second plurality of embeddings generated by the first machine learning model from a third batch of images, wherein the first distribution of similarities is generated based on pairs of embeddings that are associated with a common class and different experiments; generating a second distribution of similarities between the first plurality of embeddings and a second plurality of embeddings, wherein the second distribution of similarities is generated based on pairs of embeddings that are associated with different classes and different experiments; and computing a perturbation consistency based on the first distribution of similarities and the second distribution of similarities.

7 . The computer-implemented method of claim 1 , wherein normalizing the first plurality of outputs comprises:
computing a mean of the first plurality of outputs and a standard deviation of the first plurality of outputs; subtracting the mean from an output in the first plurality of outputs to produce a centered output; and dividing the centered output by the standard deviation to produce a normalized output.

8 . The computer-implemented method of claim 1 , further comprising cropping the first batch of images prior to inputting the first batch of images into the first machine learning model.

9 . The computer-implemented method of claim 1 , wherein the plurality of layers includes a convolutional layer and a fully connected layer.

10 . The computer-implemented method of claim 1 , wherein the first batch of images is generated from a plate of cells associated with the first perturbation experiment and the first set of experimental conditions corresponds to the plate of cells.

11 . One or more non-transitory computer readable media storing instructions that, when executed by one or more processors via a connection with an interconnect bus, cause the one or more processors to perform steps of:
training a machine learning model to generate a trained machine learning model by:
inputting a first batch of images into a first machine learning model that includes a plurality of layers having a set of parameters, wherein the first batch of images has been sampled from a first plurality of images generated via a first perturbation experiment comprising a first set of experimental conditions;
for at least one layer included in the plurality of layers:
computing, by a batch normalization layer of the plurality of layers, a first set of batch normalization statistics to account for a first batch effect in the first batch of images from the first perturbation experiment associated with a first plurality of outputs generated by the at least one layer based on the first batch of images of the first perturbation experiment; and
normalizing the first plurality of outputs to account for a first batch effect in the first batch of images from the first perturbation experiment based on the first set of batch normalization statistics to generate a first plurality of batch normalized outputs;
generating an updated set of parameters by updating the set of parameters of the plurality of layers b

Assignees

• Recursion Pharmaceuticals Inc

Inventors

• Earnshaw Berton Allen
• Sypetkowski Maciej

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/084

  Backpropagation, e.g. using gradient descent · CPC title

• G06N20/00 Primary

  Machine learning · CPC title

• G06N3/045 Primary

  Combinations of networks · CPC title