Multi-objective reinforcement learning with experimental feedback for protein design

Patent metadata

Field	Value
Publication number	US-2025322902-A1
Application number	US-202519174661-A
Country	US
Kind code	A1
Filing date	Apr 9, 2025
Priority date	Apr 11, 2024
Publication date	Oct 16, 2025
Grant date	—

Abstract

A method for designing proteins using multi-objective reinforcement learning can include generating, by one or more processors using a machine model, based on an initial protein sequence data structure, a plurality of protein sequences, the machine learning model configured based on reinforcement learning from a plurality of reward metrics including at least one reward metric associated with experimental data regarding example sequence data, scoring, by the one or more processors, using a plurality of scoring functions, the plurality of protein sequences, to select a subset of protein sequences of the plurality of protein sequences, and outputting one or more selected protein sequences of the subset of selected protein sequences.

First claim

What is claimed is:

1 . A method, comprising:
generating, by one or more processors using a machine learning model, based on an initial protein sequence data structure, a plurality of protein sequences, the machine learning model configured based on reinforcement learning from a plurality of reward metrics including at least one reward metric associated with experimental data regarding example sequence data; scoring, by the one or more processors, using a plurality of scoring functions, the plurality of protein sequences, to select a subset of protein sequences of the plurality of protein sequences; and outputting one or more selected protein sequences of the subset of selected protein sequences.

2 . The method of claim 1 , wherein:
the machine learning model comprises a language model; and determining, by the machine learning model, each protein sequence of the plurality of protein sequences comprises generating, by the language model, one or more protein sequence elements based on the initial protein sequence data structure.

3 . The method of claim 1 , wherein the machine learning model is configured based on reinforcement learning by a plurality of agents, each agent of the plurality of agents associated with a different reward metric of the plurality of reward metrics than each other agent of the plurality of agents.

4 . The method of claim 1 , wherein the machine learning model comprises a pre-trained language model fine-tuned based on the plurality of reward metrics.

5 . The method of claim 1 , wherein the plurality of scoring functions comprise at least a similarity function based on a database of example sequence data, a folding function, and a stability function, and the method comprises scoring using the stability function responsive to an output of at least one of the similarity function or the folding function satisfying a corresponding threshold.

6 . The method of claim 1 , wherein generating the plurality of protein sequences comprises generating a plurality of protein sequence elements for each protein sequence of the plurality of protein sequences, wherein each protein sequence element of the plurality of protein sequence elements represents at least one of a codon or a protein residue.

7 . The method of claim 1 , wherein the plurality of scoring functions comprise at least one function based on at least one of a guanine-cytosine (GC) content or a molecular weight of each protein sequence of the plurality of protein sequences.

8 . The method of claim 1 , wherein the plurality of reward metrics comprise at least one evolutionary conservation metric.

9 . The method of claim 1 , wherein the plurality of reward metrics comprise at least one molecular simulation metric.

10 . The method of claim 1 , further comprising asynchronously performing, by the one or more processors in parallel using a plurality of parallel computing resources, at least one of the generating of the plurality of protein sequences or the scoring of the plurality of protein sequences.

11 . The method of claim 1 , wherein the plurality of scoring functions comprise an activity function to determine an activity of at least one protein sequence of the plurality of protein sequences.

12 . A system, comprising:
one or more processors to:
generate, using a machine learning model, based on an initial protein sequence data structure, a plurality of protein sequences, the machine learning model configured based on reinforcement learning from a plurality of reward metrics including at least one reward metric associated with experimental data regarding example sequence data;
score, using a plurality of scoring functions, the plurality of protein sequences, to select a subset of protein sequences of the plurality of protein sequences; and
output one or more selected protein sequences of the subset of selected protein sequences.

13 . The system of claim 12 , wherein:
the machine learning model comprises a language model; and the one or more processors are to determine each protein sequence of the plurality of protein sequences by generating, using the language model, one or more protein sequence elements based on the initial protein sequence data structure.

14 . The system of claim 12 , wherein the machine learning model is configured based on reinforcement learning by a plurality of agents, each agent of the plurality of agents associated with a different reward metric of the plurality of reward metrics than each other agent of the plurality of agents.

15 . The system of claim 12 , wherein:
the machine learning model comprises a pre-trained language model fine-tuned based on the plurality of reward metrics; and the plurality of reward metrics comprise at least one evolutionary conservation metric and at least one molecular simulation metric.

16 . The system of claim 12 , wherein the plurality of scoring functions comprise at least a similarity function based on a database of example sequence data, a folding function, and a stability function, and the plurality of scoring functions further comprises scoring using the stability function responsive to an output of at least one of the similarity function or the folding function satisfying a corresponding threshold.

17 . The system of claim 12 , wherein the one or more processors comprise a plurality of parallel processing units to asynchronously perform at least one of the generation of the plurality of protein sequences or the scoring of the plurality of protein sequences.

18 . A method, comprising:
generating, by each of a plurality of reinforcement learning agents, for each protein sequence of a plurality of examples of protein sequences, a reward score for an objective function, the reward score generated based on a different metric for each agent of the plurality of reinforcement learning agents;
evaluating the objective function using each reward score to generate an output of the objective function; and
updating a language model based on the output.

19 . The method of claim 18 , wherein a metric of a first agent of the plurality of reinforcement learning agents corresponds to a structure of the protein sequence, and the metric of a second agent of the plurality of reinforcement learning agents corresponds to kinetics of the protein sequence.

20 . The method of claim 18 , wherein updating the language model comprises evaluating a Kullback-Leibler divergence with respect to a previous state of the language model.

Assignees

• Uchicago Argonne Llc

Inventors

• Ramanathan Arvind
• Dharuman Gautham
• Ma Heng
• Setty Priyanka Varadaraja
• Ward Logan Timothy
• Gokdemir Ozan
• Brace Alexander
• Hippe Kyle
• Anandkumar Anima

Classifications

• G16B40/20

  Supervised data analysis · CPC title

• G06N20/00

  Machine learning · CPC title

• G16B15/20 Primary

  Protein or domain folding · CPC title