Dynamic memory architecture for use with large language model(s)

Patent metadata

Field	Value
Publication number	US-12489722-B2
Application number	US-202318540373-A
Country	US
Kind code	B2
Filing date	Dec 14, 2023
Priority date	Dec 14, 2023
Publication date	Dec 2, 2025
Grant date	Dec 2, 2025

Abstract

Processor(s) can receive a first natural language (NL) based input as part of a dialog between a user of a client device and an automated assistant that is accessible at the client device; process, using a first machine learning (ML) model, the first NL based input to generate a first inference associated with the user and a confidence indicator for the first inference, the confidence indicator for the first inference indicative of a degree of confidence that the first inference is true; store the first inference and the confidence indicator for the first inference in a database; process, using a second ML model, the first inference and additional data to generate a modified confidence indicator for the first inference and store the modified confidence indicator in the database; and determine, based at least on the modified confidence indicator for the first inference, whether to process, using a first large language model (LLM), the first inference with a second NL based input to generate a first NL based response.

First claim

What is claimed is:

1 . A method implemented by one or more processors, the method comprising:
receiving a first natural language (NL) based input as part of a dialog between a user of a client device and an automated assistant that is accessible at the client device; processing, using a first machine learning (ML) model, the first NL based input to generate a first inference associated with the user and a confidence indicator for the first inference, the confidence indicator for the first inference indicative of a degree of confidence that the first inference is true; storing the first inference and the confidence indicator for the first inference in a database; processing, using a second ML model, the first inference and additional data to generate a modified confidence indicator for the first inference, the modified confidence indicator for the first inference being indicative of a degree of confidence that the first inference is true based on the first inference and the additional data; storing the modified confidence indicator for the first inference in the database in association with the first inference; and determining, based at least on the modified confidence indicator for the first inference stored in the database, whether to process, using a first large language model (LLM), the first inference with a second NL based input to generate a first NL based response.

2 . The method of claim 1 , further comprising:
receiving the second NL based input as part of a dialog between the user of the client device and the automated assistant; responsive to determining, based at least on the modified confidence indicator for the first inference, to process, using the first LLM, the first inference with the second NL based input to generate the first NL based response:
modifying the second NL based input based on the first inference to generate a modified second NL based input;
processing, using the first LLM, the modified second NL based input to generate the first NL based response; and causing the first NL based response to be rendered at the client device.

3 . The method of claim 2 , wherein modifying the second NL based input based on the first inference to generate a modified second NL based input comprises prepending or postpending the first inference to the second NL based input.

4 . The method of claim 2 , wherein causing the first NL based response to be rendered at the client device comprises causing the first NL based response to be visually rendered at the client device via a display of the client device and/or comprises causing the first NL based response to be audibly rendered at the client device via one or more speakers of the client device.

5 . The method of claim 2 , wherein causing the first NL based response to be rendered at the client device comprises transmitting data to the client device that is operable for causing the client device to render the first NL based response.

6 . The method of claim 1 , further comprising:
receiving the second NL based input as part of a dialog between the user of the client device and the automated assistant; responsive to determining, based at least on the modified confidence indicator for the first inference stored in the database, not to process, using the first LLM, the first inference with the second NL based input to generate a first NL based response:
processing, using the first LLM, the second NL based input without the first inference to generate the first NL based response; and
causing the first NL based response to be rendered at the client device.

7 . The method of claim 1 , further comprising:
processing, using the first ML model, a third NL based input to generate a second inference associated with the user and a confidence indicator for the second inference, the confidence indicator for the second inference indicative of a degree of confidence that the second inference is true; and storing the second inference and the confidence indicator for the second inference in the database, wherein the additional data comprises the second inference.

8 . The method of claim 7 , further comprising:
receiving a fourth NL based input as part of a dialog between the user of the client device and the automated assistant; processing, using the first ML model, the fourth NL based input to generate a third inference associated with the user; storing the third inference in the database; processing, using the second ML model, the first inference, the modified confidence indicator for the first inference and the third inference to generate a further modified confidence indicator for the first inference, the further modified confidence indicator for the first inference being indicative of a degree of confidence that the first inference is true based on the first inference, the modified confidence indicator for the first inference and the third inference; storing the further modified confidence indicator for the first inference in the database in association with the first inference; receiving a fifth NL based input as part of a dialog between the user of the client device and the automated assistant; responsive to determining, based at least on the further modified confidence indicator for the first inference, to process, using the first LLM, the first inference with the fifth NL based input to generate a second NL based response:
modifying the fifth NL based input based on the first inference to generate a modified fifth NL based input;
processing, using the first LLM, the modified fifth NL based input to generate the second NL based response; and causing the second NL based response to be rendered at the client device.

9 . The method of claim 1 , further comprising:
receiving a third NL based input as part of a dialog between the user of the client device and the automated assistant; and storing the third NL based input, wherein the additional data comprises at least a portion of the third NL based input.

10 . The method of claim 1 , further comprising:
obtaining one or more contextual signals associated with one or more of: a dialog between the user of the client device and the automated assistant, the user of the client device, or the client device; and determining, based on the one or more contextual signals, a current context, wherein the additional data comprises the current context.

11 . The method of claim 1 , further comprising:
obtaining one or more contextual signals associated with one or more of: the user of the client device, or the client device, wherein the additional data comprises the one or more contextual signals.

12 . The method of claim 1 , wherein the second ML model is a second LLM, and wherein processing, using the second ML model, the first inference and the additional data to generate a modified confidence indicator for the first inference comprises:
providing the first inference and the additional data to the second LLM in an LLM prompt; and receiving the modified confidence indicator for the first inference as an LLM output.

13 . A method implemented by one or more processors, the method comprising:
receiving a first natural language (NL) based input as part of a dialog between a user of a client device and an automated assistant that is accessible at the client device; processing, using a first machine learning (ML) model, the first NL based input to generate a first inference associated with the user; storing the first inference in a database; processing, using a second ML model, the first inference and additional data to generate the second inference associated with the user and a confidence ind

Assignees

• Google Llc

Inventors

• Beltrone Lauren
• Tolins Jackson
• Parra Maria
• Echevarria Carla
• Isert Carsten
• Zöchbauer Patrick

Classifications

• G06F40/35 Primary

  Discourse or dialogue representation · CPC title

• G06N20/00

  Machine learning · CPC title

• H04L51/02 Primary

  using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages · CPC title