Systems and methods for training a neural network model using knowledge from pre-trained large language models

Patent metadata

Field	Value
Publication number	US-12499312-B2
Application number	US-202318335898-A
Country	US
Kind code	B2
Filing date	Jun 15, 2023
Priority date	Jun 6, 2023
Publication date	Dec 16, 2025
Grant date	Dec 16, 2025

Abstract

Embodiments described herein provide a training framework for generative NLP models. Specifically, the training input, e.g., in the form of a sequence of tokens representing a user-agent dialogue, may be randomly masked for a few spans, which can be one or more tokens, one or more words, one or more sentences, or one or more paragraphs. These masked spans are replaced with their embeddings generated from pre-trained large language models are then used for training the NLP model.

First claim

What is claimed is:

1 . A method for pretraining a neural network based natural language processing (NLP) model, comprising:
receiving, at a server and via a communication interface over a network, a training input including tokens of a prior user-agent dialogue; randomly replacing one or more spans in the training input with one or more corresponding span embeddings obtained by embedding the one or more spans via a pretrained large language model (LLM), thereby resulting in a masked training input; training the neural network based NLP model implemented on one or more hardware processors at the server using a pre-processed dataset including the masked training input; and generating, by the trained neural network based NLP model, an agent response in response to a user utterance.

2 . The method of claim 1 , wherein at least one span of the one or more spans includes a sentence or a paragraph.

3 . The method of claim 1 , wherein the one or more corresponding span embeddings have a same dimension of the neural network based NLP model.

4 . The method of claim 1 , wherein the one or more corresponding span embeddings are precomputed by the pretrained LLM prior to runtime.

5 . The method of claim 1 , wherein the prior user-agent dialogue comprises a prior user utterance and at least one prior system response generated in response to the prior user utterance based on at least one or more knowledge documents.

6 . The method of claim 1 , wherein the one or more corresponding span embeddings are retrieved via an application programming interface (API) at the server, which connects the server to an external server that hosts the pretrained LLM.

7 . The method of claim 1 , wherein the training the neural network based NLP model using the pre-processed dataset including the masked training input further comprises:
predicting, by the neural network based NLP model, a next token subsequent to a subset of tokens in response to the masked training input; computing a causal language modeling loss based on the predicted next token and a ground-truth token from tokens of the prior user-agent dialogue; and updating parameters of the neural network based NLP model based on the causal language modeling loss.

8 . A system for pretraining a neural network based natural language processing (NLP) model, the system comprising:
a memory that stores the neural network based NLP model and a plurality of processor executable instructions; a communication interface that receives a training input including tokens of a prior user-agent dialogue; and one or more hardware processors that read and execute the plurality of processor-executable instructions from the memory to perform operations comprising:
randomly replacing one or more spans in the training input with one or more corresponding span embeddings obtained by embedding the one or more spans via a pretrained large language model (LLM), thereby resulting in a masked training input;
training the neural network based NLP model implemented on one or more hardware processors at the server using a pre-processed dataset including the masked training input; and
generating, by the trained neural network based NLP model, an agent response in response to a user utterance.

9 . The system of claim 8 , wherein at least one span of the one or more spans includes a sentence or a paragraph.

10 . The system of claim 8 , wherein the one or more corresponding span embeddings have a same dimension of the neural network based NLP model.

11 . The system of claim 8 , wherein the one or more corresponding span embeddings are precomputed by the pretrained LLM prior to runtime.

12 . The system of claim 8 , wherein the prior user-agent dialogue comprises a prior user utterance and at least one prior system response generated in response to the prior user utterance based on at least one or more knowledge documents.

13 . The system of claim 8 , wherein the one or more corresponding span embeddings are retrieved via an application programming interface (API) at the server, which connects the server to an external server that hosts the pretrained LLM.

14 . The system of claim 8 , wherein the operation of training the neural network based NLP model using the pre-processed dataset including the masked training input further comprises:
predicting, by the neural network based NLP model, a next token subsequent to a subset of tokens in response to the masked training input; computing a causal language modeling loss based on the predicted next token and a ground-truth token from tokens of the prior user-agent dialogue; and updating parameters of the neural network based NLP model based on the causal language modeling loss.

15 . A non-transitory machine-readable medium comprising a plurality of machine-executable instructions which, when executed by one or more processors, are adapted to cause the one or more processors to perform operations comprising:
receiving, at a server and via a communication interface over a network, a training input including tokens of a prior user-agent dialogue; randomly replacing one or more spans in the training input with one or more corresponding span embeddings obtained by embedding the one or more spans via a pretrained large language model (LLM), thereby resulting in a masked training input; and training the neural network based NLP model implemented on one or more hardware processors at the server using a pre-processed dataset including the masked training input; and generating, by the trained neural network based NLP model, an agent response in response to a user utterance.

16 . The non-transitory machine-readable medium of claim 15 , wherein at least one span of the one or more spans includes a sentence or a paragraph.

17 . The non-transitory machine-readable medium of claim 15 , wherein the one or more corresponding span embeddings have a same dimension of the neural network based NLP model.

18 . The non-transitory machine-readable medium of claim 15 , wherein the one or more corresponding span embeddings are precomputed by the pretrained LLM prior to runtime.

19 . The non-transitory machine-readable medium of claim 15 , wherein the prior user-agent dialogue comprises a prior user utterance and at least one prior system response generated in response to the prior user utterance based on at least one or more knowledge documents.

20 . The non-transitory machine-readable medium of claim 15 , wherein the one or more corresponding span embeddings are retrieved via an application programming interface (API) at the server, which connects the server to an external server that hosts the pretrained LLM.

Assignees

• Salesforce Inc

Inventors

• Pentyala Shiva Kumar
• Choubey Prafulla Kumar
• Harinath Shashank
• Asur Sitaram
• Wu Chien-Sheng Jason
• Alexander Zachary
• Xiong Caiming

Classifications

• G06N3/08 Primary

  Learning methods · CPC title

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06F40/35

  Discourse or dialogue representation · CPC title

• G06F40/30

  Semantic analysis · CPC title

• G06N3/045

  Combinations of networks · CPC title