Cross-lingual natural language understanding model for multi-language natural language understanding (mNLU)

Patent metadata

Field	Value
Publication number	US-12596891-B2
Application number	US-202218070128-A
Country	US
Kind code	B2
Filing date	Nov 28, 2022
Priority date	Nov 28, 2022
Publication date	Apr 7, 2026
Grant date	Apr 7, 2026

Abstract

A cross-lingual Natural Language Understanding (NLU) framework includes a cross-lingual NLU model that can be trained and tuned on a base language and one or more target languages, and subsequently be used to infer utterances in any of the supported languages. The present technique enables minimal, targeted fine-tuning of the cross-lingual NLU model in each language to be supported without negatively impacting prediction performance in other languages. Accordingly, the present technique reduces the resource costs in developing and maintaining a multi-language NLU (mNLU) framework and improves the scalability of the mNLU framework to support different languages.

First claim

What is claimed is:

1 . A natural language understanding (NLU) framework, comprising:
at least one memory configured to store a cross-lingual NLU model and a base language test dataset, wherein the cross-lingual NLU model includes an intent-entity model and a cross-lingual word vector distribution model, wherein the intent-entity model defines intents and includes sample utterances for each intent; and at least one processor configured to execute stored instructions to cause the NLU framework to perform actions comprising:
performing an initial iterative training and tuning of the cross-lingual NLU model until the NLU framework is configured to infer the base language test dataset with at least a first predefined level of prediction performance;
determining a target language test dataset from the base language test dataset;
selecting a set of target language test utterances from the target language test dataset and adding the selected set of target language test utterances to the intent-entity model as new target language sample utterances; and
performing additional iterative training and tuning of the cross-lingual NLU model until the NLU framework is configured to infer the target language test dataset with at least a second predefined level of prediction performance.

2 . The NLU framework of claim 1 , wherein, to determine the target language test dataset from the base language test dataset, the at least one processor is configured to execute stored instructions to cause the NLU framework to perform actions comprising:
performing a machine translation of base language test utterances of the base language test dataset to yield target language test utterances of the target language test dataset; and modifying one or more of the target language test utterances of the target language test dataset based on input received from a developer to yield the target language test dataset.

3 . The NLU framework of claim 1 , wherein, to perform the initial iterative training and tuning of the cross-lingual NLU model, the at least one processor is configured to execute stored instructions to cause the NLU framework to perform actions comprising:
(A) compiling meaning representations of a search space of the cross-lingual NLU model based on the sample utterances of the intent-entity model using the cross-lingual word vector distribution model, in accordance with a set of NLU framework parameters of the cross-lingual NLU model; (B) inferencing base language test utterances of the base language test dataset using the search space and the cross-lingual word vector distribution model of the cross-lingual NLU model, in accordance with the set of NLU framework parameters of the cross-lingual NLU model; (C) evaluating a base language prediction performance of the cross-lingual NLU model; and then (D) in response to determining that the base language prediction performance of the cross-lingual NLU model is less than the first predefined level of prediction performance, modifying the cross-lingual NLU model and returning to step A.

4 . The NLU framework of claim 3 , wherein, to select the set of target language test utterances from the target language test dataset and add the selected set of target language test utterances to the intent-entity model, the at least one processor is configured to execute stored instructions to cause the NLU framework to perform actions comprising:
inferencing target language test utterances of the target language test dataset using the search space and the cross-lingual word vector distribution model of the cross-lingual NLU model, in accordance with the set of NLU framework parameters of the cross-lingual NLU model; identifying a set of intents of the intent-entity model for which the cross-lingual NLU model demonstrates poorer prediction performance in the target language than in the base language; selecting the set of target language test utterances that correspond to the identified set of intents in the target language test dataset; and adding the selected set of target language test utterances as the new target language sample utterances in the intent-entity model.

5 . The NLU framework of claim 4 , wherein, to perform the additional iterative training and tuning of the cross-lingual NLU model, the at least one processor is configured to execute stored instructions to cause the NLU framework to perform actions comprising:
(E) compiling additional meaning representations of the search space of the cross-lingual NLU model based on the new target language sample utterances of the intent-entity model using the cross-lingual word vector distribution model, in accordance with the set of NLU framework parameters of the cross-lingual NLU model; (F) inferencing the target language test utterances of the target language test dataset using the search space and the cross-lingual word vector distribution model of the cross-lingual NLU model, in accordance with the set of NLU framework parameters of the cross-lingual NLU model; (G) evaluating a target language prediction performance of the cross-lingual NLU model; and then (H) in response to determining that the target language prediction performance of the cross-lingual NLU model is less than the second predefined level of prediction performance, modifying the cross-lingual NLU model and returning to step E.

6 . The NLU framework of claim 5 , wherein, to modify the cross-lingual NLU model, the at least one processor is configured to execute stored instructions to cause the NLU framework to perform actions comprising:
modifying at least one of the set of NLU framework parameters of the cross-lingual NLU model.

7 . The NLU framework of claim 6 , wherein the set of NLU framework parameters comprise a set of focus-attention-magnification (FAM) coefficients, a set of re-expression rules, a set of class compatibility rules, or a set of class scoring coefficients, or any combination thereof.

8 . The NLU framework of claim 5 , wherein, to modify the cross-lingual NLU model, the at least one processor is configured to execute stored instructions to cause the NLU framework to perform actions comprising:
adding one or more domain-specific terms and corresponding vector representations to a cross-lingual vocabulary model of the cross-lingual NLU model, wherein, during inferencing, the NLU framework is configured to determine the corresponding vector representations for the one or more domain-specific terms in test utterances using the cross-lingual vocabulary model.

9 . The NLU framework of claim 1 , wherein the base language test dataset includes base language test utterances, each labeled to indicate a respective intent, one or more respective entities, or a combination thereof, of each of the base language test utterances.

10 . The NLU framework of claim 1 , wherein the cross-lingual word vector distribution model comprises GOOGLE's Universal Sentence Encoder (GUSE) or Language-agnostic bidirectional encoder representations from transformers (BERT) sentence embedding (LaBSE).

11 . A method of operating a natural language understanding (NLU) framework, comprising:
performing an initial iterative training and tuning of a cross-lingual NLU model until the NLU framework is configured to infer a base language test dataset with at least a first predefined level of prediction performance, wherein the cross-lingual NLU model includes an intent-entity model, a cross-lingual word vector distribution model, and a set of NLU framework parameters, and wherein the intent-entity model defines intents and includes sample utterances for each intent; determining a target language test dataset from the base language test dataset;

Assignees

• Servicenow Inc

Inventors

• Jain Mayank
• Subbarao Murali B

Classifications

• G06F40/55 Primary

  Rule-based translation · CPC title

• G06F40/279

  Recognition of textual entities · CPC title

• G06N5/04

  Inference or reasoning models · CPC title

• G06F40/30

  Semantic analysis · CPC title

• G06F40/58 Primary

  Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation · CPC title