System and method for managing and optimizing lookup source templates in a natural language understanding (NLU) framework

Patent metadata

Field	Value
Publication number	US-12511491-B2
Application number	US-202217579260-A
Country	US
Kind code	B2
Filing date	Jan 19, 2022
Priority date	Jan 21, 2021
Publication date	Dec 30, 2025
Grant date	Dec 30, 2025

Abstract

A natural language understanding (NLU) framework includes a lookup source framework that enables the lookup sources to be created and applied to understanding utterances. Each lookup source is associated with a respective lookup source template that defines the compile-time and inference-time behavior of the lookup source. For example, a lookup source template indicates which plugins are used by the lookup source, and may define property values that determine the operational behavior of each of these plugins during compile-time and/or inference-time operation of the lookup source. The lookup source framework includes a template manager that manages lookup source templates and determines a suitable lookup source template for each lookup source. The lookup source framework includes a lookup source template optimization subsystem that can apply a suitable optimization plugin to automatically determine attribute values to be included in an optimized lookup source template of a lookup source of the lookup source system.

First claim

What is claimed is:

1 . A lookup source framework, comprising:
at least one memory configured to store a template manager, a preprocessing subsystem, a producer subsystem, and a lookup source template optimization subsystem; and at least one processor configured to execute stored instructions to cause the lookup source framework to perform actions comprising:
receiving a request to generate a source data representation of a lookup source from source data of a client, wherein the source data is stored in a database prior to receiving the request;
determining, via the template manager, a lookup source template for the lookup source by:
(A) generating a test source data representation of the lookup source from test source data based on a set of attribute values;
(B) generating, via the lookup source template optimization subsystem, test segmentations of labeled training data using the test source data representation of the lookup source;
(C) determining, via the lookup source template optimization subsystem, a ratio of correct segmentations to a total number of segmentations in the test segmentations of the labeled training data;
(D) in response to determining that the ratio is less than a predefined threshold, applying an optimization plugin of the lookup source template optimization subsystem to modify the set of attribute values and then returning to step A; and
(E) in response to determining that the ratio is greater than or equal to the predefined threshold, storing, via the lookup source template optimization subsystem, the set of attribute values as the lookup source template in the at least one memory;

preprocessing, via the preprocessing subsystem, the source data of the client using one or more preprocessors indicated by the lookup source template to identify a set of state values, each representing a portion of the source data;
applying, via the producer subsystem, one or more producers indicated by the lookup source template to the set of state values to derive a set of produced state values; and
compiling the source data representation of the lookup source having a plurality of states, including original states having corresponding state values from the set of state values and produced states having corresponding state values from the set of produced state values.

2 . The lookup source framework of claim 1 , wherein the lookup source template defines a plurality of attributes, wherein the plurality of attributes comprises a language attribute indicating a language of the lookup source and a data source type attribute indicating a data source type of the lookup source.

3 . The lookup source framework of claim 2 , wherein the plurality of attributes comprises one or more preprocessor attributes respectively indicating the one or more preprocessors of the preprocessing subsystem, including at least one tokenizer of the preprocessing subsystem.

4 . The lookup source framework of claim 2 , wherein the plurality of attributes comprises one or more producer attributes respectively indicating the one or more producers of the producer subsystem, wherein each of the one or more producer attributes includes at least one respective property value defining an aspect of operation for each of the one or more producers, wherein the at least one respective property value of each of the one or more producers comprises a lower threshold value, an upper threshold value, an edit distance, a transduction procedure, a scoring adjustment, or any combination thereof.

5 . The lookup source framework of claim 1 , wherein, after compiling the source data representation of the lookup source, the at least one processor is configured to execute the stored instructions to cause the lookup source framework to perform actions comprising:
performing lookup source inference of a user utterance to determine a plurality of segmentations of the user utterance, wherein each of the plurality of segmentations indicates how tokens of the user utterance have been matched to tokens of the source data represented by the compiled source data representation of the lookup source.

6 . The lookup source framework of claim 5 , wherein the at least one memory is configured to store a matcher subsystem of the lookup source framework, and wherein, to perform the lookup source inference, the at least one processor is configured to execute the stored instructions to cause the lookup source framework to perform actions comprising:
applying, via the matcher subsystem, one or more matchers indicated by the lookup source template to attempt to match the tokens of the user utterance to the tokens of the source data represented by the compiled source data representation.

7 . The lookup source framework of claim 6 , wherein the lookup source template defines a plurality of attributes, wherein the plurality of attributes comprises one or more matcher attributes respectively indicating the one or more matchers of the matcher subsystem, wherein each of the one or more matcher attributes includes at least one respective property value defining an aspect of operation for each of the one or more matchers during the lookup source inference, wherein the at least one respective property value of each of the one or more matchers comprises a lower threshold value, an upper threshold value, an edit distance, a transduction procedure, a scoring adjustment, or any combination thereof.

8 . The lookup source framework of claim 5 , wherein the at least one memory is configured to store a postprocessor subsystem of the lookup source framework, wherein, to perform the lookup source inference, the at least one processor is configured to execute the stored instructions to cause the lookup source framework to perform actions comprising:
applying, via the postprocessor subsystem, one or more postprocessors indicated by the lookup source template to reformat segments, to aggregate segments, or any combination thereof, to determine the plurality of segmentations during the lookup source inference of the user utterance.

9 . The lookup source framework of claim 1 , wherein the at least one memory is configured to store a plurality of lookup source templates, and wherein, to determine the lookup source template for the lookup source, the at least one processor is configured to execute the stored instructions to cause the template manager to perform actions comprising:
selecting the lookup source template for the lookup source from the plurality of lookup source templates stored in the at least one memory.

10 . The lookup source framework of claim 9 , wherein the at least one memory is configured to store a lookup source configuration of the lookup source, and wherein, to select the lookup source template for the lookup source, the at least one processor is configured to execute the stored instructions to cause the template manager to perform actions comprising:
in response to determining that the lookup source configuration defines a lookup source template attribute value, selecting the lookup source template for the lookup source from the plurality of lookup source templates based on the lookup source template attribute value of the lookup source configuration, wherein the lookup source template attribute value comprises a setting of the lookup source template.

11 . The lookup source framework of claim 10 , wherein the at least one memory is configured to store an intent-entity model that associates intents with entities and sample utterances, and wherein, in response to determining that the lookup source configuration does not define the lookup source template attribute value, the at least one processor is configured to execute the stored instr

Assignees

• Servicenow Inc

Inventors

• Narendula Rammohan
• Sapugay Edwin
• Naboka Maxim
• Chadalavada Swathi Kumar

Classifications

• G06F40/284

  Lexical analysis, e.g. tokenisation or collocates · CPC title

• G06F40/186

  Templates · CPC title

• H04L51/02

  using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages · CPC title

• G06F40/295

  Named entity recognition · CPC title

• G06F16/3344

  using natural language analysis · CPC title