Search-based natural language intent detection, selection, and execution for multi-agent automation systems

Patent metadata

Field	Value
Publication number	US-12299016-B2
Application number	US-202418421408-A
Country	US
Kind code	B2
Filing date	Jan 24, 2024
Priority date	Dec 29, 2017
Publication date	May 13, 2025
Grant date	May 13, 2025

Abstract

Improved intelligent personal assistant (IPA) software agents are disclosed that are configured to interact with various people, service providers, files, and/or smart devices. More particularly, this disclosure relates to an improved Natural Language Processing (NLP) Intent Determination Service (IDS) that is able to determine the likely best action to take in response to generic user commands and queries. The improved NLP IDS disclosed is said to be ‘search-based’ because, rather than attempt to parse incoming user commands and queries up front, the incoming user commands and queries are searched against a pre-generated database of exemplary user commands (e.g., having associated action or parsing identifiers) to determine the most relevant search result(s). The associated system actions and known grammar/parsing rules of the most relevant search result(s) may then be used to process the incoming user command or query—without having to actually parse the incoming user command or query from scratch.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
detecting, by a computing device, a voice command having a natural language dialogue from a user;
determining an input text string for the natural language dialogue from the voice command;
executing a search of a document store of commands issuable by the computing device using the input text string, wherein the document store of commands includes one or more inexact matches to the input text string;
identifying, based on the executed search, document metadata for a plurality of search results ranked from the one or more inexact matches, wherein the plurality of search results identify at least one of an action or a grammar usable for parsing the voice command for a user intent;
parsing the voice command using the plurality of search results and natural language processing (NLP);
determining, using a machine learning (ML) model, the user intent for the voice command based on the parsing and a previous voice command by the user;
determining a plurality of actions capable of being executed by the computing device within a threshold distance to the computing device;
selecting one of the plurality of actions for execution by the computing device based on the user intent; and
executing, by the computing device, one or more application programming interface (API) calls that cause the computing device to perform the one of the plurality of actions within the threshold distance.

2. The computer-implemented method of claim 1 , wherein the voice command comprises a sentence, a phrase, or an utterance, and wherein the document store of commands comprises operations executable in response to at least one of a user input, text converted from spoken words, audio data, video data, or a search query.

3. The computer-implemented method of claim 1 , wherein, prior to the determining the user intent, the computer-implemented method further comprises:
determining one or more command variants for the input text string from the voice command; and
executing an additional search of the document store of commands for the input text string using the one or more command variants as an additional parameter of the search.

4. The computer-implemented method of claim 3 , wherein the determining the one or more command variants comprises:
determining at least one command in the document store of commands tagged in association with the voice command or the input text string; and
varying words in the input text string based on corresponding text strings associated with the at least one command.

5. The computer-implemented method of claim 1 , wherein the selecting one of the plurality of actions uses at least one of an applied language model, an artificial intelligence model, or a string-matching technique, and wherein the selecting is further based on a ranking of corresponding relevance scores for the plurality of actions.

6. The computer-implemented method of claim 5 , wherein the selecting the one of the plurality of actions comprises requesting that the user resolve which of at least two of the plurality of actions corresponds to the voice command, and wherein the computer-implemented method further comprises:
updating the at least one of the applied language model, the artificial intelligence model, or the string-matching technique based on the one of the plurality of actions selected from the at least two of the plurality of actions.

7. The computer-implemented method of claim 1 , wherein, prior to the executing the search, the computer-implemented method further comprises:
identifying at least one of a part of speech of the voice command, a meaning of the voice command, or semantics between one or more words in the input text string,
wherein the search is further executed based on the at least one of the part of speech, the meaning, or the semantics.

8. The computer-implemented method of claim 1 , wherein the voice command is received via one of a messaging service, an Internet of Things (IoT) sensor associated with the computing device, an intent determination service (IDS), or a software program implemented with the computing device.

9. A system comprising:
a non-transitory memory; and
one or more hardware processors coupled to the non-transitory memory and configured to read instructions from the non-transitory memory to cause the system to perform operations comprising:
detecting, by an Internet-of-Things (IoT) device, a voice command having a natural language dialogue from a user;
determining an input text string for the natural language dialogue from the voice command;
executing a search of a document store of commands issuable by the IoT device using the input text string, wherein the search includes one or more inexact matches to the input text string;
identifying, based on the executed search, document metadata for a plurality of search results ranked from the one or more inexact matches, wherein the plurality of search results identify at least one of an action or a grammar usable for parsing the voice command for a user intent;
parsing the voice command using the plurality of search results and a natural language processing (NLP) system;
determining, using a machine learning (ML) model, a user intent for the voice command based on the parsing and a previous voice command by the user;
determining a plurality of actions capable of being executed by the IoT device in an environment of the IoT device;
selecting one of the plurality of actions for execution by the IoT device based on the user intent and the plurality of actions; and
executing, by the IoT device, one or more application programming interface (API) calls that effectuate the one of the plurality of actions by the IoT device.

10. The system of claim 9 , wherein the voice command comprises a sentence, a phrase, or an utterance, and wherein the document store of commands comprises operations executable in response to at least one of a user input, text converted from spoken words, audio data, video data, or a search query.

11. The system of claim 9 , wherein, prior to the determining the user intent, the operations further comprise:
determining one or more command variants for the input text string from the voice command; and
executing an additional search of the document store of commands for the input text string using the one or more command variants as an additional parameter of the search.

12. The system of claim 11 , wherein the determining the one or more command variants comprises:
determining at least one command in the document store of commands tagged in association with the voice command or the input text string; and
varying, by a natural language processor, words in the input text string based on corresponding text strings associated with the at least one command.

13. The system of claim 9 , wherein the selecting one of the plurality of actions uses at least one of an applied language model, an artificial intelligence model, or a string-matching technique, and wherein the selecting is further based on a ranking of corresponding relevance scores for the plurality of actions.

14. The system of claim 13 , wherein the selecting the one of the plurality of actions comprises requesting that the user resolve which of at least two of the plurality of actions is aligned with the voice command, and wherein the operations further comprise:
updating the at least one of the applied language model, the artificial intelligence model, or the string-matching technique based on the one of the plurality of actions selected from the at least two of the plurality of actions.

15.

Assignees

• Entefy Inc

Inventors

• Ghafourifar Alston
• Ghafourifar Mehdi

Classifications

• G06F16/3334

  Selection or weighting of terms from queries, including natural language queries · CPC title

• G06F40/205

  Parsing · CPC title

• G06F40/117

  Tagging; Marking up (details of markup languages G06F40/143); Designating a block; Setting of attributes (style sheets, e.g. eXtensible Stylesheet Language Transformation [XSLT], G06F40/154) · CPC title

• G06F16/3344

  using natural language analysis · CPC title

• G06F40/253

  Grammatical analysis; Style critique · CPC title