Intelligent contextually aware digital assistants

Patent metadata

Field	Value
Publication number	US-9542648-B2
Application number	US-201414250322-A
Country	US
Kind code	B2
Filing date	Apr 10, 2014
Priority date	Apr 10, 2014
Publication date	Jan 10, 2017
Grant date	Jan 10, 2017

Abstract

One embodiment of the present invention provides a system for providing context-based web services for a user. During operation, the system receives a sentence as input from a user. The system performs natural language processing on the sentence to determine one or more parameters. The system retrieves data from a foreground knowledge graph containing contextual data for the user and from a background knowledge graph containing background information corresponding to the parameters. The system determines a set of arguments based on the parameters and/or data from the foreground knowledge graph and/or data from the background knowledge graph. The system then selects an action module based on results of the natural language processing and/or the set of arguments. The system passes the arguments to the action module. The action module then uses the arguments to respond to a question or interact with web services to perform an action for the user.

First claim

What is claimed is:

1. A computer-executable method for providing context-based web services for a user, comprising:
receiving a sentence as input from a user interacting with a visual interface that includes an animated agent;
performing natural language processing on the sentence to determine one or more parameters;
retrieving data from a foreground knowledge graph that contains contextual data for the user and from a background knowledge graph that contains background information corresponding to the one or more parameters, wherein the background knowledge graph is different from the foreground knowledge graph;
determining a set of arguments based on the one or more parameters and data from the foreground and background knowledge graphs;
passing the set of arguments to an action module selected based on results of the natural language processing and the set of arguments;
using the set of arguments, by the selected action module, to interact with web services to perform an action for the user and provide a response to the user, wherein providing the response involves using a text-speech translator to produce audio for the response, using a viseme extractor to determine mouth positions of the animated agent for synchronous display with the audio, and animating the animated agent based on animation tags inserted into the response; and
converting general and domain-specific knowledge into modifications to the background knowledge graph, which involves obtaining a document set on a particular subject based on performing a web search, and modifying the background knowledge graph using results from analyzing the document set using a content analysis module and a semantic meaning extraction system.

2. The method of claim 1 , wherein performing an action for the user further comprises completing an online sales transaction.

3. The method of claim 1 , wherein performing natural language processing to determine one or more parameters further comprises:
determining a sentence structure of the sentence;
determining whether there is an entry in a database corresponding to the sentence structure;
responsive to determining that there is an entry in the database corresponding to the sentence structure, retrieving information from the entry in the database; and
extracting parameters from the sentence based on information retrieved from the database entry.

4. The method of claim 1 , wherein performing natural language processing on the sentence to determine one or more parameters further comprises:
determining a sentence structure of the sentence;
determining whether there is an entry in a database corresponding to the sentence structure;
responsive to determining that there is no entry in the database corresponding to the sentence structure, engaging in a dialogue to elicit one or more parameters;
determining mapping of the one or more parameters to properties on an object; and
storing information that includes the mapping and the one or more parameters in a database.

5. The method of claim 1 , wherein changes in the contextual data of the foreground knowledge graph triggers performing an action based on the user's context.

6. The method of claim 1 , further comprising:
adding contextual data to the foreground knowledge graph based on detected user activity and/or user communications;
disambiguating another input sentence that requires information from the background knowledge graph based on the contextual data from the foreground knowledge graph; and
performing another action for the user based at least on a portion of the contextual data added to the foreground knowledge graph and the information from the background knowledge graph.

7. The method of claim 1 , wherein one or more modules perform parameterized queries and modifications on the foreground knowledge graph and the background knowledge graph.

8. A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for providing context-based web services for a user, the method comprising:
receiving a sentence as input from a user interacting with a visual interface that includes an animated agent;
performing natural language processing on the sentence to determine one or more parameters;
retrieving data from a foreground knowledge graph that contains contextual data for the user and from a background knowledge graph that contains background information corresponding to the one or more parameters, wherein the background knowledge graph is different from the foreground knowledge graph;
determining a set of arguments based on the one or more parameters, data from the foreground and background knowledge graphs;
passing the set of arguments to an action module selected based on results of the natural language processing and the set of arguments;
using the set of arguments, by the selected action module, to interact with web services to perform an action for the user and provide a response to the user,
wherein providing the response involves using a text-speech translator to produce audio for the response, using a viseme extractor to determine mouth positions of the animated agent for synchronous display with the audio, and animating the animated agent based on animation tags inserted into the response; and
converting general and domain-specific knowledge into modifications to the background knowledge graph, which involves obtaining a document set on a particular subject based on performing a web search, and modifying the background knowledge graph using results from analyzing the document set using a content analysis module and a semantic meaning extraction system.

9. The non-transitory computer-readable storage medium of claim 8 , wherein performing an action for the user further comprises completing an online sales transaction.

10. The non-transitory computer-readable storage medium of claim 8 , wherein performing natural language processing to determine one or more parameters further comprises:
determining a sentence structure of the sentence;
determining whether there is an entry in a database corresponding to the sentence structure;
responsive to determining that there is an entry in the database corresponding to the sentence structure, retrieving information from the entry in the database; and
extracting parameters from the sentence based on information retrieved from the database entry.

11. The non-transitory computer-readable storage medium of claim 8 , wherein performing natural language processing on the sentence to determine one or more parameters further comprises:
determining a sentence structure of the sentence;
determining whether there is an entry in a database corresponding to the sentence structure;
responsive to determining that there is no entry in the database corresponding to the sentence structure, engaging in a dialogue to elicit one or more parameters;
determining mapping of the one or more parameters to properties on an object; and
storing information that includes the mapping and the one or more parameters in a database.

12. The non-transitory computer-readable storage medium of claim 8 , wherein changes in the contextual data of the foreground knowledge graph triggers performing an action based on the user's context.

13. The non-transitory computer-readable storage medium of claim 8 , wherein the method further comprises:
adding contextual data to the foreground knowledge graph based on detected user activity and/or user communications;
disambiguating another input sentence that requires information from the background knowledge graph based on the contextual data fro

Assignees

• Palo Alto Res Ct Inc

Inventors

• Roberts Michael

Classifications

• G10L2015/226

  using non-speech characteristics · CPC title

• G10L15/1815

  Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning · CPC title

• G06F16/90332

  Natural language query formulation or dialogue systems · CPC title

• G06F40/30

  Semantic analysis · CPC title

• G10L15/22 Primary

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title