Semantic representations using structural ontology for assistant systems

Patent metadata

Field	Value
Publication number	US-11688022-B2
Application number	US-202016998423-A
Country	US
Kind code	B2
Filing date	Aug 20, 2020
Priority date	Oct 18, 2019
Publication date	Jun 27, 2023
Grant date	Jun 27, 2023

Abstract

In one embodiment, a method includes receiving a user input comprising a natural-language utterance by an assistant xbot from a client system associated with a user, determining a semantic representation of the user input based on a structural ontology defining a labeling syntax for parsing the natural-language utterance to semantic units comprising actions, objects, and attributes, wherein the semantic representation embeds at least one object within at least one action and declares at least one attribute of the embedded object to be acted upon, sending a request based on the semantic representation to an agent for executing a task corresponding to the user input, receiving results of the executed task mapped to a structure determined by the structural ontology from the agent, and sending from the assistant xbot to the client system instructions for presenting a response based on the results of the executed task.

First claim

What is claimed is:

1. A method comprising, by one or more computing systems:
receiving, by an assistant xbot from a client system associated with a user, a user input comprising a natural-language utterance;
determining a semantic representation of the user input based on a structural ontology, wherein the structural ontology defines a labeling syntax for parsing the natural-language utterance to semantic units comprising a plurality of actions, objects, and attributes, and wherein the semantic representation embeds at least one object within at least one action and declares at least one attribute of the at least one embedded object to be acted upon;
sending, to an agent, a request for executing a task corresponding to the user input, wherein the request is based on the semantic representation;
receiving, from the agent, results generated by the agent from executing the task, wherein the results are mapped to a structure determined by the structural ontology; and
sending, from the assistant xbot to the client system, instructions for presenting a response to the user input, wherein the response is based on the results.

2. The method of claim 1 , wherein the structural ontology defines that each of the plurality of actions operates on one or more of the plurality of objects.

3. The method of claim 1 , wherein the structural ontology defines that each of the plurality of objects comprises one or more of the plurality of attributes.

4. The method of claim 1 , wherein the structural ontology defines that each of the plurality of attributes declares a restriction on an action or object.

5. The method of claim 1 , wherein the assistant xbot is associated with a plurality of modules configured to process the user input, and wherein the semantic representation is executable by each module of the plurality of modules.

6. The method of claim 1 , wherein the structural ontology defines that the plurality of objects are hierarchically organized into a plurality of super-types and a plurality of sub-types, and where the plurality of super-types are associated with the plurality of sub-types, respectively, each sub-type inheriting one or more attributes of its associated super-type.

7. The method of claim 1 , wherein the assistant xbot is associated with a plurality of modules configured to execute the semantic representation, and wherein a value associated with each of the at least one action, the at least one object, and the at least one attribute is determined by each module of the plurality of modules executing the semantic representation.

8. The method of claim 1 , further comprising:
generating, by a natural-language generation module associated with the assistant xbot, the response based on the structural ontology and the results.

9. The method of claim 1 , wherein the structural ontology defines that the semantic units further comprise a plurality of methods and enums, wherein each of the plurality of methods comprises an action with its lifetime tied to an object, and wherein each of the plurality of enums comprises an object representing one out of a plurality of objects.

10. The method of claim 1 , wherein the structural ontology further defines a graph structure comprising one or more core sub-graphs and one or more generic sub-graphs, wherein the one or more core sub-graphs are not accessible by third-party agents, and wherein the one or more generic sub-graphs are accessible by the third-party agents.

11. The method of claim 1 , further comprising:
resolving, by a reasoning module associated with the assistant xbot, one or more entities associated with the at least one object based on the at least one attribute of the semantic representation.

12. The method of claim 1 , wherein the agent is a first-party agent associated with the assistant xbot.

13. The method of claim 1 , wherein the agent is a third-party agent associated with an external computing system, and wherein the structure of the results comprises one or more customized semantic units.

14. The method of claim 13 , wherein sending the request to the agent and receiving the results from the agent are both via an application programming interface (API), wherein the API defines a mapping from the one or more customized semantic units to the structure determined by the structural ontology.

15. The method of claim 13 , wherein the request sent to the agent comprises the semantic representation as modified by a dialog manager associated with the assistant xbot, wherein the results comprise the semantic representation as modified by the agent, and wherein the request and the results use the labeling syntax of the structural ontology.

16. The method of claim 13 , wherein the semantic units are each associated with a default namespace defined by the one or more computing systems, and wherein the one or more customized semantic units are associated with a personalized namespace defined by the third-party agent.

17. The method of claim 1 , wherein the at least one action has one or more arguments, wherein each argument specifies a constraint for an object to satisfy such that the at least one action can act upon the object, and wherein at least one of the one or more arguments specifies a constraint that the at least one object satisfies.

18. One or more computer-readable non-transitory storage media embodying software that comprises instructions that when executed by one or more processors, cause the one or more processors to:
receive, by an assistant xbot from a client system associated with a user, a user input comprising a natural-language utterance;
determine a semantic representation of the user input based on a structural ontology, wherein the structural ontology defines a labeling syntax for parsing the natural-language utterance to semantic units comprising a plurality of actions, objects, and attributes, and wherein the semantic representation embeds at least one object within at least one action and declares at least one attribute of the at least one embedded object to be acted upon;
send, to an agent, a request for executing a task corresponding to the user input, wherein the request is based on the semantic representation;
receive, from the agent, results generated by the agent from executing the task, wherein the results are mapped to a structure determined by the structural ontology; and
send, from the assistant xbot to the client system, instructions for presenting a response to the user input, wherein the response is based on the results.

19. A system comprising: one or more processors; and a non-transitory memory coupled to the processors comprising instructions executable by the processors, the processors operable when executing the instructions to:
receive, by an assistant xbot from a client system associated with a user, a user input comprising a natural-language utterance;
determine a semantic representation of the user input based on a structural ontology, wherein the structural ontology defines a labeling syntax for parsing the natural-language utterance to semantic units comprising a plurality of actions, objects, and attributes, and wherein the semantic representation embeds at least one object within at least one action and declares at least one attribute of the at least one embedded object to be acted upon;
send, to an agent, a request for executing a task corresponding to the user input, wherein the request is based on the semantic representation;
receive, from the agent, results generated by the agent from executing the task, wherein the results are ma

Assignees

• Meta Platforms Inc

Inventors

• Aghajanyan Armen
• Gupta Sonal
• Moran Brian
• Levin Theodore Frank
• Nakatsu Crystal Annette Naomi Su Hua
• Difranco Daniel
• Christensen Jonathan David
• Labuda Kirk
• Kumar Anuj

Classifications

• G06Q10/40 Primary

  Business processes related to social networking or social networking services · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/098

  Distributed learning, e.g. federated learning · CPC title

• G06Q10/1093

  Calendar-based scheduling for persons or groups · CPC title

• G10L2015/223

  Execution procedure of a spoken command · CPC title