Semantic Representations Using Structural Ontology for Assistant Systems

Patent metadata

Field	Value
Publication number	US-2021117624-A1
Application number	US-202016998423-A
Country	US
Kind code	A1
Filing date	Aug 20, 2020
Priority date	Oct 18, 2019
Publication date	Apr 22, 2021
Grant date	—

Abstract

In one embodiment, a method includes receiving a user input comprising a natural-language utterance by an assistant xbot from a client system associated with a user, determining a semantic representation of the user input based on a structural ontology defining a labeling syntax for parsing the natural-language utterance to semantic units comprising actions, objects, and attributes, wherein the semantic representation embeds at least one object within at least one action and declares at least one attribute of the embedded object to be acted upon, sending a request based on the semantic representation to an agent for executing a task corresponding to the user input, receiving results of the executed task mapped to a structure determined by the structural ontology from the agent, and sending from the assistant xbot to the client system instructions for presenting a response based on the results of the executed task.

First claim

What is claimed is:

1 . A method comprising, by one or more computing systems:
receiving, by an assistant xbot from a client system associated with a user, a user input comprising a natural-language utterance; determining a semantic representation of the user input based on a structural ontology, wherein the structural ontology defines a labeling syntax for parsing the natural-language utterance to semantic units comprising a plurality of actions, objects, and attributes, and wherein the semantic representation embeds at least one object within at least one action and declares at least one attribute of the embedded object to be acted upon; sending, to an agent, a request for executing a task corresponding to the user input, wherein the request is based on the semantic representation; receiving, from the agent, results of the executed task, wherein the results are mapped to a structure determined by the structural ontology; and sending, from the assistant xbot to the client system, instructions for presenting a response to the user input, wherein the response is based on the results of the executed task.

2 . The method of claim 1 , wherein the structural ontology defines that each of the plurality of actions operates on one or more of the plurality of objects.

3 . The method of claim 1 , wherein the structural ontology defines that each of the plurality of objects comprises one or more of the plurality of attributes.

4 . The method of claim 1 , wherein the structural ontology defines that each of the plurality of attributes declares a restriction on an action or object.

5 . The method of claim 1 , wherein the semantic representation is executable by each module associated with the assistant xbot configured to process the user input.

6 . The method of claim 1 , wherein the structural ontology defines that the plurality of objects are hierarchically organized into a plurality of super-types and sub-types, each sub-type inheriting one or more attributes of its associated super-type.

7 . The method of claim 1 , wherein a value associated with each of the at least one action, the at least one object, and the at least one attribute is determined by each module executing the semantic representation.

8 . The method of claim 1 , further comprising:
generating, by a natural-language generation module associated with the assistant xbot, the response based on the structural ontology and the results of the executed task.

9 . The method of claim 1 , wherein the structural ontology defines that the semantic units further comprise a plurality of methods and enums, wherein each of the plurality of methods comprises an action with its lifetime tied to an object, and wherein each of the plurality of enums comprises an object representing one out of a plurality of objects.

10 . The method of claim 1 , wherein the structural ontology further defines a graph structure comprising one or more core sub-graphs and one or more generic sub-graphs, wherein the one or more core sub-graphs are not accessible by third-party agents, and wherein the one or more generic sub-graphs are accessible by the third-party agents.

11 . The method of claim 1 , further comprising:
resolving, by a reasoning module associated with the assistant xbot, one or more entities associated with the at least one object based on the at least one attribute of the semantic representation.

12 . The method of claim 1 , wherein the agent is a first-party agent associated with the assistant xbot.

13 . The method of claim 1 , wherein the agent is a third-party agent associated with an external computing system, and wherein the structure of the execution results comprises one or more customized semantic units.

14 . The method of claim 13 , wherein sending the request to the agent and receiving the execution results from the agent are both via an application programming interface (API), wherein the API defines a mapping from the one or more customized semantic units to the structure determined by the structural ontology.

15 . The method of claim 13 , wherein the request sent to the agent comprises the semantic representation as modified by a dialog manager associated with the assistant xbot, wherein the execution results comprise the semantic representation as modified by the agent, and wherein the request and the execution results use the labeling syntax of the structural ontology.

16 . The method of claim 13 , wherein the semantic units are each associated with a default namespace defined by the computing systems, and wherein the one or more customized semantic units are associated with a personalized namespace defined by the third-party agent.

17 . The method of claim 1 , wherein the at least one action has one or more arguments, wherein each argument specifies a constraint for an object to satisfy such that the at least one action can act upon the object, and wherein at least one of the arguments specifies a constraint that the at least one object satisfies.

18 . One or more computer-readable non-transitory storage media embodying software that is operable when executed to:
receive, by an assistant xbot from a client system associated with a user, a user input comprising a natural-language utterance; determine a semantic representation of the user input based on a structural ontology, wherein the structural ontology defines a labeling syntax for parsing the natural-language utterance to semantic units comprising a plurality of actions, objects, and attributes, and wherein the semantic representation embeds at least one object within at least one action and declares at least one attribute of the embedded object to be acted upon; send, to an agent, a request for executing a task corresponding to the user input, wherein the request is based on the semantic representation; receive, from the agent, results of the executed task, wherein the results are mapped to a structure determined by the structural ontology; and send, from the assistant xbot to the client system, instructions for presenting a response to the user input, wherein the response is based on the results of the executed task.

19 . A system comprising: one or more processors; and a non-transitory memory coupled to the processors comprising instructions executable by the processors, the processors operable when executing the instructions to:
receive, by an assistant xbot from a client system associated with a user, a user input comprising a natural-language utterance; determine a semantic representation of the user input based on a structural ontology, wherein the structural ontology defines a labeling syntax for parsing the natural-language utterance to semantic units comprising a plurality of actions, objects, and attributes, and wherein the semantic representation embeds at least one object within at least one action and declares at least one attribute of the embedded object to be acted upon; send, to an agent, a request for executing a task corresponding to the user input, wherein the request is based on the semantic representation; receive, from the agent, results of the executed task, wherein the results are mapped to a structure determined by the structural ontology; and send, from the assistant xbot to the client system, instructions for presenting a response to the user input, wherein the response is based on the results of the executed task.

Assignees

• Facebook Inc

Inventors

• Aghajanyan Armen
• Gupta Sonal
• Moran Brian
• Levin Theodore Frank
• Nakatsu Crystal Annette Naomi Su Hua
• Difranco Daniel
• Christensen Jonathan David
• Labuda Kirk
• Kumar Anuj

Classifications

• G06Q10/40 Primary

  Business processes related to social networking or social networking services · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06V20/41

  Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49) · CPC title

• G06V40/174

  Facial expression recognition · CPC title