Natural language unification based robotic agent control

Patent metadata

Field	Value
Publication number	US-11062142-B2
Application number	US-201816020566-A
Country	US
Kind code	B2
Filing date	Jun 27, 2018
Priority date	Jun 29, 2017
Publication date	Jul 13, 2021
Grant date	Jul 13, 2021

Abstract

In some examples, natural language unification based robotic agent control may include ascertaining, by a robotic agent, an image of an object or an environment, and ascertaining a plurality of natural language insights for the image. A semantic relatedness may be determined between each insight of the plurality of insights, and a semantic relatedness graph may be generated for the plurality of insights. For each insight of the plurality of insights, at least one central concept may be identified. Based on the semantic relatedness graph and the identified at least one central concept, the plurality of insights may be clustered to generate at least one insights cluster. For insights included in the least one insights cluster, a unified insight may be generated. Further, an operation associated with the robotic agent, the object, or the environment may be controlled by the robotic agent and based on the unified insight.

First claim

What is claimed is:

1. A natural language unification based robotic agent control apparatus comprising:
an insight analyzer, executed by at least one hardware processor, to
ascertain, by a robotic agent, an image of an object or an environment, and
ascertain a plurality of natural language insights for the image;

a semantic relatedness analyzer, executed by the at least one hardware processor, to
determine semantic relatedness between each insight of the plurality of insights;

a semantic relatedness graph generator, executed by the at least one hardware processor, to
generate, based on the determined semantic relatedness, a semantic relatedness graph for the plurality of insights;

a central concepts identifier, executed by the at least one hardware processor, to
identify, for each insight of the plurality of insights, at least one central concept;

an insights cluster generator, executed by the at least one hardware processor, to
cluster, based on the semantic relatedness graph and the identified at least one central concept, the plurality of insights to generate at least one insights cluster;

a unified insights generator, executed by the at least one hardware processor, to
generate, for insights included in the least one insights cluster, a unified insight;

an eminence score generator, executed by the at least one hardware processor, to
generate an eminence score for each unified insight of a plurality of unified insights including the unified insight; and

a robotic agent controller, executed by the at least one hardware processor, to
control, by the robotic agent and based on the eminence score and the unified insight, an operation associated with the robotic agent, the object, or the environment.

2. The apparatus according to claim 1 , wherein the semantic relatedness analyzer is to determine semantic relatedness between each insight of the plurality of insights by:
determining, based on the determined semantic relatedness, whether an insight of the plurality of insights is a duplicate of another insight of the plurality of insights; and
based on a determination that the insight of the plurality of insights is the duplicate of the other insight of the plurality of insights, removing the insight of the plurality of insights to generate a set of non-redundant insights.

3. The apparatus according to claim 1 , wherein the unified insights generator is to generate, for the insights included in the least one insights cluster, the unified insight by:
analyzing, for the insights included in the at least one insights cluster,
dependency relationships between the identified at least one central concept,
semantic relatedness relationships between the identified at least one central concept, and
ontological relationships between the identified at least one central concept; and

generating, based on the dependency relationships, the semantic relatedness relationships, and the ontological relationships, the unified insight.

4. The apparatus according to claim 3 , wherein the unified insights generator is to generate, for the insights included in the least one insights cluster, the unified insight by:
extracting, for the insights included in the least one insights cluster, subject, predicate, and object tuples; and
generating, based on the extraction of the subject, predicate, and object tuples, the unified insight.

5. The apparatus according to claim 4 , wherein the unified insights generator is to generate, for the insights included in the least one insights cluster, the unified insight by:
merging the extracted subject, predicate, and object tuples; and
generating, based on the merged subject, predicate, and object tuples, the unified insight.

6. The apparatus according to claim 5 , wherein the unified insights generator is to extract, for the insights included in the least one insights cluster, subject, predicate, and object tuples, and merge the extracted subject, predicate, and object tuples by:
generating dependency parse trees for the insights included in the least one insights cluster; and
merging, based on the dependency parse trees, the extracted subject, predicate, and object tuples.

7. The apparatus according to claim 1 , wherein the semantic relatedness analyzer is to determine semantic relatedness between each insight of the plurality of insights by:
identifying terms of an insight; and
determining, for each term of the identified terms, a relevance to all other terms of the insight.

8. The apparatus according to claim 1 ,
wherein the eminence score generator is to
rank each unified insight of the plurality of unified insights according to the eminence scores, and

wherein the robotic agent controller is to control, by the robotic agent and based on the unified insight, the operation associated with the robotic agent, the object, or the environment by:
controlling, by the robotic agent and based on a highest ranked unified insight, the operation associated with the robotic agent, the object, or the environment.

9. The apparatus according to claim 1 , wherein the unified insights generator is to generate, for the insights included in the least one insights cluster, the unified insight by:
identifying, for the insights included in the at least one insights cluster, an insight including a highest number of concept terms;
designating the insight including the highest number of concept terms as a base insight; and
expanding the base insight to generate the unified insight.

10. The apparatus according to claim 1 , wherein the unified insights generator is to generate, for the insights included in the least one insights cluster, the unified insight by:
determining, for each of the insights included in the at least one insights cluster, subject, predicate, and object tuples;
generating a semantic relatedness graph for predicates of the determined subject, predicate, and object tuples;
determining, for the semantic relatedness graph generated for the predicates of the determined subject, predicate, and object tuples, whether an edge includes a weight that is less than a specified weight; and
based on a determination that the edge includes the weight that is less than the specified weight, removing the edge with respect to the unified insight.

11. The apparatus according to claim 1 , wherein the unified insights generator is to generate, for the insights included in the least one insights cluster, the unified insight by:
determining, for each of the insights included in the at least one insights cluster, subject, predicate, and object tuples;
generating a semantic relatedness graph for predicates of the determined subject, predicate, and object tuples;
determining, for the semantic relatedness graph generated for the predicates of the determined subject, predicate, and object tuples, whether an edge includes a weight that is greater than a specified weight; and
based on a determination that the edge includes the weight that is greater than the specified weight, utilizing the edge to generate the unified insight.

12. A method for natural language unification based robotic agent control, the method comprising:
ascertaining, by a robotic agent, an image of an object or an environment;
ascertaining, by at least one processor, a plurality of natural language insights for the image;
determining, by the at least one processor, semantic relatedness between each insight of the plurality of insights;
generating, by the at least one processor, based on the determined semantic relatedness, a semantic relatedness graph for the plurality of insights;
identifying, by the at least one processor, for each insigh

Assignees

• Accenture Global Solutions Ltd
• Accenture Gobal Solutions Ltd

Inventors

• Misra Janardan
• Podder Sanjay
• Rawat Divya
• Ghosh Bhaskar
• DUBASH Neville

Classifications

• G06F40/30 Primary

  Semantic analysis · CPC title

• G06V20/35 Primary

  Categorising the entire scene, e.g. birthday party or wedding scene · CPC title

• G06V20/10

  Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) · CPC title

• G06V30/274

  Syntactic or semantic context, e.g. balancing · CPC title

• G06V20/20

  in augmented reality scenes · CPC title