System and method for semantic processing of natural language commands

Patent metadata

Field	Value
Publication number	US-9764477-B2
Application number	US-201414557005-A
Country	US
Kind code	B2
Filing date	Dec 1, 2014
Priority date	Dec 1, 2014
Publication date	Sep 19, 2017
Grant date	Sep 19, 2017

Abstract

A system, method and computer-readable storage devices are for processing natural language commands, such as commands to a robotic arm, using a Tag & Parse approach to semantic parsing. The system first assigns semantic tags to each word in a sentence and then parses the tag sequence into a semantic tree. The system can use statistical approach for tagging, parsing, and reference resolution. Each stage can produce multiple hypotheses, which are re-ranked using spatial validation. Then the system selects a most likely hypothesis after spatial validation, and generates or outputs a command. In the case of a robotic arm, the command is output in Robot Control Language (RCL).

First claim

We claim:

1. A method comprising:
receiving a natural language command, the natural language command addressed to a robotic arm to instruct the robotic arm to perform a function;
assigning, via a sequence tagger, a part of speech, a semantic tag and a label to each word in the natural language command to yield a tagged natural language command;
semantically parsing, via a processor and a semantic parsor, the tagged natural language command to yield a parsed natural language command, wherein a data set used to train the semantic parsor does not include any tag from which the sequence tagger selects for assigning the semantic tag;
identifying a command type for the parsed natural language command; and
moving the robotic arm according to the command type and according to a spatial validation of a physical context of the natural language command when applied to the robotic arm.

2. The method of claim 1 , wherein the semantic tag identifies entity types and event types in the natural language command.

3. The method of claim 1 , further comprising, after identifying the command type:
Performing the spatial validation, based on the command type, for the physical context of the natural language command when applied to the robotic arm; and
if the spatial validation indicates that a certainty of the command type meets a certainty threshold, executing the natural language command; and
if the spatial validation indicates that the certainty of the command type does not meet the certainty threshold, prompting for clarification of the natural language command.

4. The method of claim 3 , wherein the spatial validation is based on a condition of a target area associated with the natural language command.

5. The method of claim 4 , wherein the target area comprises a working area of a robotic arm, and wherein the condition of the target area comprises presence and positions of objects in the working area.

6. The method of claim 5 , further comprising:
performing reference resolution on the natural language command to uniquely identify one of the objects in the working area.

7. The method of claim 1 , wherein the natural language command is directed to one of a physical robotic appendage or a virtual robotic appendage.

8. The method of claim 7 , further comprising outputting the parsed natural language command according to the command type and according to a robot control language.

9. The method of claim 1 , wherein the semantic tag is assigned using a maximum entropy sequence tagger.

10. The method of claim 1 , wherein a semantic parser trained on a non-lexical semantic tree parses the tagged natural language command.

11. A system comprising:
a processor; and
a computer-readable storage medium having instructions stored which, when executed by the processor, cause the processor to perform operations comprising:
receiving a natural language command, the natural language command addressed to a robotic arm to instruct the robotic arm to perform a function;
assigning, via a sequence tagger, a part of speech, a semantic tag and a label to each word in the natural language command to yield a tagged natural language command;
semantically parsing, via a semantic parsor, the tagged natural language command to yield a parsed natural language command, wherein a data set used to train the semantic parsor does not include any tag from which the sequence tagger selects for assigning the semantic tag;
identifying a command type for the parsed natural language command; and
moving the robotic arm according to the command type and according to a spatial validation of a physical context of the natural language command when applied to the robotic arm.

12. The system of claim 11 , wherein the semantic tag identifies entity types and event types in an input command.

13. The system of claim 11 , the computer-readable storage medium having additional instructions stored which, when executed by the processor, cause the processor to perform operations comprising, after identifying the command type:
performing the spatial validation, based on the command type, for the physical context of the natural language command when applied to the robotic arm; and
if the spatial validation indicates that a certainty of the command type meets a certainty threshold, executing the natural language command; and
if the spatial validation indicates that the certainty of the command type does not meet the certainty threshold, prompting for clarification of the natural language command.

14. The system of claim 13 , wherein the spatial validation is based on a condition of a target area associated with the natural language command.

15. The system of claim 14 , wherein the target area comprises a working area of a robotic arm, and wherein the condition of the target area comprises presence and positions of objects in the working area.

16. The system of claim 15 , the computer-readable storage medium having additional instructions stored which, when executed by the processor, cause the processor to perform operations comprising:
performing reference resolution on the natural language command to uniquely identify one of the objects in the working area.

17. A non-transitory computer-readable storage device having instructions stored which, when executed by a computing device, cause the computing device to perform operations comprising:
receiving a natural language command, the natural language command addressed to a robotic arm to instruct the robotic arm to perform a function;
assigning, via a sequence tagger, a part of speech, a semantic tag and a label to each word in the natural language command to yield a tagged natural language command;
semantically parsing, and a semantic parsor, the tagged natural language command to yield a parsed natural language command, wherein a data set used to train the semantic parsor does not include any tag from which the sequence tagger selects for assigning the semantic tag;
identifying a command type for the parsed natural language command; and
moving the robotic arm according to the command type and according to a spatial validation of a physical context of the natural language command when applied to the robotic arm.

18. The non-transitory computer-readable storage device of claim 17 , wherein the natural language command is directed to one of a physical robotic appendage or a virtual robotic appendage.

19. The non-transitory computer-readable storage device of claim 18 , having additional instructions stored which, when executed by the computing device, cause the computing device to perform operations comprising:
outputting the parsed natural language command according to the command type and according to a robot control language.

20. The non-transitory computer-readable storage device of claim 17 , wherein the semantic tag is assigned using a maximum entropy sequence tagger.

Assignees

• At & T Ip I Lp

Inventors

• Stoyanchev Svetlana
• Bangalore Srinivas
• Chen John
• Jung Hyuckchul

Classifications

• G06F40/30

  Semantic analysis · CPC title

• B25J13/003 Primary

  by means of an audio-responsive input (audible safety signals B25J19/061) · CPC title

• G06F17/2785

  Physics · mapped topic