Robotic system with verbal interaction

Patent metadata

Field	Value
Publication number	US-9403279-B2
Application number	US-201313916863-A
Country	US
Kind code	B2
Filing date	Jun 13, 2013
Priority date	Jun 13, 2013
Publication date	Aug 2, 2016
Grant date	Aug 2, 2016

Abstract

A method and apparatus for moving an object. A verbal instruction for moving the object is received. The verbal instruction is converted into text. A logical representation of the verbal instruction is generated. A movement of a robotic system that corresponds to the verbal instruction for moving the object using a model of an environment in which the object and the robotic system are located is identified. A set of commands used by the robotic system for the movement of the robotic system is identified. The set of commands is sent to the robotic system.

First claim

What is claimed is:

1. An apparatus comprising:
an instruction processor configured to
receive a verbal instruction for moving an object;
determine whether moving the object conflicts with a policy that includes rules that define a type of object that may be moved, a weight limit for the object, and a movement limit that includes one or more of a reach limit of a robotic system or a speed at which the robotic system may move;
in response to a determination that moving the object does not conflict with the policy, identify a movement of the robotic system that corresponds to the verbal instruction for moving the object using a model of an environment in which the object and the robotic system are located;
identify a set of commands used by the robotic system for the movement of the robotic system identified;
send the set of commands to the robotic system; and
identify a movement of the object using a sensor system as the object moves on a conveyor belt;
wherein the movement of the object is used by the instruction processor to identify a resulting movement of the robotic system.

2. The apparatus of claim 1 , wherein the instruction processor comprises:
a dialog manager configured to:
receive the verbal instruction for moving the object;
convert the verbal instruction into text; and
generate a logical representation of the verbal instruction; and

a controller configured to
identify the movement of the robotic system that corresponds to the verbal instruction for moving the object from the model of the environment in which the object and robotic system are located;
identify the set of commands used by the robotic system for the movement of the robotic system identified; and
send the set of commands to the robotic system.

3. The apparatus of claim 2 , wherein the instruction processor further comprises:
a robotic interface configured to place the set of commands into a format used by the robotic system.

4. The apparatus of claim 1 , wherein the model of the environment comprises information about objects in the environment.

5. The apparatus of claim 4 further comprising:
a sensor system configured to generate the information about the environment.

6. The apparatus of claim 4 , wherein the information comprises at least one of an identification of the objects in the environment, name mapping of the objects from first names used by an operator to second names used by the robotic system, position mapping from first positions of the objects used by the operator to second positions used by the robotic system, or descriptions of the objects.

7. The apparatus of claim 1 , wherein the instruction processor is configured to provide feedback on processing of the verbal instruction.

8. The apparatus of claim 1 , wherein the instruction processor is configured to determine whether the verbal instruction is performable and is configured to identify the set of commands used by the robotic system for the movement of the robotic system when the verbal instruction is performable.

9. The apparatus of claim 8 , wherein a controller determines whether the verbal instruction is performable using at least one of the policy, a world model, or a conversation.

10. The apparatus of claim 1 , wherein the verbal instruction is generated by one of a human operator and a computer system.

11. The apparatus of claim 1 , wherein the robotic system is selected from one of a physical robotic system and a simulated robotic system.

12. A pick and place robotic system comprising:
a robotic system configured to move an object from a first location to a second location;
a dialog manager configured to:
receive a verbal instruction for moving the object from the first location to the second location;
convert the verbal instruction into text; and generate a logical representation of the verbal instruction;
compare the logical representation, which is a first logical representation, of the verbal instruction to a second logical representation for a conversation;
determine whether an inconsistency is present between the first logical representation and the second logical representation;
provide first feedback to the operator when the inconsistency is present;
determine whether moving the object conflicts with a policy that includes rules that define a type of object that may be moved, a weight limit for the object, and a movement limit that includes one or more of a reach limit of the robotic system or a speed at which the robotic system may move;

a controller configured to:
identify a movement of the robotic system in response to a determination that the object moving the object does not conflict with the policy, wherein the movement of the robotic system corresponds to the verbal instruction for moving the object from the first location to the second location from a model of an environment in which the object and the robotic system are located;
identify a set of commands used by the robotic system for the movement of the robotic system identified; and
send the set of commands to the robotic system;

a robotic interface configured to:
receive the set of commands from the controller;
place the commands in a format used by the robotic system; and
send the set of commands to the robotic system; and

a sensor system configured to identify a movement of the object using a sensor system as the object moves on a conveyor belt;
wherein the movement of the object is used by the controller to identify a resulting movement of the robotic system.

13. The pick and place robotic system of claim 12 further comprising:
a sensor system configured to generate information about objects in the environment for the model of the environment.

14. The pick and place robotic system of claim 12 , wherein the robotic system is selected from one of a physical robotic system and a simulated robotic system.

15. A method for moving an object, the method comprising:
receiving a verbal instruction for moving the object;
determining, based on the verbal instruction, whether moving the object conflicts with a policy includes rules that define a type of object that may be moved, a weight limit for the object, and a movement limit that includes one or more of a reach limit of a robotic system or a speed at which the robotic system may move;
in response to a determination that moving the object does not conflict with the policy, identifying a movement of the robotic system that corresponds to the verbal instruction for moving the object using a model of an environment in which the object and the robotic system are located;
identifying a set of commands used by the robotic system for the movement of the robotic system identified;
sending the set of commands to the robotic system; and
identifying a movement of the object using a sensor system as the object moves on a conveyor belt;
wherein the movement of the object is used to identify a resulting movement of the robotic system.

16. The method of claim 15 further comprising:
placing the set of commands into a format used by the robotic system.

17. The method of claim 15 , wherein the model of the environment comprises information about objects in the environment and further comprising:
generating the information about the environment using a sensor system.

18. The method of claim 17 , wherein the information comprises at least one of an identification of the objects in the environment, name mapping of the objects from first names used by an operator to second names used by the robotic system, position mapping from f

Assignees

• Boeing Co

Inventors

• Smith Scott D G
• Provine Ronald Carl
• Mendez Mario A

Classifications

• Y10S901/02

  Arm motion controller · CPC title

• G10L15/22

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title

• B25J13/003 Primary

  by means of an audio-responsive input (audible safety signals B25J19/061) · CPC title