System and method for flexible human-machine collaboration

Patent metadata

Field	Value
Publication number	US-9586315-B2
Application number	US-201615001003-A
Country	US
Kind code	B2
Filing date	Jan 19, 2016
Priority date	Sep 2, 2014
Publication date	Mar 7, 2017
Grant date	Mar 7, 2017

Abstract

Methods and systems for enabling human-machine collaborations include a generalizable framework that supports dynamic adaptation and reuse of robotic capability representations and human-machine collaborative behaviors. Specifically, a method of enabling user-robot collaboration includes providing a composition of a robot capability that models a robot's functionality for performing a type of task action and user interaction capabilities; specializing the robot capability with an information kernel to provide a specialized robot capability, the information kernel encapsulating a set of task-related parameters associated with the type of task action; providing an instance of the specialized robot capability as a robot capability element that controls the robot's functionality based on the set of task-related parameters; providing instances of the user interaction capabilities as interaction capability elements; executing the robot capability element to receive user input via the user interaction capability elements; and controlling, based on the user input and the set of task-related parameters, the robot's functionality to perform a task action of the type of task action in collaboration with the user input.

First claim

What is claimed is:

1. A computer-implemented method for enabling generalizable user-robot collaboration, comprising:
providing a composition of a robot capability and one or more user interaction capabilities, wherein the robot capability models at least one functionality of a robot for performing a type of task action;
specializing the robot capability with an information kernel to provide a specialized robot capability, wherein the information kernel encapsulates a set of task-related parameters associated with the type of task action;
providing an instance of the specialized robot capability as a robot capability element that controls the at least one functionality of the robot based on the set of task-related parameters;
providing one or more instances of the one or more user interaction capabilities as one or more interaction capability elements;
executing the robot capability element to receive user input via the one or more user interaction capability elements; and
controlling, based on the user input and the set of task-related parameters, the at least one functionality of the robot to perform at least one task action of the type of task action in collaboration with the user input.

2. The method of claim 1 , further comprising:
providing, via the one or more interaction capability elements, one or more user interfaces (UIs); and
receiving the user input via the one or more UIs.

3. The method of claim 1 , wherein the one or more user interaction capabilities include an admittance control interaction capability, the method further comprising:
providing an instance of the admittance control interaction capability as an admittance control interaction capability element; and
receiving, via the admittance control interaction capability element, user manipulation of the robot as the user input.

4. The method of claim 3 , further comprising:
providing, based on the admittance control interaction capability element, an admittance control instructor interface that receives the user manipulation of the robot as a user demonstrated motion;
determining a start pose of the robot;
acquiring trajectory data of the user demonstrated motion;
generalizing the trajectory data based on the start pose of the robot to form a tool movement primitive (TMP); and
storing the TMP.

5. The method of claim 1 , wherein the type of task action includes motion constraint and the set of task-related parameters in the information kernel includes a set of tool parameters associated with a type of tool, and wherein executing the robot capability element further comprises:
detecting that a tool of the type of tool is attached to the robot; and
constraining, based on the set of tool parameters associated with the type of tool, one or more motions of the robot in performing the at least one task action responsive to the user input, thereby enforcing the motion constraint.

6. The method of claim 5 , wherein constraining the one or more motions of the robot further comprises:
obtaining a start pose of the robot; and
constraining, based on the start pose of the robot and the set of tool parameters associated with the type of tool, the one or more motions of the robot.

7. The method of claim 5 , wherein the set of tool parameters includes a tool geometry and a tool behavior constraint, the method further comprising:
generating, based on the tool geometry and the tool behavior constraint, a tool-constrained workspace of the robot that is a subset of a maximum workspace of the robot; and
constraining the one or more motions of the robot, thereby constraining the robot to the tool-constrained workspace.

8. The method of claim 5 , wherein the set of task-related parameters in the information kernel includes a perceptual template, and wherein constraining the one or more motions of the robot further comprises:
obtaining a set of workpiece characteristics of at least one target workpiece; and
constraining, based on the initial pose of the robot and the set of workpiece characteristics, the one or more motions of the robot.

9. The method of claim 1 , wherein the type of task action includes motion instant replay and the set of task-related parameters in the information kernel includes a TMP, and wherein executing the robot capability element further comprises:
determining, via the one or more user interaction capabilities, a start pose of the robot; and
controlling, based on the start pose of the robot and a tool motion trajectory of the TMP, one or more motions of the robot in performing the at least one task action.

10. The method of claim 9 , wherein the set of task-related parameters in the information kernel includes a perceptual template, and wherein controlling the one or more motions of the robot further comprises:
obtaining a set of workpiece characteristics of at least one workpiece in a workspace of the robot; and
grounding, based on the set of workpiece characteristics, the one or more motions of the robot in performing the at least one task action.

11. The method of claim 1 , wherein the composition of the robot capability and the one or more user interaction capabilities includes one or more links between the robot capability and the one or more user interaction capabilities, the method further comprising:
providing one or more instances of the one or more links as one or more link elements that connect the robot capability element to the one or more user interaction capability elements.

12. The method of claim 11 , wherein providing the one or more instances of the one or more links further comprises:
deriving an UI requirement of the robot capability element based on an interface of the robot capability element, wherein the interface includes at least one of an input interface or an output interface; and
providing, based on the UI requirement, at least one instance of at least one link as at least one link element that connect the robot element to the at least one user interaction capability element.

13. The method of claim 1 , wherein executing the robot capability element further comprises:
performing a runtime evaluation of the set of task-related parameters in the information kernel during runtime; and
performing, based on the runtime evaluation of the set of task-related parameters, the at least one task action in collaboration with the user input.

14. The method of claim 1 , further comprising:
providing a collaborative behavior that includes the composition of the robot capability and the one or more user interaction capabilities;
selecting a plurality of robot capabilities based on a robot capability composition requirement of the collaborative behavior;
composing the plurality of robot capabilities into a composition of robot capabilities as the robot capability; and
associating the composition of robot capabilities with the collaborative behavior.

15. The method of claim 14 , wherein composing the plurality of robot capabilities further comprises:
composing two robot capabilities of the plurality of robot capabilities using at least one of a parallel composition operator or a serial composition operator.

16. The method of claim 14 , further comprising:
deriving a plurality of UI requirements of the composition of robot capabilities by performing a compatibility function on the composition of robot capabilities; and
determining at least one user interaction capability that meets the plurality of UI requirements; and
generating a mapping between the composition of robot capabilities and the at least one user interaction capability based on the pluralit

Assignees

• Univ Johns Hopkins

Inventors

• Guerin Kelleher
• Hager Gregory D
• Riedel Sebastian

Classifications

• G05B2219/40391

  Human to robot skill transfer · CPC title

• G05B2219/36422

  During teaching shut off, disable motor to move arm easy · CPC title

• G05B2219/33051

  BBC behavior based control, stand alone module, cognitive, independent agent · CPC title

• B25J9/1661

  characterised by task planning, object-oriented languages · CPC title

• B25J9/163

  learning, adaptive, model based, rule based expert control · CPC title