Control Interface for Robotic Humanoid Avatar System and Related Methods

Patent metadata

Field	Value
Publication number	US-2016046023-A1
Application number	US-201514827701-A
Country	US
Kind code	A1
Filing date	Aug 17, 2015
Priority date	Aug 15, 2014
Publication date	Feb 18, 2016
Grant date	—

Abstract

A system and method for controlling a humanoid robot from a remote location are provided. One or more sensing devices are used to sense positions and movements of a user at an inhabiter station. A computing device generates a virtual character based upon the sensed positions and movements of the user and provides a plurality of outputs for controlling the humanoid robot based upon motion of the virtual character. The computing device includes a master controller to generate the virtual character and its behaviors and to transmit instructions to control motion of the humanoid robot to a slave controller located at the humanoid robot, providing smooth and natural motion of the humanoid robot. The method permits switching between multiple virtual characters in a single scene to control and alter the behavior of one or more humanoid robots located in several geographic locations.

First claim

What is claimed is:

1 . A system for controlling a humanoid robot from a remote location, the system comprising:
one or more sensing devices to sense positions and movements of the user; and a computing device in communication with the one or more sensing devices, the computing device comprising one or more processors and memory, and including computer-executable instructions stored in the memory that, upon execution by the one or more processors, generate a virtual character based upon the sensed positions and movements of the user and provide a plurality of outputs for controlling the humanoid robot based upon motion of the virtual character.

2 . The system of claim 1 , wherein the one or more processors of the computing device include a master controller operative to generate the virtual character and to translate motion of the virtual character into instructions to control motion of the humanoid robot, the master controller in communication with a slave controller located at the humanoid robot to transmit the instructions to the slave controller.

3 . The system of claim 2 , wherein the master controller includes a graphics rendering engine and a physics engine including instructions operative to translate motions of the user into a three-dimensional animation of the virtual character.

4 . The system of claim 1 , wherein the virtual character is generated by creating an animation from the sensed positions and motions of the user.

5 . The system of claim 4 , wherein the animation comprises an interpolation between key frames or between time-positions or a combination thereof.

6 . The system of claim 1 , wherein the computing device is operative to provide a real time mapping between motions of the user and motions of the humanoid robot.

7 . The system of claim 1 , wherein the computing device is operative to determine a torque or force to apply to a joint of the humanoid robot based on one or more accelerations of one or more joints of the virtual character.

8 . The system of claim 1 , wherein a number of degrees of freedom of motion of the virtual character is greater than a number of degrees of freedom of motion of the humanoid robot.

9 . The system of claim 1 , wherein the virtual character has a number of joints, each joint corresponding to a joint of the user, and the computing device is operative to assign a uniform coordinate system to each joint of the virtual character.

10 . The system of claim 9 , wherein the computing device is operative to store in memory data comprising at least one of pitch, yaw, and roll orientation for each joint, the joints including at least one of a shoulder joint, an elbow joint, a wrist joint, a finger joint, a neck joint, a waist joint, a hip joint, a knee joint, and an ankle joint.

11 . The system of claim 1 , wherein the virtual character has a number of joints, each joint corresponding to a joint of the user, and the computing device is operative to store an orientation of each joint of the virtual character as a quaternion, and includes instructions to transform the orientation of each joint of the virtual character to an orientation of each joint of the humanoid robot stored as an Euler angle.

12 . The system of claim 11 , wherein the transform instruction comprises mapping the set of orientations for each joint of the virtual character to the set of orientations for each joint of the humanoid robot.

13 . The system of claim 11 , further comprising constructing a new set of orientations for each joint of the virtual character by applying a rotation operation to each joint.

14 . The system of claim 11 , further comprising mapping multiple joints of the virtual character to a single joint of the humanoid robot.

15 . The system of claim 14 , wherein the mapping comprises generating a reduced set having a reduced number of orientations of the virtual set equal to the number of joints of the humanoid robot, the reduced set including a quaternion equal to a top rotation of a hierarchy of joints and a rotation of a leaf joint of the humanoid robot.

16 . The system of claim 15 , further comprising converting the reduced set of orientations to a converted set of orientations of the joints of the humanoid robot utilizing a Denavit-Hartenberg convention applicable to a corresponding joint of the humanoid robot.

17 . The system of claim 16 , further comprising extracting an Euler angle for the corresponding joint of the humanoid robot.

18 . The system of claim 1 , wherein:
the computing device provides a plurality of selectable micro-poses on the display and receives a selected micro-pose via input from the user; and the computing device causes the humanoid robot to perform the selected micro-pose.

19 . The system of claim 1 , wherein the one or more sensing devices comprise a physical input device coupled to the computing device to input physical motions of the user to the computing device.

20 . The system of claim 19 , wherein the physical input interface comprises a joystick device, a magnetic tracking device, a motion sensing device, an image-based motion recognition device, a depth-based motion recognition device, an infrared (IR) motion sensing device, or combinations thereof.

21 . The system of claim 1 , wherein the one or more sensing devices comprise one or more reflective markers positioned on a head of the user, and an infrared (IR) image sensor coupled to the computing device to determine a head position of the user based upon tracking of the one or more reflective markers.

22 . The system of claim 1 , wherein the computing device controls head motion of the humanoid robot based upon a head position of the user.

23 . The system of claim 1 , wherein the computing device controls arm motion of the humanoid robot based upon an arm position of the user.

24 . The system of claim 1 , wherein the computing device provides corresponding pluralities of outputs for controlling a plurality of humanoid robots in different locations.

25 . The system of claim 24 , wherein the computing device provides a menu stored in the memory and displayable on a display for selecting a current humanoid robot from the plurality thereof.

26 . The system of claim 1 , wherein the computing device provides a plurality of virtual characters configurations stored in memory and receives a selected virtual character configuration from the user.

27 . The system of claim 26 , wherein the computing device is operative to switch between selected virtual characters in real time upon input of an updated virtual character selection from a user.

28 . The system of claim 26 , wherein any of the plurality of virtual characters can be used to control the humanoid robot.

29 . The system of claim 26 , wherein the computing device includes stored in the memory a plurality of motions of the virtual character, each motion representative of a user behavior.

30 . The system of claim 1 , further comprising a display coupled to the computing device to provide a third person view of the virtual character as a representation of a user and a first person view from the at least one humanoid robot.

31 . The system of claim 1 , wherein the display comprises a first display device coupled to the computing device to provide the third person view of the virtual cha

Assignees

• Univ Central Florida Res Found

Inventors

• Nagendran Arjun
• Kelly Brian

Classifications

• G05B2219/40264

  Human like, type robot arm · CPC title

• G05B2219/40116

  Learn by operator observation, symbiosis, show, watch · CPC title

• G05B2219/36442

  Automatically teaching, teach by showing · CPC title

• G05B2219/36436

  Arm follows movement of handheld device, camera detects, analyses motion · CPC title

• B25J9/0081

  with leader teach-in means · CPC title