Simulated local demonstration data for robotic demonstration learning

Patent metadata

Field	Value
Publication number	US-11820014-B2
Application number	US-202016880869-A
Country	US
Kind code	B2
Filing date	May 21, 2020
Priority date	May 21, 2020
Publication date	Nov 21, 2023
Grant date	Nov 21, 2023

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using simulated local demonstration data for robotic demonstration learning. One of the methods includes receiving perceptual data of a workcell of a robot to be configured to execute a task according to a skill template, wherein the skill template specifies one or more subtasks required to perform the skill, wherein at least one of the subtasks is a demonstration subtask that relies on learning visual characteristics of the workcell. A virtual model is generated of a portion of the workcell. A training system generates simulated local demonstration data from the virtual model of the portion of the workcell and tunes a base control policy for the demonstration subtask using the simulated local demonstration data generated from the virtual model of the portion of the workcell.

First claim

What is claimed is:

1. A computer-implemented method comprising:
receiving perceptual data of a workcell at a first site, the workcell having a physical robot to be configured to execute a task according to a skill template, wherein the skill template specifies a plurality of subtasks required to perform a skill, wherein the plurality of subtasks includes a nondemonstration subtask and at least one demonstration subtask that relies on force feedback and visual characteristics of the workcell, and wherein the demonstration subtask is associated with a pretrained base control policy that was previously trained using at least one different robot located at a different site;
generating actual local demonstration data from a workcell demonstration of the demonstration subtask by operating the physical robot in the workcell while capturing sensor data generated by a plurality of sensors in the workcell of the physical robot, wherein the actual local demonstration data comprises task state representations generated at a base update rate corresponding to a rate at which the physical robot receives commands for execution, and
wherein generating each task state representation comprises processing sensor output of each sensor of the plurality of sensors by a different respective neural network of a plurality of neural networks to generate, by each neural network, a different portion of the task state representation, wherein each portion of the task state representation is generated at a different rate, and wherein generating the task state representation comprises reading, at the base update rate, the most recently updated output of each of the plurality of neural networks processing the sensor data, wherein the actual local demonstration data includes robot state data representing force sensor data obtained during a workcell demonstration of the demonstration subtask;
generating, from the perceptual data, a virtual model of a portion of the workcell;
performing a plurality of simulated training iterations using a virtual robot in the virtual model of the portion of the workcell to generate, by a training system, simulated local demonstration data from the virtual model of the portion of the workcell, including:
simulating respective outputs of the plurality of sensors in the workcell capturing data at different respective rates;
generating, at each time step corresponding to the base update rate, a respective simulated task state representation using the simulated outputs of the plurality of sensors capturing data at different respective rates, including processing the simulated sensor output of each sensor of the plurality of sensors by a different respective neural network of the plurality of neural networks used to generate the actual local demonstration data to generate, by each respective neural network, a different portion of the simulated task state representation, wherein each portion of the simulated task state representation is generated at a different rate, and wherein generating the simulated task state representation comprises obtaining, at the base update rate, the most recently updated output of each of the plurality of neural networks processing the simulated sensor data; and
providing, to a current control policy subsystem at each time step corresponding to the base update rate, a most recently generated simulated task state representation having portions generated by different respective neural networks processing simulated sensor data at different rates to generate one or more commands to be performed by the virtual robot, and
updating the simulated outputs of the plurality of sensors according to the one or more commands generated by the current control policy subsystem; and

tuning the current control policy subsystem for the demonstration subtask using the simulated local demonstration data generated from the virtual model of the portion of the workcell.

2. The method of claim 1 , wherein tuning the base control policy at least partially overlaps with a user collecting the actual local demonstration data from the robot.

3. The method of claim 1 , wherein the perceptual data comprises a camera image, depth camera data, lidar scan data, or laser scan data of the workcell.

4. The method of claim 1 , wherein generating the virtual model of the workcell comprises generating a plurality of different perspectives of a target of the demonstration subtask defined in the skill template.

5. The method of claim 1 , wherein generating the virtual model of the workcell comprises generating a three-dimensional representation of a portion of the workcell represented by the perceptual data.

6. The method of claim 1 , wherein the robot state data of the actual local demonstration data includes joint data of the robot representing joint angles of the robot during the workcell demonstration.

7. A system comprising:
one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
receiving perceptual data of a workcell at a first site, the workcell having a physical robot to be configured to execute a task according to a skill template, wherein the skill template specifies a plurality of subtasks required to perform a skill, wherein the plurality of subtasks includes a nondemonstration subtask and at least one demonstration subtask that relies on force feedback and visual characteristics of the workcell, and wherein the demonstration subtask is associated with a pretrained base control policy that was previously trained using at least one different robot located at a different site;
generating actual local demonstration data from a workcell demonstration of the demonstration subtask by operating the physical robot in the workcell while capturing sensor data generated by a plurality of sensors in the workcell of the physical robot, wherein the actual local demonstration data comprises task state representations generated at a base update rate corresponding to a rate at which the physical robot receives commands for execution, and
wherein generating each task state representation comprises processing sensor output of each sensor of the plurality of sensors by a different respective neural network of a plurality of neural networks to generate, by each neural network, a different portion of the task state representation, wherein each portion of the task state representation is generated at a different rate, and wherein generating the task state representation comprises reading, at the base update rate, the most recently updated output of each of the plurality of neural networks processing the sensor data, wherein the actual local demonstration data includes robot state data representing force sensor data obtained during a workcell demonstration of the demonstration subtask;
generating, from the perceptual data, a virtual model of a portion of the workcell;
performing a plurality of simulated training iterations using a virtual robot in the virtual model of the portion of the workcell to generate, by a training system, simulated local demonstration data from the virtual model of the portion of the workcell including:
simulating respective outputs of the plurality of sensors in the workcell capturing data at different respective rates;
generating, at each time step corresponding to the base update rate, a respective simulated task state representation using the simulated outputs of the plurality of sensors capturing data at different respective rates, including processing the simulated sensor output of each sensor of the plurality of sensors by a different respective neural network of the plurality of neural networks used to generate the actual local d

Assignees

• Intrinsic Innovation Llc

Inventors

• Kolluri Bala Venkata Sai Ravi Krishna
• Schaal Stefan
• Schönherr Ralf Oliver Michael
• Davis Benjamin M
• Ye Ning

Classifications

• B25J9/163 Primary

  learning, adaptive, model based, rule based expert control · CPC title

• B25J9/1605

  Simulation of manipulator lay-out, design, modelling of manipulator · CPC title

• B25J9/1617

  Cellular, reconfigurable manipulator, e.g. cebot · CPC title

• B25J9/161

  Hardware, e.g. neural networks, fuzzy logic, interfaces, processor · CPC title

• B25J9/1661

  characterised by task planning, object-oriented languages · CPC title