Systems, devices, and methods for the distribution and collection of multimodal data associated with robots

Patent metadata

Field	Value
Publication number	US-10994417-B2
Application number	US-201916593736-A
Country	US
Kind code	B2
Filing date	Oct 4, 2019
Priority date	Dec 1, 2015
Publication date	May 4, 2021
Grant date	May 4, 2021

Abstract

A dynamic representation of a robot in an environment is produced, one or more observer agent collects data, and respective values of one or more metrics for the robot are computed based at least in part on the collected data. Tasks for the robot to perform are generated. Ratings and challenge questions are generated. A server may produce a user interface and a value of a metric based on collected observer data.

First claim

The invention claimed is:

1. A method of assessing qualities of a robot, comprising:
causing, by a processor, a robot to perform a task in an environment;
capturing, by a camera coupled to the processor, media data of the robot performing the task in the environment, the media data including a video feed;
generating, by the processor, a dynamic representation of the robot performing the task in the environment from the captured media data;
providing, by the processor, the dynamic representation of the robot performing the task in the environment to an observer agent;
requesting, by the processor, a judged level of behavior of the robot, to be input by the observer agent;
receiving the judged level of behavior of the robot from the observer agent, wherein the judged level of behavior of the robot is determined by the observer agent observing the dynamic representation of the robot performing the task in the environment; and
training a machine learning model using the judged level of behavior of the robot.

2. The method of claim 1 , wherein the captured media data includes an audio feed.

3. The method of claim 1 , further comprising, distributing, by the processor, an interface for the observer agent.

4. The method of claim 3 , wherein the interface is configured to accept input from a touch display, a mouse, a keyboard, or a virtual reality device.

5. The method of claim 1 , further comprising, receiving by the processor, from the observer agent, a score derived from the judged level of behavior.

6. The method of claim 1 , wherein the observer agent considers at least one of an intelligence, motion, or action to determine the judged level of behavior.

7. The method of claim 1 , wherein the task is a grasping maneuver.

8. A method of assessing qualities of a robot, comprising:
causing, by a processor, a robot to perform a task in an environment;
capturing, by a camera coupled to the processor, media data of the robot performing the task in the environment, the media data including a video feed;
generating, by the processor, a dynamic representation of the robot performing the task in the environment from the captured media data;
providing, by the processor, the dynamic representation of the robot performing the task in the environment to an observer agent;
generating, by the processor, a question for the observer agent, wherein the question relates to the appropriateness of the robot's behavior in performing the task;
receiving, by the processor, an answer to the question from the observer agent, wherein the answer is determined by the observer agent observing the dynamic representation of the robot performing the task in the environment; and
training a machine learning model using the received answer.

9. The method of claim 8 , wherein the captured media data includes an audio feed.

10. The method of claim 8 , further comprising, distributing, by the processor, an interface for the observer agent.

11. The method of claim 10 , wherein the interface is configured to accept input from a touch display, a mouse, a keyboard, or a virtual reality device.

12. The method of claim 8 , wherein the appropriateness of the robot's behavior is determined in relation to at least one of the environment and the task.

13. The method of claim 8 , wherein the task is a grasping maneuver.

14. A system of assessing qualities of a robot, comprising:
a robot configured to perform a task in an environment;
a camera configured to capture media data of the robot performing the task in the environment, the media data including video data;
a processor communicatively coupled to the robot and the camera, the processor configured to generate a dynamic representation of the robot performing the task in the environment from the captured media data;
an observer agent communicatively coupled to the processor, the observer agent configured to receive the dynamic representation from the processor; and
an interface coupled to the observer agent, wherein the interface is configured to display a request to input a judged level of behavior of the robot by the observer agent,
wherein the judged level of behavior of the robot is determined by the observer agent observing the dynamic representation of the robot performing the task in the environment, and wherein the judged level of behavior of the robot is used to train a machine learning model.

15. The system of claim 14 , wherein the captured media data includes an audio feed.

16. The system of claim 14 , wherein the interface is a coupled to a web service.

17. The system of claim 14 , wherein the interface is configured to accept input from a touch display, a mouse, a keyboard, or a virtual reality device.

18. The system of claim 14 , wherein the judged level of behavior is related to a level of friendliness of the robot.

19. The system of claim 14 , wherein the observer agent considers at least one of an intelligence, motion, or action to determine the judged level of behavior.

20. The system of claim 14 , wherein the task is a grasping maneuver.

Assignees

• Kindred Sys Inc

Inventors

• Bergstra James Sterling
• Gildert Suzanne
• Rose George Samuel

Classifications

• B25J9/1656 Primary

  characterised by programming, planning systems for manipulators · CPC title

• G05B2219/40405

  Leader-follower position control · CPC title

• G05B2219/39354

  Operation, work space observer · CPC title