Method and system for training a robot

Patent metadata

Field	Value
Publication number	US-12427660-B2
Application number	US-202118042490-A
Country	US
Kind code	B2
Filing date	Aug 12, 2021
Priority date	Aug 24, 2020
Publication date	Sep 30, 2025
Grant date	Sep 30, 2025

Abstract

A method and system are provided for training a robot to recognize objects in the workspace of the robot. Objects in the workspace are identified by the user, and the robot determines candidate objects. Feedback may be used in order for the user to confirm whether the candidate object determined by the robot system matches the object intended by the user. Gripping information for the object may also be identified by the user to train the robot how to grip the object.

First claim

The invention claimed is:

1. A method of training a robot system by a user, comprising:
determining a candidate object by the robot system within a workspace of a robot;
displaying the candidate object to the user;
providing a first identifying input to the robot system, the first identifying input being indicative of whether the candidate object matches an intended object;
providing a second identifying input to the robot system, the second identifying input being indicative of a gripping location on the intended object; and
displaying the gripping location, including accuracy information, to the user.

2. The method according to claim 1 , further comprising:
assigning a name to the candidate object by the user;
storing image data of the candidate object and the name in a memory of the robot system;
cross-referencing future candidate objects with the image data; and displaying the name with the future candidate objects which correspond to the image data.

3. The method according to claim 1 , wherein the candidate object is displayed with highlighting or a boundary line around the candidate object on a display screen.

4. The method according to claim 1 , further comprising providing a third identifying input to the robot system, the third identifying input being indicative of a region within the workspace of the robot including the intended object, wherein the robot system determines the candidate object from the region as likely corresponding to the intended object.

5. The method according to claim 1 , wherein the candidate object is displayed within an image of the workspace on a touch screen, and the user identifies the intended object by touching the touch screen.

6. The method according to claim 1 , wherein the user identifies the intended object by gesturing within the workspace.

7. The method according to claim 6 , wherein the gesturing comprises the user positioning a hand or finger within the workspace, the robot system determining a location and/or orientation of the hand or finger to determine the candidate object.

8. The method according to claim 6 , wherein the gesturing comprises the user positioning a wand within the workspace, the robot system determining a location and/or orientation of the wand to determine the candidate object.

9. The method according to claim 8 , wherein the wand is physically unattached to the robot system.

10. The method according to claim 8 , wherein the robot system visually tracks the location and/or orientation of the hand or finger or wand.

11. The method according to claim 8 , wherein the wand comprises an IMU and wirelessly transmits the location and/or orientation to the robot system.

12. The method according to claim 1 , wherein the user identifies the intended object with a word.

13. The method according to claim 8 , further comprising providing a third identifying input to the robot system, the third identifying input being indicative of a gripping force to be applied to the intended object.

14. The method according to claim 2 , wherein the candidate object is displayed with highlighting or a boundary line around the candidate object on a display screen.

15. The method according to claim 2 , further comprising identifying to the robot system by the user a region within the workspace of the robot including the intended object, wherein the robot system determines the candidate object from the region as likely corresponding to the intended object.

16. The method according to claim 2 , wherein the candidate object is displayed within an image of the workspace on a touch screen, and the user identifies the intended object by touching the touch screen.

17. The method according to claim 2 , wherein the user identifies the intended object by gesturing within the workspace.

18. The method according to claim 7 , wherein the robot system visually tracks the location and/or orientation of the hand or finger.

19. The method according to claim 1 , further comprising:
in response to receiving confirmation that the gripping location is correct, training the robot system to recognize the gripping location for use when gripping the candidate object.

Assignees

• Abb Schweiz Ag
• Abb Schweiz

Inventors

• Schoenborn Sandro
• Dan Andrei
• Birke Robert René Maria
• Sommer Philipp
• Locher Thomas

Classifications

• B25J9/1697

  Vision controlled systems · CPC title

• B25J9/1656

  characterised by programming, planning systems for manipulators · CPC title

• B25J9/163 Primary

  learning, adaptive, model based, rule based expert control · CPC title