Bare hand robot path teaching

Patent metadata

Field	Value
Publication number	US-10406686-B2
Application number	US-201314652111-A
Country	US
Kind code	B2
Filing date	Dec 13, 2013
Priority date	Dec 14, 2012
Publication date	Sep 10, 2019
Grant date	Sep 10, 2019

Abstract

A system for generating instructions for operating a robot to perform work on a workpiece has a 3D model or models of a robot scene that provides robot scene data. The system also has data of one or both hands of a gesture made with the one or both hands and a computation device that has program code configured to process the robot scene data and the gesture data to generate an instruction to operate the robot.

First claim

What is claimed is:

1. A system for generating instructions to a robot to perform work on a workpiece comprising:
a non-transitory computer readable medium structured to store the following:
a 3D model or 3D models of a robot scene providing robot scene data;
a plurality of gesture data, each of said plurality of gesture data corresponding to one or more gestures made with one or both of a human's hands and a work operation that is to be performed by said robot on said workpiece, wherein said one or more gestures and said work operation for at least one of said plurality of gesture data is different than said one or more gestures and said work operation for another of said plurality of gesture data;
one or more gestures and associated said work operation being different for each of said plurality of gesture data, wherein an index gesture of the one or more gestures includes an index finger of a hand pointing in a direction, said index gesture having a corresponding index gesture data from said plurality of gesture data; and

a computation device having program code, the computation device configured to process said robot scene data and said plurality of gesture data, the computation device operable to:
identify the index gesture data, using at least a detection of the index gesture, from the plurality of gesture data;
generate, using the work operation of said index gesture data and the robot scene data, an index gesture instruction for operating said robot to perform work on said workpiece, where the index gesture instruction is formulated to enable movement of the robot to a point that corresponds to an intersection of a direction of the index finger with the workpiece, and
output for said robot said index gesture instruction to control said robot to perform said work operation of said at least one selected gesture data on said workpiece.

2. The system of claim 1 wherein said one or more gestures made with one or both hands is a series of gestures, and wherein said computation device is operable to generate said instruction by assembling said work operation for each of said at least one selected gesture data for said series of gestures.

3. The system of claim 2 wherein said computation device is operable to recognize at least some of said gestures in said series of gestures provide an indication that a next at least one gesture is to be identified by said computation device as said at least one selected gesture data for use in generating said instruction.

4. The system of claim 1 further comprising one or more cameras, wherein said robot scene and said one or more gestures made with said one or both hands are received by said computation device in one or more images captured by said one or more cameras, and said computation device has program code configured to process said one or more images to obtain said robot scene data and identify said at least one selected gesture data.

5. The system of claim 1 further comprising one or more cameras wherein said robot scene and said one or more gestures made with said one or both hands, said one or more gestures referenced to said robot scene, are received by said computation device in separate images from said one or more cameras and said computation device has program code configured to process said separate images to obtain said scene data and identify said at least one selected gesture data.

6. The system of claim 1 wherein said instruction is used to operate said robot as soon as said instruction is generated by said computation device.

7. The system of claim 1 wherein said robot scene data is obtained by a vision system or computer added design data.

8. The system of claim 1 wherein said one or more gestures comprises two or more gestures made with one or both hands, and wherein said computation device is operable to assemble into the instruction each of said work operation for each of said at least one selected gesture data associated with said two or more gestures.

9. The system of claim 1 wherein said computation device is operable to identify the at least one selected gesture data from a detection of a single gesture made with one or both hands, and wherein said computation device is operable to generate with said robot scene data and said at least one selected gesture data for said single gesture two or more instructions to control said robot to perform said work operation on said workpiece.

10. The system of claim 1 wherein said one or more gestures made with one or both hands comprises a plurality of gestures, wherein each of said plurality of gestures are assigned to a particular instruction to operate said robot.

11. The system of claim 1 wherein said one or more gestures made with one or both hands is a series of gestures, and wherein said computation device is operable to output at least one instruction that controls said robot that is generated by said computation device from said robot scene data and data from said at least one selected gesture data for one of said one or more gestures in said series.

12. The system of claim 1 wherein said computation device is operable to identify said at least one selected gesture data associated with one of said hands holding an object when making said gesture and wherein said instruction outputted by said computation device causes said robot to hold said workpiece.

13. The system of claim 9 wherein said computation device is configured to output one of said two or more instructions to control the operation of said robot as soon as that one of said two or more instruction is generated by said computation device.

14. The system of claim 1 , wherein said plurality of gesture data includes a first gesture data and a second gesture data, said one or more gestures being the same for said first gesture data and said second gesture data, said work operation of said first gesture data being different than said work operation of said second gesture data, and
wherein said computation device is operable to identify either the first gesture data or the second gesture data as the at least one selected gesture data based on a context in which the said one more gestures is made.

Assignees

• Boca Remus
• Nidamarthi Srinivas
• Fuhlbrigge Thomas A
• Abb Schweiz Ag

Inventors

• Boca Remus
• Nidamarthi Srinivas
• Fuhlbrigge Thomas A

Classifications

• B25J9/1656 Primary

  characterised by programming, planning systems for manipulators · CPC title

• G05B2219/35444

  Gesture interface, controlled machine observes operator, executes commands · CPC title

• B25J9/1664 Primary

  characterised by motion, path, trajectory planning · CPC title

• B25J9/1697

  Vision controlled systems · CPC title