Learning device, learning method, learning model, detection device and grasping system

Patent metadata

Field	Value
Publication number	US-11565407-B2
Application number	US-202117326250-A
Country	US
Kind code	B2
Filing date	May 20, 2021
Priority date	May 31, 2017
Publication date	Jan 31, 2023
Grant date	Jan 31, 2023

Abstract

An estimation device includes a memory and at least one processor. The at least one processor is configured to acquire information regarding a target object. The at least one processor is configured to estimate information regarding a location and a posture of a gripper relating to where the gripper is able to grasp the target object. The estimation is based on an output of a neural model having as an input the information regarding the target object. The estimated information regarding the posture includes information capable of expressing a rotation angle around a plurality of axes.

First claim

The invention claimed is:

1. A system comprising:
an end effector configured to manipulate an object;
at least one processor configured to generate, by inputting information of the object into a neural network model, information of at least one of a location or a posture for manipulating the object by the end effector;
wherein the end effector manipulates the object based on the generated information, and
the neural network model is learned by using data generated based on at least one of a virtual object or an augmented object.

2. The system according to claim 1 , wherein at least one of the virtual object or the augmented object is generated based on information of a real object acquired by a detector.

3. The system according to claim 2 , wherein the detector acquiring the information of the real object is at least one of at least one camera, at least one camera which obtains distance information, or at least one detection device capable of three-dimensionally measuring the real object.

4. The system according to claim 1 , wherein the data includes information of at least one of a location or a posture for manipulating at least one of the virtual object or the augmented object using an end effector that manipulates at least one of the virtual object or the augmented object.

5. The system according to claim 1 , wherein the data is generated by using at least one of a virtual reality technique or an augmented reality technique.

6. The system according to claim 1 further comprising a controller that controls the end effector based on the generated information.

7. The system according to claim 1 , wherein a detector acquiring the information of the object inputted into the neural network is installed on the end effector or an arm.

8. The system according to claim 1 , wherein a detector acquiring the information of the object inputted into the neural network is at least one of at least one camera, at least one camera which obtains distance information, or at least one detection device capable of three-dimensionally measuring the object.

9. The system according to claim 1 , wherein the generated information of the posture includes information capable of expressing rotation angles around axes.

10. The system according to claim 1 , wherein the at least one processor inputs the information of the object into the neural network model and generates the information of at least one of locations or postures for manipulating the object by the end effector.

11. The system according to claim 1 , wherein the end effector grasps the object based on the generated information.

12. The system according to claim 1 , wherein the end effector is a gripper.

13. A method of learning a neural network model which is inputted information of an object to output information of at least one of a location or a posture for manipulating the object by an end effector comprising:
learning, by one or more processors, the neural network model based on data generated by using at least one of a virtual object or an augmented object.

14. The method according to claim 13 , wherein the at least one of the virtual object or the augmented object is generated based on information of a real object acquired by a detector.

15. The method according to claim 14 , wherein the detector acquiring the information of the real object is at least one of at least one camera, at least one camera which obtains distance information, or a detection device capable of three-dimensionally measuring the real object.

16. The method according to claim 13 , wherein the data includes information of at least one of a location or a posture for manipulating at least one of the virtual object or the augmented object using an end effector that manipulates at least one of the virtual object or the augmented object.

17. The method according to claim 13 , wherein the data is generated by using at least one of a virtual reality technique or an augmented reality technique.

18. The method according to claim 13 , wherein the generated information of the posture includes information capable of expressing rotation angles around axes.

19. A manipulating method comprising:
inputting, by one or more processors, information of an object into a neural network model that has been learned using at least one of a virtual object or an augmented object;
generating, by the one or more processors, information of at least one of a location or a posture for manipulating the object by an end effector;
manipulating, by the one or more processors, the object by the end effector based on the generated information.

20. The method according to claim 19 , wherein the at least one of the virtual object or the augmented object is generated based on information of a real object acquired by a detector.

21. The method according to claim 19 , wherein the generated information of the posture includes information capable of expressing rotation angles around axes.

22. The method according to claim 19 , wherein the generated information includes information of at least one of locations or postures for manipulating the object by the end effector.

23. The method according to claim 19 , wherein the manipulating is grasping the object based on the generated information.

24. A non-transitory computer readable medium storing therein a program which executes a method, when executed by one or more processors, the method comprising:
inputting information of an object into a neural network model that has been learned using at least one of a virtual object or an augmented object;
generating information of at least one of a location or a posture for manipulating the object by an end effector; and
manipulating the object by the end effector based on the generated information.

Assignees

• Preferred Networks Inc

Inventors

• Kusano Hitoshi
• KUME Ayaka
• Matsumoto Eiichi

Classifications

• B25J9/1669

  characterised by special application, e.g. multi-arm co-operation, assembly, grasping · CPC title

• B25J9/1635

  flexible-arm control · CPC title

• B25J9/1612 Primary

  characterised by the hand, wrist, grip control · CPC title

• B25J9/163

  learning, adaptive, model based, rule based expert control · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title