System and method for object shape identification and pose measurement using multi-fingered hand

Patent metadata

Field	Value
Publication number	US-12246448-B2
Application number	US-202017617992-A
Country	US
Kind code	B2
Filing date	Jun 11, 2020
Priority date	Jun 11, 2019
Publication date	Mar 11, 2025
Grant date	Mar 11, 2025

Abstract

A system and method for identifying a shape and pose of an object is provided. The system includes a controller and a grasping device. The grasping device includes a plurality of fingers that are moveable relative to a base. The fingers include one or more tactile sensors attached thereto. The tactile sensors are configured to collect data, such as a 2D image or 3D point cloud, based on points of contact with the object when the object is grasped. The grasping device is configured to roll the object within the fingers and collect additional data. The controller may combine the data and determine the shape and pose of the object based on the combined data.

First claim

What is claimed is:

1. A system for determining a physical characteristic and pose of an object, the system comprising:
a controller and a processor with instructions stored thereon;
a grasping device in communication with the controller, the grasping device configured to grasp an object, wherein the grasping device includes a plurality of fingers attached to a base and the fingers are moveable relative to the base; and
one or more tactile sensors attached to the fingers, the tactile sensors configured to detect first data associated with a physical characteristic of the object when the object is grasped by the grasping device;
wherein the controller is configured to receive the data from the tactile sensors and to store the data;
wherein the fingers are configured to roll the object within the fingers while the object is being grasped, and the tactile sensors are configured to provide second data to the controller associated with additional physical characteristics of the object;
wherein the controller is configured to obtain the second data associated with additional physical characteristics while the fingers are grasping the object and rolling the object; and
wherein the controller is configured to combine the first data and the second data into combined data, and configured to determine a shape and pose of the object from the combined data.

2. The system of claim 1 , wherein the first data and the second data are in the form of 2D images.

3. The system of claim 1 , wherein the first data and the second data are in the form of 3D point clouds.

4. The system of claim 1 , wherein the first data and the second data are obtained without visualization techniques.

5. The system of claim 1 , wherein the shape of the object is determined contemporaneously with grasping and manipulation of the object.

6. The system of claim 1 , wherein the plurality of fingers are pivotally attached to the base of the grasping device.

7. The system of claim 6 , wherein the grasping device is configured to manipulate the fingers relative to known predetermined coordinates of the base.

8. The system of claim 6 , wherein the fingers each include a tip portion coupled to and moveable relative to a proximal portion, wherein the proximal portion is attached to the base.

9. The system of claim 8 , wherein the tactile sensors are attached to the tip portions.

10. The system of claim 1 , wherein the tactile sensors are vision-based tactile sensors having a camera configured to measure three-dimensional force, wherein the camera captures deformation of a sensor layer.

11. The system of claim 1 , wherein the tactile sensor is a resistance or capacitive changing sensor or a piezoelectric changing sensor.

12. A method for identifying a shape and pose of an object, the method comprising:
grasping an object with a grasping device, wherein the grasping device includes a plurality of fingers having tactile sensors attached thereto;
detecting a first physical characteristic at the tactile sensors and collecting a first data corresponding to the first physical characteristic;
receiving, at a controller, the first data from tactile sensors;
manipulating the fingers of the grasping device and manipulating the object within the fingers of the grasping device;
detecting a second physical characteristic at the tactile sensors and collecting a second data corresponding to the second physical characteristic;
wherein the second physical characteristics and second data are obtained while the object is being grasped and manipulated by the fingers of the grasping device;
receiving, at the controller, the second data from the tactile sensors;
combining the first and second data into combined data at the controller; and
at the controller, identifying the shape and pose of the object based on the combined data;
wherein the all of the first and second data resulting in the combined data is provided by the tactile sensors and without visualization of the object.

13. The method of claim 12 , further comprising moving some of the fingers out of contact with the object repeatedly and detecting additional data from the tactile sensors at different positions on the object.

14. The method of claim 12 , further comprising rolling the object within the fingers to obtain the first and second data.

15. The method of claim 12 , further comprising manipulating the fingers relative to a base portion of the grasping device, the base portion having known predetermined coordinates.

16. The method of claim 12 , wherein the first and second data are obtained from vision-based tactile sensors, wherein the method includes obtaining an image of sensor layer deformation.

17. The method of claim 12 , wherein the first and second data are obtained from a resistance or capacitive changing sensor or a piezoelectric changing sensor.

18. The method of claim 12 , further comprising obtaining a 2d image or 3d point cloud from the tactile sensors.

19. The method of claim 12 , further comprising obtaining shape information of the object contemporaneously with the grasping of the object.

20. The method of claim 12 , further comprising automatically creating, at the controller, a model of the object based on the first and second data.

Assignees

• Magna Int Inc
• Tang Chinpei
• Zak Alexander

Inventors

• Tang Chinpei
• Zak Alexander

Classifications

• B25J13/089

  Determining the position of the robot with reference to its environment · CPC title

• B25J13/084

  Tactile sensors (in general G01L5/16, G01L5/22) · CPC title

• G05B2219/40575

  Camera combined with tactile sensors, for 3-D · CPC title

• G05B2219/39518

  Rolling contact between fingers, robot arms and object · CPC title

• B25J9/1612 Primary

  characterised by the hand, wrist, grip control · CPC title