Visuotactile operators for proximity sensing and contact control

Patent metadata

Field	Value
Publication number	US-2023264367-A1
Application number	US-202318103825-A
Country	US
Kind code	A1
Filing date	Jan 31, 2023
Priority date	Feb 24, 2022
Publication date	Aug 24, 2023
Grant date	—

Abstract

A method for identifying and manipulating objects may include obtaining, from an image sensor, image sensor data; identifying, using the image sensor data, a location of an object; controlling a robotic element, which includes the image sensor, to move towards the location of the object; determining a slippage based on contact between the image sensor and the object; and controlling a movement of the robotic element based on the determined slippage.

First claim

What is claimed is:

1 . A method for identifying and manipulating objects, the method comprising:
obtaining, from an image sensor, image sensor data; identifying, using the image sensor data, a location of an object; controlling a robotic element, which includes the image sensor, to move towards the location of the object; determining a slippage based on contact between the image sensor and the object; and controlling a movement of the robotic element based on the determined slippage.

2 . The method of claim 1 , wherein the identifying the location of the object comprises identifying a bounding box for the object in an image that is represented by the image sensor data.

3 . The method of claim 2 , wherein the identifying the bounding box comprises predicting coordinates of the bounding box in the image.

4 . The method of claim 2 , wherein the identifying the bounding box comprises identifying a centroid of the bounding box and determining a distance between the centroid and a center of the image sensor.

5 . The method of claim 1 , wherein the determining the slippage comprises measuring a deformation of a surface of the image sensor when the image sensor is in contact with the object.

6 . The method of claim 1 , wherein the determining the slippage comprises determining a marker flow and determining an object flow, and determining a slip field as a difference between the object flow and the marker flow.

7 . The method of claim 6 , wherein the determining the marker flow comprises identifying movement of at least one marker, and wherein determining the object flow comprises determining a motion of the object in relation to the image sensor.

8 . The method of claim 6 , further comprising combining the marker flow and the object flow using a convolutional neural network architecture.

9 . An electronic device for performing image authentication, the electronic device comprising:
at least memory storing instructions; and at least one processor configured to execute the instructions to:
obtain, from an image sensor, image sensor data;
identify, using the image sensor data, a location of an object;
control a robotic element, which includes the image sensor, to move towards the location of the object;
determine a slippage based on contact between the image sensor and the object; and
control a movement of the robotic element based on the determined slippage.

10 . The electronic device of claim 9 , wherein the at least one processor is further configured to identify a bounding box for the object in an image that is represented by the image sensor data.

11 . The electronic device of claim 10 , wherein the at least one processor is further configured to predict coordinates of the bounding box in the image.

12 . The electronic device of claim 10 , wherein the at least one processor is further configured to identify a centroid of the bounding box and determine a distance between the centroid and a center of the image sensor.

13 . The electronic device of claim 9 , wherein the at least one processor is further configured to measure a deformation of a surface of the image sensor when the image sensor is in contact with the object.

14 . The electronic device of claim 9 , wherein the at least one processor is further configured to determine a marker flow and determine an object flow, and determine a slip field as a difference between the object flow and the marker flow.

15 . The electronic device of claim 14 , wherein the at least one processor is further configured to identify movement of at least one marker, and wherein determining the object flow comprises determining a motion of the object in relation to the image sensor.

16 . The electronic device of claim 14 , wherein the at least one processor is further configured to combine the marker flow and the object flow using a convolutional neural network architecture.

17 . A non-transitory computer readable storage medium that stores instructions to be executed by at least one processor to perform a method for identifying and manipulating objects, the method comprising:
obtaining, from an image sensor, image sensor data; identifying, using the image sensor data, a location of an object; controlling a robotic element, which includes the image sensor, to move towards the location of the object; determining a slippage based on contact between the image sensor and the object; and controlling a movement of the robotic element based on the determined slippage.

18 . The non-transitory computer readable storage medium of claim 17 , wherein the identifying the location of the object comprises identifying a bounding box for the object in an image that is represented by the image sensor data.

19 . The non-transitory computer readable storage medium of claim 18 , wherein the identifying the bounding box comprises predicting coordinates of the bounding box in the image.

20 . The non-transitory computer readable storage medium of claim 18 , wherein the identifying the bounding box comprises identifying a centroid of the bounding box and determining a distance between the centroid and a center of the image sensor.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Hogan Francois
• Tremblay Jean-Francois
• Baghi Bobak Hamed
• Jenkin Michael
• Siddiqi Kaleem
• Dudek Gregory Lewis

Classifications

• B25J13/083 Primary

  fitted with slippage detectors · CPC title

• B25J9/1697

  Vision controlled systems · CPC title

• B25J9/1612

  characterised by the hand, wrist, grip control · CPC title

• G05B2219/40575

  Camera combined with tactile sensors, for 3-D · CPC title

• G05B2219/40625

  Tactile sensor · CPC title