Vision-based tactile measurement method and apparatus, chip, and storage medium

Patent metadata

Field	Value
Publication number	US-12214487-B2
Application number	US-202117369837-A
Country	US
Kind code	B2
Filing date	Jul 7, 2021
Priority date	May 16, 2019
Publication date	Feb 4, 2025
Grant date	Feb 4, 2025

Abstract

A vision-based tactile measurement method is provided, performed by a computer device (e.g., a chip) connected to a tactile sensor, the tactile sensor including a sensing face and an image sensing component, and the sensing face being provided with a marking pattern. The method includes: obtaining an image sequence collected by the image sensing component of the sensing face, each image of the image sequence comprising one instance of the marking pattern; calculating a difference feature of the marking patterns in adjacent images of the image sequence; and processing the difference feature of the marking patterns using a feedforward neural network to obtain a tactile measurement result, a quantity of hidden layers in the feedforward neural network being less than a threshold.

First claim

What is claimed is:

1. A vision-based tactile measurement method performed by a computer device connected to a tactile sensor, the tactile sensor comprising a sensing face and an image sensing component, and the sensing face being provided with a marking pattern; and the method comprising:
obtaining an image sequence collected by the image sensing component of the sensing face that is in physical contact with a surface of an object, each image of the image sequence comprising one instance of the marking pattern;
calculating a difference feature of the marking patterns in adjacent images of the image sequence; and
processing the difference feature of the marking patterns using a feedforward neural network to obtain a tactile measurement result, further comprising:
calling a hidden layer in the feedforward neural network to perform feature extraction on the difference feature of the marking patterns to obtain a feature representation of the surface of the object, the feature presentation further including a curvature prediction on the surface of the object; and
calling an output layer in the feedforward neural network to process the feature representation to obtain the tactile measurement result, the tactile measurement result comprising a local curvature radius of the surface of the object.

2. The method according to claim 1 , wherein n hidden neurons are provided in the hidden layer, n being an integer;
the hidden layer is constructed based on hidden neurons of a logistic sigmoid function; and
the output layer is constructed based on output neurons of a normalized exponential softmax function or linear output neurons.

3. The method according to claim 1 , wherein the feedforward neural network comprises: a location estimation model, and the location estimation model comprises a first hidden layer and a first output layer;
the calling the hidden layer in the feedforward neural network to perform feature extraction on the difference feature of the marking patterns to obtain a feature representation of the surface of the object comprises:
calling the first hidden layer in the location estimation model to perform feature extraction on the difference feature of the marking patterns to obtain a feature representation of a contact location on the surface of the object; and
the calling the output layer in the feedforward neural network to process the feature representation to obtain the tactile measurement result comprises:
calling the first output layer in the location estimation model to process the feature representation of the contact location, to obtain the contact location on the surface of the object.

4. The method according to claim 1 , wherein the feedforward neural network comprises: a contact force estimation model, and the contact force estimation model comprises a second hidden layer and a second output layer;
the calling the hidden layer in the feedforward neural network to perform feature extraction on the difference feature of the marking patterns to obtain a feature representation of the surface of the object comprises:
calling the second hidden layer in the contact force estimation model to perform feature extraction on the difference feature of the marking patterns to obtain a feature representation of the contact force on the surface of the object; and
the calling the output layer in the feedforward neural network to process the feature representation to obtain the tactile measurement result comprises:
calling the second output layer in the contact force estimation model to process the feature representation of the contact force, to obtain three-dimensional information of the contact force on the surface of the object, the three-dimensional information comprising at least one of a magnitude and a direction.

5. The method according to claim 1 , wherein the feedforward neural network comprises: a surface classification model and at least two curvature estimation models; and
the processing the difference feature of the marking patterns using a feedforward neural network to obtain a tactile measurement result comprises:
calling the surface classification model to perform surface recognition on the difference feature of the marking patterns to obtain a surface type of the surface of the object; and
calling a target curvature estimation model in the at least two curvature estimation models based on the surface type to perform the curvature prediction on the surface of the object, to obtain the local curvature radius of the surface of the object.

6. The method according to claim 5 , wherein the curvature estimation model comprises: a spherical surface estimation model and a cylindrical surface estimation model; and the calling a target curvature estimation model in the at least two curvature estimation models based on the surface type to perform the curvature prediction on the surface of the object, to obtain the local curvature radius of the surface of the object comprises:
in a case that the surface type is a spherical surface, calling the spherical surface estimation model to perform first curvature prediction on the spherical surface, to obtain the local curvature radius of the spherical surface; and
in a case that the surface type is a cylindrical surface, calling the cylindrical surface estimation model to perform second curvature prediction on the cylindrical surface, to obtain the local curvature radius of the cylindrical surface.

7. The method according to claim 5 , wherein the surface classification model comprises a third hidden layer and a third output layer; and the calling the surface classification model to perform surface recognition on the difference feature of the marking patterns to obtain a surface type of the surface of the object comprises:
calling the third hidden layer in the surface classification model to perform surface recognition on the difference feature of the marking patterns to obtain a feature representation of the surface type; and
calling the third output layer in the surface classification model to process the feature representation of the surface type, to obtain the surface type of the surface of the object.

8. The method according to claim 6 , wherein the spherical surface estimation model comprises a fourth hidden layer and a fourth output layer; and the calling the spherical surface estimation model to perform first curvature prediction on the spherical surface, to obtain the local curvature radius of the spherical surface comprises:
calling the fourth hidden layer in the spherical surface estimation model to perform the first curvature prediction on the spherical surface, to obtain a feature representation of the curvature prediction of the spherical surface; and
calling the fourth output layer in the spherical surface estimation model to process the feature representation of the curvature prediction of the spherical surface, to obtain the local curvature radius of the spherical surface.

9. The method according to claim 6 , wherein the cylindrical surface estimation model comprises a fifth hidden layer and a fifth output layer; and the calling the cylindrical surface estimation model to perform second curvature prediction on the cylindrical surface, to obtain the local curvature radius of the cylindrical surface comprises:
calling the fifth hidden layer in the cylindrical surface estimation model to perform the second curvature prediction on the cylindrical surface, to obtain a feature representation of the curvature prediction of the cylindrical surface; and
calling the fifth output layer in the cylindrical surface estimation model to process the feature representation of the curvature prediction of the cylindrical surface, to obtain the local curvature radius of t

Assignees

• Tencent Tech Shenzhen Co Ltd

Inventors

• Zheng Yu
• Xu Zhongjin
• Zhang Zhengyou

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06N3/0499

  Feedforward networks · CPC title

• G06V10/225

  based on a marking or identifier characterising the area · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/776

  Validation; Performance evaluation · CPC title