Gesture information processing method and apparatus, electronic device, and storage medium

Patent metadata

Field	Value
Publication number	US-12422935-B2
Application number	US-202318227857-A
Country	US
Kind code	B2
Filing date	Jul 28, 2023
Priority date	Jan 13, 2020
Publication date	Sep 23, 2025
Grant date	Sep 23, 2025

Abstract

This application provides a gesture information processing method and apparatus, an electronic device, and a storage medium. The method includes: acquiring an electromyography signal sample generated by an electromyography signal collection target object in connection with performing multiple gestures; dividing the electromyography signal sample through a sliding window having a fixed window value and a fixed stride into different electromyography signals of the target object; and applying the different electromyography signals to a first neural network model to determine gesture information matching the multiple gestures performed by the target object.

First claim

What is claimed is:

1. A gesture information processing method performed by an electronic device, the method comprising:
acquiring an electromyography signal sample generated by an electromyography signal collection target object in connection with performing one of multiple gestures and a corresponding gesture information label;
dividing the electromyography signal sample through a sliding window having a fixed window value and a fixed stride into different electromyography signals of the target object; and
applying the different electromyography signals to a first neural network model to determine gesture information matching the one of multiple gestures performed by the target object, further including:
recognizing the different electromyography signals based on the gesture information label;
determining probabilities of gesture information represented by the different electromyography signals using the first neural network model; and
weighting the probabilities of the gesture information represented by the different electromyography signals to determine the gesture information matching the one of multiple gestures performed by the target object.

2. The method of claim 1 , wherein method further comprises:
determining a dynamic noise threshold associated with the multiple gestures performed by the target object; and
denoising the different electromyography signals according to the dynamic noise threshold to form dynamic noise electromyography signals matching the dynamic noise threshold.

3. The method of claim 1 , wherein the method further comprises:
determining features of corresponding electromyography signals when the gesture information usage environment of the gesture recognition model is a powered skeleton;
determining output sizes of different networks in a first neural network model of the gesture recognition model and a quantity of corresponding convolution kernels according to the features of the electromyography signals; and
adjusting quantities of a temporal convolutional layer network, a spatial convolutional layer network, a fully connected layer network and a residual module network of the first neural network model according to features of the powered skeleton, so as to match initialization parameters of the gesture recognition model with the features of the powered skeleton.

4. The method of claim 1 , wherein the first neural network model is trained by:
obtaining a first training sample set, wherein the first training sample set comprises at least one group of electromyography signal samples and a corresponding set of gesture information labels;
determining initial parameters of the first neural network model in the gesture recognition model;
iteratively updating the initial parameters of the first neural network model by applying the first training sample set to the first neural network model according to the updated parameters of the first neural network model until after a loss function associated with the first neural network model meets a corresponding convergence condition.

5. The method of claim 4 , wherein the first training sample set is generated by:
acquiring an electromyography signal sample matching a training sample collection object and a corresponding set of gesture information labels; and
dividing the electromyography signal sample through the sliding window having the fixed window value and the fixed stride to form the first training sample set.

6. The method of claim 4 , wherein the determining initial parameters of the first neural network model comprises:
determining the output sizes of different networks in the first neural network model and the quantity of the corresponding convolution kernels according to features of electromyography signals in the first training sample set; and
adjusting the quantities of the different networks in the first neural network model including a temporal convolutional layer network, a spatial convolutional layer network, a fully connected layer network and a residual module network according to an application environment of the gesture recognition model, so as to match initialization parameters of the gesture recognition model with the application environment.

7. The method of claim 4 , wherein the iteratively updating the initial parameters of the first neural network model further comprises:
determining a classification loss and a contrastive loss corresponding to the first neural network model;
determining a fusion loss matching the first neural network model according to the classification loss and the contrastive loss as the loss function associated with the first neural network model; and
training the first neural network model by applying the first training sample set to the first neural network model according to the updated parameters of the first neural network model iteratively until after the fusion loss indicates that the corresponding convergence condition is reached.

8. An electronic device, comprising:
a memory configured to store executable instructions; and
a processor configured to run the executable instructions stored in the memory to perform a gesture information processing method including:
acquiring an electromyography signal sample generated by an electromyography signal collection target object in connection with performing one of multiple gestures and a corresponding gesture information label;
dividing the electromyography signal sample through a sliding window having a fixed window value and a fixed stride into different electromyography signals of the target object; and
applying the different electromyography signals to a first neural network model to determine gesture information matching the one of multiple gestures performed by the target object, further including:
recognizing the different electromyography signals based on the gesture information label;
determining probabilities of gesture information represented by the different electromyography signals using the first neural network model; and
weighting the probabilities of the gesture information represented by the different electromyography signals to determine the gesture information matching the one of multiple gestures performed by the target object.

9. The electronic device of claim 8 , wherein method further comprises:
determining a dynamic noise threshold associated with the multiple gestures performed by the target object; and
denoising the different electromyography signals according to the dynamic noise threshold to form dynamic noise electromyography signals matching the dynamic noise threshold.

10. The electronic device of claim 8 , wherein the method further comprises:
determining features of corresponding electromyography signals when the gesture information usage environment of the gesture recognition model is a powered skeleton;
determining output sizes of different networks in a first neural network model of the gesture recognition model and a quantity of corresponding convolution kernels according to the features of the electromyography signals; and
adjusting quantities of a temporal convolutional layer network, a spatial convolutional layer network, a fully connected layer network and a residual module network of the first neural network model according to features of the powered skeleton, so as to match initialization parameters of the gesture recognition model with the features of the powered skeleton.

11. The electronic device of claim 8 , wherein the first neural network model is trained by:
obtaining a first training sample set, wherein the first training sample set comprises at least one group of electromyography signal samples and a corresponding set of gesture info

Assignees

• Tencent Tech Shenzhen Co Ltd

Inventors

• Hong Xiaolin
• Zheng Qingqing
• Wang Xinmin
• Ma Kai
• Zheng Yefeng

Classifications

• G06T5/70

  Denoising; Smoothing · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title