Emg-based speech detection and communication

Patent metadata

Field	Value
Publication number	US-2025085780-A1
Application number	US-202418887848-A
Country	US
Kind code	A1
Filing date	Sep 17, 2024
Priority date	Feb 14, 2021
Publication date	Mar 13, 2025
Grant date	—

Abstract

Systems and methods are provided for performing operations comprising: detecting, by one or more electromyograph (EMG) electrodes of an EMG communication device, subthreshold muscle activation signals of one or more muscles associated with speech production, the subthreshold muscle activation signals being generated in response to inner speech of a user; applying a machine learning technique to the subthreshold muscle activation signals to estimate one or more speech features corresponding to the subthreshold muscle activation signals, the machine learning technique being trained to establish a relationship between a plurality of training subthreshold muscle activation signals and ground truth speech features; generating visual or audible output based on the one or more speech features; and causing the visual or audible output to be processed by a messaging application to engage a feature of the messaging application.

First claim

1 . (canceled)

2 . A method comprising:
detecting a trigger word by a microphone of an EMG communication device; and transitioning the EMG communication device between an overt calibration mode to perform overt calibration of a machine learning technique and a covert calibration mode in response to detecting the trigger word.

3 . The method of claim 2 , comprising:
in response to detecting the trigger word, detecting, by one or more EMG electrodes of the EMG communication device, subthreshold muscle activation signals of one or more muscles associated with speech production, the subthreshold muscle activation signals being generated in response to inner speech of a user; and applying the machine learning technique to the subthreshold muscle activation signals to estimate one or more speech features corresponding to the subthreshold muscle activation signals, the machine learning technique being trained to establish a relationship between a plurality of training subthreshold muscle activation signals and ground truth speech features.

4 . The method of claim 3 , comprising:
generating at least one of a visual or audible output based on the one or more speech features; and causing the at least one of the visual or audible output to be processed by a messaging application to engage a feature of the messaging application.

5 . The method of claim 2 , further comprising synthesizing one or more speech features to generate audible output, wherein a feature of a messaging application comprises a voice message transmission feature in which the audible output is transmitted from the messaging application of a user to another messaging application of another user.

6 . The method of claim 2 , further comprising processing one or more speech features by a neural network to generate audible output.

7 . The method of claim 6 , wherein the neural network comprises a WaveNet network.

8 . The method of claim 2 , further comprising processing one or more speech features by a neural network classifier to associate the one or more speech features with one or more words or phonemes.

9 . The method of claim 2 , wherein the EMG communication device comprises a wearable collar device, the wearable collar device comprising one or more EMG electrodes, a microphone, and a communication device, the EMG communication device being in communication with a mobile device that implements a messaging application.

10 . The method of claim 2 , further comprising training the machine learning technique by performing training operations comprising:
computing a deviation between an estimated set of speech features and a first set of ground truth speech features associated with a first set of subthreshold muscle activation signals; and updating one or more parameters of the machine learning technique based on the computed deviation.

11 . The method of claim 2 , further comprising training the machine learning technique by performing operations comprising:
receiving training data comprising a first set of subthreshold muscle activation signals and a first set of ground truth speech features associated with the first set of subthreshold muscle activation signals; processing the first set of subthreshold muscle activation signals by the machine learning technique to generate an estimated set of speech features; computing a deviation between the estimated set of speech features and the first set of ground truth speech features associated with the first set of subthreshold muscle activation signals; and updating one or more parameters of the machine learning technique based on the computed deviation.

12 . The method of claim 2 , wherein the covert calibration mode performs operations comprising:
generating an instruction to a user to perform inner speech corresponding to a stimulus, the stimulus comprising a word or phrase; detecting a second set of subthreshold muscle activation signals by one or more EMG electrodes in response to generating the instruction; processing the second set of subthreshold muscle activation signals by the machine learning technique to generate a second set of estimated speech features; comparing the second set of estimated speech features with a second set of training speech features corresponding to the word or phrase; and computing a deviation between the second set of estimated speech features with the second set of training speech features.

13 . The method of claim 12 , further comprising training a neural network to map a first set of subthreshold muscle activation signals to the second set of subthreshold muscle activation signals.

14 . A system comprising:
one or more processors configured to perform operations comprising:
detecting a trigger word by a microphone of an EMG communication device; and
transitioning the EMG communication device between an overt calibration mode to perform overt calibration of a machine learning technique and a covert calibration mode in response to detecting the trigger word.

15 . A non-transitory computer-readable medium encoded with computer-readable instructions that, when executed by one or more processors, configure the one or more processors to perform operations comprising:
detecting a trigger word by a microphone of an EMG communication device; and transitioning the EMG communication device between an overt calibration mode to perform overt calibration of a machine learning technique and a covert calibration mode in response to detecting the trigger word.

16 . The non-transitory computer-readable medium of claim 15 , the operations comprising:
in response to detecting the trigger word, detecting, by one or more EMG electrodes of the EMG communication device, subthreshold muscle activation signals of one or more muscles associated with speech production, the subthreshold muscle activation signals being generated in response to inner speech of a user; and applying the machine learning technique to the subthreshold muscle activation signals to estimate one or more speech features corresponding to the subthreshold muscle activation signals, the machine learning technique being trained to establish a relationship between a plurality of training subthreshold muscle activation signals and ground truth speech features.

17 . The non-transitory computer-readable medium of claim 16 , comprising:
generating at least one of a visual or audible output based on the one or more speech features; and causing the at least one of the visual or audible output to be processed by a messaging application to engage a feature of the messaging application.

18 . The non-transitory computer-readable medium of claim 15 , the operations comprising synthesizing one or more speech features to generate audible output, wherein a feature of a messaging application comprises a voice message transmission feature in which the audible output is transmitted from the messaging application of a user to another messaging application of another user.

19 . The non-transitory computer-readable medium of claim 15 , the operations comprising processing one or more speech features by a neural network to generate audible output.

20 . The non-transitory computer-readable medium of claim 19 , wherein the neural network comprises a WaveNet network.

21 . The non-transitory computer-readable medium of claim 15 , further comprising processing one or more speech features by a neural network classifier to associate the one or more speech features with one or more words or

Assignees

• Snap Inc

Inventors

• Ziv Assif

Classifications

• H04M1/7243

  with interactive means for internal management of messages · CPC title

• G10L2015/088

  Word spotting · CPC title

• G10L2015/0638

  Interactive procedures · CPC title

• G10L2015/025

  Phonemes, fenemes or fenones being the recognition units · CPC title

• G06F1/163 Primary

  Wearable computers, e.g. on a belt · CPC title