Vibrational devices as sound sensors

Patent metadata

Field	Value
Publication number	US-10628484-B2
Application number	US-201815981016-A
Country	US
Kind code	B2
Filing date	May 16, 2018
Priority date	May 17, 2017
Publication date	Apr 21, 2020
Grant date	Apr 21, 2020

Abstract

A device includes a coil and magnetic mass movable next to the coil in response to vibrations to generate a back electromotive force signal. An amplifier generates, from the back EMF signal, a vibration signal. A processing device converts the vibration signal to time-frequency domain signal as two-dimensional matrix of frequencies mapped against time slots. Pre-process voiced data of the time-frequency domain signal to generate a reduced-noise signal. Average signal values within a frequency window, and that exist at a first time slot, of the reduced-noise signal to generate a complex frequency coefficient. Shift the frequency window across the frequencies to generate multiple complex frequency coefficients that identify speech energy concentration. Replicate signal values at a fundamental frequency within the voiced data to multiple harmonic frequencies to generate an expanded voice source signal. Combine the speech energy concentration with the expanded voice source signal to recreate original speech.

First claim

What is claimed is:

1. A device comprising:
a metal coil made of a number of turns;
a magnetic mass to move past the number of turns of the metal coil in response to vibrations from human speech, to generate a reverse electromotive force (“back EMF”) signal within the metal coil;
an amplifier circuit coupled to the metal coil to generate, from the back EMF signal, a vibration signal; and
a processing device coupled to the amplifier circuit, the processing device to:
convert the vibration signal to a time-frequency domain signal, which comprises a two-dimensional matrix of a plurality of frequencies mapped against a plurality of time slots;
pre-process voiced data of the time-frequency domain signal, to generate a reduced-noise signal;
average signal values within a frequency window of the plurality of frequencies, and that exist at a first time slot of the plurality of time slots, of the reduced-noise signal to generate a complex frequency coefficient;
shift the frequency window across the plurality of frequencies that exist in the reduced-noise signal at the first time slot, to generate a plurality of complex frequency coefficients that identify speech energy concentration within the reduced-noise signal;
replicate signal values at a fundamental frequency within the voiced data of the time-frequency domain signal to one or more harmonic multiple of the fundamental frequency, to generate an expanded voice source signal of the time-frequency domain signal; and
combine the speech energy concentration with the expanded voice source signal to recreate original speech detected within the back EMF signal.

2. The device of claim 1 , wherein the metal coil and the magnetic mass are located within a vibration motor.

3. The device of claim 1 , wherein the frequency window is one of a first length between 10 and 20 hertz or a second length that causes remaining noise within the frequency window to average to approximately zero, and wherein the frequency window is shifted up to 10 kilohertz of the reduced-noise signal.

4. The device of claim 1 , wherein the processing device, to perform frequency domain equalization on the time-frequency domain signal, is further to:
determine an inverse gain as a ratio between first frequency coefficients of a reference signal, received by a reference microphone, and second frequency coefficients of the back EMF signal; and
multiply the inverse gain times a set of third frequency coefficients of the time-frequency domain signal.

5. The device of claim 1 , wherein to pre-process the voiced data of the time-frequency domain signal, the processing device is to:
isolate voiced components in the time-frequency domain signal that are associated with a first harmonic frequency; and
apply spectral subtraction of known background noise to the voiced components, to generate the reduced-noise signal.

6. The device of claim 1 , wherein to average the signal values, the processing device is further to average the signal values of the reduced-noise signal within the frequency window that exist over a time window comprising a subset of the plurality of time slots, and wherein the processing device is further to shift the time window across the plurality of time slots while shifting the frequency window across the plurality of frequencies to generate the plurality of complex frequency coefficients.

7. The device of claim 6 , wherein to average the signal values, the processing device is further to normalize signal values, which exist at each frequency of the plurality of frequencies, as between zero (“0”) and one (“1”) across the time window, and wherein the time window is between 300 and 400 milliseconds.

8. The device of claim 1 , wherein the processing device is further to zero out speech energy located outside of time-frequency areas of the speech energy concentration identified within the reduced-noise signal.

9. The device of claim 1 , wherein the processing device is further to:
replicate the signal values at the fundamental frequency within the voiced data of the time-frequency domain signal to multiple harmonic frequencies of the fundamental frequency;
identify unvoiced data within the time-frequency domain signal; and
replicate the unvoiced data to a plurality of higher frequencies to complete the expanded voice source signal.

10. The device of claim 1 , wherein, to combine the speech energy concentration with the expanded voice source signal, the processing device is to:
apply an exponential decay function to data of the expanded voice source signal located along the one or more harmonic multiple of the fundamental frequency to generate a modified voice source signal;
apply the speech energy concentration as an energy mask to the modified voice source signal, to generate a resultant time-frequency domain signal;
apply a deferred spectral subtraction of background noise to the resultant time-frequency domain signal, to generate a resultant reduced-noise signal; and
convert the resultant reduced-noise signal to the time frequency domain to recreate the original speech.

11. A method comprising:
detecting, using a processing device coupled to an electromechanical device, a vibration signal generated by the electromechanical device in response to human speech;
converting, by the processing device, the vibration signal to a time-frequency domain signal, which comprises a two-dimensional matrix of a plurality of frequencies mapped against a plurality of time slots;
pre-processing, by the processing device, voiced data of the time-frequency domain signal to generate a reduced-noise signal;
averaging, by the processing device, signal values within a frequency window of the plurality of frequencies, and that exist at a first time slot of the plurality of time slots, of the reduced-noise signal to generate a complex frequency coefficient;
shifting, by the processing device, the frequency window across the plurality of frequencies that exist in the reduced-noise signal at the first time slot, to generate a plurality of complex frequency coefficients that identify speech energy concentration within the reduced-noise signal;
replicating, by the processing device, signal values at a fundamental frequency within the voiced data of the time-frequency domain signal to one or more harmonic multiple of the fundamental frequency, to generate an expanded voice source signal of the time-frequency domain signal; and
combining, by the processing device, the speech energy concentration with the expanded voice source signal to recreate original speech detected within the vibration signal.

12. The method of claim 11 , further comprising playing the original speech out of a speaker coupled to the processing device.

13. The method of claim 11 , wherein the frequency window is one of a first length between 10 and 20 hertz or a second length that causes remaining noise within the frequency window to average to approximately zero, and wherein the frequency window is shifted up to 10 kilohertz of the reduced-noise signal.

14. The method of claim 11 , further comprising:
determining an inverse gain as a ratio between first frequency coefficients of a reference signal, received by a reference microphone, and second frequency coefficients of the vibration signal; and
multiplying the inverse gain times a set of third frequency coefficients of the time-frequency domain signal, to perform frequency domain equalization on the time-frequency domain signal.

15. The method of claim 11 , wherein pre-processing the voiced data of the time-frequency domain signal comprises:
isolating voiced components in the

Assignees

• Univ Illinois
• Board Of Trustees Of The Univ Of Illinois

Inventors

• Choudhury Romit Roy
• Roy Nirupam

Classifications

• H04R29/004

  for microphones (H04R29/007 takes precedence) · CPC title

• G10L21/00

  Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility (G10L19/00 takes precedence) · CPC title

• G10L15/26

  Speech to text systems (G10L15/08 takes precedence) · CPC title

• H02P25/034

  Voice coil motors (voice coil motors driven by DC H02P7/025) · CPC title

• H04R1/2869

  Reduction of undesired resonances, i.e. standing waves within enclosure, or of undesired vibrations, i.e. of the enclosure itself · CPC title