Haptic communication using dominant frequencies in speech signal

Patent metadata

Field	Value
Publication number	US-2018301002-A1
Application number	US-201815949853-A
Country	US
Kind code	A1
Filing date	Apr 10, 2018
Priority date	Apr 17, 2017
Publication date	Oct 18, 2018
Grant date	—

Abstract

Embodiments relate to performing haptic communication using frequency decomposition of speech where dominant frequencies of a speech is detected at a speech source and then sent to a signal generator to actuate actuators mapped to the dominant frequencies. The digitized version of the speech is segmented into a plurality of frames and then a predetermined number of dominant frequencies are detected from each frame. The dominant frequencies of frequencies are sent over to the signal generator so that the actuators corresponding to the dominant frequencies are activated for a time period corresponding to the frame.

First claim

1 . A method, comprising:
generating a digitized version of a speech signal; processing the digitized speech signal to generate processed speech signal; detecting dominant frequencies in the processed speech signal to generate dominant frequency information; and providing the detected dominant frequencies to generate actuator signals for activating cutaneous actuators on a receiving user's skin.

2 . The method of claim 1 , wherein processing the digitized speech signal comprises:
segmenting the digitized signal into frames, each frame having a predetermined length, and performing spectral estimation on each of the frames to generate a frequency domain spectrum, wherein the dominant frequencies are detected in each of the frame by at least determining a first dominant frequency at which a magnitude of the frequency domain spectrum is highest.

3 . The method of claim 2 , wherein processing the digitized speech signal further comprises setting to zero the frequency domain spectrum below a first frequency point and beyond a second frequency point higher than the first frequency point.

4 . The method of claim 2 , wherein processing the digitized speech signal further comprises:
detecting next dominant frequencies having respective magnitude of the frequency domain spectrum that are next highest after the first dominant frequency in each of the frames, and are separated from one or more prior dominant frequencies including the first dominant frequency by at least a predetermined frequency distance.

5 . The method of claim 2 , wherein the spectral estimation is performed using Fast Fourier Transform (FFT).

6 . The method of claim 2 , wherein each of the detected dominant frequencies is mapped to a respective one of the cutaneous actuators that are activated on the receiving user's skin.

7 . The method of claim 2 , further comprising generating the actuator signals that activates cutaneous actuators mapped to the detected dominant frequencies for a period corresponding to each frame.

8 . The method of claim 7 , wherein the period is longer or shorter than a length of the frame.

9 . A haptic system comprising:
an analog-to-digital converter configured to generate a digitized version of a speech signal; a frequency decomposition encoder configured to:
process the digitized speech signal to generated processed speech signal, and
detect dominant frequencies in the processed speech signal to generate dominant frequency information; and
a communication module configured to provide the detected dominant frequencies to generate actuator signals for activating cutaneous actuators on a receiving user's skin.

10 . The haptic system of claim 9 , wherein the frequency decomposition encoder comprises:
a frame splitter configured to segment the digitized signal into frames, each frame having a predetermined length, a transform module configured to perform spectral estimation on each of the frames to generate a frequency domain spectrum, and a peak detector configured to detected at least a first dominant frequency at which a magnitude of the frequency domain spectrum is highest in each of the frame.

11 . The haptic system of claim 10 , wherein the peak detector is further configured to set to zero the frequency domain spectrum below a first frequency point and beyond a second frequency point higher than the first frequency point.

12 . The haptic system of claim 10 , wherein the peak detector is further configured to detect next dominant frequencies having respective magnitudes in the frequency domain spectrum that are next highest after the first dominant frequency in each of the frames, and are separated from one or more prior dominant frequencies including the first dominant frequency by at least a predetermined frequency distance.

13 . The haptic system of claim 10 , wherein the spectral estimation is performed using Fast Fourier Transform (FFT).

14 . The haptic system of claim 10 , wherein each of the detected dominant frequencies is mapped to a respective one of the cutaneous actuators that are activated on the receiving user's skin.

15 . The haptic system of claim 10 , wherein the actuator signals for activating cutaneous actuators mapped to the detected dominant frequencies for a period corresponding to each frame.

16 . The haptic system of claim 15 , wherein the period is longer or shorter than a length of the frame.

17 . A non-transitory computer-readable storage medium storing instructions, the instructions when executed by a processor cause the processor to:
generate a digitized version of a speech signal; process the digitized speech signal to generate processed speech signal; detect dominant frequencies in the processed speech signal to generate dominant frequency information; and provide the detected dominant frequencies to generate actuator signals for activating cutaneous actuators on a receiving user's skin.

18 . The non-transitory computer-readable storage medium of claim 17 , wherein instructions to detect the dominant frequencies comprises instructions that cause the processor to:
segment the digitized signal into frames, each frame having a predetermined length, and perform spectral estimation on each of the frames to generate a frequency domain spectrum, wherein the dominant frequencies are detected in each of the frame by at least determining a first dominant frequency at which a magnitude of the frequency domain spectrum is highest.

19 . The non-transitory computer-readable storage medium of claim 18 , wherein each of the detected dominant frequencies is mapped to a respective one of the cutaneous actuators that are activated on the receiving user's skin.

20 . The non-transitory computer-readable storage medium of claim 19 , further storing instructions that cause the processor to generate the actuator signals that activates cutaneous actuators mapped to the detected dominant frequencies for a period corresponding to the frame.

Assignees

• Facebook Inc

Inventors

• Turcott Robert

Classifications

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/084

  Backpropagation, e.g. using gradient descent · CPC title

• G06F3/016

  Input arrangements with force or tactile feedback as computer generated output to the user · CPC title

• G06F3/167

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title