Systems and methods for generating haptic effects associated with an envelope in audio signals

Patent metadata

Field	Value
Publication number	US-9576445-B2
Application number	US-201314078442-A
Country	US
Kind code	B2
Filing date	Nov 12, 2013
Priority date	Sep 6, 2013
Publication date	Feb 21, 2017
Grant date	Feb 21, 2017

Abstract

Systems and methods for generating haptic effects associated with envelopes in audio signals are disclosed. One disclosed system for outputting haptic effects includes a processor configured to: receive an audio signal; determine an envelope associated with the audio signal; determine a haptic effect based in part on the envelope; and output a haptic signal associated with the haptic effect.

First claim

What is claimed:

1. A system for outputting haptic effects comprising:
a processor configured to:
receive an audio signal;
determine an envelope signal associated with the audio signal, wherein determining the envelope signal comprises:
determining a Fast Fourier Transform of the audio signal within two or more time windows;
determining a magnitude for the Fast Fourier Transform for each time window;

determine a haptic effect based in part on the envelope signal;
output a haptic signal associated with the haptic effect;

an audio output device configured to receive the audio signal and output an audible effect;
a data store configured to receive the haptic signal and store the haptic effect in a haptic track on the data store; and
a haptic output device in communication with the processor, the haptic output device configured to receive the haptic signal and output the haptic effect.

2. The system of claim 1 , wherein the haptic effect comprises one or more of: a variation in coefficient of friction, a simulated texture, or a vibration.

3. The system of claim 1 , wherein determining an envelope signal associated with the audio signal further comprises:
determining a normalized signal by normalizing the magnitudes; and
determining the positive values from the normalized signal.

4. The system of claim 3 , wherein the time windows are overlapping.

5. The system of claim 4 , wherein the envelope signal is the positive values of the normalized signal.

6. The system of claim 1 , wherein determining an envelope signal associated with the audio signal comprises:
determining a power spectral density of the audio signal within two or more time windows;
determining a sum signal by summing the power spectral density values for each time window;
sampling the sum signal to determine a sample signal; and
interpolating and normalizing the sample signal.

7. The system of claim 6 , wherein the envelope signal is the interpolated and normalized sample signal.

8. The system of claim 1 , wherein the processor is further configured to filter the envelope signal.

9. The system of claim 8 , wherein filtering the envelope signal comprises one or more of: filtering the envelope signal using a threshold and filtering the envelope signal using a level of haptic silence.

10. A method for outputting haptic effects comprising:
receiving, by a processor, an audio signal;
determining, by the processor, an envelope signal associated with the audio signal, wherein determining the envelope signal comprises:
determining a Fast Fourier Transform of the audio signal within two or more time windows;
determining a magnitude for the Fast Fourier Transform for each time window;

determining a haptic effect based in part on the envelope signal;
storing a haptic signal associated with the haptic effect in a haptic track on a data store;
outputting the haptic signal to a haptic output device configured to receive the haptic signal and output the haptic effect; and
receiving the audio signal by an audio output device configured to receive the audio signal and output an audible effect.

11. The method of claim 10 , wherein the haptic effect comprises one or more of: a variation in coefficient of friction, a simulated texture, or a vibration.

12. The method of claim 10 , wherein determining an envelope signal associated with the audio signal further comprises:
determining a normalized signal by normalizing the magnitudes; and
determining the positive values from the normalized signal.

13. The method of claim 12 , wherein the time windows are overlapping.

14. The method of claim 12 , wherein the envelope signal is the positive values of the normalized signal.

15. The method of claim 10 , wherein determining an envelope signal associated with the audio signal comprises:
determining a power spectral density of the audio signal within two or more time windows;
determining a sum signal by summing the power spectral density values for each time window;
sampling the sum signal to determine a sample signal; and
interpolating and normalizing the sample signal.

16. The method of claim 15 , wherein the envelope signal is the interpolated and normalized sample signal.

17. The method of claim 10 , wherein the processor is further configured to filter the envelope signal.

18. The method of claim 17 , wherein filtering the envelope signal comprises one or more of: filtering the envelope signal using a threshold and filtering the envelope signal using a level of haptic silence.

19. A non-transitory computer readable medium comprising program code, which when executed by a processor, is configured to cause the processor to:
receive an audio signal;
determine an envelope signal associated with the audio signal, wherein determining the envelope signal comprises:
determining a Fast Fourier Transform of the audio signal within two or more time windows;
determining a magnitude for the Fast Fourier Transform for each time window;

determine a haptic effect based in part on the envelope signal;
storing a haptic signal associated with the haptic effect in a haptic track on a data store;
output the haptic signal to a haptic output device configured to receive the haptic signal and output the haptic effect; and
output the audio signal to an audio output device configured to output an audible effect.

20. The non-transitory computer readable medium of claim 19 , wherein the haptic effect comprises one or more of: a variation in coefficient of friction, a simulated texture, or a vibration.

21. The non-transitory computer readable medium of claim 19 , wherein determining an envelope signal associated with the audio signal further comprises:
determining a normalized signal by normalizing the magnitudes; and
determining the positive values from the normalized signal.

22. The non-transitory computer readable medium of claim 21 , wherein the time windows are overlapping.

23. The non-transitory computer readable medium of claim 21 , wherein the envelope signal is the positive values of the normalized signal.

24. The non-transitory computer readable medium of claim 19 , wherein determining an envelope signal associated with the audio signal comprises:
determining a power spectral density of the audio signal within two or more time windows;
determining a sum signal by summing the power spectral density values for each time window;
sampling the sum signal to determine a sample signal; and
interpolating and normalizing the sample signal.

25. The non-transitory computer readable medium of claim 24 , wherein the envelope signal is the interpolated and normalized sample signal.

26. The non-transitory computer readable medium of claim 19 , wherein the processor is further configured to filter the envelope signal.

27. The non-transitory computer readable medium of claim 26 , wherein filtering the envelope signal comprises one or more of: filtering the envelope signal using a threshold and filtering the envelope signal using a level of haptic silence.

Assignees

• Immersion Corp
• Immersion Corp

Inventors

• Cruz-Hernandez Juan Manuel
• Saboune Jamal

Classifications

• G06F3/016 Primary

  Input arrangements with force or tactile feedback as computer generated output to the user · CPC title

• G10L21/16

  Transforming into a non-visible representation (devices or methods enabling ear patients to replace direct auditory perception by another kind of perception A61F11/04) · CPC title

• G06F3/165

  Management of the audio stream, e.g. setting of volume, audio stream path · CPC title

• G08B6/00 Primary

  Tactile signalling systems, e.g. tactile personal calling systems · CPC title

• H04L27/265

  Fourier transform demodulators, e.g. fast Fourier transform [FFT] or discrete Fourier transform [DFT] demodulators (H04L27/26524 takes precedence) · CPC title