Sound to haptic effect conversion system using mapping

Patent metadata

Field	Value
Publication number	US-9818271-B2
Application number	US-201615152699-A
Country	US
Kind code	B2
Filing date	May 12, 2016
Priority date	Aug 31, 2012
Publication date	Nov 14, 2017
Grant date	Nov 14, 2017

Abstract

A system generates a haptic signal. The system receives an audio signal, pre-processes the audio signal by modifying the audio signal to create a modified audio signal, and maps the modified audio signal to a haptic signal. The system then sends the haptic signal to an actuator to generate one or more haptic effects.

First claim

We claim:

1. A non-transitory computer-readable medium having instructions stored thereon that, when executed by a processor, cause the processor to generate a haptic signal, the processor:
receiving an audio signal;
pre-processing the audio signal by modifying the audio signal to create a modified audio signal and adjusting one or more parameters based on a measurement of the modified audio signal by configuring the one or more parameters to enable haptic feedback for foreground events and reduce or mute haptic feedback for background events;
mapping the modified audio signal to a haptic signal based on the one or more parameters; and
sending the haptic signal to an actuator to generate one or more haptic effects.

2. The non-transitory computer-readable medium of claim 1 , wherein the modifying of the audio signal comprises under-sampling the audio signal.

3. The non-transitory computer-readable medium of claim 1 , wherein the modifying of the audio signal comprises fusing an average of two or more channels of the audio signal.

4. The non-transitory computer-readable medium of claim 1 , wherein the modifying of the audio signal comprises filtering the audio signal.

5. The non-transitory computer-readable medium of claim 1 , wherein one or more frequencies in the haptic signal are selected based on one or more characteristics of the actuator.

6. The non-transitory computer-readable medium of claim 5 , wherein the modified audio signal comprises one or more audio sub-signals with a frequency within a low range, wherein the one or more audio sub-signals are mapped to corresponding haptic signals with the one or more frequencies.

7. The non-transitory computer-readable medium of claim 5 , wherein the modified audio signal comprises one or more audio sub-signals with a frequency within a middle range, wherein the one or more audio sub-signals are mapped to corresponding haptic signals with the one or more frequencies.

8. The non-transitory computer-readable medium of claim 5 , wherein the modified audio signal comprises one or more audio sub-signals with a frequency within a high range, wherein the one or more audio sub-signals are mapped to corresponding haptic signals with the one or more frequencies.

9. The non-transitory computer-readable medium of claim 1 , wherein the mapping of the modified audio signal to the haptic signal comprises:
mapping the modified audio signal to one or more haptic signals; and
combining the one or more haptic signals into the haptic signal.

10. The non-transitory computer-readable medium of claim 1 , wherein the mapping of the modified audio signal to the haptic signal comprises modifying the haptic signal based on one or more characteristics of the actuator.

11. The non-transitory computer-readable medium of claim 10 , wherein the modifying of the haptic signal comprises filtering the haptic signal.

12. The non-transitory computer-readable medium of claim 10 , wherein the mapping of the haptic signal is based on a dynamic range, a frequency response, an amplitude, or a physical location of the actuator.

13. The non-transitory computer-readable medium of claim 1 , wherein the processor further maps the haptic signal to the actuator based on one or more characteristics of the actuator.

14. The non-transitory computer-readable medium of claim 13 , wherein the one or more characteristics of the actuator include a dynamic range, a frequency response, an amplitude, or a physical location of the actuator.

15. The non-transitory computer-readable medium of claim 1 , wherein the sending of the haptic signal to the actuator comprises sending the haptic signal to a haptic effect player, wherein the haptic effect player sends the haptic signal to the actuator to generate the one or more haptic effects.

16. A computer-implemented method for generating a haptic signal, comprising:
receiving an audio signal;
pre-processing the audio signal by modifying the audio signal to create a modified audio signal and adjusting one or more parameters based on a measurement of the modified audio signal by configuring the one or more parameters to enable haptic feedback for foreground events and reduce or mute haptic feedback for background events;
mapping the modified audio signal to a haptic signal based on the one or more parameters; and
sending the haptic signal to an actuator to generate one or more haptic effects.

17. The method of claim 16 , wherein the modifying of the audio signal comprises under-sampling the audio signal.

18. The method of claim 16 , wherein the modifying of the audio signal comprises fusing an average of two or more channels of the audio signal.

19. The method of claim 16 , wherein the modifying of the audio signal comprises filtering the audio signal.

20. A haptic signal generation system comprising:
a memory configured to store a haptic module;
a processor configured to execute the haptic module stored on the memory; and
an actuator configured to output one or more haptic effects;
wherein the haptic module is configured to:
receive an audio signal;
pre-process the audio signal by modifying the audio signal to create a modified audio signal and adjusting one or more parameters based on a measurement of the modified audio signal by configuring the one or more parameters to enable haptic feedback for foreground events and reduce or mute haptic feedback for background events;
map the modified audio signal to a haptic signal based on the one or more parameters; and
send the haptic signal to the actuator to generate the one or more haptic effects.

Assignees

• Immersion Corp

Inventors

• Cruz-Hernandez Juan Manuel
• Ullrich Christopher J

Classifications

• G08B6/00 Primary

  Tactile signalling systems, e.g. tactile personal calling systems · CPC title

• G06F3/016 Primary

  Input arrangements with force or tactile feedback as computer generated output to the user · CPC title

• G10L21/06

  Transformation of speech into a non-audible representation, e.g. speech visualisation or speech processing for tactile aids (G10L15/26 takes precedence) · CPC title

• B06B1/00

  Methods or apparatus for generating mechanical vibrations of infrasonic, sonic, or ultrasonic frequency · CPC title

• G06F3/16

  Sound input; Sound output (speech processing G10L) · CPC title