Sound to haptic effect conversion system using mapping

Patent metadata

Field	Value
Publication number	US-10339772-B2
Application number	US-201715789250-A
Country	US
Kind code	B2
Filing date	Oct 20, 2017
Priority date	Aug 31, 2012
Publication date	Jul 2, 2019
Grant date	Jul 2, 2019

Abstract

A system generates a haptic signal. The system receives an audio signal, pre-processes the audio signal by modifying the audio signal to create a modified audio signal, and maps the modified audio signal to a haptic signal. The system then sends the haptic signal to an actuator to generate one or more haptic effects.

First claim

We claim:

1. A non-transitory computer-readable medium having instructions stored thereon that, when executed by a processor, cause the processor to perform the operations of:
receiving an audio signal;
pre-processing the audio signal to establish a set of audio regions according to actuator characteristics such that each audio region of the set of audio regions has corresponding actuator characteristics;
separating the audio signal into a plurality of sub-signal sets such that each sub-signal set is associated with a corresponding one of the set of audio regions and includes corresponding sub-signals;
mapping a sub-signal of one of the sub-signal sets of the plurality of sub-signal sets to a haptic signal; and
sending the haptic signal to an actuator having the actuator characteristics corresponding to the audio region associated with the one sub-signal set.

2. The non-transitory computer-readable medium of claim 1 , wherein mapping the sub-signal to the haptic signal further comprises mapping each sub-signal of the one sub-signal set to respective haptic signals, and
wherein sending the haptic signal to the actuator further comprises combining the respective haptic signals into a combined haptic signal and sending the combined haptic signal to the actuator.

3. The non-transitory computer-readable medium of claim 1 , further comprising dividing each audio region of the set of audio regions into a plurality of audio frequency regions, wherein each audio frequency region of the plurality of audio frequency regions in each audio region is associated with a corresponding haptic frequency region.

4. The non-transitory computer-readable medium of claim 3 , further comprising separating sub-signals of each of the plurality of sub-signal sets into corresponding ones of the plurality of audio frequency regions in the audio region corresponding to the sub-signal set.

5. The non-transitory computer-readable medium of claim 3 , wherein mapping the sub-signal to the haptic signal further includes mapping respective sub-signals of each of the plurality of sub-signal sets to corresponding haptic signals in the haptic frequency region corresponding to the audio frequency region of the respective sub-signal.

6. The non-transitory computer-readable medium of claim 1 , wherein establishing the plurality of audio regions is based on actuator characteristics including at least one of a dynamic range, a frequency response, an amplitude, and a physical location of the actuator.

7. The non-transitory computer-readable medium of claim 3 , wherein
frequency ranges of the plurality of audio frequency regions match frequency ranges of the haptic frequency regions corresponding to each audio frequency region.

8. The non-transitory computer-readable medium of claim 3 , wherein
frequency ranges of the plurality of audio frequency regions differ from frequency ranges of the haptic frequency regions corresponding to each audio frequency region.

9. The non-transitory computer-readable medium of claim 7 , wherein the frequency ranges of the haptic frequency regions differ from the frequency ranges of the plurality of audio frequency regions according to the actuator characteristics of the actuators associated with the plurality of audio regions.

10. A method for generating haptics, comprising:
receiving an audio signal;
pre-processing the audio signal to establish a set of audio regions according to actuator characteristics such that each audio region of the set of audio regions has corresponding actuator characteristics;
separating the audio signal into a plurality of sub-signal sets such that each sub-signal set is associated with a corresponding one of the set of audio regions and includes corresponding sub-signals;
mapping a sub-signal of one of the sub-signal sets of the plurality of sub-signal sets to a haptic signal; and
sending the haptic signal to an actuator having the actuator characteristics corresponding to the audio region associated with the one sub-signal set.

11. The method of claim 10 , wherein mapping the sub-signal to the haptic signal further comprises mapping each sub-signal of the one sub-signal set to respective haptic signals, and
wherein sending the haptic signal to the actuator further comprises combining the respective haptic signals into a combined haptic signal and sending the combined haptic signal to the actuator.

12. The method of claim 10 , further comprising dividing each audio region of the set of audio regions into a plurality of audio frequency regions, wherein each audio frequency region of the plurality of audio frequency regions in each audio region is associated with a corresponding haptic frequency region.

13. The method of claim 12 , further comprising separating the sub-signals of each of the plurality of sub-signal sets into corresponding ones of the plurality of audio frequency regions in the audio region corresponding to the sub-signal set.

14. The method of claim 12 , wherein mapping the sub-signal to the haptic signal further includes mapping respective sub-signals of each of the plurality of sub-signal sets to haptic signals in the haptic frequency region corresponding to the audio frequency region of the respective sub-signal.

15. The method of claim 10 , wherein establishing the plurality of audio regions is based on actuator characteristics including at least one of a dynamic range, a frequency response, an amplitude, and a physical location of the actuator.

16. The method of claim 12 , wherein
frequency ranges of the plurality of audio frequency regions match frequency ranges of the haptic frequency regions corresponding to each audio frequency region.

17. The method of claim 12 , wherein
frequency ranges of the plurality of audio frequency regions differ from frequency ranges of the haptic frequency regions corresponding to each audio frequency region.

18. The method of claim 16 , wherein the frequency ranges of the haptic frequency regions differ from the frequency ranges of the plurality of audio frequency regions according to the actuator characteristics of the actuators associated with the plurality of audio regions.

19. A haptic signal generation system, comprising:
a memory configured to store a haptic conversion module;
a processor configured to execute instructions of the haptic conversion module; and
an actuator configured to output at least one haptic effect;
wherein the haptic conversion module includes instructions configured for:
receiving an audio signal,
pre-processing the audio signal to establish a set of audio regions according to actuator characteristics such that each audio region of the set of audio regions has corresponding actuator characteristics,
separating the audio signal into a plurality of sub-signal sets such that each sub-signal set is associated with a corresponding one of the set of audio regions and includes corresponding sub-signals,
mapping a sub-signal of one of the sub-signal sets of the plurality of sub-signal sets to a haptic signal, and
sending the haptic signal to an actuator having the actuator characteristics corresponding to the audio region associated with the one sub-signal set.

20. The haptic signal generation system of claim 19 , wherein mapping the sub-signal to the haptic signal further comprises mapping each sub-signal of the one sub-signal set to respective haptic signals, and
wherein sending the haptic signal to the actuator further comprises combining the respective haptic signals into a combined haptic signal

Assignees

• Immersion Corp

Inventors

• Cruz-Hernandez Juan Manuel
• Ullrich Christopher J

Classifications

• G08B6/00 Primary

  Tactile signalling systems, e.g. tactile personal calling systems · CPC title

• G06F3/016 Primary

  Input arrangements with force or tactile feedback as computer generated output to the user · CPC title

• G10L21/06

  Transformation of speech into a non-audible representation, e.g. speech visualisation or speech processing for tactile aids (G10L15/26 takes precedence) · CPC title

• B06B1/00

  Methods or apparatus for generating mechanical vibrations of infrasonic, sonic, or ultrasonic frequency · CPC title

• G06F3/16

  Sound input; Sound output (speech processing G10L) · CPC title