Sound to haptic effect conversion system using multiple actuators

Patent metadata

Field	Value
Publication number	US-10074246-B2
Application number	US-201715635395-A
Country	US
Kind code	B2
Filing date	Jun 28, 2017
Priority date	Apr 4, 2012
Publication date	Sep 11, 2018
Grant date	Sep 11, 2018

Abstract

A haptic conversion system is provided that analyzes an audio signal, generates a plurality of haptic signals based on the analysis of the audio signal, and plays the generated plurality of haptic signals through a plurality of actuators to produce one or more haptic effects. The haptic conversion system maps the generated plurality of haptic signals to the plurality of actuators based on one or more audio characteristics of the audio signal. Each generated haptic signal includes one or more haptic parameters, and is played at its mapped actuator to generate the one or more haptic effects.

First claim

We claim:

1. A non-transitory computer-readable medium having instructions stored thereon that, when executed by a processor, cause the processor to execute a method of producing haptic effects, the method comprising:
outputting audio effects by a plurality of actuators, each actuator of the plurality of actuators being located in a different region of a device, wherein at least two of the audio effects are output from different parts of a surface of the device, wherein at least one audio effect is output at a location where a finger of a user interacts with the surface of the device;
generating haptic effects based on the audio effects; and
outputting the haptic effects at the plurality of actuators, wherein at least two of the haptic effects are output from different parts of the surface of the device, wherein the plurality of actuators output the haptic effects and the audio effects at the same time.

2. The non-transitory computer-readable medium of claim 1 , wherein the haptic effects are generated based on an encoding scheme.

3. The non-transitory computer-readable medium of claim 2 , wherein the encoding scheme comprises a Dolby Digital encoding scheme.

4. The non-transitory computer-readable medium of claim 2 , wherein haptic encoding is performed as part of audio encoding by adding low frequency content to audio information.

5. The non-transitory computer-readable medium of claim 1 , wherein the device further includes one or more additional actuators located on a back portion of the device.

6. The non-transitory computer-readable medium of claim 5 , wherein the one or more additional actuators comprise full body actuators configured to provide one or more haptic effects at different frequencies.

7. The non-transitory computer-readable medium of claim 1 , wherein generating the haptic effects comprises:
determining sets of values of audio characteristics of an audio signal, each of the sets of values of audio characteristics comprising at least a frequency, a magnitude, or a duration,
mapping the sets of values of audio characteristics to the actuators, wherein each of the sets of values of audio characteristics is mapped to a different one of the actuators, and
generating haptic signals based on the sets of values of audio characteristics, wherein the haptic signals include a haptic signal generated for each of the sets of values of audio characteristics, and
wherein outputting the haptic effects comprises
sending the haptic signals to the actuators based on the mapping of the sets of values of audio characteristics to the actuators, and
playing the haptic signals at the actuators to generate the haptic effects.

8. The non-transitory computer-readable medium of claim 7 , wherein generating the haptic effects further comprises:
selecting one or more pre-defined haptic signals from a universal haptic layer library based on the sets of values of audio characteristics of the audio signal, and
mapping the one or more pre-defined haptic signals to the actuators, wherein each of the one or more pre-defined haptic signals is mapped to a corresponding actuator, and
wherein outputting the haptic effects further comprises:
sending each of the one or more pre-defined haptic signals to the corresponding actuator.

9. The non-transitory computer-readable medium of claim 7 , wherein generating the haptic effects further comprises:
generating one or more secondary haptic signals based on a stored definition, wherein the stored definition comprises a conditional statement that comprises a condition and a statement, wherein the condition is the occurrence of a generation of one or more primary haptic signals that are sent to a primary actuator, and the statement is an instruction to generate the one or more secondary haptic signals that are sent to a secondary actuator, and
mapping the one or more secondary haptic signals to the secondary actuator, and
wherein outputting the haptic effects further comprises
sending each of the one or more secondary haptic signals to the secondary actuator based on the stored definition.

10. The non-transitory computer-readable medium of claim 7 , wherein generating the haptic effects further comprises:
dividing at least one haptic signal into haptic signal components, and
mapping each of the haptic signal components to corresponding actuators, and
wherein outputting the haptic effects further comprises
sending the haptic signal components to the corresponding actuators.

11. The non-transitory computer-readable medium of claim 7 , wherein generating the haptic effects further comprises:
combining at least two of the haptic signals into a combined haptic signal, and
mapping the combined haptic signal to a mapped actuator, and
wherein outputting the haptic effects further comprises
sending the combined haptic signal to the mapped actuator.

12. The non-transitory computer-readable medium of claim 7 , wherein haptic parameters of the haptic signals comprise at least one of an amplitude parameter, a duration parameter, or a frequency parameter.

13. The non-transitory computer-readable medium of claim 7 , wherein determining sets of values comprises identifying amplitude peaks of the audio signal.

14. The non-transitory computer-readable medium of claim 7 , wherein the sets of values of audio characteristics comprise at least one of an amplitude of an audio data frame, a frequency of an audio data frame, and a duration of an audio data frame.

15. The non-transitory computer-readable medium of claim 7 , wherein the mapping is at least one of system-defined and user-defined.

16. The non-transitory computer-readable medium of claim 7 , wherein characteristics of each actuator comprise an ability to generate a haptic effect with at least one of: a specified frequency; a specified amplitude; or a specified duration.

17. The non-transitory computer-readable medium of claim 7 , wherein a user experiences synchronized playback of the haptic effects.

18. The non-transitory computer-readable medium of claim 7 ,
wherein the audio signal comprises channels;
wherein generating the haptic signals further comprises generating distinct haptic signals for the channels of the audio signal;
wherein sending of the haptic signals to the respective one of the actuators sends a distinct haptic signal to a respective distinct actuator.

19. A computer-implemented method for producing haptic effects, the computer-implemented method comprising:
outputting audio effects by a plurality of actuators, each actuator of the plurality of actuators included in a different region of a device, wherein at least two of the audio effects are output from different parts of a surface of the device, wherein at least one audio effect is output at a location where a finger of a user interacts with the surface of the device;
generating haptic effects based on the audio effects; and
outputting the haptic effects at the plurality of actuators, wherein at least two of the haptic effects are output from different parts of the surface of the device, wherein the plurality of actuators output the haptic effects and the audio effects at the same time.

20. A haptic conversion system comprising:
a memory configured to store a haptic conversion module;
a processor configured to execute the haptic conversion module stored on the memory; and
a plurality of actuators, each actuator included in a different region of a device;
wherein the haptic conversion module is configured to
output audio effects by the

Assignees

• Immersion Corp

Inventors

• Da Costa Henry
• Timone Herve Thu
• Heubel Robert
• Rank Stephen
• Cruz-Hernandez Juan Manuel
• Grant Danny

Classifications

• G06F3/16

  Sound input; Sound output (speech processing G10L) · CPC title

• G06F3/016 Primary

  Input arrangements with force or tactile feedback as computer generated output to the user · CPC title

• G08B6/00 Primary

  Tactile signalling systems, e.g. tactile personal calling systems · CPC title