Sound to haptic effect conversion system using multiple actuators

Patent metadata

Field	Value
Publication number	US-10467870-B2
Application number	US-201816125041-A
Country	US
Kind code	B2
Filing date	Sep 7, 2018
Priority date	Apr 4, 2012
Publication date	Nov 5, 2019
Grant date	Nov 5, 2019

Abstract

A haptic conversion system is provided that analyzes an audio signal, generates a plurality of haptic signals based on the analysis of the audio signal, and plays the generated plurality of haptic signals through a plurality of actuators to produce one or more haptic effects. The haptic conversion system maps the generated plurality of haptic signals to the plurality of actuators based on one or more audio characteristics of the audio signal. Each generated haptic signal includes one or more haptic parameters, and is played at its mapped actuator to generate the one or more haptic effects.

First claim

We claim:

1. A method of producing haptic effects, the method comprising:
outputting a plurality of audio effects by a plurality of actuators, each actuator of the plurality of actuators being located in a different region of a device, wherein a first audio effect of the plurality of audio effects is output from a surface of the device by a first actuator of the plurality of actuators and a second audio effect is output from the surface of the device by a second actuator of the plurality of actuators;
generating a first haptic effect based on the first audio effect by
determining a set of values of audio characteristics of an audio signal corresponding to the first audio effect, the set of values of audio characteristics being generated from an audio data frame and comprising at least one of an amplitude of the audio data frame, a frequency of the audio data frame, and a duration of the audio data frame,
mapping the set of values of audio characteristics to the first actuator,
generating a haptic signal based on the set of values of audio characteristics, wherein the first haptic signal corresponds to the first haptic effect; and

outputting the first haptic effect by the first actuator by
sending the haptic signal to the first actuator based on the mapping of the set of values of audio characteristics to the first actuator, and
playing the haptic signal at the first actuator to generate the first haptic effect.

2. The method of claim 1 , wherein the first haptic effect and the first audio effect are output by the first actuator at the same time.

3. The method of claim 2 , wherein generating the first haptic effect includes encoding the haptic signal corresponding to the first haptic effect by adding the haptic signal to the audio signal corresponding to the first audio effect.

4. The method of claim 1 , further comprising:
generating a second haptic effect based on the second audio effect; and
outputting the second haptic effect by the second actuator.

5. The method of claim 1 , wherein the device further includes one or more additional actuators located on a back portion of the device.

6. The method of claim 5 , wherein the one or more additional actuators comprise full body actuators configured to provide one or more haptic effects at different frequencies.

7. The method of claim 1 ,
wherein generating the first haptic effect further comprises
selecting one or more pre-defined haptic signals from a universal haptic layer library based on the set of values of audio characteristics of the first audio signal, and
mapping the one or more pre-defined haptic signals to the first actuator, and

wherein outputting the first haptic effect further comprises
sending the one or more pre-defined haptic signals to the first actuator.

8. The method of claim 1 , further comprising:
generating one or more secondary haptic signals based on a stored definition, the stored definition comprising a conditional statement that comprises a condition and a statement, wherein the condition is the occurrence of a generation of the haptic signal and the statement is an instruction to generate the one or more secondary haptic signals,
mapping the one or more secondary haptic signals to a secondary actuator, and
outputting the one or more secondary haptic signals to the secondary actuator.

9. The method of claim 1 , wherein generating the first haptic effect further comprises
dividing the haptic signal into haptic signal components, and
mapping one of the haptic signal components to the first actuator, and
wherein outputting the first haptic effect further comprises sending the one of the haptic signal components to the first actuator.

10. The method of claim 1 , wherein the first haptic effect and the first audio effect are output at a location of a finger of a user interacting with the surface of the device.

11. The method of claim 1 , wherein haptic parameters of the haptic signal comprise at least one of an amplitude parameter, a duration parameter, and a frequency parameter.

12. The method of claim 1 , wherein determining the set of values comprises identifying amplitude peaks of the audio signal.

13. The method of claim 1 , wherein the mapping is system-defined.

14. The method of claim 1 , wherein characteristics of the first actuator comprise an ability to generate the haptic effect with at least one of a specified frequency, a specified amplitude, or a specified duration.

15. A haptic conversion system comprising:
a memory configured to store a haptic conversion module;
a processor configured to execute the haptic conversion module stored on the memory; and
a plurality of actuators, each actuator disposed at a different region of a device,
wherein the haptic conversion module is configured to
output a plurality of audio effects by the plurality of actuators, wherein a first audio effect of the plurality of audio effects is output from a surface of the device by a first actuator of the plurality of actuators and a second audio effect is output from the surface of the device by a second actuator of the plurality of actuators,
generate a first haptic effect based on the first audio effect by
determining a set of values of audio characteristics of an audio signal corresponding to the first audio effect, the set of values of audio characteristics being generated from an audio data frame and comprising at least one of an amplitude of the audio data frame, a frequency of the audio data frame, and a duration of the audio data frame,
mapping the set of values of audio characteristics to the first actuator,
generating a haptic signal based on the set of values of audio characteristics, wherein the haptic signal corresponds to the first haptic effect, and

cause output of the first haptic effect by the first actuator by
sending the haptic signal to the first actuator based on the mapping of the set of values of audio characteristics to the first actuator, and
playing the haptic signal at the first actuator to generate the first haptic effect.

16. The system of claim 15 , wherein the processor is further configured to cause the first haptic effect and the first audio effect to be output by the first actuator at the same time.

17. The system of claim 15 , wherein the processor is further configured to
generate a second haptic effect based on the second audio effect, and
cause output of the second haptic effect by the second actuator.

18. The system of claim 15 , wherein the first haptic effect and the first audio effect are output at a location of a finger of a user interacting with the surface of the device.

19. A method of producing haptic effects, the method comprising:
outputting a plurality of audio effects by a plurality of actuators, each actuator of the plurality of actuators being located in a different region of a device, wherein a first audio effect of the plurality of audio effects is output from a surface of the device by a first actuator of the plurality of actuators and a second audio effect is output from the surface of the device by a second actuator of the plurality of actuators;
generating a first haptic effect based on the first audio effect;
generating one or more secondary haptic signals based on a stored definition, the stored definition comprising a conditional statement that comprises a condition and a statement, wherein the condition is the occurrence of a generation of the haptic signal and the statement is an instruction to generate the one or more secondary haptic signals;
mapping th

Assignees

• Immersion Corp

Inventors

• Da Costa Henry
• Timone Herve Thu
• Heubel Robert
• Rank Stephen
• Cruz-Hernandez Juan Manuel
• Grant Danny

Classifications

• G06F3/016 Primary

  Input arrangements with force or tactile feedback as computer generated output to the user · CPC title

• G06F3/16

  Sound input; Sound output (speech processing G10L) · CPC title

• G08B6/00 Primary

  Tactile signalling systems, e.g. tactile personal calling systems · CPC title