Haptic effect conversion system using granular synthesis

Patent metadata

Field	Value
Publication number	US-10606358-B2
Application number	US-201816140107-A
Country	US
Kind code	B2
Filing date	Sep 24, 2018
Priority date	Jun 14, 2012
Publication date	Mar 31, 2020
Grant date	Mar 31, 2020

Abstract

A system is provided that converts an input, such as audio data, into one or more haptic effects. The system applies a granular synthesis algorithm to the input in order to generate a haptic signal. The system subsequently outputs the one or more haptic effects based on the generated haptic signal. The system can also shift a frequency of the input, and also filter the input, before the system applies the granular synthesis algorithm to the input.

First claim

We claim:

1. A computer-implemented method to generate a haptic effect comprising:
receiving an input signal corresponding to a user gesture captured by a sensor;
selecting a first grain comprising a first enveloped waveform;
selecting a second grain comprising a second enveloped waveform;
interpolating between the first grain and the second grain according to the input signal to generate an interpolated grain; and
generating a haptic effect based on the interpolated grain.

2. The computer-implemented method of claim 1 , wherein the user gesture includes contact with a touchscreen.

3. The computer-implemented method of claim 1 , wherein the user gesture includes hand movement captured by a motion sensor.

4. The computer-implemented method of claim 1 , wherein the first grain and the second grain are predefined.

5. The computer-implemented method of claim 1 , wherein interpolating between the first grain and the second grain is performed by at least one of a linear interpolator, a logarithmic interpolator, a circular interpolator, a triangular interpolator, and a random interpolator.

6. The computer-implemented method of claim 1 , wherein interpolating between the first grain and the second grain includes interpolating different parameters of the first grain and the second grain according to different interpolations.

7. The computer-implemented method of claim 1 , further comprising:
receiving multiple input signals corresponding to multiple user gestures captured by the sensor; and
transforming the multiple input signals into the input signal.

8. The computer-implemented method of claim 7 , wherein interpolating between the first grain and the second grain includes interpolating different parameters of the first grain and the second grain according to different interpolations.

9. A system for generating a haptic effect, the system comprising:
a sensor configured to capture user gestures;
a memory configured to store a haptic conversion module;
a processor configured to execute the haptic conversion module stored on the memory,
wherein the haptic conversion module is configured to
receive an input signal corresponding to a user gesture captured by the sensor;
select a first grain comprising a first enveloped waveform;
select a second grain comprising a second enveloped waveform;
interpolate between the first grain and the second grain according to the input signal to generate an interpolated grain; and
generate a haptic effect based on the interpolated grain.

10. The system of claim 9 , wherein the user gesture includes contact with a touchscreen.

11. The system of claim 9 , wherein the sensor is a motion sensor and the user gesture includes hand movement captured by the motion sensor.

12. The system of claim 9 , wherein the first grain and the second grain are predefined.

13. The system of claim 9 , wherein the haptic conversion module is further configured to interpolate between the first grain and the second grain with at least one of a linear interpolator, a logarithmic interpolator, a circular interpolator, a triangular interpolator, and a random interpolator.

14. The system of claim 9 , wherein the haptic conversion module is further configured to interpolate different parameters of the first grain and the second grain according to different interpolations.

15. The system of claim 9 , wherein the haptic conversion module is further configured to:
receive multiple input signals corresponding to multiple user gestures captured by the sensor; and
transform the multiple input signals into the input signal.

16. A computer-implemented method for converting an input waveform into one or more haptic effects, the method comprising:
receiving the input waveform, wherein the input waveform is a multimodal waveform including audio and video content;
segmenting the input waveform into a plurality of waveforms, wherein each waveform of the plurality of waveforms comprises at least a portion of the input waveform;
combining each waveform of the plurality of waveforms with an envelope to generate a grain, wherein a plurality of grains are generated;
combining the plurality of grains to generate a cloud, wherein the cloud comprises a representation of the plurality of grains; and
generating the one or more haptic effects based on the cloud.

17. The computer-implemented method of claim 16 , further comprising modifying the input waveform before segmenting the input waveform, wherein modifying the input waveform includes shifting a frequency of the input waveform.

18. The computer-implemented method of claim 16 , further comprising modifying the input waveform before segmenting the input waveform, wherein modifying the input waveform includes increasing a magnitude of the input waveform.

Assignees

• Immersion Corp

Inventors

• Cruz Hernandez Juan Manuel
• Modarres Ali
• Wu Liwen
• Birnbaum David

Classifications

• G06F3/016 Primary

  Input arrangements with force or tactile feedback as computer generated output to the user · CPC title

• G08B6/00

  Tactile signalling systems, e.g. tactile personal calling systems · CPC title

• A63F13/424

  involving acoustic input signals, e.g. by using the results of pitch or rhythm extraction or voice recognition · CPC title

• A63F13/285

  Generating tactile feedback signals via the game input device, e.g. force feedback · CPC title