Haptic effect conversion system using granular synthesis

Patent metadata

Field	Value
Publication number	US-9733710-B2
Application number	US-201614995858-A
Country	US
Kind code	B2
Filing date	Jan 14, 2016
Priority date	Jun 14, 2012
Publication date	Aug 15, 2017
Grant date	Aug 15, 2017

Abstract

A system is provided that converts an input, such as audio data, into one or more haptic effects. The system applies a granular synthesis algorithm to the input in order to generate a haptic signal. The system subsequently outputs the one or more haptic effects based on the generated haptic signal. The system can also shift a frequency of the input, and also filter the input, before the system applies the granular synthesis algorithm to the input.

First claim

We claim:

1. A non-transitory computer-readable medium having instructions stored thereon that, when executed by a processor, cause the processor to convert an input signal into a haptic effect, the converting comprising:
receiving the input signal;
segmenting the input into a plurality of waveforms, wherein each waveform comprises at least a portion of the input signal;
combining each waveform with an envelope to generate a first grain and a second grain, wherein the first and second grains are each mapped to a particular position on a touchscreen;
receiving a position signal that indicates a position on the touchscreen, wherein the generation of the second grain is based on the position signal;
combining the first and second grains to generate a cloud, wherein the cloud comprises a representation of the first and second grains; and
generating the haptic effect based on the cloud.

2. The non-transitory computer-readable medium of claim 1 , wherein the first and second grains are organized within a grain sequence and wherein each grain has a position within the grain sequence.

3. The non-transitory computer-readable medium of claim 1 , wherein the second grain is mapped to the position on the touchscreen.

4. The non-transitory computer-readable medium of claim 1 , wherein the position signal is generated by a sensor configured to detect a form of energy.

5. The non-transitory computer-readable medium of claim 4 , wherein the form of energy comprises a force, pressure or strain.

6. The non-transitory computer-readable medium of claim 1 , further comprising using the input signal and the position signal to modulate a magnitude of the haptic effect based on an input pressure on the touchscreen.

7. The non-transitory computer-readable medium of claim 1 , wherein the touchscreen comprises a plurality of pixels, wherein each pixel represents a position on the touchscreen.

8. A system for converting an input signal into a haptic effect, the system comprising:
a memory configured to store a haptic conversion module;
a processor configured to execute the haptic conversion module stored on the memory; and
a touchscreen coupled to the processor;
wherein the haptic conversion module is configured to receive the input signal;
wherein the haptic conversion module is further configured to segment the input signal into a plurality of waveforms, wherein each waveform comprises at least a portion of the input signal;
wherein the haptic conversion module is further configured to combine each waveform with an envelope to generate a first grain and a second grain, wherein the first and second grains are each mapped to a particular position on the touchscreen;
wherein the haptic conversion module is further configured to receive a position signal that indicates a position on the touchscreen, wherein the generation of the second grain is based on the position signal
wherein the haptic conversion module is further configured to combine the first and second grains to generate a cloud, wherein the cloud comprises a representation of the first and second grains; and
wherein the haptic conversion module is further configured to generate the haptic effect based on the cloud.

9. The system of claim 8 , wherein the first and second grains are organized within a grain sequence and wherein each grain has a position within the grain sequence.

10. The system of claim 8 , wherein the second grain is mapped to the position on the touchscreen.

11. The system of claim 8 , wherein the position signal is generated by a sensor configured to detect a form of energy.

12. The system of claim 11 , wherein the form of energy comprises a force, pressure or strain.

13. The system of claim 8 , further comprising using the input signal and the position signal to modulate a magnitude of the haptic effect based on an input pressure on the touchscreen.

14. The system of claim 8 , wherein the touchscreen is comprises of a plurality of pixels, wherein each pixel represents a position on the touchscreen.

15. A method to convert an input signal into a haptic effect, the converting comprising:
receiving the input signal;
segmenting the input into a plurality of waveforms, wherein each waveform comprises at least a portion of the input signal;
combining each waveform with an envelope to generate a first grain and a second grain, wherein the first and second grains are each mapped to a particular position on a touchscreen;
receiving a position signal that indicates a position on the touchscreen, wherein the generation of the second grain is based on the position signal;
combining the first and second grains to generate a cloud, wherein the cloud comprises a representation of the first and second grains; and
generating the haptic effect based on the cloud.

16. The method of claim 15 , wherein the first and second grains are organized within a grain sequence and wherein each grain has a position within the grain sequence.

17. The method of claim 15 , wherein the second grain is mapped to the position on the touchscreen.

18. The method of claim 15 , wherein the position signal is generated by a sensor configured to detect a form of energy.

19. The method of claim 18 wherein the form of energy comprises a force, pressure or strain.

20. The method of claim 15 , further comprising using the input signal and the position signal to modulate a magnitude of the haptic effect based on an input pressure on the touchscreen.

Assignees

• Immersion Corp

Inventors

• Cruz-Hernandez Juan Manuel
• Modarres Ali
• Wu Liwen
• Birnbaum David

Classifications

• G06F3/016 Primary

  Input arrangements with force or tactile feedback as computer generated output to the user · CPC title

• G08B6/00

  Tactile signalling systems, e.g. tactile personal calling systems · CPC title

• A63F13/00

  Video games, i.e. games using an electronically generated display having two or more dimensions · CPC title

• A63F13/424

  involving acoustic input signals, e.g. by using the results of pitch or rhythm extraction or voice recognition · CPC title

• A63F13/285

  Generating tactile feedback signals via the game input device, e.g. force feedback · CPC title