Haptic conversion system using segmenting and combining

Patent metadata

Field	Value
Publication number	US-9898085-B2
Application number	US-201314020461-A
Country	US
Kind code	B2
Filing date	Sep 6, 2013
Priority date	Sep 6, 2013
Publication date	Feb 20, 2018
Grant date	Feb 20, 2018

Abstract

A system is provided that converts an input into one or more haptic effects using segmenting and combining. The system receives an input. The system further segments the input into a plurality of input sub-signals. The system further converts the plurality of input sub-signals into a haptic signal. The system further generates the one or more haptic effects based on the haptic signal.

First claim

We claim:

1. A non-transitory computer-readable medium having instructions stored thereon that, when executed by a processor, cause the processor to convert an input into one or more haptic effects using segmenting and combining, the converting comprising:
receiving the input;
segmenting the input into input sub-signals, the segmenting comprising filtering the input using one or more filters, wherein each of the input sub-signals comprises a different frequency band that is output from a respective filter;
after the segmenting, prioritizing the input sub-signals based on an analysis parameter, wherein the prioritizing comprises determining for each of the input sub-signals, a value of a characteristic that is defined by the analysis parameter of the input sub-signal, and ordering the input sub-signals based on the determined values;
selecting a subset of the input sub-signals from the input sub-signals based on the prioritizing, wherein the subset is less than the input sub-signals;
calculating a haptic signal based on a combination of the selected subset of input sub-signals, the calculating the haptic signal including:
converting the subset of input sub-signals into haptic sub-signals wherein the converting includes multiplying an input sub-signal by a factor, and
summing the haptic sub-signals into the haptic signal and normalizing the haptic signal; and

generating the one or more haptic effects based on the haptic signal.

2. The computer-readable medium of claim 1 , wherein the one or more filters comprise at least one band-pass filter.

3. The computer-readable medium of claim 1 , wherein the receiving the input further comprises receiving a segment of an input signal.

4. The computer-readable medium of claim 3 , wherein the analysis parameter comprises a maximum magnitude value for each of the input sub-signals.

5. The computer-readable medium of claim 1 ,
wherein the characteristic comprises one of: a frequency, a duration, an envelope, a density, or a magnitude.

6. The computer-readable medium of claim 1 , the converting further comprising:
warping the haptic signal into a warped haptic signal, the warping based on a type of haptic output device that is configured to play the warped haptic signal;
wherein the generating the one or more haptic effects further comprises generating the one or more haptic effects based on the warped haptic signal.

7. The computer-readable medium of claim 1 , wherein the receiving the input further comprises:
receiving a multimedia file; and
extracting an input signal from the multimedia file.

8. The computer-readable medium of claim 1 , wherein the converting the subset of input sub-signals into haptic sub-signals further comprises at least one of:
multiplying an input sub-signal by a factor and a sine carrier waveform;
multiplying the input sub-signal by a factor; or
shifting frequency content from a first frequency band of the input sub-signal to a second frequency band of the input sub-signal.

9. The computer-readable medium of claim 8 wherein each of the input sub-signals is converted into a haptic sub-signal using a unique haptic conversion algorithm.

10. The computer-readable medium of claim 8 , wherein the shifting the frequency content from the first frequency band of the input sub-signal to the second frequency band of the input sub-signal further comprises performing a fast Fourier transform of the input sub-signal.

11. The computer-readable medium of claim 10 , wherein the shifting the frequency content from the first frequency band of the input sub-signal to the second frequency band of the input sub-signal further comprises at least one of:
averaging fast Fourier transform values for frequencies within the first frequency band of the input sub-signal, and assigning the average to a frequency within the second frequency band of the input sub-signal;
summing fast Fourier transform values for frequencies within the first frequency band of the input sub-signal, and assigning the sum to a frequency within the second frequency band of the input sub-signal;
selecting a fast Fourier transform value that has a highest absolute value from fast Fourier transform values for frequencies within the first frequency band of the input sub-signal, and assigning the selected fast Fourier transform value to a frequency within the second frequency band of the input sub-signal;
assigning a fast Fourier transform value for a frequency within the first frequency band of the input sub-signal to frequencies within the second frequency band of the input sub-signal; or
assigning a fast Fourier transform value for a frequency within the first frequency band of the input sub-signal to a lowest frequency within the second frequency band of the input sub-signal.

12. The computer-readable medium of claim 1 , wherein the summing of the haptic sub-signals into the haptic signal comprises at least one of:
segmenting the haptic signal into one or more time-windows, analyzing frequencies of the haptic sub-signals for each time-window, and selecting a haptic sub-signal from the haptic sub-signals as the haptic signal for each time-window, wherein the selected haptic sub-signal comprises a frequency of the frequencies; or
segmenting the input signal into one or more time windows, calculating a power spectrum density percentage contribution for each of the input sub-signals of the input sub-signals for each time window, and calculating a weighted combination of the corresponding haptic sub-signals as the haptic signal for each time-window, wherein a weight of each haptic sub-signal is based on the power spectrum density percentage contribution of each corresponding input sub-signal.

13. The computer-readable medium of claim 1 , the converting further comprising:
normalizing the haptic signal to 1 using its maximum absolute value.

14. The computer-readable medium of claim 1 , the converting further comprising:
cleaning one or more noisy vibrations from the haptic signal.

15. The computer-readable medium of claim 14 , wherein the cleaning the one or more noisy vibrations from the haptic signal further comprises:
selecting one or more sample haptic sub-signals from the haptic signal;
calculating a mean absolute value for each selected sample haptic sub-signal of the one or more selected sample haptic sub-signals;
comparing each mean absolute value with a threshold; and
removing a sample haptic sub-signal from the haptic signal when its corresponding mean absolute value is less than the threshold.

16. The computer-readable medium of claim 1 , wherein the generating the one or more haptic effects further comprising sending the haptic signal to a haptic output device to generate the one or more haptic effects.

17. The computer-readable medium of claim 16 , wherein the haptic output device comprises an actuator.

18. The computer-readable medium of claim 1 , wherein the input comprises one of an audio signal, a video signal, or an acceleration signal.

19. A computer-implemented method for converting an input into one or more haptic effects using segmenting and combining, the computer-implemented method comprising:
receiving the input;
segmenting by a processor the input into input sub-signals, the segmenting comprising filtering the input using one or more filters, wherein each of the input sub-signals comprises a different frequency band that is output from a respective filter;
after the segmenting, prioritizing by the processor the input sub-signals based on an analysis parameter, where

Assignees

• Immersion Corp

Inventors

• Saboune Jamal
• Cruz Hernandez Juan Manuel
• Bhatia Satvir Singh

Classifications

• G06F3/016 Primary

  Input arrangements with force or tactile feedback as computer generated output to the user · CPC title

• G10L21/16

  Transforming into a non-visible representation (devices or methods enabling ear patients to replace direct auditory perception by another kind of perception A61F11/04) · CPC title