Method for producing haptic signal and electronic device supporting the same

Patent metadata

Field	Value
Publication number	US-10229565-B2
Application number	US-201715825935-A
Country	US
Kind code	B2
Filing date	Nov 29, 2017
Priority date	Nov 30, 2016
Publication date	Mar 12, 2019
Grant date	Mar 12, 2019

Abstract

An electronic device includes a memory, a communication interface capable of performing communication, and a processor electrically connected to the memory and the communication interface. The processor is configured to extract a first audio signal to be output through a speaker, based on audio data received from an external electronic device and/or stored in the memory, to generate a second audio signal obtained by removing the first audio signal from the audio data, and to generate a haptic signal for outputting a haptic effect, based on the second audio signal.

First claim

What is claimed is:

1. An electronic device comprising:
a memory;
a communication interface, including communication interface circuitry; and
a processor electrically connected to the memory and the communication interface,
wherein the processor is configured to:
extract a first audio signal to be output through a speaker, based at least on audio data received from an external electronic device and/or stored in the memory;
generate a second audio signal obtained by at least removing the first audio signal from the audio data, wherein the second audio signal includes at least a noise signal of the audio data; and
generate a haptic signal for outputting a haptic effect, based at least on the second audio signal.

2. The electronic device of claim 1 , wherein the electronic device is one of a camera device, a haptic device, or a server device or is a device for controlling at least one of the camera device and the haptic device.

3. The electronic device of claim 1 , wherein the processor is configured to:
segment the audio data at a preset time period to generate a plurality of audio frames; and
perform Fourier transform on the plurality of audio frames.

4. The electronic device of claim 3 , wherein at least one of the audio frames has a section in which the audio frame overlaps another audio frame of a time period before and/or after the audio frame.

5. The electronic device of claim 3 , wherein the processor is configured to:
remove the noise signal from the plurality of audio frames via noise canceling to extract the first audio signal.

6. The electronic device of claim 3 , wherein the second audio signal comprises a signal to noise ratio (SNR) for each frequency included in the plurality of audio frames.

7. The electronic device of claim 1 , wherein the processor is configured to:
segment the second audio signal at a preset time period to generate a plurality of audio frames; and
perform sound pressure level (SPL) on the plurality of audio frames to determine intensity of the haptic effect.

8. The electronic device of claim 7 , wherein the processor is configured to:
analyze a statistical feature of the plurality of audio frames; and
determine a type and/or an output method of the haptic effect.

9. The electronic device of claim 8 , wherein the processor is configured to:
calculate Mel-frequency cepstral coefficients (MFCC) and/or a spectral slope of each of the plurality of audio frames.

10. The electronic device of claim 9 , wherein the processor is configured to:
if variance of the MFCC is greater than a preset value, set a change value of vibration according to the haptic effect to a specified value or more.

11. The electronic device of claim 9 , wherein the processor is configured to:
if variance of the MFCC is less than a preset value, set a change value of vibration according to the haptic effect to a specified value or less.

12. The electronic device of claim 1 , wherein the audio data is collected through a microphone array, and
wherein the processor is configured to:
detect an interaural time difference (ITD) and/or an interaural level difference (ILD) of the second audio signal.

13. The electronic device of claim 12 , wherein the processor is configured to:
determine a direction of the haptic effect based on the detected ITD and/or ILD.

14. The electronic device of claim 1 , wherein the processor is configured to:
receive additional information associated with the audio data from an external device; and
correct the haptic signal based at least on the additional information.

15. A haptic signal producing method performed by an electronic device, the method comprising:
extracting a first audio signal to be output through a speaker, based at least on audio data received from an external electronic device and/or stored in the memory;
generating a second audio signal obtained by at least removing the first audio signal from the audio data, wherein the second audio signal includes a noise signal of the audio data; and
generating a haptic signal for outputting an haptic effect, based at least on the second audio signal.

16. The method of claim 15 , wherein the extracting of the first audio signal includes:
removing the noise signal from a plurality of audio frames depending on a specified noise cancellation algorithm to extract the first audio signal.

17. The method of claim 15 , wherein the generating of the haptic signal includes:
segmenting the second audio signal at a preset time period to generate a plurality of audio frames; and
performing SPL on the plurality of audio frames to determine intensity of the haptic effect.

18. The method of claim 15 , wherein the generating of the haptic signal includes:
calculating MFCC and/or a spectral slope of each of a plurality of audio frames.

19. The method of claim 18 , wherein the generating of the haptic signal includes:
calculating MFCC of each of a plurality of audio frames;
if variance of the MFCC is greater than a preset value, setting a change value of vibration according to the haptic effect to a specified value or more; and
if variance of the MFCC is less than the preset value, setting the change value of vibration according to the haptic effect to the specified value or less.

20. The method of claim 15 , wherein the generating of the second audio signal includes:
receiving additional information associated with the audio data from an external device; and
correcting the haptic signal based at least on the additional information.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Cha Aran
• Moon Gong Bo
• Kudryavtsev Vadim
• Kim Kyung Tae
• Lee Sun Key
• Lee Yo Han
• Hyun Eun Jung

Classifications

• G10L21/06

  Transformation of speech into a non-audible representation, e.g. speech visualisation or speech processing for tactile aids (G10L15/26 takes precedence) · CPC title

• G10K11/16

  Methods or devices for protecting against, or for damping, noise or other acoustic waves in general (G10K11/36 takes precedence) · CPC title

• A63F13/213 Primary

  comprising photodetecting means, e.g. cameras, photodiodes or infrared cells (A63F13/219 takes precedence) · CPC title

• H04N13/332

  Displays for viewing with the aid of special glasses or head-mounted displays [HMD] · CPC title

• A63F13/00

  Video games, i.e. games using an electronically generated display having two or more dimensions · CPC title