Haptic signal synthesis and transport in a bit stream

Patent metadata

Field	Value
Publication number	US-9349378-B2
Application number	US-201414536755-A
Country	US
Kind code	B2
Filing date	Nov 10, 2014
Priority date	Nov 19, 2013
Publication date	May 24, 2016
Grant date	May 24, 2016

Abstract

Techniques for synthesizing a parameterized haptic track from a multichannel audio signal and embedding the haptic track in a multichannel audio codec bit stream. The haptic track synthesis can occur while encoding the multichannel audio signal as the multichannel audio codec bit stream. The haptic track can be synthesized in a way that allows select parameters of the haptic track to be extracted from the audio signal. The parameters can be adjusted by a user during the encoding process. For example, the parameter adjustments can be made using a authoring/monitoring tool. The parameter adjustments can be recorded as metadata that, along with the haptic track, is included in the codec bit stream. In some aspects the present technology, the adjustable parameters include center frequency, gain, decay rate, and other parameters that allow for modulation of the haptic track during decoding of the codec bit stream. By synthesizing a parameterized haptic track, A/V content creators can be provided greater control in authoring haptic content to accompany the A/V content they create.

First claim

What is claimed is:

1. A method comprising:
generating a low frequency information audio signal based on a multichannel audio signal;
synthesizing a haptic signal from the low frequency information audio signal;
wherein synthesizing the haptic signal comprises synthesizing, from the low frequency information audio signal, one or more haptic effect parameters;
providing the one or more haptic effect parameters to a tool for adjusting the one or more haptic effect parameters;
receiving haptic effect metadata reflecting one or more user adjustments made using the tool to at least one of the one or more haptic effect parameters;
embedding the haptic signal and the haptic effect metadata in a multichannel audio codec bit stream that encodes at least the multichannel audio signal;
wherein the method is performed by a computing device.

2. The method of claim 1 , wherein generating the low frequency information audio signal comprises extracting a low frequency effects audio channel signal from the multichannel audio signal.

3. The method of claim 1 , wherein generating the low frequency information audio signal comprises generating a composite audio signal based on a right audio channel signal, a left audio channel signal, a left surround audio channel audio, a right surround audio channel signal, and a low frequency effects audio channel signal.

4. The method of claim 1 , wherein one of the one or more haptic effect parameters is center frequency.

5. The method of claim 1 , wherein one of the one or more haptic effect parameters is gain.

6. The method of claim 1 , wherein one of the one or more haptic effect parameters is a modulation haptic effect parameter extracted from low frequency information audio signal based on a spectrogram analysis for controlling the temporal characteristics of haptic effects.

7. The method of claim 1 , wherein embedding the haptic signal comprises compressing the haptic signal to produce a compressed haptic signal and embedding the compressed haptic signal in the multichannel audio codec bit stream.

8. The method of claim 1 , wherein embedding the haptic effect metadata comprises translating the haptic effect metadata to a haptic effect metadata syntax suitable for embedding in the multichannel audio codec bit stream and embedding the haptic effect metadata syntax in the multichannel audio codec bit stream.

9. The method of claim 1 , wherein the haptic signal is embedded in the multichannel audio codec bit stream as a monaural channel.

10. The method of claim 1 , wherein the tool is a digital audio mixing software application.

11. The method of claim 1 , further comprising:
embedding at least a portion of the haptic effect metadata in an auxiliary data field, an addbsi data field, a skip field, or a wastebit field of the multichannel audio codec bit stream.

12. The method of claim 1 , further comprising:
formatting the haptic effect metadata for carriage in a metadata container; and
embedding the metadata container carrying the formatted haptic effect metadata in the multichannel audio codec bit stream.

13. The method of claim 12 , wherein embedding the metadata container carrying the formatted haptic effect metadata comprises embedding the metadata container carrying the formatted haptic effect metadata in skip fields of syncframes of the multichannel audio codec bit stream.

14. The method of claim 1 , further comprising:
formatting the haptic effect metadata for carriage in a dependent substream of the multichannel audio codec bit stream.

15. The method of claim 1 , further comprising:
formatting the haptic effect metadata for carriage in an independent substream of the multichannel audio codec bit stream.

16. The method of claim 1 , further comprising:
embedding at least a portion of the haptic effect metadata in a data stream bit field or a fil element of the multichannel audio codec bit stream.

17. A method comprising:
receiving a multichannel audio codec bitstream comprising a haptic track and associated haptic effect metadata;
decoding the multichannel audio codec bitstream to produce a multichannel audio signal;
sending the multichannel audio signal to one or more audio output devices;
decoding the haptic track to produce a first haptic signal;
generating a second haptic signal based on the associated haptic effect metadata;
modulating the first haptic signal with the second haptic signal to produce a third haptic signal;
sending the third haptic signal to a haptic actuator;
wherein the method is performed by one or more computing devices.

18. A non-transitory computer-readable media storing instructions which, when executed by a computing device, cause performance of a method as recited in claim 1 .

19. A non-transitory computer-readable media storing instructions which, when executed by a computing device, cause performance of a method as recited in claim 17 .

Assignees

• Dolby Lab Licensing Corp

Inventors

• Bharitkar Sunil
• Riedmiller Jeffrey
• Brown C Phillip

Classifications

• G10L21/16

  Transforming into a non-visible representation (devices or methods enabling ear patients to replace direct auditory perception by another kind of perception A61F11/04) · CPC title

• G10L19/008 Primary

  Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing · CPC title

• G08B6/00

  Tactile signalling systems, e.g. tactile personal calling systems · CPC title

• G06F3/016

  Input arrangements with force or tactile feedback as computer generated output to the user · CPC title