Coding higher-order ambisonic audio data with motion stabilization

Patent metadata

Field	Value
Publication number	US-9712936-B2
Application number	US-201514864588-A
Country	US
Kind code	B2
Filing date	Sep 24, 2015
Priority date	Feb 3, 2015
Publication date	Jul 18, 2017
Grant date	Jul 18, 2017

Abstract

In general, techniques and devices are described for motion compensation. An example a device configured to compensate motion. The device includes a memory configured to store audio data associated with a three-dimensional (3D) soundfield and one or more processors. The one or more processors are configured to receive motion information indicating one or more movements associated with a capture of one or more audio objects of a three-dimensional (3D) soundfield by a microphone array, and to adjust virtual positioning information associated with one or more microphones of a microphone array to compensate one or more movements associated with a capture of one or more audio objects of the 3D soundfield by the microphone array. The one or more processors may also be configured to generate a motion-compensated bitstream based on the adjusted virtual positioning information.

First claim

What is claimed is:

1. A method of motion compensation, the method comprising:
receiving, by a device configured to compensate motion, motion information indicating one or more movements associated with a capture of one or more audio objects of a three-dimensional (3D) soundfield by a microphone array;
adjusting, by the device configured to compensate motion, one or more higher-order ambisonics (HOA) representations of the 3D soundfield to compensate the one or more movements associated with the capture of one or more audio objects of the 3D soundfield by the microphone array; and
generating, by the device configured to compensate motion, a motion-compensated bitstream based on the HOA representations of the 3D soundfield.

2. The method of claim 1 , wherein adjusting the one or more HOA representations of the 3D soundfield comprises adjusting, by the device configured to compensate motion, virtual positioning information associated with one or more microphones of the microphone array.

3. The method of claim 1 , wherein adjusting the one or more HOA representations comprises altering, by the device configured to compensate motion, spatial information associated with the one or more HOA representations.

4. The method of claim 1 , wherein adjusting the one or more HOA representations comprises obtaining, by the device configured to compensate motion, an effects matrix associated with the one or more movements.

5. The method of claim 4 , wherein the effects matrix represents an inverse rotation operation with respect to the one or more movements.

6. The method of claim 4 , wherein adjusting the one or more HOA representations comprises applying, by the device configured to compensate motion, the effects matrix to the one or more HOA representations to obtain a motion compensated 3D soundfield.

7. The method of claim 4 , wherein obtaining the effects matrix comprises:
obtaining, by the device configured to compensate motion, rotational information associated with the one or more movements; and
calculating, by the device configured to compensate motion, the effects matrix at least in part by calculating an inverse of the rotational information.

8. The method of claim 4 ,
wherein the effects matrix comprises a set of zero entries and a set of significant entries, and
wherein the set of zero entries includes a greater number of entries than the set of significant entries.

9. The method of claim 1 , wherein adjusting the one or more HOA representations comprises adjusting, by the device configured to compensate motion, the one or more HOA representations for a subset of a plurality of audio samples associated with the 3D soundfield, such that any pair of audio samples of the subset represents an interval of the plurality of the audio samples.

10. The method of claim 9 , wherein the interval comprises one of a ten-sample interval or a twelve-sample interval.

11. The method of claim 9 , further comprising interpolating, by the device configured to compensate motion, a respective effects matrix with respect to each interval, to obtain one or more interpolated effects matrices.

12. The method of claim 11 , further comprising applying, by the device configured to compensate motion, each interpolated effects matrix to a corresponding sample included in a corresponding interval.

13. The method of claim 1 , further comprising differentiating, by the device configured to compensate motion, one or more micromovements from one or more gradual movements associated with the one or more audio objects of the 3D soundfield.

14. The method of claim 13 , wherein differentiating the micromovements from the gradual movements is based on a threshold value associated with one or more of a distance, a frequency, or an angle sharpness describing motion information associated with the capture.

15. The method of claim 1 ,
wherein receiving the motion information indicating the one or more movements associated with the capture of the one or more audio objects of the 3D soundfield by the microphone array comprises receiving, by the device configured to compensate motion, one or more of a yaw angle, a pitch angle, or a roll angle associated with the movements, and
wherein adjusting the one or more HOA representations of the 3D soundfield to compensate the movements comprises compensating, by the device configured to compensate motion, rotation information based on the received one or more of the yaw angle, the pitch angle, or the roll angle.

16. The method of claim 1 , wherein adjusting the one or more HOA representations of the 3D soundfield comprises adjusting, by the device configured to compensate motion, the one or more HOA representations of the 3D soundfield for a time-domain representation of the 3D soundfield.

17. The method of claim 16 , wherein the time-domain representation of the 3D soundfield comprises a pre-transcoding representation of the 3D soundfield.

18. The method of claim 1 , further comprising adjusting, by the device configured to compensate motion, the one or more HOA representations for all audio samples captured by the microphone array with respect to the 3D soundfield.

19. The method of claim 1 , wherein adjusting the one or more HOA representations of the 3D soundfield comprises generating, by the device configured to compensate motion, virtual re-positioning information based on the movements and actual positioning information associated with the microphone array.

20. A device configured to compensate motion, the device comprising:
a memory configured to store audio data associated with a three-dimensional (3D) soundfield; and
one or more processors coupled to the memory, the one or more processors being configured to:
receive motion information indicating one or more movements associated with a capture of one or more audio objects of the three-dimensional (3D) soundfield by a microphone array;
adjust one or more higher-order ambisonics (HOA) representations of the 3D soundfield associated with one or more microphones of a microphone array to compensate the one or more movements associated with the capture of one or more audio objects of the 3D soundfield by the microphone array; and
generate a motion-compensated bitstream based on the adjusted HOA representations of the 3D soundfield.

21. The device of claim 20 , wherein, to receive the motion information indicating the one or more movements associated with the capture of the one or more audio objects of the 3D soundfield by the microphone array, the one or more processors are configured to receive the motion information from a motion sensing device that comprises one or more of an accelerometer or a compass.

22. The device of claim 20 , wherein, to adjust the one or more HOA representations of the 3D soundfield, the one or more processors are configured to adjust virtual positioning information soundfield associated with one or more microphones of the microphone array.

23. The device of claim 20 , wherein, to adjust the one or more HOA representations, the one or more processors are configured to obtain an effects matrix that represents an inverse rotation operation with respect to the one or more movements.

24. The device of claim 20 ,
wherein the one or more processors are configured to adjust the one or more HOA representations of the 3D soundfield by adjusting the one or more HOA representations of the 3D soundfield for a time-domain representation of the 3D soundfield, and
whe

Assignees

• Qualcomm Inc

Inventors

• Peters Nils Günther

Classifications

• H04S2400/15

  Aspects of sound capture and related signal processing for recording or reproduction · CPC title

• H04R2430/21

  Direction finding using differential microphone array [DMA] · CPC title

• H04S7/303

  Tracking of listener position or orientation · CPC title

• G10L21/02 Primary

  Speech enhancement, e.g. noise reduction or echo cancellation (reducing echo effects in line transmission systems H04B3/20; echo suppression in hands-free telephones H04M9/08) · CPC title

• H04S3/008 Primary

  in which the audio signals are in digital form, i.e. employing more than two discrete digital channels (data reduction aspects thereof based on psychoacoustics G10L19/02) · CPC title