Spatialized audio output based on predicted position data

Patent metadata

Field	Value
Publication number	US-10979843-B2
Application number	US-201615094620-A
Country	US
Kind code	B2
Filing date	Apr 8, 2016
Priority date	Apr 8, 2016
Publication date	Apr 13, 2021
Grant date	Apr 13, 2021

Abstract

In a particular aspect, an audio processing device includes a position predictor configured to determine predicted position data based on position data. The audio processing device further includes a processor configured to generate an output spatialized audio signal based on the predicted position data.

First claim

What is claimed is:

1. A device comprising:
a memory configured to store instructions; and
a processor coupled to the memory, the processor configured to:
receive position data, the position data indicating a translational position, an orientation, or both associated with a headset that includes the processor at a first time;
determine predicted position data based on the position data, the predicted position data indicating a predicted translational position, a predicted orientation, or both, that is associated with the headset at a second time that is subsequent to the first time and that is based on an estimated latency associated with processing an input spatialized audio signal that includes a multi-channel representation of a three-dimensional (3D) sound field;
apply, to the input spatialized audio signal, a rotation to adjust an orientation of the 3D sound field based on the predicted orientation and a binauralization to generate a binaural audio signal; and
output the binaural audio signal to transducers of the headset for playback.

2. The device of claim 1 , wherein the processor is configured to apply the rotation via processing of the multi-channel representation of the 3D sound field based on a rotation operation.

3. The device of claim 1 , wherein the processor is further configured to apply a translation to the 3D sound field based on the predicted translational position.

4. The device of claim 1 , further comprising the transducers coupled to the processor.

5. The device of claim 4 , further comprising a display coupled to the processor.

6. The device of claim 5 , wherein the transducers are configured to generate audio associated with a virtual reality or augmented reality application, and wherein the display is configured to display visual information associated with the virtual reality or augmented reality application.

7. The device of claim 1 , further comprising one or more sensors coupled to the processor and configured to provide the position data to the processor.

8. The device of claim 7 , further comprising a camera coupled to the processor.

9. The device of claim 1 , wherein the processor is further configured to:
receive second position data, the second position data indicating an updated translational position, an updated orientation, or both, that is associated with the headset at a third time; and
determine second predicted position data based on the second position data, the second predicted position data indicating a second predicted translational position, a second predicted orientation, or both, associated with a fourth time that is subsequent to the third time.

10. The device of claim 9 , further comprising a transceiver coupled to the processor and configured to:
wirelessly transmit an indication of at least the second predicted translational position to a second device; and
receive an updated spatialized audio signal from the second device, the received updated spatialized audio signal corresponding to a translated 3D sound field that is based on the indication.

11. The device of claim 10 , wherein the updated spatialized audio signal is selected, based on the second predicted position data, from among multiple stored 3D audio signals corresponding to different positions of a playback device.

12. The device of claim 10 , wherein the processor is further configured to, prior to playback of the updated spatialized audio signal received from the second device, adjust the updated spatialized audio signal based on a second updated translational position, a second updated orientation, or both, for the fourth time.

13. The device of claim 1 , wherein the multi-channel representation of the 3D sound field corresponds to ambisonics data.

14. A method comprising:
receiving, at a processor, position data indicating a translational position, an orientation, or both, associated with a headset that includes the processor at a first time;
determining, at the processor, predicted position data based on the position data, the predicted position data indicating a predicted translational position, a predicted orientation, or both, that is associated with the headset at a second time that is subsequent to the first time and that is based on an estimated latency associated with processing an input spatialized audio signal that includes a multi-channel representation of a three-dimensional (3D) sound field;
applying, at the processor, a rotation to the input spatialized audio signal to adjust an orientation of the 3D sound field based on the predicted orientation and a binauralization to generate a binaural audio signal; and
outputting the binaural audio signal to transducers of the headset for playback.

15. The method of claim 14 , further comprising:
receiving second position data, the second position data indicating an updated translational position, an updated orientation, or both, that is associated with the headset at a third time;
determining second predicted position data based on the second position data, the second predicted position data indicating a second predicted translational position, a second predicted orientation, or both, associated with a fourth time that is subsequent to the third time;
wirelessly transmitting an indication of at least the second predicted translational position to a second device; and
receiving an updated spatialized audio signal from the second device, the received updated spatialized audio signal corresponding to a translated 3D sound field that is based on the indication.

16. The method of claim 15 , wherein the updated spatialized audio signal is selected, based on the second predicted position data, from among multiple stored 3D audio signals corresponding to different positions of a playback device.

17. The method of claim 14 , wherein the multi-channel representation of the 3D sound field corresponds to ambisonics data.

18. A device comprising:
a memory configured to store instructions; and
a processor coupled to the memory, the processor configured to:
obtain predicted position data indicating a predicted translational position, a predicted orientation, or both, that is associated with a remote device and that is based on an estimated latency associated with processing an input spatialized audio signal that includes a multi-channel representation of a three-dimensional (3D) sound field;
perform one or more modifications to the input spatialized audio signal to generate an output audio signal, the one or more modifications including a translation to adjust a position of the 3D sound field based on the predicted translational position; and
initiate wireless transmission of the output audio signal to the remote device.

19. The device of claim 18 , wherein the remote device includes a headset.

20. The device of claim 18 , wherein the one or more modifications further include a rotation to adjust an orientation of the 3D sound field based on the predicted orientation.

21. The device of claim 20 , wherein the processor is configured to perform the rotation via processing of the multi-channel representation of the 3D sound field based on a rotation operation.

22. The device of claim 20 , wherein the one or more modifications further include a binauralization.

23. The device of claim 18 , further comprising a transceiver configured to receive position data, the position data indicating a translational position, an orientation, or both, that is associated with the remote device

Assignees

• Qualcomm Inc

Inventors

• Thagadur Shivappa Shankar

Classifications

• H04R2430/23

  Direction finding using a sum-delay beam-former · CPC title

• H04R2203/12

  Beamforming aspects for stereophonic sound reproduction with loudspeaker arrays · CPC title

• H04R5/033

  Headphones for stereophonic communication {(details thereof, e.g. relating to batteries, cables or control elements H04R1/10)} · CPC title

• G06F3/167

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• H04S2420/01

  Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD] · CPC title