Spatial audio reproduction based on head-to-torso orientation

Patent metadata

Field	Value
Publication number	US-12010506-B2
Application number	US-202217847095-A
Country	US
Kind code	B2
Filing date	Jun 22, 2022
Priority date	Sep 20, 2019
Publication date	Jun 11, 2024
Grant date	Jun 11, 2024

Abstract

A media system and a method of using the media system to reproduce spatial audio based on head-to-torso orientation, are described. The method includes determining a head-to-source orientation and a head-to-torso orientation based on head orientation data generated by a head tracking device. Determining the head-to-torso orientation includes determining torso movements based on movements of the head. The torso can be determined to move when the head movements meet a head movement condition, such as a predetermined angle of movement or pattern of movement. A binaural audio filter that is based on a head-related transfer function corresponding to both the head-to-source orientation and the head-to-torso orientation is applied to an audio input signal to generate an audio output signal. The audio output signal is played to accurately recreate spatial audio having sounds emitted to the user by a sound source. Other aspects are also described and claimed.

First claim

What is claimed is:

1. A method, comprising:
detecting a head movement of a head of a user based on head orientation data generated by a head mounted device;
determining a torso movement of a torso of the user based at least in part on a downward looking camera coupled to the head mounted device, wherein the downward looking camera is arranged to track a pattern of clothing on the torso of the user, wherein determining the torso movement includes identifying and tracking, using the downward looking camera, the pattern of the clothing;
determining a head-to-torso orientation based at least in part on the head movement and the torso movement;
selecting a head-related transfer function (HRTF) according to the head-to-torso orientation; and
applying a binaural audio filter to an audio input signal to generate an audio output signal, wherein the binaural audio filter is based on the HRTF.

2. The method of claim 1 , wherein the HRTF takes into account spatial effects of the head-to-torso orientation on spatial audio produced during playback of the audio output signal.

3. The method of claim 1 , wherein the torso movement is further based on a forward-facing camera that is a part of a portable device.

4. The method of claim 1 further comprising driving, using the audio output signal, a speaker to reproduce spatial sound that accounts for spatial audio effects of the head-to-torso orientation.

5. The method of claim 1 , wherein determining the torso movement is further based on contextual information about one or more of a current state of the user or a current use of the head mounted device that exists when the head movement is detected.

6. The method of claim 5 , wherein the contextual information indicates that the user is ambulatory.

7. The method of claim 5 , wherein the contextual information indicates that the head mounted device is being used to play audio content of a movie.

8. The method of claim 1 , wherein the head movement includes the head moving, and wherein the torso movement includes the torso remaining stationary.

9. The method of claim 1 further comprising playing the audio output signal to recreate spatial audio of sounds.

10. A media system, comprising:
one or more processors configured to:
detect a head movement of a head of a user based on head orientation data generated by a device,
determine a torso movement of a torso of the user based at least in part on an image sensor that is arranged to track a pattern of clothing on the torso of the user, wherein the torso movement differs from the head movement,
determine a head-to-torso orientation based at least in part on the head movement and the torso movement,
select a head-related transfer function (HRTF) according to the head-to-torso orientation, and
apply a binaural audio filter to an audio input signal to generate an audio output signal, wherein the binaural audio filter is based on the HRTF; and
one or more speakers configured to play the audio output signal to recreate spatial audio of sounds that accounts for spatial audio effects of the head-to-torso orientation.

11. The media system of claim 10 , wherein the device is a head mounted device and the image sensor comprises a downward looking camera that is coupled to the head mounted device.

12. The media system of claim 10 , wherein the one or more processors are configured to determine the torso movement based on contextual information about one or more of a current state of the user or a current use of the device that exists when the head movement is detected.

13. The media system of claim 10 , wherein the head movement includes the head moving, and wherein the torso movement includes the torso remaining stationary.

14. A non-transitory computer readable medium containing instructions, which when executed by one or more processors of a media system, cause the media system to perform a method comprising:
detecting a head movement of a head of a user based on head orientation data generated by a device;
determining a torso movement of a of the user based at least in part on an image sensor that is arranged to track a pattern of clothing on the torso of the user, wherein the torso movement differs from the head movement;
determining a head-to-torso orientation based at least in part on the head movement and the torso movement;
selecting a head-related transfer function (HRTF) according to the head-to-torso orientation; and
applying a binaural audio filter to an audio input signal to generate an audio output signal, wherein the binaural audio filter is based on the HRTF.

15. The non-transitory computer readable medium of claim 14 , wherein the device is a head mounted device and the image sensor comprises a downward looking camera that is coupled to the head mounted device.

16. The non-transitory computer readable medium of claim 14 , wherein the method further comprises driving, using the audio output signal, a speaker to reproduce spatial sound that accounts for spatial audio effects of the head-to-torso orientation.

17. The non-transitory computer readable medium of claim 14 , wherein determining the torso movement is further based on contextual information about one or more of a current state of the user or a current use of the device that exists when the head movement is detected.

18. The non-transitory computer readable medium of claim 14 , wherein the head movement includes the head moving, and wherein the torso movement includes the torso remaining stationary.

Assignees

• Apple Inc

Inventors

• Vanne Antti J
• Satongar Darius A
• Vandyke James W
• Merimaa Juha O
• Johnson Martin E
• Huttunen Tomi A

Classifications

• H04R5/033

  Headphones for stereophonic communication {(details thereof, e.g. relating to batteries, cables or control elements H04R1/10)} · CPC title

• G06F3/012

  Head tracking input arrangements · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• H04S2420/01

  Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD] · CPC title

• H04R5/04

  Circuit arrangements, {e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments (combinations of amplifiers H03F3/68; stereophonic systems H04S)} · CPC title