Spatialized audio relative to a peripheral device

Patent metadata

Field	Value
Publication number	US-11356795-B2
Application number	US-202016904087-A
Country	US
Kind code	B2
Filing date	Jun 17, 2020
Priority date	Jun 17, 2020
Publication date	Jun 7, 2022
Grant date	Jun 7, 2022

Abstract

An audio system, method, and computer program product which includes a wearable audio device and a mobile peripheral device. Each device is capable of determining its respective absolute or relative position and orientation. Once the relative positions and orientations between the devices are known, virtual sound sources are generated at fixed positions and orientations relative to the peripheral device such that any change in position and/or orientation of the peripheral device produces a proportional change in the position and/or orientation of the virtual sound sources. Additionally, first order and second order reflected audio paths may be simulated for each virtual sound source to increase the realism of the simulated sources. Each sound path can be produced by modifying the original audio signal using head-related transfer functions (HRTFs) to simulate audio as though it were perceived by the user's left and right ears as coming from each virtual sound source.

First claim

What is claimed is:

1. A computer program product for simulating audio signals, the computer program product including a set of non-transitory computer-readable instructions stored in a memory, the set of non-transitory computer-readable instructions being executable on a processor and configured to:
obtain or receive an orientation of a wearable audio device relative to an orientation of a peripheral device within an environment;
generate a first modified audio signal, wherein the first modified audio signal is modified using a first head-related transfer function (HRTF) based at least in part on the orientation of the wearable audio device relative to the peripheral device, wherein generating the first modified audio signal comprises simulating a first direct sound originating from a first virtual sound source at a virtual position that is spatially fixed with respect to at least the orientation of the peripheral device;
generate a second modified audio signal, wherein the second modified audio signal is modified using a second head-related transfer function (HRTF) based at least in part on the orientation of the wearable audio device relative to the peripheral device, wherein generating the second modified audio signal comprises simulating a second direct sound originating from a second virtual sound source at a virtual position that is spatially fixed with respect to at least the orientation of the peripheral device;
send the first modified audio signal and the second modified audio signal to the wearable audio device, wherein the first modified audio signal is configured to be rendered using a first speaker of the wearable audio device and the second modified audio signal is configured to be rendered using a second speaker of the wearable audio device; and
change the virtual positions of the first and second virtual sound sources in response to a change in the orientation of the peripheral device, wherein the changed virtual positions of the first and second virtual sound sources are spatially fixed with respect to at least the changed orientation of the peripheral device.

2. The computer program product of claim 1 , wherein the set of non-transitory computer readable instructions are further configured to:
obtain or receive a position of the wearable audio device relative to a position of the peripheral device within the environment and wherein modifying the first modified audio signal and modifying the second modified audio signal include attenuation based at least in part on a calculated distance between the position of the wearable audio device and the position of the peripheral device.

3. The computer program product of claim 1 , wherein the first modified audio signal and the second modified audio signal are configured to simulate the first direct sound originating from the first virtual sound source proximate a center of the peripheral device.

4. The computer program product of claim 1 , wherein generating the first modified audio signal and generating the second modified audio signal include simulating a first direct sound originating from the first virtual sound source proximate a position of the peripheral device within the environment and simulating the second direct sound originating from the second virtual sound source proximate the position of the peripheral device.

5. The computer program product of claim 1 , wherein generating the first modified audio signal and generating the second modified audio signal include simulating surround sound based on virtual sound sources that are also spatially fixed with respect to at least the orientation of the peripheral device.

6. The computer program product of claim 1 , wherein generating the first modified audio signal and generating the second modified audio signal include using the first HRTF and the second HRTF, respectively, for only a subset of all available audio frequencies and/or channels.

7. The computer program product of claim 1 , wherein the first HRTF and the second HRTF are further configured to utilize localization data from a localization module within the environment corresponding to locations of a plurality of acoustically reflective surfaces within the environment.

8. The computer program product of claim 7 , wherein generating the first modified audio signal includes simulating a first primary reflected sound corresponding to a simulated reflection of the first direct sound off of a first acoustically reflective surface of the plurality of acoustically reflective surfaces; wherein the first primary reflected sound originates from a primary mirrored virtual sound source at a virtual position that is spatially fixed with respect to at least the orientation of the peripheral device.

9. The computer program product of claim 8 , wherein generating the first modified audio signal includes simulating a secondary reflected sound corresponding to a simulated reflection of the primary reflected sound off of a second acoustically reflective surface of the plurality of acoustically reflective surfaces.

10. The computer program product of claim 1 , wherein the first modified audio signal and the second modified audio signal correspond to video content displayed on the peripheral device.

11. The computer program product of claim 1 , wherein the orientation of the wearable audio device relative to the peripheral device is determined using at least one sensor, wherein the at least one sensor is located on, in, or in proximity to the wearable audio device or the peripheral device, and the at least one sensor is selected from: a gyroscope, an accelerometer, a magnetometer, a global positioning sensor (GPS), a proximity sensor, a microphone, a lidar sensor, or a camera.

12. A method of simulating audio signals, the method comprising:
receiving, via a wearable audio device from a peripheral device, a first modified audio signal, wherein the first modified audio signal is modified using a first head-related transfer function (HRTF) based at least in part on an orientation of the wearable audio device relative to an orientation of the peripheral device, wherein the first modified audio signal is configured to simulate a first direct sound originating from a first virtual sound source at a virtual position that is spatially fixed with respect to at least the orientation of the peripheral device;
receiving, via the wearable audio device from the peripheral device, a second modified audio signal, wherein the second modified audio signal is modified using a second head-related transfer function (HRTF) based at least in part on the orientation of the wearable audio device relative to the peripheral device, wherein the second modified audio signal is configured to simulate a second direct sound originating from a second virtual sound source at a virtual position that is spatially fixed with respect to at least the orientation of the peripheral device;
rendering the first modified audio signal using a first speaker of the wearable audio device;
rendering the second modified audio signal using a second speaker of the wearable audio device; and
changing the virtual positions of the first and second virtual sound sources in response to a change in the orientation of the peripheral device, wherein the changed virtual positions of the first and second virtual sound sources are spatially fixed with respect to at least the changed orientation of the peripheral device.

13. The method of claim 12 , wherein the method further comprises:
obtaining a position of the wearable audio device relative to the peripheral device within an environment and wherein modifying the first modified audio signal and modifying the second mod

Assignees

• Bose Corp

Inventors

• Freeman Eric J
• Dick David Avi
• Torres Wade P
• Tengelsen Daniel R
• Bernstein Eric Raczka

Classifications

• H04S7/304 Primary

  For headphones · CPC title

• H04S7/306 Primary

  For headphones · CPC title

• H04S2400/11

  Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence) · CPC title

• H04S2420/01

  Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD] · CPC title