Directional propagation

Patent metadata

Field	Value
Publication number	US-10602298-B2
Application number	US-201816103702-A
Country	US
Kind code	B2
Filing date	Aug 14, 2018
Priority date	May 15, 2018
Publication date	Mar 24, 2020
Grant date	Mar 24, 2020

Abstract

The description relates to parametric directional propagation for sound modeling and rendering. One implementation includes receiving virtual reality space data corresponding to a virtual reality space. The implementation can include using the virtual reality space data to simulate directional impulse responses for initial sounds emanating from multiple moving sound sources and arriving at multiple moving listeners. The implementation can include using the virtual reality space data to simulate directional impulse responses for sound reflections in the virtual reality space. The directional impulse responses can be encoded and used to render sound that accounts for a geometry of the virtual reality space.

First claim

The invention claimed is:

1. A system, comprising:
a processor; and
storage storing computer-readable instructions which, when executed by the processor, cause the processor to:
receive directional impulse responses corresponding to a virtual reality space, the directional impulse responses corresponding to multiple sound source locations and multiple listener locations in the virtual reality space, and specifying perceived arrival directions of initial sounds at individual listener locations as emitted from individual source locations based at least on geometry included in the virtual reality space;
compress the directional impulse responses using parameterized encoding to generate perceptual parameter fields; and
store the perceptual parameter fields on the storage,
the perceived arrival directions being encoded in the perceptual parameter fields, and the perceived arrival directions encoded in the perceptual parameter fields providing a basis for subsequent rendering of directional initial sounds emanating from specific source locations and arriving at specific listener locations as perceived by specific listeners and accounting for reference frames of the specific listeners in the virtual reality space.

2. The system of claim 1 , wherein the parameterized encoding uses 9D parameterization that accounts for the perceived arrival directions.

3. The system of claim 1 , wherein the perceptual parameter fields relate to both the initial sounds and sound reflections.

4. The system of claim 3 , wherein the perceptual parameter fields account for an initial sound delay before the initial sounds.

5. The system of claim 3 , wherein the perceptual parameter fields account for a reflection delay between the initial sounds and the sound reflections.

6. The system of claim 3 , wherein the perceptual parameter fields account for a decay of the sound reflections over time.

7. A system, comprising:
a processor; and
storage storing computer-readable instructions which, when executed by the processor, cause the processor to:
receive sound event input including sound source data related to a particular sound source having a particular sound source location in a virtual reality space and listener data related to a particular listener having a particular listener location in the virtual reality space;
access perceptual parameters corresponding to the virtual reality space, the perceptual parameters based at least on encoded directional impulse responses specifying arrival directionality of sounds emitted from different source locations and arriving at different listener locations in the virtual reality space;
based at least on the particular sound source location and the particular listener location, identify, in the perceptual parameters, a particular arrival directionality of sound emanating from the particular sound source location as perceived at the particular listener location; and
using the sound event input and the particular arrival directionality, render a directional sound as perceived by the particular listener.

8. The system of claim 7 , wherein the directional sound is an initial sound.

9. The system of claim 7 , wherein the perceptual parameters include initial sound arrival directions at the different listener locations and respective sound reflection directions at the different listener locations.

10. The system of claim 9 , wherein the perceptual parameters account for an occluder in the virtual reality space between the particular sound source location and the particular listener location.

11. The system of claim 7 , wherein the computer-readable instructions further cause the processor to:
render the directional sound on a per sound event basis.

12. The system of claim 7 , wherein the sound event input corresponds to multiple sound events and wherein the computer-readable instructions further cause the processor to:
render sound reflections by aggregating the sound source data from the multiple sound events.

13. The system of claim 12 , wherein the computer-readable instructions further cause the processor to:
aggregate the sound source data from the multiple sound events using directional canonical filters.

14. The system of claim 13 , wherein the directional canonical filters group the sound source data from the multiple sound events into respective sound reflection directions, and wherein the directional canonical filters are each associated with a direction.

15. The system of claim 13 , wherein the sound event input corresponds to multiple sound sources and wherein the computer-readable instructions further cause the processor to:
aggregate the sound source data with additional sound source data related to at least one additional sound source in the virtual reality space using the directional canonical filters to render the sound reflections.

16. The system of claim 13 , wherein the directional canonical filters sum a portion of the sound source data corresponding to a decay time.

17. A system, comprising:
a processor; and
storage storing computer-readable instructions which, when executed by the processor, cause the processor to:
receive sound event input that includes a specific sound source location and a specific listener location of a specific listener in a virtual reality space;
access perceptual parameter fields associated with the virtual reality space, the perceptual parameter fields specifying arrival directions of initial sound emanating from different source locations in the virtual reality space as perceived at different listener locations in the virtual reality space;
based at least on the specific sound source location and the specific listener location included in the sound event input, identify, in the perceptual parameter fields, a specific arrival direction of initial sound emanating from the specific sound source location as perceived at the specific listener location; and
based at least on the specific arrival direction of initial sound, produce rendered sound accounting for a reference frame of the specific listener in the virtual reality space.

18. The system of claim 17 , wherein the computer-readable instructions further cause the processor to:
generate a visual representation of the virtual reality space and produce the rendered sound based at least in part on a voxel map for the virtual reality space.

19. The system of claim 18 , wherein the perceptual parameter fields are generated based at least in part on the voxel map.

20. The system of claim 18 , wherein the voxel map includes an occluder located between the specific sound source location and the specific listener location, and the rendered sound accounts for the occluder.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Raghuvanshi Nikunj
• Snyder John

Classifications

• H04S5/005

  of the pseudo five- or more-channel type, e.g. virtual surround · CPC title

• H04S7/303 Primary

  Tracking of listener position or orientation · CPC title

• H04S2400/11

  Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence) · CPC title

• H04S3/008

  in which the audio signals are in digital form, i.e. employing more than two discrete digital channels (data reduction aspects thereof based on psychoacoustics G10L19/02) · CPC title

• G10K15/08

  Arrangements for producing a reverberation or echo sound {(modifying acoustic properties to change reverberation time G10K11/002)} · CPC title