Enhanced spatial impression for home audio

Patent metadata

Field	Value
Publication number	US-9560445-B2
Application number	US-201414158796-A
Country	US
Kind code	B2
Filing date	Jan 18, 2014
Priority date	Jan 18, 2014
Publication date	Jan 31, 2017
Grant date	Jan 31, 2017

Abstract

Technologies pertaining to provision of customized audio to each listener in a plurality of listeners are described herein. A sensor outputs data that is indicative of locations of multiple listeners in an environment. The data is processed to determine locations and orientations of the respective heads of the multiple listener in the environment. Based on the locations and orientations of heads of the listeners in the environment, for each listener, respective customized audio signals are generated. The customized audio signals are transmitted to respective beamforming transducers. The beamforming transducers directionally output customized beams for the first listener and the second listener based upon the customized audio signals and locations of the heads of the listeners.

First claim

What is claimed is:

1. A method, comprising:
receiving data that is indicative of locations of respective ears of a first listener and ears of a second listener in an environment;
receiving a binaural audio signal that comprises a first audio signal that is to be directed to left ears and a second audio signal that is to be directed to right ears;
dynamically generating left audio signals and right audio signals based upon:
the data that is indicative of locations of the respective ears of the first listener and the ears of the second listener,
a binaural late reverberation signal that is to be provided to both the first listener and the second listener, and
the binaural audio signal,
wherein the left audio signals represent audio to be output by a first beamforming transducer, and the right audio signals represent audio to be output by a second beamforming transducer;

transmitting the left audio signals to the first beamforming transducer; and
transmitting the right audio signals to the second beamforming transducer, wherein audio beams output by the first beamforming transducer and the second beamforming transducer responsive to receipt of the left audio signals and the right audio signals, respectively, include cancelling components that de-correlate audio at the ears of the first listener and the ears of the second listener and provide both shared and customized spatial audio effects for the first listener and the second listener, the shared spatial audio effects based upon the binaural late reverberation signal, the customized spatial audio effects based upon the binaural audio signal and the data that is indicative of the locations of the respective ears of the first listener and the ears of the second listener.

2. The method of claim 1 , the left audio signals comprising a first left audio signal and a second left audio signal that is different from the first left audio signal, the first beamforming transducer directing a first left audio beam to the first listener based upon the first left audio signal, and the first beamforming transducer directing a second left audio beam to the second listener based upon the second left audio signal.

3. The method of claim 2 , further comprising:
transmitting the data that is indicative of the locations of the ears of the first listener and the ears of the second listener to the first beamforming transducer.

4. The method of claim 3 , the right audio signals comprising a first right audio signal and a second right audio signal that is different from the first right audio signal, the second beamforming transducer directing a first right audio beam to the first listener based upon the first right audio signal, and the second beamforming transducer directing a second right audio beam to the second listener based upon the second right audio signal.

5. The method of claim 4 , further comprising:
transmitting the data that is indicative of the locations of the ears of the first listener and the ears of the second listener to the second beamforming transducer.

6. The method of claim 1 , further comprising:
receiving a video stream from a video camera, the first listener and the second listener captured in the video stream;
detecting the first listener and the second listener in the video stream; and
computing the data that is indicative of the locations of the respective ears of the first listener and the ears of the second listener based upon the detecting of the first listener and the second listener in the video stream.

7. The method of claim 6 , further comprising:
receiving data from a depth sensor; and
computing the data that is indicative of the locations of the respective ears of the first listener and the ears of the second listener based upon the data received from the depth sensor.

8. The method of claim 1 , configured for execution by a video game console.

9. The method of claim 1 , wherein the data that is indicative of the locations of the respective ears of the first listener and the ears of the second listener comprises an image that captures the first listener and the second listener, the method comprising:
recognizing existence of faces of the first and second listeners, respectively, in the image;
responsive to recognizing the existence of the faces in the image, estimating respective poses of the faces in the image; and
estimating the locations of the respective ears of the first listener and the ears of the second listener based upon the respective poses of the faces in the image.

10. The method of claim 1 , the left audio signals and the right audio signals configured to cause the first beamforming transducer and the second beamforming transducer, respectively, to emit audio over an ultrasonic carrier frequency.

11. An audio system, comprising:
a computing apparatus that is in communication with a sensor, a first beamforming transducer, and a second beamforming transducer, the computing apparatus comprises:
at least one processor; and
memory that stores instructions that, when executed by the at least one processor, causes the at least one processor to perform acts comprising:
determining, based upon data output by the sensor, locations and orientations of respective heads of a first listener and a second listener relative to locations of the first beamforming transducer and the second beamforming transducer;
receiving a first audio signal for the first listener and a second audio signal for the second listener, the first audio signal being different from the second audio signal;
generating customized audio signals for the first listener and customized audio signals for the second listener, wherein the customized audio signals for the first listener is based upon the first audio signal and the location and orientation of the head of the first listener, the customized audio signals for the first listener includes a binaural late reverberation signal, and wherein the customized audio signals for the second listener is based upon the second audio signal and the location and orientation of the head of the second listener, the customized audio signals for the second listener includes the binaural late reverberation signal; and
transmitting the customized audio signals to the first beamforming transducer and the second beamforming transducer.

12. The audio system of claim 11 , wherein the customized audio signals for the first listener comprise a first left customized signal and a first right customized signal, the customized audio signals for the second listener comprise a second left customized signal and a second right customized signal, wherein transmitting the customized audio signals to the first beamforming transducer and the second beamforming transducer comprises:
simultaneously transmitting the first left customized signal and the second left customized signal to the first beamforming transducer; and
simultaneously transmitting the first right customized signal and the second right customized signal to the second beamforming transducer.

13. The audio system of claim 12 , the first beamforming transducer comprises a first plurality of speakers, the second beamforming transducer comprises a second plurality of speakers, the acts further comprising:
transmitting the locations of the respective heads of the first listener and the second listener to the first beamforming transducer and the second beamforming transducer, wherein responsive to receiving the customized audio signals and the locations of the respective heads of the first listener and the second listener, the first beamforming transducer directs a first left audio beam to the first listener and a secon

Assignees

• Microsoft Corp
• Microsoft Technology Licensing Llc

Inventors

• Raghuvanshi Nikunj
• Morris Daniel
• Wilson Andrew D
• Rui Yong
• Tan Desney S
• Wing Jeannette M

Classifications

• H04R2203/12

  Beamforming aspects for stereophonic sound reproduction with loudspeaker arrays · CPC title

• H04R2201/403

  Linear arrays of transducers · CPC title

• H04R3/002 Primary

  Damping circuit arrangements for transducers, e.g. motional feedback circuits · CPC title

• H04S7/303 Primary

  Tracking of listener position or orientation · CPC title

• H04R2217/03

  Parametric transducers where sound is generated or captured by the acoustic demodulation of amplitude modulated ultrasonic waves · CPC title