Media-compensated pass-through and mode-switching

Patent metadata

Field	Value
Publication number	US-11740859-B2
Application number	US-202217660951-A
Country	US
Kind code	B2
Filing date	Apr 27, 2022
Priority date	Jun 14, 2016
Publication date	Aug 29, 2023
Grant date	Aug 29, 2023

Abstract

Media input audio data corresponding to a media stream and microphone input audio data from at least one microphone may be received. A first level of at least one of a plurality of frequency bands of the media input audio data, as well as a second level of at least one of a plurality of frequency bands of the microphone input audio data, may be determined. Media output audio data and microphone output audio data may be produced by adjusting levels of one or more of the first and second plurality of frequency bands based on the perceived loudness of the microphone input audio data, of the microphone output audio data, of the media output audio data and the media input audio data. One or more processes may be modified upon receipt of a mode-switching indication.

First claim

The invention claimed is:

1. A method, comprising:
receiving media input audio data corresponding to a media stream;
receiving microphone input audio data from at least one microphone;
computing a media input excitation function of the media input audio data;
computing a microphone input excitation function of the microphone input audio data;
determining, based at least in part on the microphone input excitation function and according to a specific loudness function that models non-linearities in a human's perception of loudness, a microphone input specific loudness of the microphone input audio data, the microphone input specific loudness corresponding with a perceived loudness of the microphone input audio data;
determining, based at least in part on the media input excitation function and the microphone input excitation function, a microphone partial specific loudness corresponding to the perceived loudness of the microphone input audio data in a presence of the media input audio data;
determining frequency- and time-varying media processing gains to apply to the media input audio data to produce media output audio data;
determining frequency- and time-varying microphone processing gains to apply to the microphone input audio data to produce microphone output audio data, wherein the media processing gains and the microphone processing gains are determined such that a first difference between the microphone input specific loudness and a perceived loudness of the microphone output audio data in a presence of the media output audio data is less than a second difference between the microphone input specific loudness and the microphone partial specific loudness;
applying the determined media processing gains to the media input audio data to produce the media output audio data;
applying the determined microphone processing gains to the microphone input audio data to produce the microphone output audio data and
mixing the media output audio data and the microphone output audio data to produce mixed audio data.

2. The method of claim 1 , wherein the perceived loudness of the microphone output audio data in the presence of the media output audio data is substantially equal to the perceived loudness of the microphone input audio data.

3. The method of claim 1 , wherein applying the determined media processing gains and applying the determined microphone processing gains involve boosting the levels of one or more frequency bands of the microphone input audio data and attenuating the levels of one or more frequency bands of the media input audio data.

4. The method of claim 1 , further comprising providing the mixed audio data to speakers of a headset.

5. The method of claim 1 , further comprising:
receiving a mode-switching indication; and
modifying at least one of the receiving, determining or mixing process based, at least in part, on the mode-switching indication.

6. The method of claim 5 , wherein the modifying involves increasing a relative loudness of the microphone output audio data, relative to a loudness of the media output audio data.

7. The method of claim 6 , wherein increasing the relative loudness of the microphone output audio data involves suppressing the media input audio data or pausing the media stream.

8. The method of claim 5 , wherein the mode-switching indication is based, at least in part, on at least one of an indication of a head movement or an indication of an eye movement.

9. The method of claim 5 , wherein the mode-switching indication is based, at least in part, on inertial sensor data.

10. The method of claim 9 , wherein the inertial sensor data corresponds to movement of a headset.

11. One or more non-transitory media having software stored thereon, the software including instructions to control one or more devices for:
receiving media input audio data corresponding to a media stream;
receiving microphone input audio data from at least one microphone;
computing a media input excitation function of the media input audio data as a time-smoothed power of the media input audio data with a frequency-variant perceptual weighting;
computing a microphone input excitation function of the microphone input audio data as a time-smoothed power of the microphone input audio data with a frequency-variant perceptual weighting;
determining, based at least in part on the microphone input excitation function, a microphone input specific loudness of the microphone input audio data, the microphone input specific loudness corresponding with a perceived loudness of the microphone input audio data;
determining, based at least in part on the media input excitation function and the microphone input excitation function, a microphone partial specific loudness corresponding to the perceived loudness of the microphone input audio data in a presence of the media input audio data;
determining frequency- and time-varying media processing gains to apply to the media input audio data to produce media output audio data;
determining frequency- and time-varying microphone processing gains to apply to the microphone input audio data to produce microphone output audio data, wherein the media processing gains and the microphone processing gains are determined such that a first difference between the microphone input specific loudness and a perceived loudness of the microphone output audio data in a presence of the media output audio data is less than a second difference between the microphone input specific loudness and the microphone partial specific loudness;
applying the determined media processing gains to the media input audio data to produce the media output audio data;
applying the determined microphone processing gains to the microphone input audio data to produce the microphone output audio data and
mixing the media output audio data and the microphone output audio data to produce mixed audio data.

12. The one or more non-transitory media of claim 11 , wherein the perceived loudness of the microphone output audio data in the presence of the media output audio data is substantially equal to the perceived loudness of the microphone input audio data.

13. The one or more non-transitory media of claim 11 , further comprising:
receiving a mode-switching indication; and
modifying at least one of the receiving, determining or mixing process based, at least in part, on the mode-switching indication.

14. The one or more non-transitory media of claim 13 , wherein the modifying involves increasing a relative loudness of the microphone output audio data, relative to a loudness of the media output audio data.

15. The one or more non-transitory media of claim 14 , wherein increasing the relative loudness of the microphone output audio data involves suppressing the media input audio data or pausing the media stream.

16. The one or more non-transitory media of claim 13 , wherein the mode-switching indication is based, at least in part, on at least one of an indication of a head movement or an indication of an eye movement.

17. The one or more non-transitory media of claim 13 , wherein the mode-switching indication is based, at least in part, on inertial sensor data corresponding to movement of a headset.

18. An audio processing apparatus, comprising:
an interface system; and
a control system configured for:
receiving, via the interface system, media input audio data corresponding to a media stream;
receiving, via the interface system, microphone input audio data from a microphone system that includes at least one microphone;

computing a media inp

Assignees

• Dolby Laboratories Licensing Corp

Inventors

• Alexander Mark
• Li Chunjian
• Lando Joshua Brandon
• Seefeldt Alan J
• Brown C Phillip
• Breebaart Dirk Jeroen

Classifications

• H04S2420/07

  Synergistic effects of band splitting and sub-band processing · CPC title

• H04S2400/15

  Aspects of sound capture and related signal processing for recording or reproduction · CPC title

• H04S2400/11

  Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence) · CPC title

• H04S7/304

  For headphones · CPC title

• H04R2430/03

  Synergistic effects of band splitting and sub-band processing · CPC title