Estimating room acoustic properties using microphone arrays

Patent metadata

Field	Value
Publication number	US-10440498-B1
Application number	US-201816180165-A
Country	US
Kind code	B1
Filing date	Nov 5, 2018
Priority date	Nov 5, 2018
Publication date	Oct 8, 2019
Grant date	Oct 8, 2019

Abstract

An audio analysis system receives a first recording of a speech signal from an origin audio assembly and a second recording of at least a portion of the speech signal from a receiving audio assembly. The speech signal originates from a speaking user of the origin audio assembly and the second recording is recorded by a receiving audio assembly operated by a different user. Both the origin audio assembly and the receiving audio assembly are located within a room. The audio analysis system selects one or more audio frames in the first recording and one or more audio frames in the second recording that both occur over the same time period. The audio analysis system determines a transfer function for the room based in part on the selected one or more audio frames in the first recording and the selected one or more audio frames in the second recording.

First claim

What is claimed is:

1. A method comprising:
receiving a first recording of a speech signal from an origin audio assembly and a second recording that includes at least a portion of the speech signal, wherein the speech signal originates from a speaking user of the origin audio assembly and the second recording is recorded at a receiving audio assembly operated by a different user and both the origin audio assembly and the receiving audio assembly are located within a room;
selecting one or more audio frames in the first recording and one or more audio frames in the second recording that both occur over a same time period; and
determining a transfer function for the room based in part on the selected one or more audio frames in the first recording and the selected one or more audio frames in the second recording.

2. The method of claim 1 , wherein the origin audio assembly includes a first microphone array and the receiving audio assembly includes a second microphone array, wherein the first microphone array generates the first recording and the second microphone array generates the second recording.

3. The method of claim 1 , wherein the selected one or more audio frames in the first recording and the selected one or more audio frames in the second recording have a noise level below a threshold value.

4. The method of claim 1 , wherein selecting the one or more audio frames in the first recording and the one or more audio frames in the second recording that both occur over a same time period comprises:
identifying audio frames from the first recording and the second recording that occur over a same time period while a change in relative position between the origin audio assembly and the receiving audio assembly is below a threshold value; and
selecting the identified audio frames in the first recording and the second recording.

5. The method of claim 1 , wherein the transfer function for the room describes at least an effect of the room on the speech signal recorded by the origin audio assembly.

6. The method of claim 1 , further comprising:
estimating one or more room parameters from the transfer function of the room, the room parameters including one or more of the following:
directional information describing a position of the origin audio assembly and the receiving audio assembly;
a spatial distribution of reverberations of the speech signal;
a direction of arrival for a direct reverberation of the speech signal;
a strength of arrival of reverberations of the speech signal from different directions within the room; and
a lateral energy fraction for the room.

7. The method of claim 1 , further comprising:
synthesizing a virtual representation of the speaking user within the room based in part on the transfer function.

8. A device comprising:
a microphone array comprising a plurality of acoustic sensors, wherein the microphone array records a first recording of a speech signal originating from a speaking user; and
a controller configured to:
receive a second recording of the speech signal recorded at a receiving audio assembly that is remote from the device, wherein the device and the receiving audio assembly are located within a room;
select one or more audio frames in the first recording and one or more audio frames in the second recording that both occur over a same time period; and
determine a transfer function for the room based in part on the selected one or more audio frames in the first recording and the selected one or more audio frames in the second recording.

9. The device of claim 8 , wherein the receiving audio assembly includes a second microphone array which generates the second recording.

10. The device of claim 8 , wherein the selected one or more audio frames in the first recording and the selected one or more audio frames in the second recording have a noise level below a threshold value.

11. The device of claim 8 , wherein the controller is further configured to:
identify audio frames from the first recording and the second recording recorded that occur over a same time period while a change in relative position between the device and the receiving audio assembly is below a threshold value; and
select the identified audio frames in the first recording and the second recording.

12. The device of claim 8 , wherein the transfer function for the room describes at least an effect of the room on the speech signal recorded by the microphone array.

13. The device of claim 8 , wherein the controller is further configured to:
estimate one or more room parameters from the transfer function of the room, the room parameters comprising:
directional information describing a position of the device and the receiving audio assembly;
a spatial distribution of reverberations of the speech signal;
a direction of arrival for a direct reverberation of the speech signal;
a strength of arrival of reverberations of the speech signal from different directions within the room; and
a lateral energy fraction for the room.

14. The device of claim 8 , wherein the controller is further configured to:
synthesize a virtual representation of the speaking user within the room based in part on the transfer function.

15. A non-transitory computer-readable medium configured to store computer-readable instructions that, when executed by a processor, cause the processor to:
receive a first recording of a speech signal from an origin audio assembly and a second recording that includes at least a portion of the speech signal, wherein the speech signal originates from a speaking user of the origin audio assembly and the second recording is recorded at a receiving audio assembly operated by a different user and both the origin audio assembly and the receiving audio assembly are located within a room;
select one or more audio frames in the first recording and one or more audio frames in the second recording that both occur over a same time period; and
determine a transfer function for the room based in part on the selected one or more audio frames in the first recording and the selected one or more audio frames in the second recording.

16. The non-transitory computer-readable medium of claim 15 , wherein the origin audio assembly includes a first microphone array and the receiving audio assembly includes a second microphone array, wherein the first microphone array generates the first recording and the second microphone array generates the second recording.

17. The non-transitory computer-readable medium of claim 15 , wherein the selected one or more audio frames in the first recording and the selected one or more audio frames in the second recording have a noise level below a threshold value and selecting the one or more audio frames in the first recording and the one or more audio frames in the second recording that both occur over a same time period comprises instructions to:
identify audio frames from the first recording and the second recording recorded that occur over a same time period while a change in relative position between the origin audio assembly and the receiving audio assembly is below a threshold value; and
select the identified audio frames in the first recording and the second recording.

18. The non-transitory computer-readable medium of claim 15 , wherein the transfer function for the room describes at least an effect of the room on the speech signal recorded by the origin audio assembly.

19. The non-transitory computer-readable medium of claim 15 , further comprising instructions to:
estimate on

Assignees

• Facebook Tech Llc

Inventors

• Amengual Garí Sebastià Vicenç
• Brimijoin Ii William Owen
• Robinson Philip

Classifications

• H04R2430/20

  Processing of the output signals of the acoustic transducers of an array for obtaining a desired directivity characteristic (H04R2203/12 takes precedence) · CPC title

• G01H7/00 Primary

  Measuring reverberation time {; room acoustic measurements} · CPC title

• H04R2430/03

  Synergistic effects of band splitting and sub-band processing · CPC title

• H04S7/301

  Automatic calibration of stereophonic sound system, e.g. with test microphone · CPC title

• H04S7/305 Primary

  Electronic adaptation of stereophonic audio signals to reverberation of the listening space (H04S7/301 takes precedence) · CPC title