Precise tracking of sound angle of arrival at a microphone array under air temperature variation

Patent metadata

Field	Value
Publication number	US-9686605-B2
Application number	US-201414282633-A
Country	US
Kind code	B2
Filing date	May 20, 2014
Priority date	May 20, 2014
Publication date	Jun 20, 2017
Grant date	Jun 20, 2017

Abstract

A video conference endpoint detects a face and determines a face angle of the detected face relative to a reference direction based on images captured with a camera. The endpoint determines an angle of arrival of sound (i.e., a sound angle) received at a microphone array that transduces the sound relative to the reference direction based on the transduced sound and a sound speed parameter indicative of a speed of sound in air. The endpoint compares the face angle against the sound angle, and adjusts the sound speed parameter so as to reduce the angle difference if the compare indicates an angle difference greater than zero between the face and sound angles.

First claim

What is claimed is:

1. A method comprising:
detecting a face and determining a face angle of the detected face relative to a reference direction based on images of a scene including one or more faces captured by a camera;
transducing sound received at a microphone array;
determining a sound angle of arrival of the sound received at the microphone array relative to the reference direction based on the transduced sound and a sound speed parameter indicative of a speed of sound in air;
comparing the face angle against the sound angle; and
if the comparing indicates an angle difference greater than zero between the face and sound angles, adjusting the sound speed parameter so as to reduce the angle difference.

2. The method of claim 1 , wherein if the comparing indicates the sound angle is less than or greater than the face angle, the adjusting includes increasing or decreasing the sound speed parameter to reduce the angle difference, respectively.

3. The method of claim 2 , wherein:
the adjusting includes incrementally adjusting the sound speed parameter; and
the method further comprises repeating, the determining a sound angle, the comparing, and the adjusting using the incrementally adjusted sound speed parameter until the comparing indicates the angle difference is within a predetermined angle range.

4. The method of claim 2 , further comprising:
if the comparing indicates the angle difference is within a predetermined angle range, performing the adjusting; and
if the comparing indicates that the angle difference is not within the predetermined angle range, not performing the adjusting.

5. The method of claim 4 , wherein:
the detecting a face includes detecting the face with a detected face confidence level;
the determining a sound angle includes determining the sound angle with a determined sound angle confidence level; and
the method further comprises:
performing the comparing and the adjusting if the detected face confidence level and the determined sound angle confidence levels are each a relatively high confidence level; and
not performing the comparing and the adjusting if the detected face confidence level and the determined sound angle confidence levels are not each a relatively high confidence level.

6. The method of claim 2 , wherein the method further comprises:
performing the comparing and the adjusting if the face angle is within a predetermined solid angle that represents an angle confidence range centered on the sound angle; and
not performing the comparing and the adjusting if the face angle is not within the predetermined angle confidence range centered on the sound angle.

7. The method of claim 6 , wherein the method further comprises:
performing the comparing and the adjusting if the face angle of only one detected face is within the predetermined angle confidence range; and
not performing the comparing and the adjusting if face angles of multiple detected faces are within the predetermined angle confidence range.

8. The method of claim 2 , further comprising:
constructing a memory table that maps calibrated temperatures that are representative of the parameter indicative of the speed of sound in air to corresponding angle differences between face angles and corresponding sound angles determined based on an assumed temperature, wherein:
the comparing indicates an angle difference between the determined face and sound angles; and
the adjusting includes accessing a calibrated temperature from the memory table based on the indicated angle difference such that a sound angle determined based on the accessed calibrated temperature matches the determined face angle within a predetermined range.

9. The method of claim 1 , wherein the determining a sound angle includes:
determining a time-of-arrival (TOA) difference between TOAs of the sound received at individual microphones in the microphone array based on the transduced sound; and
determining the sound angle based on the TOA difference and the sound speed parameter.

10. The method of claim 1 , wherein the sound speed parameter is one of air temperature and the speed of sound in air.

11. An apparatus comprising:
a video camera configured to capture images of a scene;
a microphone array configured to transduce sound received at the microphone array; and
a processor, coupled to the video camera and the microphone array, configured to:
detect a face and determine a face angle of the detected face relative to a reference direction based on the captured images;
determine a sound angle of arrival of sound received at the microphone array relative to the reference direction based on the transduced sound and a sound speed parameter indicative of a speed of sound in air;
compare the face angle against the sound angle; and
if the compare indicates an angle difference greater than zero between the face and sound angles, adjust the sound speed parameter so as to reduce the angle difference.

12. The apparatus of claim 11 , wherein if the compare indicates the sound angle is less than or greater than the face angle, the processor is configured to increase or decrease the sound speed parameter to reduce the angle difference, respectively.

13. The apparatus of claim 12 , wherein the processor is further configured to:
incrementally adjust the sound speed parameter; and
repeat the determine a sound angle, the compare, and the adjust operations using the incrementally adjusted sound speed parameter until the compare indicates the angle difference is within a predetermined angle range.

14. The apparatus of claim 12 , wherein the processor is configured to:
if the compare indicates the angle difference is within a predetermined angle range, perform the adjust operation; and
if the compare indicates the angle difference is not within a predetermined angle range, not perform the adjust operation.

15. The apparatus of claim 12 , wherein the processor is configured to:
perform the compare and the adjust operations if the face angle is within a predetermined solid angle that represents an angle confidence range centered on the sound angle; and
not perform the compare and the adjust operations if the face angle is not within the predetermined angle confidence range centered on the sound angle.

16. The apparatus of claim 15 , wherein the processor is configured to:
perform the compare and the adjust operation if the face angle of only one detected face is within the predetermined angle confidence range; and
not perform the compare and the adjust operations if the face angles of multiple detected faces are within the predetermined angle confidence range.

17. A non-transitory processor readable medium storing instructions that, when executed by a processor, cause the processor to:
detect a face and determine a face angle of the detected face relative to a reference direction based on images of a scene captured by a camera;
access transduced sound produced by a microphone array responsive to sound received at the microphone array;
determine a sound angle of arrival of the sound received at the microphone array relative to the reference direction based on the transduced sound and a sound speed parameter indicative of a speed of sound in air;
compare the face angle against the sound angle; and
if an angle difference is greater than zero between the face and sound angles, adjust the sound speed parameter so as to reduce the angle difference.

18. The non-transitory processor readable medium of claim 17 , wherein the instructions include instructions to cause

Assignees

• Cisco Tech Inc

Inventors

• Sun Haohai

Classifications

• H04R3/005 Primary

  for combining the signals of two or more microphones (specially adapted for hearing aids H04R25/407) · CPC title

• H04R1/32 Primary

  for obtaining desired directional characteristic only {(specially adapted for hearing aids H04R25/40)} · CPC title

• G06K9/00228

  Physics · mapped topic

• G06V40/161

  Detection; Localisation; Normalisation · CPC title