Audio focusing via multiple microphones

Patent metadata

Field	Value
Publication number	US-9596437-B2
Application number	US-201313972781-A
Country	US
Kind code	B2
Filing date	Aug 21, 2013
Priority date	Aug 21, 2013
Publication date	Mar 14, 2017
Grant date	Mar 14, 2017

Abstract

Various technologies are applied to focus audio received from a plurality of microphones of a mobile device. A camera can be used to portray a scene, and a selection within the scene can focus audio to a desired audio focus region. Techniques can account for movement of the mobile device or an object being tracked. Pre-computed audio filters can be used to customize the audio focus process to account for a particular mobile device geometry.

First claim

What is claimed is:

1. A method implemented at least in part by a computing system, the method comprising:
downloading an audio filter set for a plurality of audio focus regions according to a stored mobile device type indicating a type of a mobile device, wherein the audio filter set comprises a plurality of pluggable audio filters operable to accept input from a plurality of microphones and output enhanced audio as focused within the audio focus regions, wherein the plurality of pluggable audio filters are pre-computed based on combinations of azimuth ranges and elevation ranges based on microphone placement on the mobile device of the stored mobile device type, the plurality of pluggable audio filters being pre-computed, region-specific audio filters;
displaying a scene currently viewed by a camera;
receiving an indication of a location within the scene;
based on the location, determining an azimuth and elevation;
translating the azimuth and elevation into an audio focus region;
selecting from the pre-computed, region-specific audio filters based on the audio focus region; and
enhancing audio received by the plurality of microphones based on the audio focus region, wherein enhancing the audio comprises applying the selected pre-computed, region-specific audio filters to audio received from respective of the plurality of microphones.

2. The method of claim 1 wherein:
the camera is integrated into a device; and
the method further comprises:
tracking movement of the device via one or more positional sensors; and
adjusting the audio focus region based on the movement.

3. The method of claim 1 further comprising:
tracking movement of an object appearing within the audio focus region; and
adjusting the audio focus region based on the movement of the object.

4. The method of claim 3 wherein:
the object comprises a face; and
tracking movement of the object comprises applying face detection.

5. The method of claim 1 wherein selecting the pre-computed, region-specific audio filters comprises:
identifying an entry in a table of pre-computed, region-specific audio filters.

6. The method of claim 1 wherein translating the azimuth and elevation into an audio focus region comprises:
translating the azimuth and elevation into an audio focus region identifier.

7. The method of claim 1 wherein:
determining the audio focus region comprises, based on the location, determining an azimuth range identifier and an elevation range identifier combination; and
selecting the pre-computed, region-specific audio filters comprises mapping the azimuth range identifier and the elevation range identifier combination to the pre-computed, region-specific audio filters.

8. The method of claim 1 wherein:
enhancing audio received by the plurality of microphones comprises applying audio filters to emphasize sound originating from the audio focus region.

9. The method of claim 1 wherein:
enhancing audio received by the plurality of microphones comprises applying audio filters to dampen sound originating from the audio focus region.

10. The method of claim 1 wherein:
receiving an indication of a location within the scene comprises receiving a two-dimensional area.

11. The method of claim 1 wherein:
the location within the scene corresponds to a location of a participant in a conference call; and
the method further comprises presenting the enhanced audio during the conference call.

12. The method of claim 1 further comprising:
receiving an indication of a second location within the scene;
translating the second location into a second audio focus region; and
further enhancing audio received by the plurality of microphones based on the second audio focus region.

13. A mobile device comprising:
a stored mobile device type indicating a type of the mobile device;
a camera;
a plurality of microphones;
a display configured to display a scene viewed by the camera;
a downloading tool configured to download an audio filter set for a plurality of audio focus regions according to the stored mobile device type, wherein the audio filter set comprises a plurality of pluggable audio filters operable to accept input from the microphones and output enhanced audio as focused within the audio focus regions, wherein the plurality of pluggable audio filters are pre-computed based on combinations of azimuth ranges and elevation ranges based on microphone placement on the mobile device of the stored mobile device type, the plurality of pluggable audio filters being pre-computed, region-specific audio filters;
an audio focus tool comprising a region translator configured to receive a location on the display and translate the location on the display into an audio focus region; and
an audio filter selector configured to choose from among the plurality of pluggable audio filters based on the audio focus region.

14. The mobile device of claim 13 further comprising:
a table associating audio focus regions with respective sets of audio filters to be applied to audio streams originating from the microphones when a given audio focus region is indicated;
wherein the plurality of pluggable audio filters are represented in the table.

15. The mobile device of claim 13 further comprising:
one or more positional sensors; and
an audio focus region adjustment tool configured to adjust the audio focus region based on detected movement of the mobile device as indicated by the one or more positional sensors.

16. The mobile device of claim 13 further comprising:
an object tracker configured to track movement of an object in an audio focus region; and
an audio focus region adjustment tool configured to adjust the audio focus region according to changes in position detected by the object tracker.

17. One or more nonvolatile memory components having encoded thereon computer-executable instructions causing a computing system to perform a method comprising:
receiving a determination of a mobile device type of a mobile device comprising a plurality of microphones;
based on the mobile device type of the mobile device, with a downloading tool, downloading a pluggable audio filter set for a plurality of audio focus regions to the mobile device, wherein the pluggable audio filter set comprises a plurality of pluggable audio filters operable to accept input from the plurality of microphones and output enhanced audio as focused within the audio focus regions, wherein the plurality of pluggable audio filters are pre-computed based on combinations of azimuth ranges and elevation ranges based on microphone placement on the mobile device of the mobile device type, the plurality of pluggable audio filters being pre-computed, region-specific audio filters;
receiving video output from a camera on a mobile device;
displaying the video output from the camera on the mobile device on a display;
receiving an indication of a location on the display;
translating the location on the display into an audio focus region identifier, based at least on a combination of an azimuth range and an elevation range indicated by the location on the display;
based on the audio focus region identifier, selecting a set of beamformer audio filters out of the pluggable audio filter set configured to enhance audio coming from an audio focus region associated with the audio focus region identifier,
enhancing audio streams originating from respective of the plurality of microphones of the mobile device, wherein the enhancing comprises applying the set of beamformer audio filters to respective of the audio streams; a

Assignees

• Zad Issa Syavosh
• Fleck Rod G
• Microsoft Technology Licensing Llc

Inventors

• Zad Issa Syavosh
• Fleck Rod G

Classifications

• H04R29/004

  for microphones (H04R29/007 takes precedence) · CPC title

• H04R2499/15

  Transducers incorporated in visual displaying devices, e.g. televisions, computer displays, laptops · CPC title

• H04R2460/07

  Use of position data from wide-area or local-area positioning systems in hearing devices, e.g. program or information selection · CPC title

• H04R2499/11

  Transducers incorporated or for use in hand-held devices, e.g. mobile phones, PDA's, camera's · CPC title

• H04R2410/01

  Noise reduction using microphones having different directional characteristics · CPC title