Position directed acoustic array and beamforming methods

Patent metadata

Field	Value
Publication number	US-9747917-B2
Application number	US-201313918752-A
Country	US
Kind code	B2
Filing date	Jun 14, 2013
Priority date	Jun 14, 2013
Publication date	Aug 29, 2017
Grant date	Aug 29, 2017

Abstract

Methods and systems are provided for receiving desired sounds. The system includes a position sensor configured to determine an occupant position of an occupant engaging in speech within a defined space and transmit the speaking occupant position. A plurality of microphones are configured to receive sound from within the defined space and transmit audio signals corresponding to the received sound. A processor, in communication with the position sensor and the microphones, is configured to receive the speaking occupant position and the audio signals, apply a beamformer to the audio signals to direct a microphone beam toward the occupant position, and generate a beamformer output signal.

First claim

What is claimed is:

1. An adaptive beamforming method for receiving desired sounds, said method comprising:
determining an occupant position of an occupant engaging in speech within a defined space in a vehicle, said determining comprising
transmitting waves in an ultrasonic range and/or a radio frequency (RF) range with a plurality of transmitters,
receiving reflected waves, and
analyzing signals corresponding to the transmitted and reflected waves to determine the occupant position;

determining when the occupant is engaging in speech by identifying motion of a mouth of the occupant engaging in speech by using a sensor in the vehicle and searching for distinguishing features in spatio-temporal and/or velocity features of signals corresponding to the reflected waves, and generating speech detection based on the determination, the speech detection including detection of when the occupant is speaking based on the detected motion of the mouth of the occupant;
receiving sound from within the defined space with a plurality of microphones;
generating a plurality of audio signals corresponding to the received sound at each of said plurality of microphones;
applying a beamformer to the audio signals to direct a microphone beam toward the determined occupant position and generate a beamformer output signal, wherein:
the applying of the beamformer comprises modifying the beamformer based simultaneously on both the determined occupant position and the speech detection, including the detection of when the occupant is speaking based on the detected motion of the mouth of the occupant; and
the applying of the beamformer comprises applying a sidelobe canceller to the audio signals to direct a microphone beam toward the determined occupant position and generate a beamformer output signal, by modifying the sidelobe canceller based on both the determined occupant position and the speech detection, including the detection of when the occupant is speaking based on the detected motion of the mouth of the occupant; and

applying a blocking matrix to the audio signals via the sidelobe canceller, and generating a plurality of blocking matrix output signals, each of which corresponds to one of the audio signals.

2. A method as set forth in claim 1 further comprising:
applying an adaptive noise cancellation process to the blocking matrix output signals to generate an adaptive noise cancellation output signal; and
summing the beamformer output signal and the adaptive noise cancellation output signal to generate a speaking occupant output signal.

3. A system for receiving desired sounds, said system comprising:
a position sensor configured to determine an occupant position of an occupant engaging in speech within a defined space and transmit the speaking occupant position, said position sensor comprising:
a signal generator,
a plurality of transmitters electrically coupled to said signal generator and configured to generate waves in an ultrasonic range and/or an RF range,
a plurality of receivers configured to receive reflected waves and generate a plurality of received signals corresponding to the received reflected waves, and
a processing unit electrically coupled to said receivers and configured to receive the received signals, determine the occupant position, and identify motion of a mouth of the occupant engaging in speech by searching for distinguishing features in spatio-temporal and/or velocity features of signals corresponding to the reflected waves, and generating speech detection based on the determination, the speech detection including detection of when the occupant is speaking based on the detected motion of the mouth of the occupant;

a plurality of microphones configured to receive sound from within the defined space and transmit audio signals corresponding to the received sound; and
a processor in communication with said position sensor and said microphones and configured to receive the speaking occupant position and the audio signals, apply a beamformer to the audio signals to direct a microphone beam toward the determined occupant position, and generate a beamformer output signal, wherein the processor is configured to apply the beamformer by modifying the beamformer based simultaneously on both the determined occupant position and the speech detection, including the detection of when the occupant is speaking based on the detected motion of the mouth of the occupant.

4. A system as set forth in claim 3 wherein said processor is also configured to apply a blocking matrix to each of the audio signals to exclude other positions not associated with the occupant engaging in speech to generate a plurality of blocking matrix output signals corresponding to each of the audio signals, apply an adaptive noise cancellation process to the blocking matrix output signals to generate an adaptive noise cancellation output signal, and sum the beamformer output signal and the adaptive noise cancellation output signal to generate a speaking occupant output signal.

5. A system as set forth in claim 3 wherein said position sensor comprises a camera.

6. A vehicle comprising:
a passenger compartment;
an audio system comprising
a position sensor configured to determine an occupant position of an occupant engaging in speech within the passenger compartment and transmit the speaking occupant position, said position sensor comprising
a signal generator,
a plurality of transmitters electrically coupled to said signal generator and configured to generate waves in an ultrasonic range and/or an RF range,
a plurality of receivers configured to receive reflected waves and generate a plurality of received signals corresponding to the received reflected waves, and
a processing unit electrically coupled to said receivers and configured to receive the received signals, determine the occupant position, and identify motion of a mouth of the occupant engaging in speech by searching for distinguishing features in spatio-temporal and/or velocity features of signals corresponding to the reflected waves, and generating speech detection based on the determination, the speech detection including detection of when the occupant is speaking based on the detected motion of the mouth of the occupant;

a plurality of microphones configured to receive sound from within the defined space and transmit audio signals corresponding to the received sound; and
a processor in communication with said position sensor and said microphones and configured to receive the speaking occupant position and the audio signals, apply a beamformer to the audio signals to direct a microphone beam toward the occupant position, and generate a beamformer output signal, apply a blocking matrix to each of the audio signals to exclude other positions not associated with the occupant engaging in speech to generate a plurality of blocking matrix output signals corresponding to each of the audio signals, apply an adaptive noise cancellation process to the blocking matrix output signals to generate an adaptive noise cancellation output signal, and sum the beamformer output signal and the adaptive noise cancellation output signal to generate a speaking occupant output signal, wherein the processor is configured to apply the beamformer by modifying the beamformer based simultaneously on both the determined occupant position and the speech detection, including the detection of when the occupant is speaking based on the detected motion of the mouth of the occupant; and

at least one of a speech recognition system and a telephony system in communication with said audio system for receiving said speaking occupant output signal.

7. A vehicle as set forth in claim 6 wherein said plurality of transmitters comprise a plurality of ultrasonic transmit

Assignees

• Gm Global Tech Operations Llc

Inventors

• Tzirkel-Hancock Eli
• Bilik Igal
• Laifenfeld Moshe

Classifications

• G01S13/06

  Systems determining position data of a target · CPC title

• G10L21/00 Primary

  Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility (G10L19/00 takes precedence) · CPC title

• H04R3/005 Primary

  for combining the signals of two or more microphones (specially adapted for hearing aids H04R25/407) · CPC title

• G10L21/0208

  Noise filtering · CPC title

• G01S15/06

  Systems determining the position data of a target · CPC title