Active control system for a vehicular dual microphone and a control method thereof

Patent metadata

Field	Value
Publication number	US-10645494-B1
Application number	US-201916440266-A
Country	US
Kind code	B1
Filing date	Jun 13, 2019
Priority date	Dec 13, 2018
Publication date	May 5, 2020
Grant date	May 5, 2020

Abstract

An active control system of a vehicular dual microphone and a control method thereof control a beamforming area of a microphone installed adjacent to each of a driver seat and a passenger seat to be directed toward a person (utterer) who utters a voice command. The control system and method use sitting height information and seat position information of the driver or the passenger and use passenger seat riding information. The control system determines output from a mouth shape change detection unit and the user and seat position detection unit, controls microphone beamforming of the driver seat according to the determined microphone beamforming control mode, and selectively adjusts a specific microphone gain according to the determined microphone beamforming control mode to separately control microphone beamforming of the driver seat and the passenger seat.

First claim

What is claimed is:

1. An active control system of a vehicular dual microphone, the system comprising:
a mouth shape change detection unit installed inside a vehicle and configured to capture an image of a driver sitting on a driver seat of the vehicle and/or a passenger sitting on a passenger seat and to detect a change in a mouth shape of the driver and/or the passenger;
a user and seat position detection unit configured to detect riding state information of the driver and/or the passenger;
an infotainment system configured to determine a microphone beamforming control mode according to image data and user and seat position information, output from the mouth shape change detection unit and the user and seat position detection unit, to control microphone beamforming of the driver seat according to the determined microphone beamforming control mode, and to selectively adjust a specific microphone gain according to the determined microphone beamforming control mode to separately control microphone beamforming of the driver seat and the passenger seat; and
a microphone beamforming controller configured to adjust an installation angle of a first microphone installed adjacent to the driver seat to be directed toward the driver or installation angles of the first microphone and a second microphone, which are installed adjacent to the driver seat and the passenger seat, respectively, to be respectively directed toward the driver and the passenger according to control of the infotainment system,
wherein the infotainment system controller includes a beamforming processor configured to
determine the microphone beamforming control mode based on the riding information included in the user and seat information, and
when only the driver rides in the vehicle, output a beamforming control signal for directing an installation angle of the first microphone, installed adjacent to the driver seat, toward the driver, or
when both the driver and the passenger ride in the vehicle, output a beamforming control signal for respectively directing installation angles of the first and second microphones, which are installed adjacent to the driver seat and the passenger seat, respectively, toward the driver and the passenger.

2. The system of claim 1 , wherein the mouth shape change detection unit includes:
at least one camera installed in the vehicle and configured to capture an image of a driver sitting on the driver seat of the vehicle or a passenger sitting on the passenger seat positioned at one side of the driver seat according to control of the infotainment system; and
a first network communication unit configured to transmit data of the image captured by the camera to the outside.

3. The system of claim 1 , wherein the riding state information includes information on a seating height of the driver and/or the passenger, seat position information of the driver seat and/or the passenger seat of the vehicle, and information acquired by selectively combining at least one of pieces of the passenger riding information of the passenger seat of the vehicle.

4. The system of claim 1 , wherein the user and seat position detection unit includes:
a first database configured to store and manage user and seat information, which selectively includes sitting height information of the driver or the passenger, seat position information of the driver seat or the passenger seat of the vehicle, and passenger riding information of the passenger seat of the vehicle, using a database method according to control of the infotainment system;
a first controller configured to store or read the user and seat information stored in the first database according to control of the infotainment system; and
a second network communication unit configured to transmit the user and seat information to the outside according to control of the first controller.

5. The system of claim 1 , wherein the infotainment system includes:
a third network communication unit configured to data-communicate with the mouth shape change detection unit, the user and seat position detection unit, or the microphone beamforming controller;
a second controller configured to determine a microphone beamforming control mode based on the riding information included in the user and seat information when a voice command for voice recognition is uttered in the vehicle and to output a determined microphone control signal;
a display unit configured to display the image data and/or driving information according to control of the second controller; and
a second database configured to store and manage the beamforming control signal using a database method based on a predicted mouth position value of the driver, obtained by considering the user and seat information, and to read the signal according to control of the second controller.

6. The system of claim 5 , wherein the second controller includes:
the beamforming processor configured to determine a microphone beamforming control mode based on the riding information included in the user and seat information and to output a beamforming control signal for directing an installation angle of the first microphone, installed adjacent to the driver seat, toward the driver or to output a beamforming control signal for respectively directing installation angles of the first and second microphones, which are installed adjacent to the driver seat and the passenger seat, respectively, toward the driver and the passenger according to the determined microphone beamforming control mode;
an image data processor configured to process image data transmitted from the mouth shape change detection unit and to detect a change in a mouth position and a mouth shape of the driver or the passenger who rides in the driver seat or the passenger seat; and
a seat position information processor configured to receive the user and seat position information transmitted from the user and seat position detection unit and to separately detect sitting height information of the driver or the passenger, seat position information of the driver seat or the passenger seat of the vehicle, and passenger riding information of the passenger seat of the vehicle.

7. The system of claim 1 , wherein the microphone beamforming controller includes:
a fourth network communication unit configured to receive a microphone beamforming control signal output from the infotainment system;
first and second microphone angle adjustment units configured to output a microphone installation angle adjustment signal for adjusting a microphone beamforming area of the driver seat or the passenger seat according to the microphone beamforming control signal output from the fourth network communication unit; and
first and second servo motors installed at the driver seat of the vehicle and the passenger seat of the vehicle and configured to adjust the installation angle of a microphone unit including at least the first or second microphone with a user voice command being input thereto, in one or more arbitrary directions of an upward direction, a downward direction, a left direction, and a right direction, according to the microphone installation angle adjustment signal.

8. An active control method of a vehicular dual microphone, the method comprising:
capturing an image of a driver sitting on a driver seat of a vehicle and/or a passenger sitting on a passenger seat and detecting a change in a mouth shape of the driver and/or the passenger;
detecting riding state information of the driver and/or the passenger;
determining a microphone beamforming control mode according to image data obtained by detecting the change in the mouth shape of the driver and/or the passenger and the user and seat position information, controlling microphone beamforming of the d

Assignees

• Hyundai Motor Co Ltd
• Kia Motors Corp

Inventors

• Oh Hyung Suk
• Kim Hong Ju

Classifications

• H04R1/406 Primary

  microphones · CPC title

• H04R2499/13

  Acoustic transducers and sound field adaptation in vehicles · CPC title

• G10L15/25

  using position of the lips, movement of the lips or face analysis · CPC title

• G10L15/22

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title

• H04R29/005

  Microphone arrays · CPC title