Methods and systems for cockpit speech recognition acoustic model training with multi-level corpus data augmentation

Patent metadata

Field	Value
Publication number	US-2020335084-A1
Application number	US-201916388647-A
Country	US
Kind code	A1
Filing date	Apr 18, 2019
Priority date	Apr 18, 2019
Publication date	Oct 22, 2020
Grant date	—

Abstract

A method for initializing a device for performing acoustic speech recognition (ASR) using an ASR model, by a computer system including at least one processor and a system memory element. The method includes obtaining a plurality of voice data articulations of predetermined phrases, by the at least one processor via a user interface. The plurality of voice data articulations includes a first quantity of audio samples of actual articulated voice data, and each of the plurality of voice data articulations includes one of the audio samples including acoustic frequency components. The method further includes performing a plurality of augmentations to the plurality of voice data articulations of predetermined phrases, to generate a corpus audio data set that includes the first quantity of audio samples and a second quantity of audio samples including augmented versions of the first quantity of audio samples.

First claim

What is claimed is:

1 . A method for initializing a device for performing acoustic speech recognition (ASR) using an ASR model, by a computer system comprising at least one processor and a system memory element, the method comprising:
obtaining a plurality of voice data articulations of predetermined phrases, by the at least one processor via a user interface, wherein the plurality of voice data articulations comprises a first quantity of audio samples of actual articulated voice data, and wherein each of the plurality of voice data articulations comprises one of the audio samples including acoustic frequency components; performing a plurality of augmentations to the plurality of voice data articulations of predetermined phrases, to generate a corpus audio data set that includes the first quantity of audio samples and a second quantity of audio samples comprising augmented versions of the first quantity of audio samples, by:
performing a first level augmentation by processing each of the plurality of voice data articulations to enhance a first subset of the acoustic frequency components and to suppress a second subset of the acoustic frequency components, to generate transformed voice data articulations including a plurality of voice transformations; and
performing a second level augmentation by processing the transformed voice data articulations, by:
combining the transformed voice data articulations with noise-based audio data, to generate combined voice data articulations, by the at least one processor; and
adjusting levels of the noise-based audio data for each of the combined voice data articulations to generate the corpus audio data set including various noise levels, by the at least one processor, wherein each audio sample of the corpus audio data set includes one of the plurality of voice transformations and one of the various noise levels; and

training the ASR model to perform ASR, using the corpus audio data set, by the at least one processor.

2 . The method of claim 1 , wherein the device is implemented in an aircraft, and wherein obtaining the plurality of voice articulations is performed using a headset comprising a microphone and a speaker that is communicatively coupled with the aircraft.

3 . The method of claim 1 , wherein performing the first level augmentation comprises utilizing a voice random transformation algorithm that selects the first and second subsets at random.

4 . The method of claim 1 , wherein the first subset of the acoustic frequency components comprises frequency components of the same frequency range.

5 . The method of claim 1 , wherein the second subset of the acoustic frequency components comprises frequency components of the same frequency range.

6 . The method of claim 1 , wherein the device is implemented in an aircraft, and wherein adjusting the noise-based audio data comprises adjusting cockpit noise profile data.

7 . The method of claim 1 , wherein the device is implemented in an aircraft, and wherein the method further comprises receiving a flight crew or air traffic control voice communication and automatically recognizing words spoken in the voice communication using the ASR model.

8 . The method of claim 7 , further comprising automatically performing an aircraft function based on the recognized words spoken.

9 . The method of claim 1 , further comprising generating an updated ASR model using a further plurality of voice data articulations of the predetermined phrases subsequently received via the user interface.

10 . The method of claim 1 , further comprising initiating an upload of the ASR model into the device for performing the ASR, by the at least one processor.

11 . A computer system for performing acoustic speech recognition (ASR) using an ASR model, comprising:
a system memory element; a user interface; and at least one processor, wherein the at least one processor is configured to:
obtain a plurality of voice data articulations of predetermined phrases via the user interface, wherein the plurality of voice data articulations comprises a first quantity of audio samples of actual articulated voice data, and wherein each of the plurality of voice data articulations comprises one of the audio samples including acoustic frequency components;
perform a plurality of augmentations to the plurality of voice data articulations of predetermined phrases, to generate a corpus audio data set that includes the first quantity of audio samples and a second quantity of audio samples comprising augmented versions of the first quantity of audio samples, by:
performing a first level augmentation by processing each of the plurality of voice data articulations to enhance a first subset of the acoustic frequency components and to suppress a second subset of the acoustic frequency components, to generate transformed voice data articulations including a plurality of voice transformations; and
performing a second level augmentation by processing the transformed voice data articulations, by:
combining the transformed voice data articulations with noise-based audio data, to generate combined voice data articulations; and
adjusting levels of the noise-based audio data for each of the combined voice data articulations to generate the corpus audio data set including various noise levels, wherein each audio sample of the corpus audio data set includes one of the plurality of voice transformations and one of the various noise levels; and

train the ASR model to perform ASR, using the corpus audio data set.

12 . The computer system of claim 11 , wherein the computer system in an aircraft, and wherein the at least one processor is configured to obtain the plurality of voice articulations via electronic communication with a headset comprising a microphone and a speaker that is communicatively coupled with the aircraft.

13 . The computer system of claim 11 , wherein the at least one computer processor is configured to perform the first level augmentation comprises utilizing a voice random transformation algorithm that selects the first and second subsets at random.

14 . The computer system of claim 11 , wherein the first subset of the acoustic frequency components comprises frequency components of the same frequency range.

15 . The computer system of claim 11 , wherein the second subset of the acoustic frequency components comprises frequency components of the same frequency range.

16 . The computer system of claim 11 , wherein the computer system is implemented in an aircraft, and wherein the at least one processor is configured to adjust cockpit noise profile data.

17 . The computer system of claim 11 , wherein the computer system is implemented in an aircraft, and wherein the at least one processor is further configured to receive a flight crew or air traffic control voice communication and automatically recognize words spoken in the voice communication using the ASR model.

18 . The computer system of claim 17 , wherein the at least one processor is further configured to command automatically performing an aircraft function based on the recognized words spoken.

19 . The computer system of claim 11 , wherein the at least one processor is further configured to generate an updated ASR model using a further plurality of voice data articulations of the predetermined phrases subsequently received via the user interface.

20 . An aircraft comprising the computer system of claim 11 .

Assignees

• Honeywell Int Inc

Inventors

• Wang Luning
• Yang Wei
• Dai Zhiyong

Classifications

• G10L15/22

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title

• G10L2015/223

  Execution procedure of a spoken command · CPC title

• G10L15/063 Primary

  Training · CPC title

• B64C13/16

  actuated automatically, e.g. responsive to gust detectors · CPC title

• B64D43/00

  Arrangements or adaptations of instruments · CPC title