Speech recognition in a vehicle

Patent metadata

Field	Value
Publication number	US-11893978-B2
Application number	US-202117400624-A
Country	US
Kind code	B2
Filing date	Aug 12, 2021
Priority date	Aug 12, 2021
Publication date	Feb 6, 2024
Grant date	Feb 6, 2024

Abstract

An audio sample including speech and ambient sounds is transmitted to a vehicle computer. Recorded audio is received from the vehicle computer, the recorded audio including the audio sample broadcast by the vehicle computer and recorded by the vehicle computer and recognized speech from the recorded audio. The recognized speech and text of the speech are input to a machine learning program that outputs whether the recognized speech matches the text. When the output from the machine learning program indicates that the recognized speech does not match the text, the recognized speech and the text are included in a training dataset for the machine learning program.

First claim

The invention claimed is:

1. A system, comprising a computer including a processor and a memory, the memory storing instructions executable by the processor to:
transmit, via a network, an audio sample including speech and ambient sounds to a vehicle computer;
broadcast the audio sample in a vehicle of the vehicle computer;
record, from the vehicle computer, audio from the audio sample;
recognize, by the vehicle computer, speech in the recorded audio from the audio sample;
receive, via the network from the vehicle computer, the recorded audio including the audio sample;
receive, via the network from the vehicle computer, the recognized speech from the recorded audio;
identify text of the recognized speech from the recorded audio;
input the recognized speech and the identified text of the recognized speech to a machine learning program that outputs whether the recognized speech matches the identified text; and
when the output from the machine learning program indicates that the recognized speech does not match the text include the recognized speech and the text in a training dataset for the machine learning program;
wherein the machine learning program is trained to output, from the recorded audio, a detection of a vibration from a vehicle subsystem indicating potential maintenance for the vehicle subsystem; and
identify a variation in the vehicle subsystem based on the output detected vibration from the recorded audio and a vibration predicted by a prognostics model of the vehicle subsystem.

2. The system of claim 1 , wherein the instructions further include instructions to generate the audio sample with a second machine learning program trained to output audio samples including speech and ambient sounds.

3. The system of claim 2 , wherein the audio sample includes speech of a voice command to actuate one or more vehicle subsystems.

4. The system of claim 1 , wherein the instructions further include instructions to receive, from the vehicle computer, a message indicating that the vehicle computer did not recognize speech from the recorded audio broadcast and to include the audio sample and the message in the training dataset.

5. The system of claim 1 , wherein the instructions further include instructions to receive a message from the vehicle computer indicating that a vehicle in which the vehicle computer is located is stationary, and then transmit the audio sample to the vehicle computer based on receiving the message indicating that the vehicle in which the vehicle computer is located is stationary.

6. The system of claim 1 , wherein the instructions further include instructions to transmit the detected variation to the vehicle computer.

7. The system of claim 1 , wherein the instructions further include instructions to input the detected vibration in the training dataset for the machine learning program.

8. The system of claim 1 , wherein the vehicle computer is further programmed to, upon receiving the audio sample, to actuate a speaker to broadcast the audio sample and to actuate a microphone to record the broadcast audio sample.

9. The system of claim 8 , wherein the vehicle computer is further programmed to input the recorded audio to a speech recognition program trained to output speech recognized from the recorded audio.

10. The system of claim 8 , wherein the vehicle computer is further programmed to actuate each of a plurality of speakers to broadcast the audio sample and to actuate the microphone to record the audio sample broadcast from each of the plurality of speakers.

11. The system of claim 1 , wherein the instructions further include instructions to transmit the audio sample to each of a plurality of vehicle computers, each vehicle computer located in a respective vehicle, and to receive recorded audio and recognized speech from each of the plurality of vehicle computers.

12. The system of claim 11 , wherein the instructions further include instructions to input respective recognized speech and text of the speech from each of the plurality of vehicle computers to the training dataset of the machine learning program.

13. The system of claim 1 , wherein the instructions further include instructions to retrain the machine learning program with the training dataset and to transmit the retrained machine learning program to the vehicle computer.

14. A method, comprising:
transmitting, via a network, an audio sample including speech and ambient sounds to a vehicle computer;
broadcasting the audio sample in a vehicle of the vehicle computer;
recording, from the vehicle computer, audio from the audio sample;
recognizing, by the vehicle computer, speech in the recorded audio from the audio sample;
receiving, via the network from the vehicle computer, the recorded audio including the audio sample:
receiving, via the network from the vehicle computer, the recognized speech from the recorded audio;
identifying text of the recognized speech from the recorded audio;
inputting the recognized speech and the identified text of the recognized speech to a machine learning program that outputs whether the recognized speech matches the identified text; and
when the output from the machine learning program indicates that the recognized speech does not match the text, including the recognized speech and the text in a training dataset for the machine learning program;
wherein the machine learning program is trained to output, from the recorded audio, a detection of a vibration from a vehicle subsystem indicating potential maintenance for the vehicle subsystem; and
identifying a variation with the vehicle subsystem based on the output detected vibration from the recorded audio and a vibration predicted by a prognostics model of the vehicle subsystem.

15. The method of claim 14 , further comprising generating the audio sample with a second machine learning program trained to output audio samples including speech and ambient sounds.

16. The method of claim 14 , further comprising receiving, from the vehicle computer, a message indicating that the vehicle computer did not recognize speech from the recorded audio broadcast and including the audio sample and the message in the training dataset.

17. The method of claim 14 , further comprising receiving a message from the vehicle computer indicating that a vehicle in which the vehicle computer is located is stationary, and then transmitting the audio sample to the vehicle computer.

18. The method of claim 14 , wherein the vehicle computer is further programmed to, upon receiving the audio sample, to actuate a speaker to broadcast the audio sample and to actuate a microphone to record the broadcast audio sample.

Assignees

• Ford Global Tech Llc

Inventors

• Makke Omar
• Gusikhin Oleg
• Kadry Haysam M

Classifications

• G10L15/063 Primary

  Training · CPC title

• G06N20/00

  Machine learning · CPC title

• G10L15/20

  Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech (G10L21/02 takes precedence) · CPC title

• G10L15/22

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title

• G10L15/26

  Speech to text systems (G10L15/08 takes precedence) · CPC title