Artificial intelligence-based text-to-speech system and method

Patent metadata

Field	Value
Publication number	US-2022148564-A1
Application number	US-202217589449-A
Country	US
Kind code	A1
Filing date	Jan 31, 2022
Priority date	May 18, 2017
Publication date	May 12, 2022
Grant date	—

Abstract

A technique improves training and speech quality of a text-to-speech (TTS) system having an artificial intelligence, such as a neural network. The TTS system is organized as a front-end subsystem and a back-end subsystem. The front-end subsystem is configured to provide analysis and conversion of text into input vectors, each having at least a base frequency, f0, a phenome duration, and a phoneme sequence that is processed by a signal generation unit of the back-end subsystem. The signal generation unit includes the neural network interacting with a pre-existing knowledgebase of phenomes to generate audible speech from the input vectors. The technique applies an error signal from the neural network to correct imperfections of the pre-existing knowledgebase of phenomes to generate audible speech signals. A back-end training system is configured to train the signal generation unit by applying psychoacoustic principles to improve quality of the generated audible speech signals.

First claim

What is claimed is:

1 . A text-to-speech (TTS) system including one or more processors and one or more memories configured to perform operations for converting text into a corrected speech signal comprising:
interacting with data of previously generated speech in a pre-existing knowledgebase of phonemes, wherein the previously generated speech has speech signal distortions; generating the corrected speech signal of the previously generated speech to correct for the speech signal distortions of the previously generated speech based upon, at least in part, interacting with the data of the previously generated speech in the pre-existing knowledgebase of phonemes; and applying the corrected speech signal to the previously generated speech for correcting the speech signal distortions of the previously generated speech in the pre-existing knowledgebase of phonemes.

2 . The TTS system of claim 1 wherein the operations further comprise converting a frequency domain signal combined from a neural network and the pre-existing knowledgebase into the corrected speech signal.

3 . The TTS system of claim 1 wherein the data in the pre-existing knowledgebase of phonemes comprises average basic acoustic signal data of how a speaker speaks derived from the recorded audible speech.

4 . The TTS system of claim 1 wherein the operations further comprise correcting for psychoacoustic perceived speech signal distortions of the pre-existing knowledgebase of phonemes.

5 . The TTS system of claim 1 wherein the operations further comprise upsampling a frequency of an input vector to another frequency of an intermediate vector.

6 . The TTS system of claim 1 wherein the operations further comprise correcting voiced phonemes of the pre-existing knowledgebase of phonemes.

7 . The TTS system of claim 1 wherein the operations further comprise correcting unvoiced phonemes of the pre-existing knowledgebase of phonemes.

8 . The TTS system of claim 1 wherein a neural network is configured based on psychoacoustic modeling of phonemes.

9 . The TTS system of claim 5 , wherein the input vector comprises at least one of a base frequency, a phoneme duration, and a phoneme sequence.

10 . A method of processing text-to-speech (TTS) comprising:
interacting with data of previously generated speech in a pre-existing knowledgebase of phonemes, wherein the previously generated speech has speech signal distortions; generating the corrected speech signal of the previously generated speech to correct for the speech signal distortions of the previously generated speech based upon, at least in part, interacting with the data of the previously generated speech in the pre-existing knowledgebase of phonemes; and applying the corrected speech signal to the previously generated speech for correcting the speech signal distortions of the previously generated speech in the pre-existing knowledgebase of phonemes.

11 . The method of claim 10 wherein the operations further comprise converting a frequency domain signal combined from a neural network and the pre-existing knowledgebase into the corrected speech signal.

12 . The method of claim 10 wherein the data in the pre-existing knowledgebase of phonemes comprises average basic acoustic signal data of how a speaker speaks derived from the recorded audible speech.

13 . The method of claim 10 wherein the operations further comprise correcting for psychoacoustic perceived speech signal distortions of the pre-existing knowledgebase of phonemes.

14 . The method of claim 10 wherein the operations further comprise upsampling a frequency of an input vector to another frequency of an intermediate vector.

15 . The method of claim 10 wherein the operations further comprise correcting voiced phonemes of the pre-existing knowledgebase of phonemes.

16 . The method of claim 10 wherein the operations further comprise correcting unvoiced phonemes of the pre-existing knowledgebase of phonemes.

17 . The method of claim 10 wherein a neural network is configured based on psychoacoustic modeling of phonemes.

18 . The method of claim 14 , wherein the input vector comprises at least one of a base frequency, a phoneme duration, and a phoneme sequence.

19 . A non-transitory computer-readable medium having program instructions which, when executed across one or more processors, causes at least a portion of the one or more processors to perform operations comprising:
interacting with data of previously generated speech in a pre-existing knowledgebase of phonemes, wherein the previously generated speech has speech signal distortions; generating the corrected speech signal of the previously generated speech to correct for the speech signal distortions of the previously generated speech based upon, at least in part, interacting with the data of the previously generated speech in the pre-existing knowledgebase of phonemes; and applying the corrected speech signal to the previously generated speech for correcting the speech signal distortions of the previously generated speech in the pre-existing knowledgebase of phonemes.

20 . The non-transitory computer-readable medium of claim 19 wherein the operations further comprise correcting for psychoacoustic perceived speech signal distortions of the pre-existing knowledgebase of phonemes.

Assignees

• Telepathy Labs Inc

Inventors

• Reber Martin
• Avijeet Vijeta

Classifications

• G06F18/217

  Validation; Performance evaluation; Active pattern learning techniques · CPC title

• G06N3/042

  Knowledge-based neural networks; Logical representations of neural networks · CPC title

• G06F18/2135

  based on approximation criteria, e.g. principal component analysis · CPC title

• G06N3/08

  Learning methods · CPC title

• G10L25/30

  using neural networks · CPC title