Artificial intelligence-based text-to-speech system and method

Patent metadata

Field	Value
Publication number	US-2023351999-A1
Application number	US-202318346657-A
Country	US
Kind code	A1
Filing date	Jul 3, 2023
Priority date	May 18, 2017
Publication date	Nov 2, 2023
Grant date	—

Abstract

A technique improves training and speech quality of a text-to-speech (TTS) system having an artificial intelligence, such as a neural network. The TTS system is organized as a front-end subsystem and a back-end subsystem. The front-end subsystem is configured to provide analysis and conversion of text into input vectors, each having at least a base frequency, f 0 , a phenome duration, and a phoneme sequence that is processed by a signal generation unit of the back-end subsystem. The signal generation unit includes the neural network interacting with a pre-existing knowledgebase of phenomes to generate audible speech from the input vectors. The technique applies an error signal from the neural network to correct imperfections of the pre-existing knowledgebase of phenomes to generate audible speech signals. A back-end training system is configured to train the signal generation unit by applying psychoacoustic principles to improve quality of the generated audible speech signal.

First claim

What is claimed is:

1 . A text-to-speech (TTS) system including one or more processors and one or more memories configured to perform operations for converting text into a corrected speech signal comprising:
training a neural network based upon, at least in part, data of previously generated speech in a pre-existing knowledgebase of phonemes, wherein the previously generated speech has an inaccuracy; generating a lossy representation of at least a portion of the data for use in the training; and applying lossy representation of at least the portion of the data to the previously generated speech for correcting the inaccuracy of the previously generated speech in the pre-existing knowledgebase of phonemes.

2 . The TTS system of claim 1 wherein the lossy representation reduces a representation of a phoneme in the pre-existing knowledgebase of phonemes resulting in a lossy representation of the phoneme where the inaccuracy represents inaudible errors.

3 . The TTS system of claim 1 wherein generating the lossy representation includes limiting derivations of a principal component analysis for at least the portion of the data.

4 . The TTS system of claim 1 wherein the lossy representation is a domain-to-frequency domain transformation of at least the portion of the data.

5 . The TTS system of claim 1 wherein applying the lossy representation includes correcting voiced phonemes of the pre-existing knowledgebase of phonemes using principal component analysis.

6 . The TTS system of claim 1 wherein applying the lossy representation includes correcting unvoiced phonemes of the pre-existing knowledgebase of phonemes using noise band/energy band thresholding.

7 . The TTS system of claim 1 wherein applying the lossy representation includes combining a limited number of frequency bands with specified band widths.

8 . A method of processing text-to-speech (TTS) comprising:
training a neural network based upon, at least in part, data of previously generated speech in a pre-existing knowledgebase of phonemes, wherein the previously generated speech has an inaccuracy; generating a lossy representation of at least a portion of the data for use in the training; and applying lossy representation of at least the portion of the data to the previously generated speech for correcting the inaccuracy of the previously generated speech in the pre-existing knowledgebase of phonemes.

9 . The method of claim 8 wherein the lossy representation reduces a representation of a phoneme in the pre-existing knowledgebase of phonemes resulting in a lossy representation of the phoneme where the inaccuracy represents inaudible errors.

10 . The method of claim 8 wherein generating the lossy representation includes limiting derivations of a principal component analysis for at least the portion of the data.

11 . The method of claim 8 wherein the lossy representation is a domain-to-frequency domain transformation of at least the portion of the data.

12 . The method of claim 8 wherein applying the lossy representation includes correcting voiced phonemes of the pre-existing knowledgebase of phonemes using principal component analysis.

13 . The method of claim 8 wherein applying the lossy representation includes correcting unvoiced phonemes of the pre-existing knowledgebase of phonemes using noise band/energy band thresholding.

14 . The method of claim 8 wherein applying the lossy representation includes combining a limited number of frequency bands with specified band widths.

15 . A non-transitory computer-readable medium having program instructions which, when executed across one or more processors, causes at least a portion of the one or more processors to perform operations comprising:
training a neural network based upon, at least in part, data of previously generated speech in a pre-existing knowledgebase of phonemes, wherein the previously generated speech has an inaccuracy; generating a lossy representation of at least a portion of the data for use in the training; and applying lossy representation of at least the portion of the data to the previously generated speech for correcting the inaccuracy of the previously generated speech in the pre-existing knowledgebase of phonemes.

16 . The non-transitory computer-readable medium of claim 15 wherein the lossy representation reduces a representation of a phoneme in the pre-existing knowledgebase of phonemes resulting in a lossy representation of the phoneme where the inaccuracy represents inaudible errors.

17 . The non-transitory computer-readable medium of claim 15 wherein generating the lossy representation includes limiting derivations of a principal component analysis for at least the portion of the data.

18 . The non-transitory computer-readable medium of claim 15 wherein applying the lossy representation includes correcting voiced phonemes of the pre-existing knowledgebase of phonemes using principal component analysis.

19 . The non-transitory computer-readable medium of claim 15 wherein applying the lossy representation includes correcting unvoiced phonemes of the pre-existing knowledgebase of phonemes using noise band/energy band thresholding.

20 . The non-transitory computer-readable medium of claim 15 wherein applying the lossy representation includes combining a limited number of frequency bands with specified band widths.

Assignees

• Telepathy Labs Inc

Inventors

• Reber Martin
• Avijeet Vijeta

Classifications

• G06N3/08

  Learning methods · CPC title

• G10L25/30

  using neural networks · CPC title

• G06N3/0499

  Feedforward networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0495

  Quantised networks; Sparse networks; Compressed networks · CPC title