Methods and systems for synthesising speech from text

Patent metadata

Field	Value
Publication number	US-12573373-B2
Application number	US-202218052861-A
Country	US
Kind code	B2
Filing date	Nov 4, 2022
Priority date	Nov 5, 2021
Publication date	Mar 10, 2026
Grant date	Mar 10, 2026

Abstract

A method for synthesising speech from text includes receiving text and encoding, by way of an encoder module, the received text. The method further includes determining, by way of an attention module, a context vector from the encoding of the received text, wherein determining the context vector comprises at least one of: applying a threshold function to an attention vector and accumulating the thresholded attention vector, or applying an activation function to the attention vector and accumulating the activated attention vector. The method further includes determining speech data from the context vector.

First claim

What is claimed is:

1 . A computer-implemented method for synthesising speech from text, the method comprising:
receiving text; encoding, by way of an encoder, the received text to obtain an encoding of the received text; determining, by way of an attention vector, a context vector from the encoding of the received text, wherein determining the context vector comprises:
applying a threshold function to the attention vector and accumulating a thresholded attention vector;
determining speech data from the context vector; and automatically converting the speech data into an output speech audio waveform.

2 . The method according to claim 1 , wherein determining the context vector comprises determining a score from the accumulated thresholded attention vector.

3 . The method according to claim 1 , wherein determining speech data from the context vector comprises decoding, by way of a decoder, the context vector.

4 . The method according to claim 3 , wherein the decoder comprises a recurrent neural network (RNN).

5 . The method according to claim 1 , wherein the encoder comprises a conformer.

6 . The method according to claim 1 , wherein the received text comprises a representation of a non-speech sound.

7 . A method according to claim 6 , wherein the non-speech sound is represented by one or more repeating tokens.

8 . A non-transitory computer-readable storage medium comprising computer readable code configured to cause a computer to perform a set of operations, comprising:
receiving text; encoding, by way of an encoder, the received text to obtain an encoding of the received text; determining, by way of an attention vector, a context vector from the encoding of the received text, wherein determining the context vector comprises:
applying a threshold function to the attention vector and accumulating a thresholded attention vector;
determining speech data from the context vector; and automatically converting the speech data into an output speech audio waveform.

9 . The non-transitory computer-readable storage medium according to claim 8 , wherein determining the context vector comprises determining a score from the accumulated thresholded attention vector.

10 . The non-transitory computer-readable storage medium according to claim 8 , wherein determining speech data from the context vector comprises decoding, by way of a decoder, the context vector.

11 . The non-transitory computer-readable storage medium according to claim 8 , wherein the decoder comprises a recurrent neural network (RNN).

12 . A computer system comprising one or more processors and memory storing instructions, configured to be executed by the one or more processors, to perform a set of operations, comprising:
receiving text; encoding, by way of an encoder, the received text to obtain an encoding of the received text; determining, by way of an attention vector, a context vector from the encoding of the received text, wherein determining the context vector comprises:
applying a threshold function to the attention vector and accumulating a thresholded attention vector;
determining speech data from the context vector; and automatically converting the speech data into an output speech audio waveform.

13 . The computer system according to claim 12 , wherein determining the context vector comprises determining a score from the accumulated thresholded attention vector.

14 . The computer system according to claim 12 , wherein determining speech data from the context vector comprises decoding, by way of a decoder, the context vector.

15 . The computer system according to claim 14 , wherein the decoder comprises a recurrent neural network (RNN).

16 . The computer system according to claim 12 , wherein the encoder comprises a conformer.

17 . The computer system according to claim 12 , wherein the received text comprises a representation of a non-speech sound.

18 . The computer system according to claim 17 , wherein the non-speech sound is represented by one or more repeating tokens.

Assignees

• Spotify Ab

Inventors

• Vaughan Felix Mathew William Chase
• Blum Harry Alexander Coultas
• Flynn John
• Qureshi Zeenat

Classifications

• G10L13/02

  Methods for producing synthetic speech; Speech synthesisers · CPC title

• G10L13/07

  Concatenation rules · CPC title

• G10L13/047

  Architecture of speech synthesisers · CPC title

• G10L25/30

  using neural networks · CPC title

• G10L13/08 Primary

  Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination · CPC title