Artificial intelligence apparatus for converting text and speech in consideration of style and method for the same

Patent metadata

Field	Value
Publication number	US-11715485-B2
Application number	US-201916490316-A
Country	US
Kind code	B2
Filing date	May 17, 2019
Priority date	May 17, 2019
Publication date	Aug 1, 2023
Grant date	Aug 1, 2023

Abstract

According to an embodiment of the present invention, there is provided an artificial intelligence (AI) apparatus for mutually converting a text and a speech, including: a memory configured to store a plurality of Text-To-Speech (TTS) engines; and a processor configured to: obtain image data containing a text, determine a speech style corresponding to the text, generate a speech corresponding to the text by using a TTS engine corresponding to the determined speech style among the plurality of TTS engines, and output the generated speech.

First claim

The invention claimed is:

1. An artificial intelligence (AI) apparatus for mutually converting a text and a speech, comprising:
a memory configured to store a plurality of Text-To-Speech (TTS) engines, wherein at least one of the plurality of TTS engines is learned using a machine learning algorithm or a deep learning algorithm; and
a processor configured to:
obtain image data containing a text,
determine a speech style corresponding to the text,
generate a speech corresponding to the text by using a TTS engine corresponding to the determined speech style among the plurality of TTS engines, and
output the generated speech,

wherein the processor is further configured to:
extract text style features from the text included in the image data,
in response to the text being handwritten, determine a text creator corresponding to an age and a gender based on the text style features,
determine a speech style corresponding to the determined text creator, and
adjust the speech style according to the text style features of the text creator being changed, and

wherein the processor is further configured to:
compare a shape of a word within handwritten text of the determined text creator with shapes of other words within the handwritten text of the determined text creator,
in response to the shape of the word within the handwritten text being different than the shapes of the other words within the handwritten text, determine a different speech style for the word that is different than a speech style determined for the other words within the handwritten text of the determined text creator, and
convert the word into an audio speech segment and output the audio speech segment according to the different speech style.

2. The AI apparatus of claim 1 , wherein each of the plurality of TTS engines includes at least one speech style feature, and
wherein the speech style feature includes at least one of a tone, a pitch, a speed, an accent, a speech volume, or a pronunciation.

3. A method for mutually converting a text and a speech, the method comprising:
obtaining image data containing a text;
determining a speech style corresponding to the text;
generating a speech corresponding to the text by using a Text-To-Speech (TTS) engine corresponding to the determined speech style, wherein the TTS engine is learned using a machine learning algorithm or a deep learning algorithm; and
outputting the generated speech,
wherein the determining the speech style comprises:
extracting text style features from the text included in the image data;
in response to the text being handwritten, determining a text creator corresponding to an age and a gender based on the text style features; and
determining the speech style corresponding to the determined text creator, and wherein the method further comprises:

adjusting the speech style according to the text style features of the text creator being changed, and
wherein the method further comprises:
comparing a shape of a word within handwritten text of the determined text creator with shapes of other words within the handwritten text of the determined text creator;
in response to the shape of the word within the handwritten text being different than the shapes of the other words within the handwritten text, determining a different speech style for the word that is different than a speech style determined for the other words within the handwritten text of the determined text creator; and
converting the word into an audio speech segment and outputting the audio speech segment according to the different speech style.

Assignees

• Lg Electronics Inc

Inventors

• Park Jisoo
• Chae Jonghoon

Classifications

• G06N3/0895

  Weakly supervised learning, e.g. semi-supervised or self-supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/082

  modifying the architecture, e.g. adding, deleting or silencing nodes or connections · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0985

  Hyperparameter optimisation; Meta-learning; Learning-to-learn · CPC title