Sound synthesis device, sound synthesis method and storage medium

Patent metadata

Field	Value
Publication number	US-2016180833-A1
Application number	US-201514969150-A
Country	US
Kind code	A1
Filing date	Dec 15, 2015
Priority date	Dec 22, 2014
Publication date	Jun 23, 2016
Grant date	—

Abstract

A sound synthesis device that includes a processor configured to perform the following: extracting intonation information from prosodic information contained in sound data and digitally smoothing the extracted intonation information to obtain smoothed intonation information; obtaining a plurality of digital sound units based on text data and concatenating the plurality of digital sound units so as to construct a concatenated series of digital sound units that corresponds to the text data; and modifying the concatenated series of digital sound units in accordance with the smoothed intonation information with respect to at least one of parameters of the concatenated series of digital sound units to generate synthesized sound data corresponding to the text data.

First claim

What is claimed is:

1 . A sound synthesis device, comprising a processor configured to perform the following:
extracting intonation information from prosodic information contained in sound data and digitally smoothing the extracted intonation information to obtain smoothed intonation information; obtaining a plurality of digital sound units based on text data and concatenating the plurality of digital sound units so as to construct a concatenated series of digital sound units that corresponds to the text data; and modifying the concatenated series of digital sound units in accordance with the smoothed intonation information with respect to at least one of parameters of the concatenated series of digital sound units to generate synthesized sound data corresponding to the text data.

2 . The sound synthesis device according to claim 1 , wherein said processor concatenates the plurality of digital sound units to construct the concatenated series of digital sound units that meets a prescribed matching condition with respect to the text data.

3 . The sound synthesis device according to claim 2 ,
wherein the sound data represents speech by a user.

4 . The sound synthesis device according to claim 1 , wherein said one of the parameters of the concatenated series of digital sound units is a pitch sequence, and said processor modifies the pitch sequence in the concatenated series of digital sound units so as to substantially match the smoothed intonation information.

5 . The sound synthesis device according to claim 1 , wherein, in obtaining said smoothed intonation information, said processor smoothes a pitch sequence in the prosodic information contained in the sound data, and extracts as said smoothed intonation information the pitch sequence that has been smoothed.

6 . The sound synthesis device according to claim 5 , wherein, in smoothing said pitch sequence in the prosodic information, said processor quantizes pitches of the pitch sequence in the prosodic information contained in the sound data, and smoothes the pitch sequence by acquiring a weighted moving average of the quantized pitches.

7 . The sound synthesis device according to claim 4 , wherein, in modifying the pitch sequence, said processor adjusts respective time scales of a pitch sequence in the prosodic information included in the sound data and of said pitch sequence in the concatenated series of digital sound units, and adjusts at least one of the pitch sequence in the prosodic information and the pitch sequence in the concatenated series of digital sound units so that periods during which pitches exist substantially match with each other.

8 . The sound synthesis device according to claim 1 , wherein said one of the parameters of the concatenated series of digital sound units is a power sequence, and said processor modifies the power sequence in the concatenated series of digital sound units so as to substantially match the smoothed intonation information.

9 . The sound synthesis device according to claim 8 , wherein, in obtaining said smoothed intonation information, said processor smoothes a power sequence in the prosodic information included in the sound data and extracts the smoothed power sequence as the smoothed intonation information, and
wherein, in modifying the power sequence in the concatenated series of digital sound units, said processor smoothes the power sequence in the concatenated series of digital sound units, acquires a sequence of ratios between the smoothed power sequence in the concatenated series of digital sound units and the smoothed power sequence in the prosodic information, and corrects the smoothed power sequence in the concatenated series of digital sound units in accordance with said sequence of ratios.

10 . The sound synthesis device according to claim 9 , wherein said processor smoothes the power sequence in the prosodic information by acquiring a weighted average of respective powers in the power sequence in the prosodic information.

11 . The sound synthesis device according to claim 9 , wherein, in modifying the power sequence in the concatenated series of digital sound units, said processor adjusts respective time scales of the power sequence in the prosodic information included in the sound data and of the power sequence in the concatenated series of digital sound units.

12 . A method of synthesizing sound performed by a processor in a sound synthesis device, the method comprising:
extracting intonation information from prosodic information contained in sound data and digitally smoothing the extracted intonation information to obtain smoothed intonation information; obtaining a plurality of digital sound units based on text data and concatenating the plurality of digital sound units so as to construct a concatenated series of digital sound units that corresponds to the text data; and modifying the concatenated series of digital sound units in accordance with the smoothed intonation information with respect to at least one of parameters of the concatenated series of digital sound units to generate synthesized sound data corresponding to the text data.

13 . A non-transitory storage medium that stores instructions executable by a processor included in a sound synthesis device, said instructions causing the processor to perform the following:
extracting intonation information from prosodic information contained in sound data and digitally smoothing the extracted intonation information to obtain smoothed intonation information; obtaining a plurality of digital sound units based on text data and concatenating the plurality of digital sound units so as to construct a concatenated series of digital sound units that corresponds to the text data; and modifying the concatenated series of digital sound units in accordance with the smoothed intonation information with respect to at least one of parameters of the concatenated series of digital sound units to generate synthesized sound data corresponding to the text data.

14 . The sound synthesis device according to claim 1 , wherein said processor is further configured to perform the following:
receiving the sound data; receiving the text data; and causing a database to be accessed to obtain said plurality of digital sound units.

15 . The method of synthesizing sound according to claim 12 , further comprising:
receiving the sound data; receiving the text data; and causing a database to be accessed to obtain said plurality of digital sound units.

16 . The non-transitory storage medium according to claim 13 , wherein said instructions cause the processor to further perform the following:
receiving the sound data; receiving the text data; and causing a database to be accessed to obtain said plurality of digital sound units.

Assignees

• Casio Computer Co Ltd

Inventors

• Tanaka Hyuta

Classifications

• G10L13/0335

  Pitch control · CPC title

• G10L13/10 Primary

  Prosody rules derived from text; Stress or intonation · CPC title

• G10L13/08

  Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination · CPC title

• G10L13/06 Primary

  Elementary speech units used in speech synthesisers; Concatenation rules · CPC title

• G10L21/013

  Adapting to target pitch · CPC title