System and method for user-specified pronunciation of words for speech synthesis and recognition

Patent metadata

Field	Value
Publication number	US-9966060-B2
Application number	US-201715445863-A
Country	US
Kind code	B2
Filing date	Feb 28, 2017
Priority date	Jun 7, 2013
Publication date	May 8, 2018
Grant date	May 8, 2018

Abstract

The method is performed at an electronic device with one or more processors and memory storing one or more programs for execution by the one or more processors. A first speech input including at least one word is received. A first phonetic representation of the at least one word is determined, the first phonetic representation comprising a first set of phonemes selected from a speech recognition phonetic alphabet. The first set of phonemes is mapped to a second set of phonemes to generate a second phonetic representation, where the second set of phonemes is selected from a speech synthesis phonetic alphabet. The second phonetic representation is stored in association with a text string corresponding to the at least one word.

First claim

What is claimed is:

1. A method for learning word pronunciations, comprising:
at an electronic device with one or more processors and memory storing one or more programs for execution by the one or more processors:
detecting an error in a speech based interaction with a digital assistant based on detecting a user input other than the speech based interaction;
in response to detecting the error, receiving a speech input from a user, the speech input including a pronunciation of one or more words;
determining, based on the pronunciation of the one or more words, a first set of phonemes from a speech recognition phonetic alphabet and a second set of phonemes from a speech synthesis phonetic alphabet;
updating one or more databases to include the first set of phonemes and the second set of phonemes in association with a text string corresponding to the one or more words; and
performing speech recognition or speech synthesis using the updated one or more databases.

2. The method of claim 1 , wherein the one or more words were received in a prior speech input provided by the user, and wherein the error is an error in speech recognition of the one or more words.

3. The method of claim 1 , wherein the one or more words were output in a speech output by the electronic device, and wherein the error is an error in speech synthesis of the one or more words.

4. The method of claim 1 , further comprising:
receiving the speech input including the one or more words;
performing speech recognition on the speech input to generate the text string corresponding to the one or more words;
determining a confidence metric of the text string; and
detecting the error based on a determination that the confidence metric does not meet a predetermined threshold.

5. The method of claim 1 , further comprising:
synthesizing a speech output including the one or more words; and
detecting the error based on an indication from the user that the one or more words were pronounced incorrectly.

6. The method of claim 1 , further comprising:
performing speech recognition on the speech input to generate the text string corresponding to the one or more words; and wherein updating the one or more databases comprises
updating a speech recognizer to associate the first set of phonemes with the text string.

7. The method of claim 6 , further comprising:
receiving a second speech input including the one or more words;
determining a third set of phonemes for the one or more words;
determining that the one or more words in the second speech input correspond to the text string based on comparing the first set of phonemes and the third set of phonemes.

8. The method of claim 1 , further comprising:
prior to receiving the speech input from the user and after detecting the error, prompting the user to provide the speech input, the speech input including a preferred pronunciation of the one or more words.

9. The method of claim 1 , further comprising:
synthesizing a speech output including the one or more words; and
displaying a textual version of the speech output on a display of the electronic device.

10. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by an electronic device, cause the device to:
detect an error in a speech based interaction with a digital assistant based on detecting a user input other than the speech based interaction;
in response to detecting the error, receive a speech input from a user, the speech input including a pronunciation of one or more words;
determine, based on the pronunciation of the one or more words, a first set of phonemes from a speech recognition phonetic alphabet and a second set of phonemes from a speech synthesis phonetic alphabet;
update one or more databases to include the first set of phonemes and the second set of phonemes in association with a text string corresponding to the one or more words; and
perform speech recognition or speech synthesis using the updated one or more databases.

11. The computer readable storage medium of claim 10 , wherein the one or more words were received in a prior speech input provided by the user, and wherein the error is an error in speech recognition of the one or more words.

12. The computer readable storage medium of claim 10 , wherein the one or more words were output in a speech output by the electronic device, and wherein the error is an error in speech synthesis of the one or more words.

13. The computer readable storage medium of claim 10 , wherein the instructions further cause the device to:
receive the speech input including the one or more words;
perform speech recognition on the speech input to generate the text string corresponding to the one or more words;
determine a confidence metric of the text string; and
detect the error based on a determination that the confidence metric does not meet a predetermined threshold.

14. The computer readable storage medium of claim 10 , wherein the instructions further cause the device to:
synthesize a speech output including the one or more words; and
detect the error based on an indication from the user that the one or more words were pronounced incorrectly.

15. An electronic device, comprising:
one or more processors; and
memory storing one or more programs, the one or more programs including instructions, which when executed by the one or more processors, cause the one or more processors to:
detect an error in a speech based interaction with a digital assistant based on detecting a user input other than the speech based interaction;
in response to detecting the error, receive a speech input from a user, the speech input including a pronunciation of one or more words;
determine, based on the pronunciation of the one or more words, a first set of phonemes from a speech recognition phonetic alphabet and a second set of phonemes from a speech synthesis phonetic alphabet;
update one or more databases to include the first set of phonemes and the second set of phonemes in association with a text string corresponding to the one or more words; and
perform speech recognition or speech synthesis using the updated one or more databases.

16. The device of claim 15 , wherein the one or more words were received in a prior speech input provided by the user, and wherein the error is an error in speech recognition of the one or more words.

17. The device of claim 15 , wherein the one or more words were output in a speech output by the electronic device, and wherein the error is an error in speech synthesis of the one or more words.

18. The device of claim 15 , wherein the instructions further cause the one or more processors to:
receive the speech input including the one or more words;
perform speech recognition on the speech input to generate the text string corresponding to the one or more words;
determine a confidence metric of the text string; and
detect the error based on a determination that the confidence metric does not meet a predetermined threshold.

19. The device of claim 15 , wherein the instructions further cause the one or more processors to:
synthesize a speech output including the one or more words; and
detect the error based on an indication from the user that the one or more words were pronounced incorrectly.

Assignees

• Apple Inc

Inventors

• Naik Devang K
• Gruber Thomas R
• Weiner Liam
• Binder Justin G
• Srisuwananukorn Charles
• Evermann Gunnar
• Williams Shaun Eric
• Chen Hong
• Napolitano Lia T

Classifications

• G10L13/027 Primary

  Concept to speech synthesisers; Generation of natural phrases from machine-based concepts (generation of parameters for speech synthesis out of text G10L13/08) · CPC title

• G10L2015/0638

  Interactive procedures · CPC title

• G10L15/26

  Speech to text systems (G10L15/08 takes precedence) · CPC title

• G10L15/063

  Training · CPC title

• G10L15/22

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title