Methods and apparatus for use in speech recognition systems for identifying unknown words and for adding previously unknown words to vocabularies and grammars of speech recognition systems

Patent metadata

Field	Value
Publication number	US-9754586-B2
Application number	US-13376208-A
Country	US
Kind code	B2
Filing date	Jun 5, 2008
Priority date	Nov 30, 2005
Publication date	Sep 5, 2017
Grant date	Sep 5, 2017

Abstract

The present invention concerns methods and apparatus for identifying and assigning meaning to words not recognized by a vocabulary or grammar of a speech recognition system. In an embodiment of the invention, the word may be in an acoustic vocabulary of the speech recognition system, but may be unrecognized by an embedded grammar of a language model of the speech recognition system. In another embodiment of the invention, the word may not be recognized by any vocabulary associated with the speech recognition system. In embodiments of the invention, at least one hypothesis is generated for an utterance not recognized by the speech recognition system. If the at least one hypothesis meets at least one predetermined criterion, a sword or more corresponding to the at least one hypothesis is added to the vocabulary of the speech recognition system. In other embodiments of the invention, before adding the word to the vocabulary of the speech recognition system, the at least one hypothesis may be presented to the user of the speech recognition system to determine if that is what the used intended when the user spoke.

First claim

We claim:

1. A speech recognition system configured for use with a portable electronic device including telephonic capabilities, the portable electronic device comprising a user interface configured to receive speech from a speaker as input, the speech recognition system comprising:
at least one processor programmed to:
determine that the received speech includes an item in an acoustic vocabulary of the speech recognition system, wherein the item is not associated with a valid command construct of an open embedded grammar of the speech recognition system, wherein the open embedded grammar includes a plurality of valid command constructs, each of which includes at least one command word for performing an action on the portable electronic device in combination with one or more words in the acoustic vocabulary of the speech recognition system;
determine a first confidence score for the item, wherein the first confidence score is determined based, at least in part, on an acoustic score for the item, wherein the acoustic score for the item is determined based, at least in part, on at least a portion of the received speech, wherein the first confidence score is determined based, at least in part, on a language model score for the item, wherein the language model score for the item is determined based, at least in part, on at least a portion of the received speech, and wherein the first confidence score is determined based, at least in part, on a sum of the acoustic score for the item and the language model score for the item;
determine a second confidence score for at least one word in a valid command construct of the open embedded grammar, wherein the second confidence score is determined based, at least in part, on an acoustic score for the at least one word in the valid command construct, wherein the acoustic score for the at least one word in the valid command construct is determined based, at least in part, on the at least a portion of the received speech, wherein the second confidence score is determined based, at least in part, on a language model score for the at least one word in the valid command construct, wherein the language model score for the at least one word in the valid command construct is determined based, at least in part, on the at least a portion of the received speech, and wherein and the second confidence score is determined based, at least in part, on a sum of the acoustic score for the at least one word in the valid command construct and the language model score for the at least one word in the valid command construct; and
add a new command construct to the open embedded grammar based, at least in part, on a comparison of the first confidence score and the second confidence score, wherein the new command construct includes the item.

2. The speech recognition system of claim 1 , wherein the portable electronic device is a cellular telephone.

3. The speech recognition system of claim 1 , wherein the at least one processor is further programmed to:
prior to adding the new command construct, request confirmation from the speaker that the new command construct corresponds to what the speaker intended, and wherein adding the new command construct is further based, at least in part, on receiving a confirmation from the speaker.

4. The speech recognition system of claim 3 , wherein the confirmation comprises confirmation of a spelling of the item and/or confirmation of a pronunciation of the item.

5. The speech recognition system of claim 1 , wherein the at least one processor is further programmed to:
instruct the speaker to illustrate the new command construct by inputting keystrokes, and wherein adding the new command construct is further based, at least in part, on the inputted keystrokes.

6. The speech recognition system of claim 1 , wherein the at least one processor is further programmed to:
detect biometric information identifying that the received speech is associated with the new command construct, and wherein adding the new command construct is further based, at least in part, on the detected biometric information.

7. The speech recognition system of claim 6 , wherein the biometric information comprises speech biometric information and/or data derived from video information.

8. The speech recognition system of claim 1 , wherein determining that received speech includes an item in an acoustic vocabulary of the speech recognition system comprises determining that the acoustic score for the item is greater than a threshold value.

9. A method for use with a speech recognition system configured for use with a portable electronic device including telephonic capabilities, the portable electronic device comprising a user interface configured to receive speech from a speaker as input, the method comprising:
determining that the received speech includes an item in an acoustic vocabulary of the speech recognition system, wherein the item is not associated with a valid command construct of an open embedded grammar of the speech recognition system, wherein the open embedded grammar includes a plurality of valid command constructs, each of which includes at least one command word for performing an action on the portable electronic device in combination with one or more words in the acoustic vocabulary of the speech recognition system;
determining a first confidence score for the item, wherein the first confidence score is determined based, at least in part, on an acoustic score for the item, wherein the acoustic score for the item is determined based, at least in part, on at least a portion of the received speech, wherein the first confidence score is determined based, at least in part, on a language model score for the item, wherein the language model score for the item is determined based, at least in part, on at least a portion of the received speech, and wherein the first confidence score is determined based, at least in part, on a sum of the acoustic score for the item and the language model score for the item;
determining a second confidence score for at least one word in a valid command construct of the open embedded grammar, wherein the second confidence score is determined based, at least in part, on an acoustic score for the at least one word in the valid command construct, wherein the acoustic score for the at least one word in the valid command construct is determined based, at least in part, on the at least a portion of the received speech, wherein the second confidence score is determined based, at least in part, on a language model score for the at least one word in the valid command construct, wherein the language model score for the at least one word in the valid command construct is determined based, at least in part, on the at least a portion of the received speech, and wherein the second confidence score is determined based, at least in part, on a sum of the acoustic score for the at least one word in the valid command construct and the language model score for the at least one word in the valid command construct; and
adding a new command construct to the open embedded grammar based, at least in part, on a comparison of the first confidence score and the second confidence score, wherein the new command construct includes the item.

10. The method of claim 9 , wherein the portable electronic device is a cellular telephone.

11. The method of claim 9 , further comprising:
prior to adding the new command construct, request confirmation from the speaker that the new command construct corresponds to what the speaker intended, and wherein adding the new command construct is further based, at least in part, on receiving a confirmation from the speaker.

12.

Assignees

• Deligne Sabine
• Gopinath Ramesh A
• Kanevsky Dimitri
• Viswanathan Mahesh
• Nuance Communications Inc

Inventors

• Deligne Sabine
• Gopinath Ramesh A
• Kanevsky Dimitri
• Viswanathan Mahesh

Classifications

• G10L15/183

  using context dependencies, e.g. language models · CPC title

• G10L2015/0631

  Creating reference templates; Clustering · CPC title

• G10L15/19 Primary

  Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules · CPC title

• G10L15/063

  Training · CPC title