Incorporating an exogenous large-vocabulary model into rule-based speech recognition

Patent metadata

Field	Value
Publication number	US-9601108-B2
Application number	US-201414158147-A
Country	US
Kind code	B2
Filing date	Jan 17, 2014
Priority date	Jan 17, 2014
Publication date	Mar 21, 2017
Grant date	Mar 21, 2017

Abstract

Incorporation of an exogenous large-vocabulary model into rule-based speech recognition is provided. An audio stream is received by a local small-vocabulary rule-based speech recognition system (SVSRS), and is streamed to a large-vocabulary statistically-modeled speech recognition system (LVSRS). The SVSRS and LVSRS perform recognitions of the audio. If a portion of the audio is not recognized by the SVSRS, a rule is triggered that inserts a mark-up in the recognition result. The recognition result is sent to the LVSRS. If a mark-up is detected, recognition of a specified portion of the audio is performed. The LVSRS result is unified with the SVSRS result and sent as a hybrid response back to the SVSRS. If the hybrid-recognition rule is not triggered, an arbitration algorithm is evoked to determine whether the SVSRS or the LVSRS recognition has a lesser word error rate. The determined recognition is sent as a response to the SVSRS.

First claim

We claim:

1. A method for providing speech recognition by a computing device, comprising:
receiving an audio stream;
processing the audio stream by performing a rule-based speech recognition of the audio stream using rule-based grammars to generate a first recognition result comprising a rule-matched recognition result of a recognized portion of the audio stream and a mark-up specifying an unrecognized portion of the audio stream;
sending the first recognition result to a large vocabulary speech recognition system that processes the first recognition result and the mark-up using a statistical-based recognition of the unrecognized portion of the audio stream; and
receiving a response, including a recognition of the unrecognized portion, from the large vocabulary speech recognition system.

2. The method of claim 1 , further comprising sending the audio stream and metadata to the large vocabulary speech recognition system.

3. The method of claim 1 , wherein the response comprises the rule-matched recognition portion of the audio stream and a second recognition result.

4. The method of claim 3 , wherein the second recognition result comprises a large vocabulary speech recognition system recognition result of the unrecognized portion.

5. The method of claim 1 , wherein the response comprises a second recognition result produced by the large vocabulary speech recognition system.

6. The method of claim 5 , further comprising producing a final result based on the first recognition result and the second recognition result.

7. The method of claim 6 , wherein producing the final result comprises replacing the mark-up in the first recognition result with at least a portion of the second recognition result.

8. The method of claim 1 , further comprising performing a task based on the first recognition result and the second recognition result.

9. A system for providing incorporation of an exogenous large-vocabulary model into rule-based speech recognition, comprising:
one or more processors; and
a memory coupled to the one or more processors, the one or more processors operable to:
receive an audio stream;
perform a rule-based speech recognition of the audio stream using a small vocabulary speech recognition system;
produce a first recognition result comprising a rule-matched recognition result of a portion of the audio stream that is recognized by the rule-based speech recognition;
insert a mark-up in the first recognition result specifying a portion of the audio stream that is unrecognized by the rule-based speech recognition;
send the first recognition result to a large vocabulary speech recognition system that applies a statistical-based recognition of the first recognition result and the unrecognized portion of the audio stream; and
receive a response from the large vocabulary speech recognition system, the response comprising a second recognition result that includes a recognition of the unrecognized portion.

10. The system of claim 9 , wherein the one or more processors are further operable to send the audio stream and metadata to the large vocabulary speech recognition system.

11. The system of claim 10 , wherein the one or more processors are operable to send the audio stream to the large vocabulary speech recognition system by streaming the audio stream to the large vocabulary speech recognition system.

12. The system of claim 10 , wherein the one or more processors are operable to simultaneously send the audio stream and perform the rule-based speech recognition of the audio stream.

13. The system of claim 9 , wherein the second recognition result comprises a large vocabulary speech recognition system recognition result.

14. The system of claim 9 , wherein the first recognition result comprises a plurality of mark-ups specifying a plurality of portions of the audio stream that are not matched with a rule.

15. The system of claim 9 , wherein the first recognition result is produced using rule-based grammars.

16. The system of claim 9 , wherein the one or more processors are further operable to produce a final result based on the first recognition result and the second recognition result.

17. The system of claim 16 , wherein producing the final result comprises replacing the mark-up in the first recognition result with at least a portion of the second recognition result.

18. The system of claim 16 , wherein the one or more processors are further operable to perform a task based on the final result.

19. A method for providing speech recognition, comprising:
receiving an audio stream;
processing the audio stream using a rule-based speech recognition of the audio stream using rule-based grammars to generate a first recognition result comprising a rule-matched recognition result of a recognized portion of the audio stream and a mark-up specifying an unrecognized portion of the audio stream;
sending the first recognition result to a remote large vocabulary speech recognition system for applying a statistical-based recognition of the first recognition result and the unrecognized portion of the audio stream;
receiving a response from the large vocabulary speech recognition system, wherein the response comprises a second recognition result produced by the large vocabulary speech recognition system;
producing a final result by replacing the mark-up in the first recognition result with at least a portion of the second recognition result; and
performing a task based on the final result.

20. The method of claim 19 , further comprising streaming the audio stream to the large vocabulary speech recognition system simultaneously with performing the rule-based speech recognition of the audio stream.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Wilson Travis
• Quazi Salman
• Vicondoa John
• Fatehpuria Pradip

Classifications

• G10L15/197

  Probabilistic grammars, e.g. word n-grams · CPC title

• G10L15/193

  Formal grammars, e.g. finite state automata, context free grammars or word networks · CPC title

• G10L15/32 Primary

  Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems · CPC title

• G10L15/18

  using natural language modelling · CPC title

• G10L15/30

  Distributed recognition, e.g. in client-server systems, for mobile phones or network applications · CPC title