Method and system for efficient spoken term detection using confusion networks

Patent metadata

Field	Value
Publication number	US-2016005398-A1
Application number	US-201514837876-A
Country	US
Kind code	A1
Filing date	Aug 27, 2015
Priority date	Mar 31, 2014
Publication date	Jan 7, 2016
Grant date	—

Abstract

Systems and methods for spoken term detection are provided. A method for spoken term detection, comprises receiving phone level out-of-vocabulary (OOV) keyword queries, converting the phone level OOV keyword queries to words, generating a confusion network (CN) based keyword searching (KWS) index, and using the CN based KWS index for both in-vocabulary (IV) keyword queries and the OOV keyword queries.

First claim

What is claimed is:

1 . A method for spoken term detection, comprising:
receiving phone level out-of-vocabulary (OOV) keyword queries; converting the phone level OOV keyword queries to words; generating a confusion network (CN) based keyword searching (KWS) index; and using the CN based KWS index for both in-vocabulary (IV) keyword queries and the OOV keyword queries, wherein the receiving, converting, generating and using steps are performed by a computer system comprising a memory and at least one processor coupled to the memory.

2 . The method according to claim 1 , wherein generating the CN based KWS index comprises constructing the CN based KWS index from a plurality of confusion networks by compiling each confusion network into a weighted finite state transducer having the same topology as the confusion network.

3 . The method according to claim 2 , wherein each weighted finite state transducer includes input labels that are words on each arc in the corresponding confusion network.

4 . The method according to claim 2 , wherein each weighted finite state transducer includes output labels that encode a start time (T start) and an end time (T end) of each arc in the corresponding confusion network as T start-T end strings.

5 . The method according to claim 2 , wherein each weighted finite state transducer includes costs that are negative log CN posteriors for each arc in the confusion network.

6 . The method according to claim 2 , wherein for each weighted finite state transducer, the method further comprises adding a new start node, S i with zero-cost epsilon-arcs connecting S i to each node in the weighted finite state transducer.

7 . The method according to claim 2 , wherein for each weighted finite state transducer, the method further comprises adding a new end node, E i with zero-cost epsilon-arcs connecting each node in the weighted finite state transducer to E i .

8 . The method according to claim 6 , further comprising obtaining a final single index by creating a new start node, S, that is connected to each S i by the zero-cost epsilon arcs.

9 . The method according to claim 7 , further comprising obtaining a final single index by creating a new end node, E, that is connected to each E i by the zero-cost epsilon arcs.

10 . The method according to claim 1 , wherein using the CN based KWS index for an IV query comprises:
converting the query into a word automaton; composing the query automaton with an index transducer; and if overlapping hits are produced, keeping only a highest scoring hit.

11 . The method according to claim 1 , wherein converting the phone level OOV keyword queries to words comprises converting the phone level OOV keyword queries to phonetic finite state acceptors, wherein phone sequences for IV terms are looked up in a recognition lexicon and phone sequences for OOV terms are generated with a grapheme-to-phoneme model.

12 . The method according to claim 11 , wherein converting the phone level OOV keyword queries to words further comprises expanding the phone level OOV keyword queries through composition with a weighted finite state transducer (WFST) that models probabilities of confusions between different phones.

13 . The method according to claim 12 , wherein converting the phone level OOV keyword queries to words further comprises extracting N-best hypotheses represented by each expanded WFST.

14 . The method according to claim 13 , wherein converting the phone level OOV keyword queries to words further comprises mapping back the N-best hypotheses to a set of N or fewer word sequences through composition with a finite state transducer that maps from phone sequences to word sequences.

15 . The method according to claim 14 , wherein using the CN based KWS index for an OOV query comprises searching for the resulting word sequences via composition with the CN based KWS index.

16 . A system for spoken term detection, comprising:
a query module capable of receiving phone level out-of-vocabulary (OOV) keyword queries; a mapping module capable of converting the phone level OOV keyword queries to words; an indexing module capable of generating a confusion network (CN) based keyword searching (KWS) index; and a search module capable of using the CN based KWS index for both in-vocabulary (IV) keyword queries and the OOV keyword queries.

17 . The system according to claim 16 , wherein the indexing module is further capable of constructing the CN based KWS index from a plurality of confusion networks by compiling each confusion network into a weighted finite state transducer having the same topology as the confusion network.

18 . The system according to claim 16 , wherein the mapping module is further capable of converting the phone level OOV keyword queries to phonetic finite state acceptors, wherein phone sequences for IV terms are looked up in a recognition lexicon and phone sequences for OOV terms are generated with a grapheme-to-phoneme model.

19 . The system according to claim 18 , wherein the mapping module is further capable of:
expanding the phone level OOV keyword queries through composition with a weighted finite state transducer (WFST) that models probabilities of confusions between different phones; extracting N-best hypotheses represented by each expanded WFST; and mapping back the N-best hypotheses to a set of N or fewer word sequences through composition with a finite state transducer that maps from phone sequences to word sequences.

20 . A computer program product for spoken term detection, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform a method comprising:
receiving phone level out-of-vocabulary (OOV) keyword queries; converting the phone level OOV keyword queries to words; generating a confusion network (CN) based keyword searching (KWS) index; and using the CN based KWS index for both in-vocabulary (IV) keyword queries and the OOV keyword queries.

Assignees

• IBM

Inventors

• Kingsbury Brian E D
• Kuo Hong-Kwang
• Mangu Lidia
• Soltau Hagen

Classifications

• G10L13/08

  Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination · CPC title

• G10L2015/085

  Methods for reducing search complexity, pruning · CPC title

• G10L15/02

  Feature extraction for speech recognition; Selection of recognition unit · CPC title

• G10L2015/025

  Phonemes, fenemes or fenones being the recognition units · CPC title

• G10L15/083 Primary

  Recognition networks (G10L15/142, G10L15/16 take precedence) · CPC title