Method for using a multi-scale recurrent neural network with pretraining for spoken language understanding tasks

Patent metadata

Field	Value
Publication number	US-9607616-B2
Application number	US-201514827669-A
Country	US
Kind code	B2
Filing date	Aug 17, 2015
Priority date	Aug 17, 2015
Publication date	Mar 28, 2017
Grant date	Mar 28, 2017

Abstract

A spoken language understanding (SLU) system receives a sequence of words corresponding to one or more spoken utterances of a user, which is passed through a spoken language understanding module to produce a sequence of intentions. The sequence of words are passed through a first subnetwork of a multi-scale recurrent neural network (MSRNN), and the sequence of intentions are passed through a second subnetwork of the multi-scale recurrent neural network (MSRNN). Then, the outputs of the first subnetwork and the second subnetwork are combined to predict a goal of the user.

First claim

We claim:

1. A spoken language understanding (SLU) method, comprising steps of:
receiving a sequence of words corresponding to one or more spoken utterances of a user;
passing the sequence of words through a spoken language understanding module to produce a sequence of intentions;
passing the sequence of words through a first subnetwork of a multi-scale recurrent neural network (MSRNN);
passing the sequence of intentions through a second subnetwork of the multi-scale recurrent neural network (MSRNN);
combining outputs of the first subnetwork and the second subnetwork to predict a goal of the user, wherein the steps are performed in a processor.

2. The method of claim 1 , wherein the sequence of words is an output of an automatic speech recognitions (ASR) system.

3. The method of claim 2 , wherein the sequence of words is a probability distribution over a set of words corresponding to the one or more spoken utterances of the user.

4. The method of claim 1 , wherein the goal is input to a dialog manager to output an action to be performed by a spoken language dialog system.

5. The method of claim 1 , wherein each intention in the sequence of intentions is a probability distribution over a set of intentions that correspond to the one or more spoken utterance of the user.

6. The method of claim 1 wherein the network parameters for the multi-scale recurrent neural network (MSRNN) are trained jointly using separate pre-trained initialization parameters for the first subnetwork and the second subnetwork.

Assignees

• Mitsubishi Electric Res Laboratories Inc

Inventors

• Watanabe Shinji
• Luan Yi
• Harsham Bret

Classifications

• G10L15/1822

  Parsing for meaning understanding · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G10L15/32

  Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems · CPC title

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G10L15/22

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title