Log-linear dialog manager that determines expected rewards and uses hidden states and actions

Patent metadata

Field	Value
Publication number	US-9311430-B2
Application number	US-201314106968-A
Country	US
Kind code	B2
Filing date	Dec 16, 2013
Priority date	Dec 16, 2013
Publication date	Apr 12, 2016
Grant date	Apr 12, 2016

Abstract

A dialog manager receives previous user actions and previous observations and current observations. Previous and current user states, previous user actions, current user actions, future system actions, and future observations are hypothesized. The user states, the user actions, and the user observations are hidden. A feature vector is extracted based on the user states, the system actions, the user actions, and the observations. An expected reward of each current action is based on a log-linear model using the feature vectors. Then, the current action that has an optimal expected reward is outputted.

First claim

We claim:

1. A dialog manager comprising the steps of:
receiving previous user actions and previous observations and current observations;
hypothesizing previous and current user states, previous user actions, current user actions, future system actions, and future observations, wherein the user states, the user actions, and the user observations are hidden;
extracting a feature vector based on the user states, the system actions, the user actions, and the observations;
determining an expected reward of each current action based on a log-linear model using the feature vectors; and
outputting the current action that has an optimal expected reward, wherein the steps are performed in a processor.

2. The dialog manager of claim 1 , wherein a probabilistic model has four variables at each time step t, including two observable variables: the system action a t , the observation o t , and two latent variables: the user action u t and the user state s t .

3. The method of claim 2 , wherein a dialog session of duration T is represented by four variable sequences
s 0:T ,a 0:T ,o 1:T ,u 1:T .

4. The method of claim 3 , wherein the dialog session is represented by a factor graph, which corresponds to a joint probability distribution

p
⁡

(

s

0
:
T

,

a

0
:
T

,

u

1
:
T

,

o

1
:
T

)

=

1

Z
θ

⁢

exp
⁡

[

∑

t
=
0

T

⁢

θ
f
T

⁢

φ
f

⁢

(

s
t

,

a
t

,

s

t
+
1

,

u

t
+
1

)

+

∑

t
=
1

T

⁢

θ
g
T

⁢

φ
g

⁡

(

u
t

,

o
t

)

]

,

where Z θ is a normalizing constant, φ f and φ g are the feature vectors, and θ f and θ g vectors of corresponding model parameters, respectively.

5. The method of claim 1 , wherein the observations are spoken words or text.

6. The method of claim 3 , wherein S, U, A, and O represent the variable spaces that is a set of all possible values for the variables s t , u t , a t , and o t , respectively.

7. The method of claim 6 , further comprising:
defining the variable spaces S, U, and A using a context-free grammar (CFG) including a set of production rules.

8. The method of claim 7 , wherein each variable space is defined as a set of all possible parse trees that can be generated by the CFG.

9. The method of claim 3 , wherein a planning part of the dialog manger determines an optimal system action a τ , given all previous system actions a 0:τ-1 and previous observations o 1:τ .

10. The method of claim 3 , further comprising:
maximizing an objective function

E

s

0
:
T

,

a

Assignees

• Mitsubishi Electric Res Lab

Inventors

• Watanabe Shinji
• Tang Hao

Classifications

• G06F40/35

  Discourse or dialogue representation · CPC title

• G06F40/30 Primary

  Semantic analysis · CPC title

• G06F16/90332 Primary

  Natural language query formulation or dialogue systems · CPC title

• G10L15/22

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title

• G10L15/1822

  Parsing for meaning understanding · CPC title