End-of-turn detection in spoken dialogues

Patent metadata

Field	Value
Publication number	US-10957320-B2
Application number	US-201916257566-A
Country	US
Kind code	B2
Filing date	Jan 25, 2019
Priority date	Jan 25, 2019
Publication date	Mar 23, 2021
Grant date	Mar 23, 2021

Abstract

Systems, computer-implemented methods, and computer program products that can facilitate predicting a source of a subsequent spoken dialogue are provided. According to an embodiment, a system can comprise a memory that stores computer executable components and a processor that executes the computer executable components stored in the memory. The computer executable components can comprise a speech receiving component that can receive a spoken dialogue from a first entity. The computer executable components can further comprise a speech processing component that can employ a network that can concurrently process a transition type and a dialogue act of the spoken dialogue to predict a source of a subsequent spoken dialogue.

First claim

What is claimed is:

1. A system, comprising:
a memory that stores computer executable components;
a processor that executes the computer executable components stored in the memory, wherein the computer executable components comprise:
a speech receiving component that receives a spoken dialogue from a first entity; and
a speech processing component that employs a neural network that concurrently processes a first classifier and a second classifier using acoustic cues from the spoken dialogue to predict a source of a subsequent spoken dialogue, wherein;
the first classifier generates a first prediction of an intention of the spoken dialogue,
the second classifier generates a second prediction of a type of turn of the spoken dialogue, and
the neural network combines the first prediction and the second prediction using a minimizing joint loss function to predict whether the source of the subsequent spoken dialogue will be the first entity or another entity.

2. The system of claim 1 , wherein the neural network is a multi-task neural network, and wherein the system further comprises a network optimizing component that optimizes the multi-task neural network by employing a plurality of speech labels to predict the source of the subsequent spoken dialogue.

3. The system of claim 2 , wherein the plurality of speech labels comprises an optimizing data set.

4. The system of claim 1 , wherein the minimizing joint loss function comprises a first loss function for the first prediction and a second loss function for the second prediction.

5. The system of claim 1 , wherein the speech processing component predicts the source of the subsequent spoken dialogue in real time during a communication session comprising the spoken dialogue.

6. The system of claim 1 , wherein the type of turn is selected from a group consisting of a turn hold, a turn switch, a smooth switch, and an overlapping switch.

7. The system of claim 1 , wherein the acoustic cues comprise timing of the spoken dialogue.

8. The system of claim 1 , wherein the acoustic cues comprise a cue selected from the group consisting of intonation, pitch change, speaking rate, and pause.

9. The system of claim 1 , wherein the other entity is a computerized spoken dialog system.

10. A computer-implemented method, comprising:
receiving, by a system operatively coupled to a processor, a spoken dialogue from a first entity; and
predicting, by the system, a source of a subsequent spoken dialogue by employing a neural network that concurrently processes a first classifier and a second classifier using acoustic cues from the spoken dialogue, wherein:
the first classifier generates a first prediction of an intention of the spoken dialogue,
the second classifier generates a second prediction of a type of turn of the spoken dialogue, and
the neural network combines the first prediction and the second prediction using a minimizing joint loss function to predict whether the source of the subsequent spoken dialogue will be the first entity or another entity.

11. The computer-implemented method of claim 10 , wherein the neural network is a multi-task neural network, and wherein the computer-implemented method further comprises optimizing, by the system, the multi-task neural network by employing a plurality of speech labels to predict the source of the subsequent spoken dialogue.

12. The computer-implemented method of claim 11 , wherein the plurality of speech labels comprises an optimizing data set.

13. The computer-implemented method of claim 10 , wherein the minimizing joint loss function comprises a first loss function for the first prediction and a second loss function for the second prediction.

14. The computer-implemented method of claim 10 , wherein the predicting the source of the subsequent spoken dialogue occurs in real time during a communication session comprising the spoken dialogue.

15. The computer-implemented method of claim 10 , wherein the type of turn is selected from a group consisting of a turn hold, a turn switch, a smooth switch, and an overlapping switch.

16. The computer-implemented method of claim 10 , wherein the acoustic cues comprise timing of the spoken dialogue.

17. The computer-implemented method of claim 10 , wherein the acoustic cues comprise a cue selected from the group consisting of intonation, pitch change, speaking rate, and pause.

18. The computer-implemented method of claim 10 , wherein the other entity is a computerized spoken dialog system.

19. A computer program product facilitating predicting a source of a subsequent spoken dialogue, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to:
receive, by the processor, a spoken dialogue from a first entity; and
predict, by the processor, the source of the subsequent spoken dialogue by employing a neural network that concurrently processes a first classifier and a second classifier using acoustic cues from the spoken dialogue, wherein:
the first classifier generates a first prediction of an intention of the spoken dialogue,
the second classifier generates a second prediction of a type of turn of the spoken dialogue, and
the neural network combines the first prediction and the second prediction using a minimizing joint loss function to predict whether the source of the subsequent spoken dialogue will be the first entity or another entity.

20. The computer program product of claim 19 , wherein the neural network is a multi-task neural network, and wherein the program instructions are further executable by the processor to cause the processor to optimize, by the processor, the multi-task neural network by employing a plurality of speech labels to predict the source of the subsequent spoken dialogue.

Assignees

• IBM
• Univ Michigan Regents

Inventors

• Polymenakos Lazaros
• Dimitriadis Dimitrios B
• Aldeneh Zakaria
• Mower Provost Emily

Classifications

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G10L15/32

  Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems · CPC title

• G10L2015/225

  Feedback of the input speech · CPC title