End-of-turn detection in spoken dialogues

Patent metadata

Field	Value
Publication number	US-11645473-B2
Application number	US-202017132036-A
Country	US
Kind code	B2
Filing date	Dec 23, 2020
Priority date	Jan 25, 2019
Publication date	May 9, 2023
Grant date	May 9, 2023

Abstract

Systems, computer-implemented methods, and computer program products that can facilitate predicting a source of a subsequent spoken dialogue are provided. According to an embodiment, a system can comprise a memory that stores computer executable components and a processor that executes the computer executable components stored in the memory. The computer executable components can comprise a speech receiving component that can receive a spoken dialogue from a first entity. The computer executable components can further comprise a speech processing component that can employ a network that can concurrently process a transition type and a dialogue act of the spoken dialogue to predict a source of a subsequent spoken dialogue.

First claim

What is claimed is:

1. A system, comprising:
a memory that stores computer executable components;
a processor that executes the computer executable components stored in the memory, wherein the computer executable components comprise:
a speech processing component that employs a neural network to process a spoken dialogue between a first entity and a second entity to predict a source of a subsequent spoken dialogue, wherein the neural network combines a first prediction of an intention of the spoken dialogue and a second prediction of a type of turn of the spoken dialogue to predict whether the source of the subsequent spoken dialogue will be the first entity or the second entity.

2. The system of claim 1 , wherein the type of turn is selected from a group consisting of a turn hold, a turn switch, a smooth switch, and an overlapping switch.

3. The system of claim 1 , wherein the intention is selected from a group consisting of an interruption, a question, a request, and a statement.

4. The system of claim 1 , wherein the neural network employs a minimizing joint loss function to combine the first prediction and the second prediction.

5. The system of claim 4 , wherein the minimizing joint loss function comprises a first loss function for the first prediction and a second loss function for the second prediction.

6. The system of claim 1 , wherein the speech processing component determines at least one of the first prediction or the second prediction based on acoustic cues from the spoken dialogue.

7. The system of claim 6 , wherein the acoustic cues comprise a cue selected from a group consisting of intonation, pitch change, speaking rate, and pause.

8. A computer-implemented method, comprising:
processing, by a system operatively coupled to a processor, via a neural network, a spoken dialogue between a first entity and a second entity to generate a first prediction of an intention of the spoken dialogue and a second prediction of a type of turn of the spoken dialogue; and
predicting, by the system, via the neural network, a source of a subsequent spoken dialogue by combining the first prediction and the second prediction.

9. The computer-implemented method of claim 8 , wherein the type of turn is selected from a group consisting of a turn hold, a turn switch, a smooth switch, and an overlapping switch.

10. The computer-implemented method of claim 8 , wherein the intention is selected from a group consisting of an interruption, a question, a request, and a statement.

11. The computer-implemented method of claim 8 , wherein the neural network employs a minimizing joint loss function to combine the first prediction and the second prediction.

12. The computer-implemented method of claim 11 , wherein the minimizing joint loss function comprises a first loss function for the first prediction and a second loss function for the second prediction.

13. The computer-implemented method of claim 8 , further comprising determining, by the system, via the neural network, at least one of the first prediction or the second prediction based on acoustic cues from the spoken dialogue.

14. The computer-implemented method of claim 13 , wherein the acoustic cues comprise a cue selected from a group consisting of intonation, pitch change, speaking rate, and pause.

15. A computer program product facilitating predicting a source of a subsequent spoken dialogue, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to:
process, via a neural network, a spoken dialogue between a first entity and a second entity to generate a first prediction of an intention of the spoken dialogue and a second prediction of a type of turn of the spoken dialogue; and
predict, via the neural network, the source of the subsequent spoken dialogue by combining the first prediction and the second prediction.

16. The computer program product of claim 15 , wherein the type of turn is selected from a group consisting of a turn hold, a turn switch, a smooth switch, and an overlapping switch.

17. The computer program product of claim 15 , wherein the intention is selected from a group consisting of an interruption, a question, a request, and a statement.

18. The computer program product of claim 15 , wherein the neural network employs a minimizing joint loss function to combine the first prediction and the second prediction.

19. The computer program product of claim 18 , wherein the minimizing joint loss function comprises a first loss function for the first prediction and a second loss function for the second prediction.

20. The computer program product of claim 15 , wherein the neural network is a multi-task neural network, and wherein the program instructions are further executable by the processor to cause the processor to determine, via the neural network, at least one of the first prediction or the second prediction based on acoustic cues from the spoken dialogue.

Assignees

• IBM
• Univ Michigan Regents

Inventors

• Polymenakos Lazaros
• Dimitriadis Dimitrios B
• Aldeneh Zakaria
• Mower Provost Emily

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G10L15/16

  using artificial neural networks · CPC title

• G10L15/22 Primary

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title

• G06F40/35 Primary

  Discourse or dialogue representation · CPC title