Conversational log replay with voice and debugging information

Patent metadata

Field	Value
Publication number	US-10574597-B2
Application number	US-201715707823-A
Country	US
Kind code	B2
Filing date	Sep 18, 2017
Priority date	Sep 18, 2017
Publication date	Feb 25, 2020
Grant date	Feb 25, 2020

Abstract

Methods, systems, and computer programs are presented for providing a user interface (UI) for monitoring and debugging an Artificial Intelligence (AI) chatting hot. One method includes operations for receiving a selection on the UI to replay an electronic conversation between a first and a second party, selecting conversation data associated with the electronic conversation from a data log having conversation data from several electronic conversations, and analyzing the conversation data to identify conversation parameters. The conversation parameters include text in each entry of the electronic conversation, timing of the entries, and debugging parameters for each entry. The method further includes an operation for causing presentation of the electronic conversation on the UI, which includes presenting the text of each entry, the audio corresponding to speech associated with each entry timed according to the timing of the entry, and the debugging parameters embedded within the presented text.

First claim

What is claimed is:

1. A method comprising:
receiving, by one or more central processing units (CPUs), a selection on a user interface to replay an electronic conversation between a first party and a chatbot, the electronic conversation comprising one or more entries having a question from the first party and an answer created by the chatbot in response to the question;
selecting, by the one or more CPUs, conversation data of the electronic conversation from a data log comprising conversation data from a plurality of electronic conversations;
analyzing, by the one or more CPUs, the conversation data to identify conversation parameters, the conversation parameters comprising text in each entry of the electronic conversation, timing of each entry in the electronic conversation, and debugging parameters for each entry; and
streaming, by the one or more CPUs, the replay of the electronic conversation on the user interface, the streaming comprising:
presenting the text of each entry of the replay of the electronic conversation;
playing audio corresponding to speech associated with each entry of the replay of the electronic conversation, the audio being timed according to the timing of the entry, the playing of the audio using a text-to-speech conversation that presents a voice of the first party using a voice font different from a real voice of the first party, the playing of the audio artificially adding emotion when the debugging parameters include detection of emotion in the entry; and
presenting the debugging parameters embedded within the presented text.

2. The method as recited in claim 1 , wherein the debugging parameters comprise a timestamp, a feed identifier which uniquely identifies the electronic conversation from other conversations, and a filtered topic identifier when detected in the electronic conversation, the filtered topic being a topic that was filtered during the electronic conversation for being offensive.

3. The method as recited in claim 1 , wherein the debugging parameters further comprise a detected emotion in one of the entries of the electronic conversation.

4. The method as recited in claim 3 , wherein playing the audio corresponding to the speech includes presenting overlapping audio when one of the parties interrupts the other party.

5. The method as recited in claim 1 , wherein playing the audio further comprises:
converting the text of an entry of the electronic conversation to speech utilizing a neutral speaker, wherein a voice of the first party is not presented on the user interface to protect privacy of the first party.

6. The method as recited in claim 1 , wherein the user interface provides an option for marking an entry as erroneous or controversial.

7. The method as recited in claim 1 , further comprising:
receiving, via the user interface, labels corresponding to the entries while presenting the electronic conversation; and
utilizing the received labels to train a machine learning program that monitors content of electronic conversations for offensive content.

8. The method as recited in claim 1 , wherein the user interface includes an option to select one or more entries during the presenting, the method further comprising:
creating a message that comprises the one or more entries, the debugging parameters corresponding to the one or more entries, and an audio clip of the one or more entries; and
transmitting the message.

9. The method as recited in claim 1 , further comprising:
automatically presenting, sequentially, a plurality of electronic conversations on the user interface.

10. The method as recited in claim 1 , wherein the debugging parameters comprise a bug on code detected during the conversation, a source of data used for forming the response by the chatbot, a skill identifier which uniquely identifies a configuration utilized when the answer was created, and metadata for debugging purposes.

11. The method as recited in claim 1 , wherein the debugging parameters comprise a detection of racism.

12. The method as recited in claim 1 , wherein playing audio further includes:
playing the audio utilizing playback parameters, the playback parameters including voices selected for speech representation, text fonts for presenting the text of each entry, and debugging parameters presented embedded within the streaming of the text.

13. The method as recited in claim 1 , wherein playing the audio further comprises:
when anger is detected in the electronic conversation, providing an angry voice font to play the audio to alert a human monitor of a possibility of problems.

14. A system comprising:
a memory comprising instructions; and
one or more CPUs, wherein the instructions, when executed by the one or more CPUs, cause the one or more CPUs to perform operations comprising:
receiving a selection on a user interface to replay an electronic conversation between a first party and a chatbot, the electronic conversation comprising one or more entries having a question from the first party and an answer created by the chatbot in response to the question;
selecting conversation data of the electronic conversation from a data log comprising conversation data from a plurality of electronic conversations;
analyzing the conversation data to identify conversation parameters, the conversation parameters comprising text in each entry of the electronic conversation, timing of each entry in the electronic conversation, and debugging parameters for each entry; and
streaming the replay of the electronic conversation on the user interface, the streaming comprising:
presenting the text of each entry of the replay of the electronic conversation;
playing audio corresponding to speech associated with each entry of the replay of the electronic conversation, the audio being timed according to the timing of the entry, the playing of the audio using a text-to-speech conversation that presents a voice of the first party using a voice font different from a real voice of the first party, the playing of the audio artificially adding emotion when the debugging parameters include detection of emotion in the entry; and

presenting the debugging parameters embedded within the presented text.

15. The system as recited in claim 14 , wherein the debugging parameters comprise a timestamp, a feed identifier which uniquely identifies the conversation from other conversations, and a filtered topic identifier when detected in the electronic conversation, the filtered topic being a topic that was filtered during the electronic conversation for being offensive.

16. The system as recited in claim 14 , wherein the debugging parameters further comprise a detected emotion on one of the entries of the electronic conversation.

17. The system as recited in claim 16 , wherein playing the audio corresponding to the speech includes presenting overlapping audio when one of the parties interrupts the other party.

18. The system as recited in claim 14 , wherein playing the audio further comprises:
converting the text of an entry of the electronic conversation to speech utilizing a neutral speaker, wherein a voice of the first party is not presented on the user interface to protect privacy of the first party.

19. A non-transitory machine-readable storage medium including instructions that, when executed by a machine, cause the machine to perform operations comprising:
receiving a selection on a user interface to replay an electronic conversation between a first party and a chatbot, the electronic conversation comprising one or mo

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Johnson Jr Joseph Edwin
• Schuerman Matthew W
• Brinkman Jr Donald Frank

Classifications

• G06F40/30

  Semantic analysis · CPC title

• G06F40/284

  Lexical analysis, e.g. tokenisation or collocates · CPC title

• G06N5/04

  Inference or reasoning models · CPC title

• H04L51/063

  Content adaptation, e.g. replacement of unsuitable content · CPC title

• G06F40/216

  using statistical methods · CPC title