Systems and methods to analyze customer contacts

Patent metadata

Field	Value
Publication number	US-11862148-B2
Application number	US-201916698449-A
Country	US
Kind code	B2
Filing date	Nov 27, 2019
Priority date	Nov 27, 2019
Publication date	Jan 2, 2024
Grant date	Jan 2, 2024

Abstract

Systems and methods to analyze contacts data. Contacts data may be encoded as text (e.g., chat logs), audio (e.g., audio recordings), and various other modalities. A computing resource service provider may implement a service to obtain audio data from a client, transcribe the audio data, thereby generating text, execute one or more natural language processing techniques to generate metadata associated with the text, processing at least the metadata to generate an output, determine whether the output matches one or more categories, and provide the output to the client. Techniques described herein may be performed as an asynchronous workflow.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
receiving, at a frontend service, a request to process a set of data for a client;
creating a job based at least in part on the request; and
as a result of the job being created, executing a step functions workflow comprising a plurality of steps to collectively:
obtain a copy of the set of data, the copy including a subset of audio data;
utilize an artificial intelligence speech-to-text service to generate transcripts for the subset of audio data, wherein a portion of the transcript is reconstructed based at least in part on fragments of the transcript that correspond to separate portions of the audio data obtained at different times;
as a result of reconstructing the portion of the transcript, use a natural language processing (NLP) service to perform a set of NLP techniques on the transcripts to generate metadata encoding one or more characteristics of the transcripts;
use a categorization service to identify one or more categories that match the subset of audio data, wherein the one or more categories are defined based at least in part on rules that evaluate content and audio characteristics of audio data;
generate an output that encodes at least the transcripts, the metadata, and the one or more categories; and
provide the output to the client.

2. The computer-implemented method of claim 1 , wherein creating the job comprises creating an entry in a database indicating the job is not yet started and the method further comprises:
detecting, from the database, that the job is not yet started;
determining a manner in which to execute the step functions workflow based at least in part on the job; and
updating status information of the job as part of executing the step functions workflow.

3. The computer-implemented method of claim 1 , wherein the set of NLP techniques includes sentiment analysis, entity detection, or key phrase detection.

4. The computer-implemented method of claim 1 , wherein at least a portion of the plurality of steps is executed asynchronously as event-driven functions.

5. A system, comprising:
one or more processors; and
memory that stores computer-executable instructions that, if executed, cause the system to:
receive, from a client of a computing resource service provider, a request to process a set of data, the set of data comprising audio data; and
as a result of receiving the request, execute a workflow to:
obtain the audio data;
use a first service to transcribe the audio data by utilizing artificial intelligence techniques, thereby generating a text-based transcript, wherein one or more turns of the text-based transcript are reconstructed based at least in part on portions of the text-based transcript that correspond to separate portions of the audio data obtained at different times;
as a result of determining that the one or more turns of the text-based transcript are reconstructed, use a second service to execute one or more natural language processing techniques, thereby generating metadata output associated with the text-based transcript;
process at least the metadata output to generate a human-readable output;
use a third service to determine whether the human-readable output matches one or more categories, wherein the one or more categories are defined based at least in part on rules that evaluate content and audio characteristics; and
make the human-readable output available to the client.

6. The system of claim 5 , wherein the instructions to obtain the audio data include instructions that, if executed, cause the system to:
assume the role associated with a client, wherein the audio data is accessible by the client via a data storage service; and
submit a second request to the data storage service for the audio data.

7. The system of claim 5 , wherein the transcript is partitioned into a plurality of turns based on who is speaking.

8. The system of claim 7 , wherein the one or more natural language processing techniques include sentiment analysis that assigns sentiments to the plurality of turns.

9. The system of claim 5 , wherein the output is encoded in a human-readable format.

10. The system of claim 5 , wherein the audio data is recorded by a customer contact service.

11. The system of claim 5 , wherein the instructions include further instruction that, if executed, cause the system to determine a sentiment score for the transcript.

12. The system of claim 5 , wherein the instructions include further instruction that, if executed, cause the system to redact, from the transcript, sensitive data.

13. A non-transitory computer-readable storage medium storing thereon executable instructions that, as a result of being executed by one or more processors of a computer system, cause the computer system to:
copy source data from a customer data store;
obtain a text-based transcript from the source data, wherein a portion of the text-based transcript is reconstructed based at least in part on at least two subportions of the text-based transcripts that correspond to separate portions of the source data obtained at different times;
as a result of reconstructing the portion of the text-based transcript, execute one or more natural language processing techniques, to determine
metadata encoding conversation characteristics associated with the transcript;
determine that one or more categories apply to the source data, wherein the one or more categories are defined based at least in part on rules that evaluate based on at least content and conversation characteristics;
generate an output based at least in part on the transcript, the metadata encoding conversation characteristics associated with the transcript, and the one or more categories; and
make the output available to the customer data store.

14. The non-transitory computer-readable storage medium of claim 13 , wherein the source data comprises chat logs and audio recordings, further wherein the text-based transcript is either a chat log or a transcript of an audio recording.

15. The non-transitory computer-readable storage medium of claim 13 , wherein the metadata encodes one or more detected entities, keywords, or phrases.

16. The non-transitory computer-readable storage medium of claim 13 , wherein the metadata encodes sentiments of speakers for one or more portions of the source data.

17. The non-transitory computer-readable storage medium of claim 16 , wherein sentiments are one of: positive, negative, neutral, or mixed.

18. The non-transitory computer-readable storage medium of claim 13 , wherein the output is a JavaScript Object Notation (JSON) file.

19. The non-transitory computer-readable storage medium of claim 13 , wherein the output includes timestamps of where the one or more categories were applicable to the source data.

20. The non-transitory computer-readable storage medium of claim 13 , wherein the instructions include further instructions that, as a result of being executed by one or more processors of a computer system, cause the computer system to emit event and metering information.

21. The system of claim 5 , wherein:
the computer-executable instructions further cause the system to parse the audio data by speaker into audio waveforms; and
the artificial intelligence techniques are utilized to map the audio waveforms to text.

Assignees

• Amazon Tech Inc

Inventors

• Sivasubramanian Swaminathan
• Philomin Vasanth
• Anbazhagan Vikram
• Singh Ashish
• Deo Atul
• Arora Anuroop
• Young Jessie
• Yadav Harsh
• Kale Priyanka Shirish

Classifications

• G10L15/1815 Primary

  Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning · CPC title

• G06F16/61

  Indexing; Data structures therefor; Storage structures · CPC title

• G06N5/04

  Inference or reasoning models · CPC title

• G06Q10/107

  Computer-aided management of electronic mailing [e-mailing] · CPC title

• G06Q30/016 Primary

  After-sales · CPC title