Automatic Alert Dispositioning using Artificial Intelligence

Patent metadata

Field	Value
Publication number	US-2024177094-A1
Application number	US-202218072042-A
Country	US
Kind code	A1
Filing date	Nov 30, 2022
Priority date	Nov 30, 2022
Publication date	May 30, 2024
Grant date	—

Abstract

The system identifies false positives about a user and closes resulting alerts before the alert wastefully consume system resources. Machine learning techniques including clustering and multi-labeling classification are used to effectively categorize prior text-based notes to efficiently identify and automatically close false positive alerts. Moreover, weak labeling, AI transformers/sentence transformation, and/or k-means cluster analysis provide a means for condensing large quantities of textual data into ML model components with improved interpretability. Customer relationship management (CRM) platform and Risk Management Supervision (RMS) note analysis captures inefficiently/ineffectively organized past work and leverages it to reduce redundancies in future expert user/supervisory/customer advisory efforts. The various ML techniques disclosed herein output numerical features that directly improve model performance for alert triaging as a whole. Streamlining the presentation of large textual data to reviewers/users (e.g., supervision principals) as individual sentences or numerical values greatly improves explainability as a byproduct.

First claim

We claim:

1 . A method for enhanced triaging of alerts generated for customer service advisors, to reduce false positive alerts using machine learning and dimensionality reduction before multi-label classification of textual notes about interactions between the customer service advisors and customers, the method comprising:
displaying, on a graphical user interface (GUI), an alert generated for a customer service advisor, the alert being for a customer with a likelihood of satisfying a first scenario from among an enumerated list of scenarios; contemporaneous with servicing of the customer, augmenting the alert displayed on the GUI with a visual indicator of a likelihood that the alert corresponding to the customer is false positive based on a machine learning (ML) model; receiving, through the GUI, feedback provided by the customer service advisor about accuracy of the likelihood that the alert is false positive; iteratively updating, based on the feedback, the ML model, wherein the ML model:
infers a likelihood that the customer was previously notified of the first scenario by clustering numerical representations of the textual notes, which represent interactions between the customer service advisor and the customer, in relation to the enumerated list of scenarios, and
outputs a first multi-classification relevance score associated with the customer, wherein the first multi-classification relevance score comprises a first score corresponding to the first scenario, and wherein each score in the multi-classification relevance score is calculated as a distance from a corresponding numerical representation to a centroid of a corresponding cluster; and
in response to determining that the first score meets a threshold value, set the visual indicator to show that the likelihood is high that the alert corresponding to the customer is false positive.

2 . The method of claim 1 , wherein the triaging to reduce false positive alerts is performed using the ML model, which was created by steps comprising:
filtering keywords in a plurality of textual notes to remove a first set of keywords that fail to correspond to any specific scenario among the enumerated list of scenarios; after the filtering of the keywords, transforming the plurality of textual notes into a text-frequency matrix, wherein the first set of keywords are omitted from the text-frequency matrix, and wherein each row of the text-frequency matrix represents a different textual note; reducing a dimension of the text-frequency matrix, wherein the text-frequency matrix is outputted as numerical representations; k-means clustering on the numerical representations outputted by the text-frequency matrix to produce a plurality of clusters, each cluster corresponding to a single scenario in the enumerated list of scenarios; and for each textual note, outputting a multi-classification relevance score that includes a quantity of scores matching a quantity of clusters in the plurality of clusters, wherein each score in the multi-classification relevance score is calculated as a distance from the corresponding numerical representation to a centroid of a corresponding cluster.

3 . The method of claim 1 , wherein the plurality of textual notes are generated from prior, no-audio, no-video, textual chat transcripts between the customer and one or more customer service advisor.

4 . The method of claim 1 , wherein the likelihood of false positive is calculated in near real-time by an AI-bot executing the ML model, wherein the customer service advisor is an AI-chat bot.

5 . The method of claim 1 , wherein the alert is generated in response to a system of record detecting the first scenario, and wherein the alert is immutably assigned a predefined label corresponding to the first scenario, the method further comprising:
receiving, through the GUI, a textual note about interactions between the customer service advisor and the customer, wherein the interactions are related to at least one scenario from the enumerated list of scenarios in addition to the first scenario; associating the textual note with the predefined label previously assigned to the alert, which was generated in response to the detecting of the first scenario; and storing, in a RMS notes data store, the textual note with the associated predefined label.

6 . The method of claim 5 , wherein the RMS notes data store contains the plurality of textual notes and their associated predefined labels, and wherein the plurality of textual notes are associated with multiple customers.

7 . The method of claim 2 , wherein the transforming of the plurality of textual notes into the text-frequency matrix comprises concatenating the plurality of textual notes into an unstructured conversation thread associated with the customer.

8 . The method of claim 2 , wherein the dimensionality reduction is performed with principal component analysis (PCA).

9 . The method of claim 2 , wherein a number of dimensions being reduced is based on a model performance objective.

10 . The method of claim 2 , wherein the k-means clustering is of a soft clustering type, the method further comprising:
after performing the dimensionality reduction, measuring the distance from centroid of each cluster; and assigning, based on the multi-classification relevance score, a relevance score to each of the clusters based on the measured distance.

11 . The method of claim 10 , wherein the multi-classification relevance score corresponds to a likelihood that a particular note belongs to a scenario.

12 . The method of claim 1 , wherein the enumerated list of scenarios comprises concentration, production credit, charges, velocity, and time-weighted rate of return (TWRR).

13 . The method of claim 1 , wherein an audio call associated with the servicing of the customer by the customer service advisor is transcribed into a textual note for processing by a NLP system.

14 . The method of claim 1 , further comprising:
aggregating, for inclusion as ML model features, numerical features associated with the customer of an alert; and updating, based on the aggregated numerical features, one or more hyperparameters of the ML model such that a plurality of clusters produced by the ML model more accurately calculate the likelihood of false positive.

15 . The method of claim 1 , further comprising:
aggregating, for inclusion as ML model features, categorical features associated with the customer of an alert; and updating, based on the aggregated categorical features, one or more hyperparameters of the ML model to more accurately calculate the likelihood of false positive.

16 . A method of training a machine learning (ML) model for enhanced triaging of alerts generated for customer service advisors that are servicing customers to reduce false positive alerts, wherein interactions between a customer and one or more customer service advisors are recorded in textual notes, and wherein an alert for servicing the customer indicates a first scenario from among an enumerated list of scenarios, method comprising:
filtering keywords in the textual notes to remove a first set of keywords that fail to correspond to any specific scenario among the enumerated list of scenarios; after the filtering of the keywords, transforming the textual notes into a text-frequency matrix, wherein the first set of keywords are omitted from the text-frequency matrix, and wherein each row of the text-frequency matrix represents a different textual note; reducing a dimension of the text-frequency matrix, wherein the text-frequency matrix is outp

Assignees

• Bank Of America

Inventors

• Li Yvonne
• Chan Franklin Kaiyuen
• Kim Min Kyung
• Ginsberg Jared Scott
• Blogg Aaron

Classifications

• G06Q30/016

  After-sales · CPC title

• G06F16/244

  Grouping and aggregation · CPC title

• G06Q30/01

  Customer relationship services · CPC title

• G06F16/285

  Clustering or classification · CPC title

• G06Q10/0635 Primary

  Risk analysis of enterprise or organisation activities · CPC title