Methods and systems for generating recommendations for contacting users through the use of a contextual multi-armed bandit framework

Patent metadata

Field	Value
Publication number	US-12045874-B2
Application number	US-202016904336-A
Country	US
Kind code	B2
Filing date	Jun 17, 2020
Priority date	Jun 17, 2020
Publication date	Jul 23, 2024
Grant date	Jul 23, 2024

Abstract

Methods and systems are described for machine learning algorithms that dynamically allocate traffic to contact strategies that are performing well, while allocating less traffic to contact strategies that are underperforming. In particular, the methods and systems discussed are for the use of a contextual multi-armed bandit framework for applications that have both immediate results and long-term results, in which immediate results are correlated with the long-term results (e.g., results related to debt collection strategies).

First claim

What is claimed is:

1. A system for generating recommendations for contacting users through use of a contextual multi-armed bandit (cMAB) framework in machine learning application, the system comprising:
cloud-based memory configured to:
store a first cMAB model comprising a first neural network; and
store a second cMAB model comprising a second neural network; and

cloud-based control circuitry configured to:
determine that a user has a debt requiring collection;
in response to determining that the user has a debt requiring collection, input user data and available contact strategies into the first cMAB model, wherein each of the available contact strategies relates to a different interaction with a user;
recommend an actual short-term contact strategy;
contact the user using the actual short-term recommended contact strategy;
store a first result of the actual short-term recommended contact strategy, wherein the first result of the actual short-term recommended contact strategy comprises a first combination of attributes including a received amount during a predetermined time period;
normalize the first result of the actual short-term recommended contact strategy into a first feature vector;
train the first cMAB model based on the first feature vector;
receive a first plurality of outputs from the trained first cMAB model, wherein each of the first plurality of outputs corresponds to a respective first probability associated with one of a first plurality of classifications, wherein each of the first plurality of classifications relates to a different short-term result of a respective contact strategy of the available contact strategies, and wherein each of the first plurality of classifications represents a first respective float-point number corresponding to the respective first probability of the different short-term result;
input the first plurality of outputs from the trained first cMAB model comprising into the second cMAB model;
recommend an actual long-term contact strategy;
contact the user using the actual long-term recommended contact strategy;
store a second result of the actual long-term recommended contact strategy, wherein the second result of the actual long-term recommended contact strategy comprises a second combination of attributes including a response time;
normalize the second result of the actual long-term recommended contact strategy into a second feature vector;
train the second cMAB model based on the second feature vector;
receive a second plurality of outputs from the trained second cMAB model, wherein each of the second plurality of outputs corresponds to a respective second probability associated with one of a second plurality of classifications, wherein each of the second plurality of classifications relates to a different long-term result of a respective contact strategy of the available contact strategies, and wherein each of the second plurality of classifications represents a second respective float-point number corresponding to the respective second probability of the different long-term result; and

cloud-based I/O circuitry configured to generate an updated recommended contact strategy for collecting the debt based on the second plurality of outputs.

2. A method of generating recommendations for contacting users through use of a contextual multi-armed bandit (cMAB) framework in machine learning application, the method comprising:
inputting, using control circuitry, user data and available contact strategies into a first cMAB model comprising a first neural network, wherein each of the available contact strategies relates to a different interaction with a user;
recommending an actual short-term contact strategy;
contacting the user using the actual short-term recommended contact strategy;
storing a first result of the actual short-term recommended contact strategy, wherein the first result of the actual short-term recommended contact strategy comprises a first combination of attributes including a received amount during a predetermined time period;
normalizing the first result of the actual short-term recommended contact strategy into a first feature vector;
training the first cMAB model based on the first feature vector;
receiving, using the control circuitry, a first plurality of outputs from the trained first cMAB model, wherein each of the first plurality of outputs corresponds to a respective first probability associated with one of a first plurality of classifications, wherein each of the first plurality of classifications relates to a different short-term result of a respective contact strategy of the available contact strategies, and wherein each of the first plurality of classifications represents a first respective float-point number corresponding to the respective first probability of the different short-term result;
inputting, using the control circuitry, the first plurality of outputs from the trained first cMAB model into a second cMAB model, wherein the second cMAB model comprises a second neural network;
recommending an actual long-term contact strategy;
contacting the user using the actual long-term recommended contact strategy;
storing a second result of the actual long-term recommended contact strategy, wherein the second result of the actual long-term recommended contact strategy comprises a second combination of attributes including a response time;
normalizing the second result of the actual long-term recommended contact strategy into a second feature vector;
training the second cMAB model based on the second feature vector;
receiving, using the control circuitry, a second plurality of outputs from the trained second cMAB model, wherein each of the second plurality of outputs corresponds to a respective second probability associated with one of a second plurality of classifications, wherein each of the second plurality of classifications relates to a different long-term result of a respective contact strategy of the available contact strategies, and wherein each of the second plurality of classifications represents a second respective float-point number corresponding to the respective second probability of the different long-term result; and
generating an updated recommended contact strategy for collecting a debt based on the second plurality of outputs.

3. The method of claim 2 , further comprising determining that the user has the debt requiring collection, wherein inputting user data and available contact strategies into the first cMAB model is performed in response to determining that the user has the debt requiring collection.

4. The method of claim 2 , further comprising:
receiving a metric related to the actual short-term recommended contact strategy;
comparing the metric to threshold metric for the actual short-term recommended contact strategy; and
in response to determining that the metric corresponds to the threshold metric, determining to train the second cMAB model based on the first result of the actual short-term recommended contact strategy.

5. The method of claim 2 , wherein each of the different short-term result is defined by a first metric and a second metric, wherein the first metric corresponds to a type of user response for the respective contact strategy, and wherein the second metric corresponds to a short-term value received.

6. The method of claim 2 , wherein each of the different short-term result is defined by a first metric and a second metric, wherein the first metric corresponds to a qualitative response for the respective contact strategy, and wherein the second metric corresponds to a quantitative response for the respective contact strategy.

7. The method of claim 2 , wherein the available contact strategies include user-initiated contact st

Assignees

• Capital One Services Llc

Inventors

• Reding Luke
• Forsyth Gilbert
• Slowey Aaron
• Flynn Christopher
• Devore Charles
• Biscarri William
• Mccarthy Ryan
• Jafreen Rifat

Classifications

• G06Q10/06375

  Prediction of business process outcome or impact based on a proposed change · CPC title

• G06Q40/03 Primary

  Credit; Loans; Processing thereof · CPC title

• H04L51/212

  using filtering or selective blocking · CPC title

• H04L51/56

  Unified messaging, e.g. interactions between e-mail, instant messaging or converged IP messaging [CPM] · CPC title

• H04L51/214

  using selective forwarding · CPC title