Systems and methods to manage feedback for a multi-arm bandit algorithm

Patent metadata

Field	Value
Publication number	US-10909567-B1
Application number	US-201916702697-A
Country	US
Kind code	B1
Filing date	Dec 4, 2019
Priority date	Dec 4, 2019
Publication date	Feb 2, 2021
Grant date	Feb 2, 2021

Abstract

Systems and methods for managing feedback for a multi-ARM bandit algorithm are disclosed. For example, the system may include one or more memory units storing instructions and one or more processors configured to execute the instructions to perform operations. The operations may include sending, to a device, a creative configured for display on the device. The operations may include receiving, from the device, feedback data associated with the creative, wherein the feedback data indicates a degree of user interaction with the creative. The operations may include checking the feedback data for a baked time associated with the creative. The operations may include sending the feedback data to a node based on the baked time. The operations may include updating, by the node, a model associated with the feedback data.

First claim

What is claimed is:

1. A system for configuring a computerized model for improving creatives, comprising:
one or more memory units storing instructions;
one or more processors that execute the instructions;
a bus that interconnects the one or more memory units and the one or more processors; and a server configured to operate a multi-armed bandit application programming interface (API) that is accessible by the one or more processors and that is configured to analyze a creative, the server also communicates with an on-line self-serve tool configured to be accessed by a user through a portal;
the one or more processors configured to interact with the mufti-armed bandit API and configured to execute the instructions to perform operations comprising:
sending, to a device, the creative configured for display on the device;
receiving, from the device, feedback data associated with the creative, the feedback data indicating a degree of user interaction with the creative;
checking the feedback data for a baked time associated with the creative; sending the feedback data to a node based on the baked time; and
dynamically updating, by the node, the multi-armed bandit API associated with the feedback data, wherein receiving the feedback data is based on a delay time associated with the feedback data, wherein the feedback data acquires the baked time when an age of the feedback data is equal to or greater than the associated delay time, wherein the node is part of a cluster, the cluster having a plurality of nodes,
wherein the node includes a listener, and when the listener detects the feedback data based on the delay time, the listener is configured to direct the feedback data to the node for processing while preventing another node from receiving the feedback data such that the plurality of nodes are enabled to work efficiently by reducing or eliminating number of operations, and by decreasing latency and reducing strain on system resources including storage devices and a network interface, and
wherein the creative includes at least one of an email, webpage advertisement, or a webpage, for a vehicle finance marketing campaign.

2. The system of claim 1 , the operations further comprising: determining a difference between the feedback data and expected feedback data, wherein the updating is based on the determined differences.

3. The system of claim 1 , the operations further comprising: determining the feedback data, does not have a baked time; and waiting a predetermined amount of time before deter reining the feedback data has a baked time, wherein the feedback data is sent to the node after determining the feedback data has a baked time.

4. The system of claim 1 , wherein the node is a first node, the operations further comprising: sending the feedback data to a second node of the cluster before sending the feedback data to the first node; and determining the feedback data cannot be processed by the second node.

5. The system of claim 4 , wherein the plurality of nodes comprises listeners that detect when a node is available and direct the feedback data to the available node.

6. The system of claim 5 , the operations further comprising detecting, at the cluster, the feedback data, and wherein the feedback data is sent from a distributed delay queue associated with the cluster.

7. The system of claim 1 , wherein the associated delay time is (i) calculated based on a distribution channel associated with the feedback data and (ii) stored in a memory component of the cluster.

8. The system of claim 1 , wherein the device is a user device, the operations further comprising storing the feedback data at a memory component of the cluster.

9. The system of claim 1 , wherein the feedback data is associated with a distribution channel and the creative is associated with the distribution channel.

10. The system of claim 9 , wherein the model comprises parameters associated with the distribution channel and the feedback data.

11. The system of claim 10 , wherein the feedback data indicates a time associated with a user interaction with the creative.

12. The system of claim 10 , wherein the model is a first model, the distribution channel is a first distribution channel, the creative is a first creative; and the operations further comprise: determining a second model associated with (i) a second creative associated with the first creative or (ii) a second distribution channel associated with the first creative, and updating the second model based on the received feedback data.

13. The system of claim 1 , wherein the operations further comprise updating the creative using the updated model.

14. A method for configuring a computerized model for improving creatives, comprising one or more processors configured to interact with a multi-armed bandit application programming interface (API) and perform the following operations:
receiving feedback data associated with a creative from a device, the feedback data indicating a degree of user interaction with the creative;
placing the feedback data into a distributed delay queue;
determining the feedback data has a baked time associated with the creative;
sending the feedback data to a cluster based on the baked time, the cluster having a plurality of nodes;
determining an available node of the plurality of nodes;
sending the feedback data to the available node; and
dynamically updating, by the node, the multi-armed bandit API associated with the feedback data, wherein receiving the feedback data is based on a delay time associated with the feedback data, wherein the feedback data acquires the baked time when an age of the feedback data is equal to or greater than the associated delay time, wherein the node is part of a cluster, the cluster having a plurality of nodes,
wherein the node includes a listener, and when the listener detects the feedback data based on the delay time, the listener is configured to direct the feedback data to the node for processing while preventing another node from receiving the feedback data such that the plurality of nodes are enabled to work efficiently by reducing or eliminating number of operations, and by decreasing latency and reducing strain on system resources including storage devices and network interface, and
wherein the creative includes at least one of an email, webpage advertisement, or a webpage.

15. A non-transitory computer-readable medium storing instructions that when executed by a processor, that is configured to interact with a multi-armed bandit application programming interface (API), cause the processor to operate a computer system for configuring a computerized model for improving creatives by performing operations, the operations comprising:
receiving feedback data from a device, the feedback data having an associated distribution channel and an associated creative, wherein the feedback data comprises an indication of a degree of user interaction with the creative on the distribution channel;
checking the feedback data for a baked time associated with the creative sending the feedback data to a node based on the baked time;
dynamically updating, by the node, the multi-armed bandit API of the associated creative, wherein receiving the feedback data is based on a delay time associated with the feedback data, wherein the feedback data acquires the baked time when an age of the feedback data is equal to or greater than the associated delay time, wherein the node is part of a cluster, the cluster having a plurality of nodes,
wherein the node includes a listener, and when the listener detects the feedback data based on the delay time, t

Assignees

• Capital One Services Llc

Inventors

• Patel Gunjan
• Konduru Anil
• Kulkarni Kirankumar

Classifications

• G06Q30/0243 Primary

  Comparative campaigns · CPC title

• G06Q30/0244 Primary

  Optimization · CPC title

• G06F18/2185

  the supervisor being an automated module, e.g. intelligent oracle · CPC title

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06N3/006

  based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO] · CPC title