System, method, and computer program product for operating dynamic shadow testing environments

Patent metadata

Field	Value
Publication number	US-11249882-B2
Application number	US-201916658840-A
Country	US
Kind code	B2
Filing date	Oct 21, 2019
Priority date	Oct 21, 2019
Publication date	Feb 15, 2022
Grant date	Feb 15, 2022

Abstract

Described are a system, method, and computer program product for operating dynamic shadow testing environments for machine-learning models. The method includes storing a testing policy including an identifier of a machine-learning model and an identifier of a transaction service. The method includes generating a shadow testing environment operating the transaction service using the machine-learning model. The method also includes receiving, at a transaction service provider system, a transaction authorization request including transaction data of a transaction associated with a payment device. The method further includes identifying the machine-learning model associated with the transaction based on a parameter of the transaction data. The method further includes determining, based on the identifier of the machine-learning model, the testing policy and the shadow testing environment. The method further includes replicating the transaction data in the shadow testing environment as input for testing the transaction service using the machine-learning model.

First claim

What is claimed is:

1. A computer-implemented method comprising:
storing, with at least one processor and in a database, at least one testing policy comprising an identifier of at least one machine-learning model and an identifier of at least one transaction service, the at least one transaction service comprising at least one computer-implemented service having at least one action triggered by one or more parameters of a payment transaction;
generating, with the at least one processor, at least one shadow testing environment operating the at least one transaction service using the at least one machine-learning model;
receiving, with the at least one processor at a transaction processing system of a transaction service provider in an electronic payment processing network, at least one transaction authorization request comprising transaction data of at least one transaction associated with at least one payment device, wherein the transaction processing system is configured to receive transaction authorization requests from at least one merchant and process payment transactions associated with said transaction authorization requests;
identifying, with the at least one processor, the at least one transaction service associated with the at least one transaction based on a parameter of the transaction data;
determining, with the at least one processor, the at least one shadow testing environment associated with the at least one transaction service; and
replicating, with the at least one processor, the transaction data in the at least one shadow testing environment as input for testing the at least one transaction service using the at least one machine-learning model,
wherein the replicating of the transaction data occurs in real-time with processing the at least one transaction at the transaction processing system.

2. The method of claim 1 , further comprising:
detecting, with the at least one processor, a modification of the at least one testing policy by a user through a computer interface;
determining, with the at least one processor, that the modification comprises at least one of a new identifier of at least one new machine-learning model and a new identifier of at least one new transaction service; and
regenerating, with the at least one processor, the at least one shadow testing environment to perform at least one of the following: operate the at least one new transaction service, and use the at least one new machine-learning model.

3. The method of claim 2 , further comprising, in response to determining that at least one of the at least one new transaction service and the at least one new machine-learning model has not yet been stored, prompting, with the at least one processor, the user through the computer interface to provide at least one of the at least one new transaction service and the at least one new machine-learning model.

4. The method of claim 1 , further comprising:
receiving, with the at least one processor at the transaction processing system, a plurality of transaction authorization requests comprising the at least one transaction authorization request; and
replicating, with the at least one processor in a respective shadow testing environment of the at least one shadow testing environment, each of the plurality of transaction authorization requests that is associated with a transaction service of the at least one testing policy.

5. The method of claim 1 , wherein generating the at least one shadow testing environment further comprises:
identifying, with the at least one processor, a set of computer resources available for operating shadow testing environments;
determining, with the at least one processor, a resource requirement of the at least one testing policy;
selecting, with the at least one processor based at least partly on the resource requirement, a subset of computer resources to operate the at least one shadow testing environment; and
initiating, with the at least one processor, the at least one shadow testing environment using the subset of computer resources.

6. The method of claim 5 , further comprising, in response to detecting a modification of the at least one testing policy:
determining, with the at least one processor and based on the modification, a new resource requirement of the at least one testing policy;
selecting, with the at least one processor based at least partly on the new resource requirement, a new subset of computer resources to operate the at least one shadow testing environment; and
initiating, with the at least one processor, the at least one shadow testing environment using the new subset of computer resources.

7. A system comprising a server comprising at least one processor, the server being at least one of programmed and configured to:
store, in a database, at least one testing policy comprising an identifier of at least one machine-learning model and an identifier of at least one transaction service, the at least one transaction service comprising at least one computer-implemented service having at least one action triggered by one or more parameters of a payment transaction;
generate at least one shadow testing environment operating the at least one transaction service using the at least one machine-learning model;
receive, at a transaction processing system of a transaction service provider in an electronic payment processing network, at least one transaction authorization request comprising transaction data of at least one transaction associated with at least one payment device, wherein the transaction processing system is configured to receive transaction authorization requests from at least one merchant and process payment transactions associated with said transaction authorization requests;
identify the at least one transaction service associated with the at least one transaction based on a parameter of the transaction data;
determine the at least one shadow testing environment associated with the at least one transaction service; and
replicate the transaction data in the at least one shadow testing environment as input for testing the at least one transaction service using the at least one machine-learning model,
wherein the replicating of the transaction data occurs in real-time with processing the at least one transaction at of the given transaction authorization request by the transaction processing system.

8. The system of claim 7 , wherein the server is further at least one of programmed and configured to:
detect a modification of the at least one testing policy by a user through a computer interface;
determine that the modification comprises at least one of a new identifier of at least one new machine-learning model and a new identifier of at least one new transaction service; and
regenerate the at least one shadow testing environment to perform at least one of the following: operate the at least one new transaction service, and use the at least one new machine-learning model.

9. The system of claim 8 , wherein the server is further at least one of programmed and configured to, in response to determining that at least one of the at least one new transaction service and the at least one new machine-learning model has not yet been stored, prompt the user through the computer interface to provide at least one of the at least one new transaction service and the at least one new machine-learning model.

10. The system of claim 7 , wherein the server is further at least one of programmed and configured to:
receive, at the transaction processing system, a plurality of transaction authorization requests comprising the at least one transaction authorization request; and
replicate, in a respective shadow testing environm

Assignees

• Visa Int Service Ass

Inventors

• Lu Ranglin
• Cheng Yinhe
• GU Yu

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06F11/3698 Primary

  Environments for analysis, debugging or testing of software · CPC title

• G06F11/3696

  Methods or tools to render software testable · CPC title

• G06Q30/06

  Buying, selling or leasing transactions · CPC title