Product evaluation using transaction details from a production system

Patent metadata

Field	Value
Publication number	US-11573875-B2
Application number	US-201916457700-A
Country	US
Kind code	B2
Filing date	Jun 28, 2019
Priority date	Jun 28, 2019
Publication date	Feb 7, 2023
Grant date	Feb 7, 2023

Abstract

Techniques are disclosed relating to accessing, by an evaluation computer system, transaction details from a subset of current transactions being processed by a production version of a transaction processing service that is implemented on a production computer system. The evaluation computer system may perform, in real-time, tests on a particular product using the transaction details. The evaluation computer system may then store output from the tests that are performed using the transaction details.

First claim

What is claimed is:

1. A method, comprising:
accessing, by an evaluation computer system, transaction details from a set of current transactions being processed by a production version of a transaction processing service that is implemented on a production computer system;
using, by the evaluation computer system, the accessed transaction details to identify transactions of particular types;
selecting, by the evaluation computer system based on a particular product to be evaluated, a subset of the identified transactions;
performing, by the evaluation computer system in parallel with the processing of the set of current transactions by the production computer system, tests on the particular product using the transaction details of the subset; and
storing, by the evaluation computer system, output from the tests performed using the transaction details.

2. The method of claim 1 , wherein the particular product is software that is an evaluation version of the transaction processing service that is an update to the production version of the transaction processing service.

3. The method of claim 2 , wherein performing the tests on the evaluation version of the transaction processing service includes evaluating a particular transaction of the subset of identified transactions to determine whether the particular transaction is valid; and
wherein the method further comprises the evaluation computer system comparing a result for the particular transaction produced by the evaluation computer system to a result for the particular transaction produced by the production computer system.

4. The method of claim 2 , further comprising:
tracking processing times for the subset of current identified transactions processed by the production version of the transaction processing service; and
comparing the tracked processing times to processing times for the evaluation version of the transaction processing service during the performed tests.

5. The method of claim 1 , wherein the particular product is hardware that is an updated version of the production computer system.

6. The method of claim 1 , wherein processing the subset of identified transactions includes exchanging, by the production computer system, messages that include the transaction details, and wherein accessing the transaction details includes:
monitoring, by the evaluation computer system, the messages exchanged on the production computer system;
copying, by the evaluation computer system, the monitored messages to the evaluation computer system; and
extracting, by the evaluation computer system, the transaction details from the copied messages.

7. The method of claim 1 , wherein performing the tests on the particular product includes the evaluation computer system using, on average, ninety-five percent of the accessed transaction details within one second from receipt from the production computer system.

8. A non-transitory, computer-readable medium storing instructions that, when executed by an evaluation computer system, cause the evaluation computer system to perform operations comprising:
identifying a set of a plurality of current transactions being processed by a production version of a transaction processing service implemented on a production computer system;
determining transaction types for ones of the set of identified transactions;
based on a particular product to be evaluated, using the determined transaction types to select a subset of the identified transactions;
accessing transaction details from the subset of the identified transactions;
using, in real-time with the processing of the current transactions, the transaction details to generate test input data for evaluation tests performed on the particular product; and
monitoring results from the evaluation tests that are performed using the transaction details.

9. The computer-readable medium of claim 8 , wherein the particular product is a software product that is executing on the evaluation computer system, and wherein the generated test input data is an input to the evaluation tests.

10. The computer-readable medium of claim 9 , wherein monitoring the results from the evaluation tests includes:
tracking a production result for a particular transaction of the subset of the identified transactions that is being processed on the production computer system; and
comparing an evaluation result for the particular transaction produced by the software product to the production result for the particular transaction.

11. The computer-readable medium of claim 10 , wherein comparing the evaluation result to the production result includes comparing a processing time for the production result to a processing time for the evaluation result.

12. The computer-readable medium of claim 8 , wherein performing the evaluation tests on the particular product includes generating the test input data for a hardware product that is coupled to communicate with the evaluation computer system.

13. The computer-readable medium of claim 12 , wherein the operations further comprise:
tracking an amount of time from when particular test input data that is based on a particular transaction of the subset of the identified transactions is sent to the hardware product to when an evaluation result based on the particular test input data is received from the hardware product; and
comparing the amount of time to an elapsed time for completing the particular transaction on the production computer system.

14. The computer-readable medium of claim 8 , wherein accessing the transaction details from the subset of the identified transactions includes snooping messages exchanged by the production computer system.

15. A system comprising:
a production computer system configured to:
process a plurality of current transactions between two or more entities that use a transaction processing service that is implemented on the production computer system; and
access, while processing the plurality of current transactions, transaction details from a set of the plurality of current transactions; and

an evaluation computer system configured to:
receive the transaction details from the set of the plurality of current transactions;
use the received transaction details to identify transactions of a particular type;
select, based on a particular product to be evaluated, a subset of the identified transactions;
use, while the production computer system is processing the plurality of current transactions, the transaction details of the subset, perform tests on the particular product under evaluation by the evaluation computer system; and
monitor output from the tests performed using the transaction details.

16. The system of claim 15 , wherein the evaluation computer system is configured to use accessed transaction details within three seconds of receipt from the production computer system in ninety-eight percent of cases.

17. The system of claim 15 , wherein to perform the tests on the particular product, the evaluation computer system is configured to:
execute an evaluation version of a software product corresponding to the particular product; and
apply, to the evaluation version of the software product, test input data that is based on the transaction details of the subset of the identified transactions.

18. The system of claim 17 , wherein the production computer system is further configured to produce, using the transaction processing service, an approval decision for a particular transaction of the subset of the plurality of current transactions; and
wherei

Assignees

• Paypal Inc

Inventors

• Wu Yuehao
• Yun Xiaohan
• Suciu Adrian Oliver
• Li Yunfeng
• Zuo Quin

Classifications

• G06F11/3698

  Environments for analysis, debugging or testing of software · CPC title

• G06F2201/87

  Monitoring of transactions · CPC title

• G06F11/3457 Primary

  Performance evaluation by simulation · CPC title

• G06Q20/10

  specially adapted for electronic funds transfer [EFT] systems; specially adapted for home banking systems · CPC title

• G06F11/3692

  for test results analysis · CPC title