Low-cost, realistic, performance test bed for an electronic trading venue

Patent metadata

Field	Value
Publication number	US-10938705-B2
Application number	US-201815897226-A
Country	US
Kind code	B2
Filing date	Feb 15, 2018
Priority date	Feb 16, 2017
Publication date	Mar 2, 2021
Grant date	Mar 2, 2021

Abstract

A system and method is disclosed for realistic performance testing of a computer network (e.g., a computer network of an electronic trading venue) based on a customizable test scenario, designed by users, executed using a plurality of independent computer nodes each connected to the computer network via its own physical connection to a network device (e.g., network switch) of the computer network. Using the system, developers may design a test scenario that includes testing parameters, which include computer executable code for transmission to and execution at the plurality of computer nodes. The computer executable code may simulate network traffic between devices of market participants and the electronic trading venue.

First claim

What is claimed is:

1. A system of performing network testing of temporal fairness in an electronic trading venue using a plurality of physical computing nodes each physically distinct from one another and connected to a network device, and each programmed to send corresponding requests to the electronic trading venue via the network device to simulate activities of a market participant, comprising:
a computer system comprising one or more physical processors programmed with computer program instructions that, when executed by the one or more physical processors, programs the computer system to:
receive an identification of a test scenario to execute, the test scenario simulating one or more activities of a plurality of market participants on the electronic trading venue;
obtain, based on the identification of the test scenario, first computer executable code that programs a target physical computing node to simulate a first market participant that transmits or receives data packets to or from the electronic trading venue via the network device, wherein the first computer executable code is stored in association with the test scenario;
obtain, based on the identification of the test scenario, second computer executable code that programs a target physical computing node to simulate a second market participant different from the first market participant that transmits or receives second data packets to or from the electronic trading venue via the network device, wherein the second computer executable code is stored in association with the test scenario;
identify at least a first set of physical computing nodes among the plurality of physical computing nodes to receive the first computer executable code, each of the first set of physical computing nodes being physically distinct from other ones of the plurality of physical computing nodes, and each connected to the network device via respective physical connections;
identify at least a second set of physical computing nodes among the plurality of physical computing nodes to receive the second computer executable code, each of the second set of physical computing nodes being physically distinct from other ones of the plurality of physical computing nodes, and each connected to the network device via respective physical connections;
transmit the first computer executable code to each of the identified first set of physical computing nodes;
transmit the second computer executable code to each of the identified second set of physical computing nodes;
monitor a first parameter of first data packets transmitted to or received from the first set of physical computing nodes
monitor a second parameter of second data packets transmitted to or received from the second set of physical computing nodes; and
determine performance of the electronic trading venue during execution of the test scenario by comparing the first parameter and second parameter.

2. The system of claim 1 , wherein:
the first parameter is a time of receipt of the first data packets measured at the network device;
the second parameter is a time of receipt of the second data packets measured at the network device; and
to determine the performance of the electronic trading venue, the computer system is programmed to determine a level of temporal fairness of the electronic trading venue as a scalar value based on comparing the first and second parameters.

3. The system of claim 1 , wherein:
the first parameter is a time of transmission of the first data packets measured at each corresponding node;
the second parameter is a time of transmission of the second data packets measured at each corresponding node; and
to determine the performance of the electronic trading venue, the computer system is programmed to determine a level of temporal fairness of the electronic trading venue as a scalar value based on comparing the first and second parameters.

4. The system of claim 1 , further comprising the network device, wherein the network device is configured to:
inject a network-level timestamp to each of the first data packets transmitted to or received from at least each of the first set of physical computing nodes;
transmit each of the first data packets injected with the network-level timestamp to its intended recipient; and
transmit a copy of each of the first data packets injected with the network-level timestamp to a test scenario storage for later retrieval and analysis.

5. The system of claim 4 , wherein the first data packets injected with the network-level timestamp are transmitted to the intended recipient and test scenario storage using port mirroring.

6. The system of claim 1 , wherein the first computer executable code, when executed, programs each of the first set of physical computing nodes to:
simulate a first trading strategy with respect to a first financial instrument trading on the electronic trading venue.

7. The system of claim 6 , wherein the second computer executable code, when executed, programs each of the second set of physical computing nodes to:
simulate a second trading strategy with respect to the first financial instrument trading on the electronic trading venue to simulate competitive activity.

8. The system of claim 1 , wherein the first computer executable code, when executed, programs each of the first set of physical computing nodes to:
implement a custom version of a communication protocol that deviates from a standard specification for the communication protocol.

9. The system of claim 8 , wherein the communication protocol comprises a TCP/IP protocol, and wherein the custom version deviates from the standard specification based on at least one of: omission of one or more required fields, transmission of packets out-of-order, transmission of bad-checksums, transmission of acks for packets not yet received, forced connection from a fixed port number, or forced transmission of packets beyond an advertised window of a receiver.

10. The system of claim 1 , wherein at least each of the first set of physical computing nodes is programmed with a test agent that programs each of the first set of physical computing nodes to receive the first computer executable code from the computer system, and execute the test scenario according to the first computer executable code.

11. The system of claim 10 , wherein the test agent programs each of the first set of physical computing nodes to generate log data that logs execution of the test scenario, and transmit the log data to a test scenario database.

12. The system of claim 1 , wherein:
the first parameter is a first plurality of time deltas that each indicate an amount of time in which the first set of physical computing nodes has temporally beaten the second set of physical computing nodes during execution of the test scenario;
the second parameter is a second plurality of time deltas that each indicate an amount of time in which the second set of physical computing nodes has temporally beaten the first set of physical computing nodes during execution of the test scenario; and
to determine the performance of the electronic trading venue, the computer system is programmed to determine a level of temporal fairness of the electronic trading venue as a scalar value based on comparing the first and second parameters.

13. The system of claim 1 , wherein the network device is further configured to:
inject a network-level timestamp to each of the first data packets transmitted to or received from at least each of the first set of physical computing nodes and the second data packets transmitted to or received from the second set of physical computing nodes;
trans

Assignees

• Thomson Reuters Global Resources Unlimited Co
• Refinitiv Us Organization Llc

Inventors

• Melton Hayden Paul

Classifications

• G06Q40/04

  Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange · CPC title

• H04L43/50 Primary

  Testing arrangements · CPC title

• H04L43/106

  using time related information in packets, e.g. by adding timestamps · CPC title

• H04L69/16

  Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP] · CPC title