Dynamic tracer message logging based on bottleneck detection

Patent metadata

Field	Value
Publication number	US-10416974-B2
Application number	US-201715727097-A
Country	US
Kind code	B2
Filing date	Oct 6, 2017
Priority date	Oct 6, 2017
Publication date	Sep 17, 2019
Grant date	Sep 17, 2019

Abstract

A monitoring system monitors processing of incoming messages by an application, and logs data related to performance of the application. The application includes a plurality of checkpoints, and the monitoring system logs data upon each message traversing the checkpoints in the application. The monitoring system is configured to dynamically modify checkpoints within the application based on latency detection of portions of the application, resulting in improved granularity/resolution of the data collected from congested portions of the application, and reducing the performance penalty of the monitoring system from portions of the application that are not congested.

First claim

The invention claimed is:

1. A computer implemented method of monitoring message processing by an application in a data transaction processing system, the method comprising:
detecting, by a processor coupled with the application, that the application has received a message of a plurality of messages for processing as each message of the plurality of messages is received by the application, the application including a plurality of checkpoints, wherein each of the plurality of checkpoints is enabled or disabled, and further wherein at least first, second and third checkpoints of the plurality of checkpoints are enabled, the third enabled checkpoint being located between the first and second enabled checkpoints in a message processing sequence;
detecting, by the processor, that a message processing latency between the first and second enabled checkpoints of the plurality of checkpoints is less than an amount of time between new messages arriving at a portion of the application defined by the first and second enabled checkpoints and further detecting that a number of messages awaiting processing by the portion of the application defined by the first and second enabled checkpoints exceeds a threshold number of messages, and based thereon, disabling, by the processor, the third enabled checkpoint; and
upon a received message of the plurality of messages traversing an enabled checkpoint of the plurality of checkpoints, storing, by the processor, in a data store, a data entry indicative of the received message, the traversed enabled checkpoint, and a time when the received message traversed the enabled checkpoint.

2. The computer implemented method of claim 1 , wherein upon a received message of the plurality of messages traversing a disabled checkpoint of the plurality of checkpoints, a data entry indicative of the traversed disabled checkpoint is not stored in the data store.

3. The computer implemented method of claim 1 , further comprising:
detecting that a message processing latency between fourth and fifth enabled checkpoints of the plurality of checkpoints satisfies another condition, the application including a sixth disabled checkpoint between the fourth and fifth enabled checkpoints in a message processing sequence; and
enabling the sixth disabled checkpoint between the fourth and fifth enabled checkpoints, such that upon a received message of the plurality of messages traversing the sixth checkpoint, a data entry indicative of the received message, the sixth checkpoint, and a time when the received message traversed the sixth checkpoint is stored in the data store.

4. The computer implemented method of claim 3 , wherein the other condition comprises one of: (i) the message processing latency between the fourth and fifth enabled checkpoints exceeds a latency threshold associated with the fourth and fifth enabled checkpoints; (ii) the message processing latency between the fourth and fifth enabled checkpoints exceeds a message processing latency between a different pair of enabled checkpoints of the plurality of checkpoints; (iii) the message processing latency between the fourth and fifth enabled checkpoints exceeds an amount of time between new messages arriving at a portion of the application defined by the fourth and fifth enabled checkpoints; or (iv) a number of messages awaiting processing by a portion of the application defined by the first and second enabled checkpoints exceeds a threshold number of messages.

5. The computer implemented method of claim 1 , wherein disabling the third checkpoint reduces an amount of data stored in the data store.

6. The computer implemented method of claim 1 , wherein disabling the third checkpoint reduces an amount of storage space necessary to store the data store.

7. The computer implemented method of claim 1 , the method further comprising, before the third checkpoint is disabled, upon a received message of the plurality of messages traversing the third checkpoint, storing, in the data store, a data entry indicative of the received message, the third checkpoint, and a time when the received message traversed the third checkpoint.

8. The computer implemented method of claim 1 , wherein detecting whether a message processing latency between two enabled checkpoints of the plurality of checkpoints is less than an amount of time between new messages arriving at a portion of the application defined by the first and second enabled checkpoints comprises determining an elapse of time between times when a received message traversed the two enabled checkpoints.

9. The computer implemented method of claim 1 , wherein the message processing latency between two enabled checkpoints is determined based on data entries stored in the data store and indicative of times when a received message traversed the two enabled checkpoints.

10. The computer implemented method of claim 1 , wherein each message is an electronic data transaction request message, and wherein the application processes the electronic data transaction request message by determining whether an attempt to match the electronic data transaction request message with at least one previously received but unsatisfied electronic data transaction request message for a transaction which is counter thereto results in at least partial satisfaction of one or both of the electronic data transaction request message and the at least one previously received but unsatisfied electronic data transaction request message.

11. The computer implemented method of claim 1 , further comprising, enabling the third checkpoint when the message processing latency between the first and second enabled checkpoints of the plurality of checkpoints ceases to be less than an amount of time between new messages arriving at a portion of the application defined by the first and second enabled checkpoints or a number of messages awaiting processing by the portion of the application defined by the first and second enabled checkpoints is less than a threshold number of messages.

12. A non-transitory computer-readable medium storing instructions for monitoring message processing by an application in a data transaction processing system that, when executed by a processor, cause the processor to:
detect that the application has received a message of a plurality of messages for processing as each message of the plurality of messages is received by the application, the application including a plurality of checkpoints, wherein each of the plurality of checkpoints is enabled or disabled, and further wherein at least first, second and third checkpoints of the plurality of checkpoints are enabled, the third enabled checkpoint being located between the first and second enabled checkpoints in a message processing sequence;
detect that a message processing latency between the first and second enabled checkpoints of the plurality of checkpoints is less than an amount of time between new messages arriving at a portion of the application defined by the first and second enabled checkpoints and further detect that a number of messages awaiting processing by the portion of the application defined by the first and second enabled checkpoints exceeds a threshold number of messages, and based thereon, disable the third enabled checkpoint; and
upon a received message of the plurality of messages traversing an enabled checkpoint of the plurality of checkpoints, store, in a data store, a data entry indicative of the received message, the traversed enabled checkpoint, and a time when the received message traversed the enabled checkpoint.

13. The computer system of claim 12 , wherein the instructions are further configured to cause the processor t

Assignees

• Chicago Mercantile Exchange Inc

Inventors

• Kavanagh Kyle Dennis
• Acuña-Rohter José Antonio
• Wong David Michael

Classifications

• G06F2201/81

  Threshold · CPC title

• G06F11/3409

  for performance assessment · CPC title

• G06F11/3476

  Data logging (G06F11/14, G06F11/2205 take precedence) · CPC title

• G06F8/443 Primary

  Optimisation · CPC title

• G06F11/3466 Primary

  Performance evaluation by tracing or monitoring · CPC title