Enhanced tracking of data flows

Patent metadata

Field	Value
Publication number	US-11544229-B1
Application number	US-202016861771-A
Country	US
Kind code	B1
Filing date	Apr 29, 2020
Priority date	Apr 29, 2020
Publication date	Jan 3, 2023
Grant date	Jan 3, 2023

Abstract

Disclosed are various embodiments for tracking the flow of data through a network environment. A monitor can detect that a data transaction event has occurred. Then, the monitor can identify data involved in the data transaction event. Next, a trace identifier can be assigned to the data involved in the data transaction event. Subsequently, a transaction data subset representing a subset of the data involved in the data transaction event that is subject to a common data processing event can be identified. Then, a span identifier can be assigned to the transaction data subset. Next, a correlation identifier can be link to a combination of the span identifier and the trace identifier. Finally, a transaction event record can be written to a distributed ledger, the transaction event record comprising the span identifier and the transaction data subset.

First claim

Therefore, the following is claimed:

1. A system, comprising:
a computing device comprising a processor and a memory; and
machine-readable instructions stored in the memory that, when executed by the processor, cause the computing device to at least:
detect that a data transaction event has occurred at a data store;
identify data involved in the data transaction event at the data store;
assign a trace identifier to the data involved in the data transaction event, wherein the trace identifier uniquely identifies a transaction event record with respect to other transaction event records;
identify a transaction data subset, the transaction data subset representing a subset of the data involved in the data transaction event that is subject to a common data processing event;
assign a span identifier to the transaction data subset, wherein the span identifier uniquely identifies the transaction data subset with respect to other transaction data subsets;
link a correlation identifier to a combination of the span identifier and the trace identifier;
search a distributed ledger for the transaction data subset to identify a previous span identifier and a previous trace identifier;
identify a correlation record based on the previous span identifier and the previous trace identifier;
write the transaction event record to the distributed ledger that is globally accessible by a user, the transaction event record comprising the trace identifier and the transaction data subset; and
write or update the correlation record at the distributed ledger, the correlation record comprising the correlation identifier, the previous span identifier, the span identifier, the previous trace identifier, and the trace identifier, wherein the correlation record is configured to maintain a history of operations performed on the transaction data subset across different data stores.

2. The system of claim 1 , wherein the machine-readable instructions that cause the computing device to detect that the data transaction event has occurred further cause the computing device to at least:
receive, through an application programming interface or a message passing interface provided by the machine-readable instructions, a notification from a data store or an application accessing the data store; and
determine from the notification that the data transaction event has occurred.

3. The system of claim 1 , wherein the machine-readable instructions that cause the computing device to detect that the data transaction event has occurred further cause the computing device to at least:
monitor a log associated with the data store; and
determine from the log that the data transaction event has occurred.

4. The system of claim 1 , wherein the machine-readable instructions further cause the computing device to at least:
generate a data identifier that identifies the subset of the data by indicating a name or location of an object or file involved in the data transaction event; and
assign the data identifier to the transaction data subset.

5. The system of claim 1 , wherein the machine-readable instructions further cause the computing device to at least:
identify metadata associated with the transaction data subset; and
link the metadata to the transaction data subset.

6. The system of claim 5 , wherein the machine-readable instructions that cause the computing device to identify the metadata associated with the transaction data subset further cause the computing device to at least:
identify a user account that initiated the data transaction event; and
include an identity of the user account in the metadata.

7. The system of claim 5 , wherein the machine-readable instructions that cause the computing device to identify the metadata associated with the transaction data subset further cause the computing device to at least:
identify a destination data store for the data in the transaction data subset; and
include an identifier of the destination data store in the metadata.

8. A computer-implemented method, comprising:
detecting that a data transaction event has occurred at a data store;
identifying data involved in the data transaction event at the data store;
assigning a trace identifier to the data involved in the data transaction event, wherein the trace identifier uniquely identifies a transaction event record with respect to other transaction event records;
identifying a transaction data subset, the transaction data subset representing a subset of the data involved in the data transaction event that is subject to a common data processing event;
assigning a span identifier to the transaction data subset, wherein the span identifier uniquely identifies the transaction data subset with respect to other transaction data subsets;
linking a correlation identifier to a combination of the span identifier and the trace identifier;
searching a distributed ledger for the transaction data subset to identify a previous span identifier and a previous trace identifier;
identifying a correlation record based on the previous span identifier and the previous trace identifier;
writing the transaction event record to the distributed ledger that is globally accessible by a user, the transaction event record comprising the trace identifier and the transaction data subset; and
writing or updating the correlation record at the distributed ledger, the correlation record comprising the correlation identifier, the previous span identifier, the span identifier, the previous trace identifier, and the trace identifier, wherein the correlation record is configured to maintain a history of operations performed on the transaction data subset across different data stores.

9. The method of claim 8 , wherein detecting that the data transaction event has occurred further comprises:
receiving, through an application programming interface or a message passing interface provided by machine-readable instructions, a notification from a data store or an application accessing the data store; and
determining from the notification that the data transaction event has occurred.

10. The method of claim 8 , wherein detecting that the data transaction event has occurred further comprises:
monitoring a log associated with the data store; and
determining from the log that the data transaction event has occurred.

11. The method of claim 8 , further comprising:
generating a data identifier that identifies the subset of the data by indicating a name or location of an object or file involved in the data transaction event; and
assigning the data identifier to the transaction data subset.

12. The method of claim 8 , further comprising:
identifying metadata associated with the transaction data subset; and
linking the metadata to the transaction data subset.

13. The method of claim 12 , wherein identifying the metadata associated with the transaction data subset further comprises:
identifying a user account that initiated the data transaction event; and
including an identity of the user account in the metadata.

14. The method of claim 12 , wherein identifying the metadata associated with the transaction data subset further comprises:
identifying a destination data store for the data in the transaction data subset; and
including an identifier of the destination data store in the metadata.

15. A non-transitory, computer-readable medium, comprising machine-readable instructions that, when executed by a processor of a computing device, cause the computing device to at least:
detect that a data transaction event has occurred at a data store;
identify data invo

Assignees

• American Express Travel Related Services Co Inc

Inventors

• Seideman Jeremy D
• Roy Amartya
• Bain Anna Korsakova
• Spahr Jaime Amanda
• Singh Paramjit
• Shrivastava Pratiti
• Nidamarthy Siva
• Sargeant Tami Sue
• Kassabov George

Classifications

• G06F11/3086

  where the reporting involves the use of self describing data formats, i.e. metadata, markup languages, human readable formats · CPC title

• H04L9/0637

  Modes of operation, e.g. cipher block chaining [CBC], electronic codebook [ECB] or Galois/counter mode [GCM] · CPC title

• G06F16/1837 Primary

  Management specially adapted to peer-to-peer storage networks (topology management mechanisms of peer-to-peer networks H04L67/1042) · CPC title

• G06F16/2358

  Change logging, detection, and notification (replication G06F16/27) · CPC title

• H04L9/50

  using hash chains, e.g. blockchains or hash trees · CPC title