Database anomaly detection

Patent metadata

Field	Value
Publication number	US-11995065-B2
Application number	US-202117566014-A
Country	US
Kind code	B2
Filing date	Dec 30, 2021
Priority date	Dec 30, 2021
Publication date	May 28, 2024
Grant date	May 28, 2024

Abstract

In one aspect, a method of detecting database anomalies, includes reading historical data in a destination database at an end of a data pipeline, determining bounds including an upper bound and a lower bound based on the read historical data, reading current data for a first specified time period in the destination database, responsive to determining the upper or the lower bound is exceeded, determining database transactions that caused the exceeding, and transmitting alerts to owners of the database transactions.

First claim

What is claimed is:

1. A method of detecting database anomalies, comprising:
reading, by one or more hardware processors, historical data in a destination database at an end of a data pipeline, wherein the destination database is a general ledger, and the data pipeline includes data that is aggregated from one or more subledgers for posting to the general ledger;
determining, by the one or more hardware processors, bounds including an upper bound and a lower bound based on the read historical data;
reading, by the one or more hardware processors, current data for a first specified time period in the destination database;
responsive to determining the upper or the lower bound is exceeded:
pulling, by the one or more hardware processors, raw data in the data pipeline and disaggregating the raw data to identify whether any database transaction is an anomaly, wherein an anomaly is detected when corrupted data is transmitted through the data pipeline and stored within the destination database;
transmitting, by the one or more hardware processors, an alert to at least one upstream database within the data pipeline in relation to the destination database where the upstream database is associated with a generation of at least one database transaction that has been identified as the anomaly; and
responsive to determining the upper or the lower bound is not exceeded by numerical values in the current data:
updating, by the one or more hardware processors, the upper bound or the lower bound based on the current data.

2. The method of claim 1 , wherein the determining bounds comprises:
reading historical data over a second prespecified time period;
determining an upper and a lower mean; and
setting the upper and lower bounds based on deviations of the upper and lower means, respectively.

3. The method of claim 1 , wherein the data pipeline includes data generated by business platforms that processed, transformed, aggregated and posted to the general ledger.

4. The method of claim 1 , wherein the first specified time period is a day.

5. The method of claim 1 , wherein the data pipeline includes a data source, a data warehouse and the destination database.

6. The method of claim 5 , wherein the data source includes access servers.

7. The method of claim 6 , wherein the access servers enable users to access networks.

8. The method of claim 1 , further comprising transmitting, by the one or more hardware processors, a second alert to an owner of a subledger in which the database transaction identified as an anomaly is posted.

9. A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a computer, cause the computer to perform operations comprising:
reading historical data in a destination database at an end of a data pipeline, wherein the destination database is a general ledger, and the data pipeline includes data that is from one or more subledgers for posting to the general ledger;
determining bounds including an upper bound and a lower bound based on the read historical data;
reading current data for a first specified time period in the destination database;
responsive to determining the upper or the lower bound is exceeded:
pulling raw data in the data pipeline and disaggregating the raw data to identify whether any database transaction is an anomaly, wherein an anomaly is detected when corrupted data is transmitted through the data pipeline and stored within the destination database;
transmitting an alert to at least one upstream database within the data pipeline in relation to the destination database where the upstream database is associated with a generation of at least one database transaction that has been identified as the anomaly, and

responsive to determining the upper or the lower bound is not exceeded by numerical values within the current data:
updating, by the one or more hardware processors, the upper or the lower bound based on the current data.

10. A computing apparatus comprising:
at least one processor; and
at least one memory storing instructions that, when executed by the at least one processor, configure the computing apparatus to perform operations comprising:
reading historical data in a destination database at an end of a data pipeline, wherein the destination database is a general ledger, and the data pipeline includes data that is from one or more subledgers for posting to the general ledger;
determining bounds including an upper bound and a lower bound based on the read historical data;
reading current data for a first specified time period in the destination database;
responsive to determining the upper or the lower bound is exceeded:
pulling raw data in the data pipeline and disaggregating the raw data to identify whether any database transaction is an anomaly, wherein an anomaly is detected when corrupted data is transmitted through the data pipeline and stored within the destination database;
transmitting an alert to at least one upstream database within the data pipeline in relation to the destination database where the upstream database is associated with a generation of at least one database transaction that has been identified as the anomaly, and

responsive to determining the upper or the lower bound is not exceeded by numerical values within the current data:
updating, by the one or more hardware processors, the upper or the lower bound based on the current data.

11. The computing apparatus of claim 10 , wherein the determining bounds comprises read historical data over a second prespecified time period, determining an upper and a lower mean, and setting the upper and lower bounds based on deviations of the upper and lower means, respectively.

12. The computing apparatus of claim 10 , wherein the data pipeline includes data generated by business platforms that processed, transformed, aggregated and posted to the general ledger.

13. The computing apparatus of claim 10 , wherein the first specified time period is a day.

14. The computing apparatus of claim 10 , wherein the data pipeline includes a data source, a data warehouse and the destination database.

15. The computing apparatus of claim 14 , wherein the data source includes access servers.

16. The computing apparatus of claim 15 , wherein the access servers enable users to access a second database.

17. The computing apparatus of claim 15 , wherein the access servers enable users to access to a network.

18. The computing apparatus of claim 10 , wherein the operations further comprise transmitting a second alert to an owner of a subledger in which the database transaction identified as an anomaly is posted.

Assignees

• Stripe Inc

Inventors

• Li Xin
• Xiang Hanhan
• Zhang Yue
• Song Guanglei
• Lin Mu
• Reiter Jeffrey
• Mah Wei Keong
• Chen Jay

Classifications

• G06F16/2365 Primary

  Ensuring data consistency and integrity · CPC title

• G06F16/2379

  Updates performed during online database operations; commit processing · CPC title

• G06F11/0754

  by exceeding limits · CPC title

• H04L63/1425

  Traffic logging, e.g. anomaly detection · CPC title

• G06Q40/12

  Accounting · CPC title