Configurable batch processing using a decoupled rule database

Patent metadata

Field	Value
Publication number	US-12210515-B1
Application number	US-202418428869-A
Country	US
Kind code	B1
Filing date	Jan 31, 2024
Priority date	Jan 31, 2024
Publication date	Jan 28, 2025
Grant date	Jan 28, 2025

Abstract

This disclosure describes techniques for performing batch processing using a custom batch processing routine (e.g., a custom filtering routine) that calls a native batch processing routine (e.g., a native filtering routine) associated with a batch processing system as well as a rule database for one or more additional filtering rules. In some cases, a custom filtering routine may be configured to: (i) receive a batch of data, (ii) call a native filtering routine to process the received batch data, and (iii) apply one or more additional filtering operations on the output of the native filtering routine. For example, the custom filtering routine may be configured to: (i) make a query to the rule database to extract one or more filtering rules, and (ii) apply the extracted filtering rules to the output of the native filtering routine to perform the additional filtering operations.

First claim

What is claimed is:

1. A method comprising:
at a first time, receiving, by a processor, a first request to update configuration data defining a batch processing workflow of a batch processing system, wherein the configuration data requires processing a first output of a first filtering routine using a first processing routine;
based on receiving the first request, updating, by the processor, the configuration data to override the first filtering routine with a second filtering routine that is configured to:
intercept a first input of the first filtering routine,
provide the first input to the first filtering routine,
receive, from the first filtering routine, the first output of the first filtering routine;
receive, from a first database that is external to the batch processing system, a filtering rule, and
determine a second output of the second filtering routine based on a applying the filtering rule to the first output;

at a second time, receiving, by the processor, a second request to execute the batch processing workflow;
based on receiving the second request, providing, by the processor, a first dataset stored on a second database as the first input to the second filtering routine;
receiving, by the processor and from the second filtering routine, a second dataset stored on the second database as the second output;
executing, by the processor, the first processing routine on the second dataset to determine a batch processing result data of the first processing routine; and
updating, by the processor, the second database based on the batch processing result data.

2. The method of claim 1 , wherein:
the configuration data comprises a first interface data structure and a first class data structure implementing the first interface data structure,
the first interface data structure comprises a first routine signature and a second routine signature,
the first class data structure implements the first routine signature using the first filtering routine, and implements the second routine signature using the first processing routine,
overriding the first filtering routine comprises implementing the first interface data structure using a second class data structure, and
the second class data structure implements the first routine signature using the second filtering routine and the second routine signature using the first processing routine.

3. The method of claim 2 , wherein:
the configuration data comprises a first interface data structure and a first class data structure implementing the first interface data structure,
the first interface data structure comprises a first routine signature and a second routine signature,
the first class data structure implements the first routine signature using the first filtering routine and the second routine signature using the first processing routine, and
overriding the first filtering routine comprises defining a second class data structure that inherits from the first class data structure and overrides the first filtering routine with the second filtering routine.

4. The method of claim 1 , further comprising:
based on receiving the first request, updating the configuration data to override the first processing routine with a second processing routine that is configured to:
intercept a second input of the first processing routine,
provide the second input to the first processing routine,
receive, from the first processing routine, a third output of the first processing routine,
receive, from a third database that is external to the batch processing system, a processing rule, and
determine a fourth output of the second processing routine based on applying the processing rule to the third output.

5. The method of claim 4 , wherein executing the first processing routine comprises:
providing the second dataset as the second input to the second processing routine; and
receiving, from the second processing routine, the batch processing result data as the fourth output of the second processing routine.

6. The method of claim 1 , further comprising:
at the second time, determining, by the processor, that a first data record of the first dataset fails to satisfy the filtering rule and storing a first indication of the first data record; and
at a third time,
receiving, by the processor from the first database, a second indication that the filtering rule has been removed from the first database, and
determining, based on the first indication and the second indication, to execute the second filtering routine on the first data record.

7. The method of claim 6 , wherein the third time is triggered by receiving a third indication of an update to the first database.

8. The method of claim 6 , wherein the third time is triggered periodically.

9. The method of claim 6 , further comprising:
executing, by the processor, the first processing routine based on the first data record to determine a second data record, and
updating, by the processor, the second database based on the second data record.

10. The method of claim 6 , wherein the first indication is stored in a log database that is internal to the batch processing system.

11. A computing system, comprising:
a processor; and
memory storing computer-executable instructions that, when executed by the processor, cause the computing system to perform operations comprising:
at a first time, receiving a first request to update configuration data defining a batch processing workflow of a batch processing system, wherein the configuration data requires processing a first output of a first filtering routine using a first processing routine;
based on receiving the first request, updating the configuration data to override the first filtering routine with a second filtering routine that is configured to:
intercept a first input of the first filtering routine,
provide the first input to the first filtering routine,
receive, from the first filtering routine, the first output of the first filtering routine;

receive, from a first database that is external to the batch processing system, a filtering rule, and determine a second output of the second filtering routine based on a applying the filtering rule to the first output;
at a second time, receiving a second request to execute the batch processing workflow;
based on receiving the second request, providing a first dataset stored on a second database as the first input to the second filtering routine;
receiving, from the second filtering routine, a second dataset stored on the second database as the second output;
executing the first processing routine on the second dataset to determine a batch processing result data of the first processing routine; and

updating the second database based on the batch processing result data.

12. The computing system of claim 11 , wherein:
the configuration data comprises a first interface data structure and a first class data structure implementing the first interface data structure,
the first interface data structure comprises a first routine signature and a second routine signature,
the first class data structure implements the first routine signature using the first filtering routine, and implements the second routine signature using the first processing routine,
overriding the first filtering routine comprises implementing the first interface data structure using a second class data structure, and
the second class data structure implements the first routine signature using the second filtering routine and the second routine signature using the first processing routine.

13. The computing system of claim 12 , wherein:
th

Assignees

• State Farm Mutual Automobile Insurance Co

Inventors

• Tucker Bruce
• Shepherd Nate
• Buchberger Tony
• Carlson Jake
• Mclaurin Hillary

Classifications

• G06F16/2386 Primary

  Bulk updating operations (data conversion details G06F16/258) · CPC title

• G06F16/2358

  Change logging, detection, and notification (replication G06F16/27) · CPC title

• G06F16/24564 Primary

  Applying rules; Deductive queries · CPC title