Configurable batch processing using a decoupled rule database

Patent metadata

Field	Value
Publication number	US-2025245214-A1
Application number	US-202418982143-A
Country	US
Kind code	A1
Filing date	Dec 16, 2024
Priority date	Jan 31, 2024
Publication date	Jul 31, 2025
Grant date	—

Abstract

This disclosure describes techniques for performing batch processing using a custom batch processing routine (e.g., a custom filtering routine) that calls a native batch processing routine (e.g., a native filtering routine) associated with a batch processing system as well as a rule database for one or more additional filtering rules. In some cases, a custom filtering routine may be configured to: (i) receive a batch of data, (ii) call a native filtering routine to process the received batch data, and (iii) apply one or more additional filtering operations on the output of the native filtering routine. For example, the custom filtering routine may be configured to: (i) make a query to the rule database to extract one or more filtering rules, and (ii) apply the extracted filtering rules to the output of the native filtering routine to perform the additional filtering operations.

First claim

What is claimed is:

1 . A method comprising:
at a first time, receiving, by a processor, a first request to update configuration data defining a batch processing workflow of a batch processing system, wherein the configuration data requires processing a first output of a first filtering routine using a first processing routine; based on receiving the first request, updating, by the processor, the configuration data to override the first processing routine with a second processing routine that is configured to:
intercept a first input of the first processing routine,
provide the first input to the first processing routine,
receive, from the first processing routine, a second output of the first processing routine,
receive, from a first database that is external to the batch processing system, a processing rule, and
determine a third output of the second processing routine based on applying the processing rule to the second output;
at a second time, receiving, by the processor, a second request to execute the batch processing workflow on a dataset; and based on receiving the second request, executing, by the processor, the batch processing workflow based on the updated configuration data.

2 . The method of claim 1 , wherein:
the configuration data comprises a first interface data structure and a first class data structure implementing the first interface data structure, the first interface data structure comprises a first routine signature and a second routine signature, the first class data structure implements the first routine signature using the first filtering routine, and implements the second routine signature using the first processing routine, overriding the first filtering routine comprises implementing the first interface data structure using a second class data structure, and the second class data structure implements the first routine signature using the first filtering routine and the second routine signature using the second processing routine.

3 . The method of claim 2 , wherein:
the configuration data comprises a first interface data structure and a first class data structure implementing the first interface data structure, the first interface data structure comprises a first routine signature and a second routine signature, the first class data structure implements the first routine signature using the first filtering routine and the second routine signature using the first processing routine, and overriding the first filtering routine comprises defining a second class data structure that inherits from the first class data structure and overrides the first processing routine with the second processing routine.

4 . The method of claim 1 , further comprising:
based on receiving the first request, updating, by the processor, the configuration data to override the first filtering routine with a second filtering routine that is configured to: intercept a second input of the first filtering routine, provide the second input to the first filtering routine, receive, from the first filtering routine, the first output of the first filtering routine; receive, from a first database that is external to the batch processing system, a filtering rule, and determine a fourth output of the second filtering routine based on a applying the filtering rule to the first output.

5 . The method of claim 4 , further comprising:
at the second time, determining, by the processor, that a first data record of the dataset fails to satisfy the filtering rule and storing a first indication of the first data record; and at a third time, receiving, by the processor from the first database, a second indication that the filtering rule has been removed from the first database, and determining, based on the first indication and the second indication, to execute the second filtering routine on the first data record.

6 . The method of claim 5 , wherein the third time is triggered by receiving a third indication of an update to the first database.

7 . The method of claim 5 , wherein the third time is triggered periodically.

8 . The method of claim 5 , wherein the first indication is stored in a log database that is internal to the batch processing system.

9 . A system, comprising:
a processor; and memory storing computer-executable instructions that, when executed by the processor, cause the system to perform operations comprising: at a first time, receiving a first request to update configuration data defining a batch processing workflow of a batch processing system, wherein the configuration data requires processing a first output of a first filtering routine using a first processing routine; based on receiving the first request, updating the configuration data to override the first processing routine with a second processing routine that is configured to:
intercept a first input of the first processing routine,
provide the first input to the first processing routine,
receive, from the first processing routine, a second output of the first processing routine,
receive, from a first database that is external to the batch processing system, a processing rule, and
determine a third output of the second processing routine based on applying the processing rule to the second output;
at a second time, receiving a second request to execute the batch processing workflow on a dataset; and based on receiving the second request, executing the batch processing workflow based on the updated configuration data.

10 . The system of claim 9 , wherein:
the configuration data comprises a first interface data structure and a first class data structure implementing the first interface data structure, the first interface data structure comprises a first routine signature and a second routine signature, the first class data structure implements the first routine signature using the first filtering routine, and implements the second routine signature using the first processing routine, overriding the first filtering routine comprises implementing the first interface data structure using a second class data structure, and the second class data structure implements the first routine signature using the first filtering routine and the second routine signature using the second processing routine.

11 . The system of claim 10 , wherein:
the configuration data comprises a first interface data structure and a first class data structure implementing the first interface data structure, the first interface data structure comprises a first routine signature and a second routine signature, the first class data structure implements the first routine signature using the first filtering routine and the second routine signature using the first processing routine, and overriding the first filtering routine comprises defining a second class data structure that inherits from the first class data structure and overrides the first processing routine with the second processing routine.

12 . The system of claim 9 , further comprising:
based on receiving the first request, updating the configuration data to override the first filtering routine with a second filtering routine that is configured to: intercept a second input of the first filtering routine, provide the second input to the first filtering routine, receive, from the first filtering routine, the first output of the first filtering routine; receive, from a first database that is external to the batch processing system, a filtering rule, and determine a fourth output of the second filtering routine based on a applying the filtering rule to the first output.

13 . Th

Assignees

• State Farm Mutual Automobile Insurance Co

Inventors

• Tucker Bruce
• Shepherd Nate
• Buchberger Tony
• Carlson Jake
• Mclaurin Hillary

Classifications

• G06F16/24564 Primary

  Applying rules; Deductive queries · CPC title

• G06F16/2358

  Change logging, detection, and notification (replication G06F16/27) · CPC title

• G06F16/2386 Primary

  Bulk updating operations (data conversion details G06F16/258) · CPC title