Systems and methods for event stream processing

Patent metadata

Field	Value
Publication number	US-9304809-B2
Application number	US-201213553652-A
Country	US
Kind code	B2
Filing date	Jul 19, 2012
Priority date	Jun 26, 2012
Publication date	Apr 5, 2016
Grant date	Apr 5, 2016

Abstract

Disclosed are systems and methods for processing events in an event stream using a map-update application. The events may be embodied as a key-attribute pair. An event is processed by one or more instances implementing either a map or an update function. A map function receives an input event from the event stream and publishes one or more events to the event stream. An update function receives an event and updates a corresponding slate and publishes zero or more events. Systems and methods are also disclosed herein for implementing a map-update application in a multithreaded architecture and for handling overloading of a particular thread or node. Systems and methods for providing access to slates updated according to update operations are also disclosed.

First claim

What is claimed is:

1. A method for performing event processing comprising:
processing an event stream by an application executed by a computer system, the application comprising a plurality of event handlers, each event handler of the plurality of event handlers comprising:
receiving an input stream from the event stream;
performing an event handling function; and
outputting to an output stream of the event stream;

detecting an overload condition for an event handler of the plurality of event handlers, the overload condition comprising one or more excess events of the input stream in excess of a capacity of the event handler of the plurality of event handlers; and
in response to the detecting of the overload condition:
processing the one or more excess events according to a degraded handling function, the degraded handling function comprising: protecting the one or more excess events from discard; and
a lower latency than the event handling function;

publishing events according to the processing of the one or more excess events according to the degraded handling function to the output stream of the event stream of the event handler of the plurality of event handlers;

detecting a degraded handling function overload condition for the degraded handling function, the degraded handling function overload condition comprising one or more degraded handling function excess events in excess of a capacity of the degraded handling function; and
in response to the detecting of the degraded handling function overload condition:
processing the one or more degraded handling function excess events according to a second degraded handling function, the second degraded handling function comprising protecting the one or more degraded handling function excess events from discard.

2. The method of claim 1 , wherein the event handling function requires a greater number of computations than the degraded handling function.

3. The method of claim 1 , wherein the event handling function accesses an external resource and the degraded handling function does not require access to the external resource.

4. The method of claim 1 , wherein the event handling function makes a network function call and the degraded handling function does not make a network function call.

5. The method of claim 1 , wherein the plurality of event handlers includes mappers and updaters, the mappers having an event handling function operable to perform a stateless computation and the updaters having an event handling function operable to perform a state dependent computation.

6. The method of claim 1 , wherein the event stream includes a plurality of events each including an attribute and value pair.

7. A system for performing event processing comprising one or more processors and one or more memory devices operably coupled to the one or more processors, the one or memory devices storing executable and operational code effective to cause the one or more processors to:
process an event stream by an application, the application comprising a plurality of event handlers, each event handler of the plurality of event handlers comprising:
receiving an input stream from the event stream;
performing an event handling function; and
outputting to an output stream of the event stream;

detect an overload condition for an event handler of the plurality of event handlers, the overload condition comprising one or more excess events of the input stream in excess of a capacity of the event handler of the plurality of event handlers; and
in response to the detecting of the overload condition:
process the one or more excess events according to a degraded handling function, the degraded handling function comprising:
protecting the one or more excess events from discard; and
a lower latency than the event handling function;

publish events according to the processing the one or more excess events according to the degraded handling function to the output stream of the event stream of the event handler of the plurality of event handlers a lower latency than the event handling function;
detect a degraded handling function overload condition for the degraded handling function, the degraded handling function overload condition comprising one or more degraded handling function excess events in excess of a capacity of the degraded handling function; and
in response to the detecting the degraded handling function overload condition:
process the one or more degraded handling function excess events according to a second degraded handling function, the second degraded handling function comprising protecting the one or more degraded handling function excess events from discard.

8. The system of claim 7 , wherein the event handling function is effective to cause the one or more processors to perform a greater number of computations than the degraded handling function.

9. The system of claim 7 , wherein the event handling function is effective to cause the one or more processors to access an external resource and the degraded handling function is not operable to cause the one or more processors to access the external resource.

10. The system of claim 7 , wherein the event handling function is effective to cause the one or more processors to make a network function call and the degraded handling function does not make a network function call.

11. The system of claim 7 , wherein the plurality of event handlers includes mappers and updaters, the mappers having an event handling function effective to cause the one or more processors to perform a stateless computation and the updaters having an event handling function effective to cause the one or more processors to perform a state dependent computation.

12. The system of claim 7 , wherein the event stream includes a plurality of events each including an attribute and value pair.

13. A computer program product for managing a transaction, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:
processing an event stream by an application executed by a computer system, the application comprising a plurality of event handlers, each event handler of the plurality of event handlers comprising:
receiving an input stream from the event stream;
performing an event handling function; and
outputting to an output stream;

detecting an overload condition for an event handler of the plurality of event handlers, the overload condition comprising one or more excess events of the input stream in excess of a capacity of the event handler of the plurality of event handlers; and
in response to the detecting the overload condition:
processing at least a portion of the one or more excess events according to a degraded handling function, the degraded handling function comprising:
protecting the one or more excess events from discard; and
a lower latency than the event handling function;

publishing events according to the processing the one or more excess events according to the degraded handling function to the output stream of the event stream of the event handler of the plurality of event handlers;
detecting a degraded handling function overload condition for the degraded handling function, the degraded handling function overload condition comprising one or more degraded handling function excess events in excess of a capacity of the degraded handling function; and
in response to the detecting the degraded handling function overload condition:
processing the one or more degraded handling function excess events according to a second degraded handling function, the second degraded handling

Assignees

• Lam Wang Chee
• Liu Lu
• Siripurapu Taraka Subrahmanya Prasad
• Rajaraman Anand
• Vacheri Zoheb
• Doan Anhai
• Wal Mart Stores Inc

Inventors

• Lam Wang Chee
• Liu Lu
• Siripurapu Taraka Subrahmanya Prasad
• Rajaraman Anand
• Vacheri Zoheb
• Doan Anhai

Classifications

• G06F9/542 Primary

  Event management; Broadcasting; Multicasting; Notifications · CPC title

• G06F9/4881

  Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues · CPC title

• G06F9/4843 Primary

  by program, e.g. task dispatcher, supervisor, operating system · CPC title