US2016188376A1 · US · A1
| Field | Value |
|---|---|
| Publication number | US-2016188376-A1 |
| Application number | US-201414583228-A |
| Country | US |
| Kind code | A1 |
| Filing date | Dec 26, 2014 |
| Priority date | Dec 26, 2014 |
| Publication date | Jun 30, 2016 |
| Grant date | — |
A practical reading order for non-experts. Skip the full description unless you need deep technical detail.
What the patent document calls the invention.
A short plain-language summary of the technical disclosure.
Who owns or filed the patent and who is credited as inventor.
Filing, priority, publication, and grant dates set the timeline.
The legal scope of protection — read this for what is actually claimed.
Technology tags used to group this patent with similar filings.
Prior art links and similar publications in this corpus.
Official abstract text for this publication.
The stream processing engine uses the Actor programming paradigm for defining the application in terms of a graph built with processing elements (PEs) that use a hash based partitioning of data, where events (key, value) are pushed towards the next element in the operator, and in case of an overloaded PE the method changes to a Producer/Consumer Model where new workers pull events from a buffer queue in order to release the amount of traffic in the overloaded PE. The programmer defines a sequential version of the PE and other parallel version that recovers the events from a buffer and, if the operator is stateless sends the result to the next PE, or if the operator is stateful sends the result to an aggregator PE before moving to the next stage of the pipeline process. Strategies for triggering changes in the graph are defined in an administrator module to provide the right amount of elasticity and load balance in the distributed stream processing engine using queues analysis of the monitoring module.
Opening claim text (preview).
What is claimed is: 1 . The method for elastic distributed stream processing engines, with a system maintaining load balance for stateless and stateful operators, comprising the steps: a. detecting overloaded operators (PEs); b. if operators are overloaded, then allocating operators to those overloaded PEs by moving idle operators from a pool of stateless operators for parallelizing the overloaded operators; changing the role of the overloaded operator to producers; changing the role of assigned parallel operators to consumers; and c. If operators are not overloaded and they have parallel operators already assigned, then deallocating parallel operators moving them back to the pool of stateless operators; changing the role of producer operators to regular operator. 2 . The method according to claim 1 , wherein the system has the capability of self-monitoring the workload of the operators, their performance and make the appropriate change(s) to improve their response time and level of utilization; which further includes the following steps: d. collecting statistics like the length of the queue of PEs, service time and level of utilization; e. if the length of the queue is greater than a threshold T and the operator works on warning mode; then calculating the number of stateless PEs to be assigned as consumers of the overloaded PE (which becomes the producer); modifying the keys to re-route the events from the producer to the consumers PEs; f. if the length of the queue is lower than a threshold T and the operator works on warning mode; then de-allocating PEs with low workload and changing the work mode of the PE from warning to normal; and g. if the length of the queue is greater than a threshold T and the operator works on normal mode; then changing the work mode to warning.
Techniques for rebalancing the load in a distributed system · CPC title
Data stream processing; Continuous queries · CPC title
Related publications grouped by family.
Answers are generated from the same data shown on this page.