Managing processing branches in an operator graph

Patent metadata

Field	Value
Publication number	US-9313110-B2
Application number	US-201414160624-A
Country	US
Kind code	B2
Filing date	Jan 22, 2014
Priority date	Jan 22, 2014
Publication date	Apr 12, 2016
Grant date	Apr 12, 2016

Abstract

Embodiments of the disclosure provide a method and system for processing data such as a stream of tuples. One embodiment is directed toward a method for managing a stream of tuples. The method can include receiving the stream of tuples to be processed by a plurality of stream operators operating on one or more computer processors. The method can include routing the stream of tuples to a plurality of output ports, the output ports each communicate with a processing branch. The method can include receiving performance data of the stream of tuples from a measuring stream operator that measures the processing branch. The method can include determining weights to assign to each output port based on the performance data. The method can also include modifying the routing of the stream of tuples through the plurality of output ports as a function of the weight of each output port.

First claim

What is claimed is:

1. A system for processing a stream of tuples, each tuple having one or more attributes, comprising:
one or more processing elements operating on a compute node having one or more computer processors, the one or more processing elements configured to processes the attributes from the stream of tuples as a single instance, each processing element having one or more stream operators;
a split operator from the one or more processing elements configured to route the stream of tuples, using a routing method, through a plurality of processing branches, each processing branch having one or more stream operators,
wherein a first processing branch of the plurality of processing branches is configured to perform a particular type of processing and produce a different processing result from a second processing branch of the plurality of processing branches, a stream operator configured to perform processing independent from another stream operator;

a measuring stream operator that is configured to:
receive the stream of tuples from one or more processing branches from the plurality of processing branches;
measure performance data of each tuple from the stream of tuples;
determine a weight to assign to each processing branch based off of the performance data;
determine the routing method for the split operator based off of the weight; and
transmit the routing method to the split operator.

2. The system of claim 1 , wherein determine the weight comprises:
determine whether the measured performance data is beyond a limit; and
determine the weight in response to the measured performance data being beyond the limit.

3. The system of claim 1 , wherein the performance data includes a processing time.

4. The system of claim 1 , wherein determine the routing method includes:
receive the weight of each processing branch;
determine the processing branch having a highest weight relative to the plurality of processing branches and the processing branch having a lowest weight relative to the plurality of processing branches; and
add a greater number of tuples to be processed by the processing branch with the highest weight than the number of tuples to be processed by the processing branch with the lowest weight.

5. The system of claim 1 , wherein the measuring stream operator occurs after a substantial bottleneck in the first processing branch.

6. The system of claim 2 , wherein the substantial bottleneck is defined by a rate of tuples.

7. The system of claim 1 , wherein the split operator is further configured to determine the routing method for the split operator by initiating a round-robin routing method to cycle a first tuple from the stream of tuples through the first processing branch and a second tuple from the stream of tuples through the second processing branch.

8. The system of claim 1 , where the wherein the measuring stream operator is further configured to route of the stream of tuples to generate an even processing time between the first processing branch and the second processing branch.

9. The system of claim 1 , wherein the measuring stream operator is further configured to measure performance data of each tuple from the stream of tuples in response to a triggering condition, the triggering condition is an event that initiates an analysis of performance data from the measuring stream operator.

10. The system of claim 9 , wherein the triggering condition occurs in response to the first processing branch having a processing time of a certain percentage above the second processing branch.

11. The system of claim 1 , wherein the weights are assigned based on the past performance of other tuples.

12. The system of claim 1 , where in the measuring stream operator that is further configured to:
synchronize the stream of tuples from the first processing branch with a stream of tuples from another processing branch running on another instance.

Assignees

• IBM

Inventors

• Branson Michael J
• Cobb Bradford L
• Santosuosso John M

Classifications

• H04L47/70

  Admission control; Resource allocation · CPC title

• H04L47/125 Primary

  by balancing the load, e.g. traffic engineering · CPC title

• H04L43/16

  Threshold monitoring · CPC title

• H04L43/08 Primary

  Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters · CPC title

• H04L43/0852

  Delays · CPC title