Dynamic stream processing within an operator graph

Patent metadata

Field	Value
Publication number	US-9391830-B2
Application number	US-201213627608-A
Country	US
Kind code	B2
Filing date	Sep 26, 2012
Priority date	Sep 26, 2012
Publication date	Jul 12, 2016
Grant date	Jul 12, 2016

Abstract

A method and system for processing a stream of tuples in a stream-based application is disclosed. The method may include a first stream operator determining whether a requirement to modify processing of a first tuple at a second stream operator exists. The method may provide for associating an indication to modify processing of the first tuple at the second stream operator if the requirement exists.

First claim

What is claimed is:

1. A method for processing a stream of tuples, comprising:
receiving a stream of tuples to be processed by a plurality of processing elements operating on one or more computer processors, each processing element having one or more stream operators, a first stream operator configured to associate an indication to tuples that require modified processing and a second stream operator configured to perform a data processing operation;
receiving a first tuple at the second stream operator;
first examining, by the second stream operator, the first tuple for the indication;
performing, by the second stream operator, the data processing operation on first data in response to the receiving the first tuple, the performing the data processing operation based on not detecting the indication during the first examining, the data processing operation modifying the first data in a first manner;
receiving a second tuple at the second stream operator;
second examining, by the second stream operator, the second tuple for the indication; and
performing, by the second stream operator, a modified data processing operation on second data in response to the receiving the second tuple, the performing the modified data processing operation based on detecting the indication during the second examining, the indication having been associated with the second tuple by the first stream operator, the modified data processing operation modifying the second data in a second manner different from the first manner, the second tuple remaining in the stream of tuples after the performing the modified data processing operation.

2. The method of claim 1 , further comprising establishing an operator graph of the one or more stream operators, the operator graph defining one or more execution paths in which the first stream operator is configured to receive tuples from one or more upstream stream operators and transmit the tuples to one or more downstream stream operators.

3. The method of claim 1 , wherein the performing the modified data processing operation comprises skipping the data processing operation, the method further comprising transmitting the second tuple from the first stream operator to the second stream operator, and transmitting the second tuple from the second stream operator to a third stream operator.

4. The method of claim 3 , further comprising receiving the second tuple at the third stream operator and processing the second tuple by the third stream operator.

5. The method of claim 1 , wherein the second stream operator is a particular group of two or more stream operators.

6. The method of claim 1 , further comprising dividing stream operators into levels, wherein the second stream operator includes all stream operators at a particular level.

7. The method of claim 1 , wherein the second stream operator is one level from the first stream operator and a third stream operator is two levels from the first stream operator.

8. The method of claim 1 , wherein the first stream operator is configured to perform a first process on a received tuple, the second stream operator is configured to perform a second process on a received tuple, and a third stream operator is configured to perform a third process on a received tuple.

9. The method of claim 1 , further comprising:
receiving the second tuple at the first stream operator;
determining by the first stream operator that a particular attribute value is present in the second tuple; and
associating, by the first stream operator, the indication with the second tuple, the associating the indication with the second tuple based on the determining that the particular attribute value is present.

10. The method of claim 1 , further comprising:
receiving the second tuple at the first stream operator;
determining by the first stream operator that a particular attribute value is missing from the second tuple; and
associating, by the first stream operator, the indication with the second tuple, the associating the indication with the second tuple based on the determining that the particular attribute value is missing.

11. The method of claim 1 , further comprising:
receiving the second tuple at the first stream operator;
determining by the first stream operator that the second tuple has been in an operator graph for a time period that exceeds a threshold time period; and
associating, by the first stream operator, the indication with the second tuple, the associating the indication with the second tuple based on the determining that the time period exceeds the threshold.

12. The method of claim 1 , further comprising:
receiving the second tuple at the first stream operator;
associating, by the first stream operator, the indication with the second tuple, wherein the associating the indication with the second tuple includes outputting by the first stream operator a third tuple to the second stream operator.

13. The method of claim 1 , wherein the data processing operation comprises two or more processing operations and wherein the modified data processing operation comprises less than all of the two or more processing operations.

14. The method of claim 1 , further comprising:
receiving the second tuple at the first stream operator;
determining, by the first stream operator, that a requirement to modify processing of the second tuple by a second stream operator exists, wherein the requirement to modify processing of the second tuple includes a performance condition; and
associating, by the first stream operator, the indication with the second tuple.

15. The method of claim 1 , wherein the detecting the indication for a tuple is selected from the group consisting of:
identifying a qualifying header associated with the tuple;
identifying a qualifying trailer associated with the tuple;
identifying qualifying metadata associated with the tuple; and
determining that the tuple satisfies qualifying criteria, the qualifying criteria established by a previously received tuple.

Assignees

• IBM

Inventors

• Branson Michael J
• Cobb Bradford L
• Santosuosso John M

Classifications

• H04L69/22 Primary

  Parsing or analysis of headers · CPC title

• H04L29/08135 Primary

  Electricity · mapped topic

• H04L67/10 Primary

  in which an application is distributed across nodes in the network (software deployment G06F8/60; multiprogramming arrangements G06F9/46) · CPC title