Reducing redundant operations in a streaming environment

Patent metadata

Field	Value
Publication number	US-2016267197-A1
Application number	US-201514644398-A
Country	US
Kind code	A1
Filing date	Mar 11, 2015
Priority date	Mar 11, 2015
Publication date	Sep 15, 2016
Grant date	—

Abstract

A stream computing application may permit one or more jobs to connect to a data stream of a different job. As more jobs connect to the data stream, the likelihood that two or more operators are performing redundant operations on a data stream may increase. A stream manager may monitor an operator graph to identify two operators performing a redundant operation. If the stream manager determines that there are redundant operators in the operator graph, the stream manager may modify the operator graph to remove the redundant operators.

First claim

1 - 17 . (canceled)

18 . A system for processing a stream of tuples comprising:
a plurality of processing elements to receive a stream of tuples, each processing element having one or more stream operators; two or more processors; and a memory containing an application that, when executed, causes a processor to:
receive a stream of tuples to be processed by a plurality of processing elements operating on one or more computer processors, each processing element having one or more stream operators, one or more of the stream operators include code configured to output tuples to one or more other stream operators, the processing elements forming an operator graph in which tuples flow between stream operators, the operator graph defining one or more execution paths for processing and routing the stream of tuples, each processing element having an associated memory space;
identify a first job, the first job including a first stream operator and a second stream operator, the first stream operator being configured to import a group of tuples from a first data stream and output tuples to the second stream operator;
identify a second job, the second job including a third stream operator and a fourth stream operator, the third stream operator being configured to import the group of tuples from the first data stream and output tuples to the fourth stream operator;
determine whether the first stream operator and the third stream operator perform one or more redundant operations on the group of tuples;
create, in response to the first stream operator and the third stream operator performing one or more redundant operations on the group of tuples, an intermediary stream operator, the intermediary stream operator being configured to perform the one or more redundant operations;
set the intermediary stream operator as exportable; and
connect the intermediary stream operator to the first and second jobs.

19 . A computer program product for processing a stream of tuples, the computer program product comprising a computer readable storage medium having program code embodied therein, the program code comprising computer readable program code configured to:
receive a stream of tuples to be processed by a plurality of processing elements operating on one or more computer processors, each processing element having one or more stream operators, one or more of the stream operators include code configured to output tuples to one or more other stream operators, the processing elements forming an operator graph in which tuples flow between stream operators, the operator graph defining one or more execution paths for processing and routing the stream of tuples, each processing element having an associated memory space; identify a first job, the first job including a first stream operator and a second stream operator, the first stream operator being configured to import a group of tuples from a first data stream and output tuples to the second stream operator; identify a second job, the second job including a third stream operator and a fourth stream operator, the third stream operator being configured to import the group of tuples from the first data stream and output tuples to the fourth stream operator; determine whether the first stream operator and the third stream operator perform one or more redundant operations on the group of tuples; create, in response to the first stream operator and the third stream operator performing one or more redundant operations on the group of tuples, an intermediary stream operator, the intermediary stream operator being configured to perform the one or more redundant operations; set the intermediary stream operator as exportable; and connect the intermediary stream operator to the first and second jobs.

Assignees

• IBM

Inventors

• Barsness Eric L
• Branson Michael J
• Santosuosso John M

Classifications

• G06F16/24568 Primary

  Data stream processing; Continuous queries · CPC title

• G06F17/30516

  Physics · mapped topic

• G06F17/30958 Primary

  Physics · mapped topic