Streaming operator with trigger

Patent metadata

Field	Value
Publication number	US-9477571-B2
Application number	US-201414158960-A
Country	US
Kind code	B2
Filing date	Jan 20, 2014
Priority date	Jan 20, 2014
Publication date	Oct 25, 2016
Grant date	Oct 25, 2016

Abstract

One or more operators in a flow graph of a streaming application may include one or more triggers that indicate when action needs to be taken for the operator. A streams manager monitors performance of a streaming application and receives a notification when a trigger in an operator fires. In response to a trigger firing, the streams manager determines an appropriate action corresponding to the trigger. When the trigger indicates an adjustment of cloud resources are needed, the streams manager formulates a cloud resource request to a cloud manager. In response, the cloud manager adjusts the cloud resources for the operator to improve performance of the streaming application. A trigger may specify a trigger action for an operator, and may additionally specify a trigger action for one or more other affected operators. The firing of a trigger in one operator can therefore result in adjusting resources to multiple operators.

First claim

The invention claimed is:

1. A computer-implemented method executed by at least one processor for managing a streaming application, the method comprising:
executing a streaming application that comprises a flow graph that includes a plurality of operators that process a plurality of data tuples, wherein a first of the plurality of operators comprises a trigger that specifies:
trigger criteria that includes data overflow of a memory in the first operator;
a corresponding trigger action to perform that comprises logging a snapshot of a virtual machine when the trigger fires due to the data overflow of the memory in the first operator; and
an affected operator that is affected by the processing in the first operator by being upstream or downstream from the first operator;

detecting when the trigger fires;
in response to detecting when the trigger fires, determining the corresponding trigger action; and
initiating at least one operation to perform the corresponding trigger action, wherein the initiation of the at least one operation comprises submitting a first request to a cloud manager to provision additional resources for a first virtual machine running the first operator and submitting a second request to the cloud manager to provision additional resources for a second virtual machine separate from the first virtual machine, wherein the second virtual machine runs the affected operator specified in the trigger.

2. The method of claim 1 wherein the initiation of the at least one operation comprises submitting a first request to a cloud manager to perform live migration of the operator to a different virtual machine.

3. The method of claim 1 wherein the initiation of the at least one operation comprises logging a snapshot of a first virtual machine running the first operator.

4. The method of claim 1 wherein the trigger criteria includes utilization of at least one resource in a first virtual machine running the first operator.

5. The method of claim 1 wherein the trigger criteria includes data rate of tuples received by the first operator.

6. The method of claim 1 wherein the trigger criteria includes data type of tuples received by the first operator.

7. The method of claim 1 wherein the trigger criteria comprises a usage threshold and the corresponding trigger action comprises doubling a number of resources in the first virtual machine running the first operator and doubling a number of resources in the second virtual machine running the affected operator.

8. The method of claim 7 wherein the resources comprise CPUs.

9. A computer-implemented method executed by at least one processor for managing a streaming application, the method comprising:
executing a streaming application that comprises a flow graph that includes a plurality of operators that process a plurality of data tuples, wherein a first of the plurality of operators comprises a trigger that specifies:
trigger criteria that comprises data overflow of a memory in the first operator;
a corresponding trigger action to perform that comprises logging a snapshot of a virtual machine when the trigger fires due to the data overflow of the memory in the first operator; and
an affected operator that is affected by the processing in the first operator by being upstream or downstream from the first operator;

detecting when the trigger fires due to the data overflow of the memory in the first operator;
in response to detecting when the trigger fires:
logging a first snapshot of the first virtual machine running the first operator; and
logging a second snapshot of a second virtual machine separate from the first virtual machine running the affected operator.

10. The method of claim 9 wherein in response to detecting when the trigger fires:
submitting a first request to a cloud manager to provision additional resources for the first virtual machine running the first operator; and
submitting a second request to the cloud manager to provision additional resources for the second virtual machine running the affected operator.

11. A computer-implemented method executed by at least one processor for managing a streaming application, the method comprising:
executing a streaming application that comprises a flow graph that includes a plurality of operators that process a plurality of data tuples, wherein a first of the plurality of operators comprises a trigger that specifies:
trigger criteria comprising:
utilization of at least one resource in a first virtual machine running the first operator;
data rate of tuples received by the first operator;
data value of tuples received by the first operator;
data type of tuples received by the first operator;
time-based criteria; and
data overflow of a memory in the first operator;

a plurality of corresponding trigger actions to perform when the trigger fires due to the trigger criteria being satisfied;
an affected operator that is affected by the processing in the first operator by being upstream or downstream from the first operator;

detecting when the trigger fires due to the trigger criteria being satisfied;
in response to detecting when the trigger fires, determining the plurality of corresponding trigger actions, wherein the plurality of corresponding trigger actions comprises:
submitting a first request to a cloud manager to provision additional resources for a first virtual machine running the first operator;
submitting a second request to the cloud manager to provision additional resources for a second virtual machine separate from the first virtual machine, wherein the second virtual machine runs the affected operator specified in the trigger;
submitting a first request to a cloud manager to perform live migration of the first operator to a different virtual machine;
logging a snapshot of a first virtual machine running the first operator;
logging a second snapshot of a second virtual machine running the affected operator specified in the trigger; and
initiating at least one operation to perform the plurality of corresponding trigger actions.

Assignees

• IBM

Inventors

• Cao Bin
• Eidem Jessica R
• Muras Brian R
• Sun Jingdong

Classifications

• H04L41/0897

  by horizontal or vertical scaling of resources, or by migrating entities, e.g. virtual resources or entities · CPC title

• H04L41/0896

  Bandwidth or capacity management, i.e. automatically increasing or decreasing capacities (flow or congestion control using dynamic resource allocation, e.g. in-call renegotiation, H04L47/76) · CPC title

• G06F9/5005

  to service a request · CPC title

• G06F11/3072 Primary

  where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting · CPC title

• G06F17/30516

  Physics · mapped topic