Injecting a fault into a stream operator in a data stream processing application

Patent metadata

Field	Value
Publication number	US-8997039-B2
Application number	US-201313867595-A
Country	US
Kind code	B2
Filing date	Apr 22, 2013
Priority date	Mar 29, 2010
Publication date	Mar 31, 2015
Grant date	Mar 31, 2015

Abstract

In one embodiment, the invention comprises partial fault tolerant stream processing applications. One embodiment of a method for implementing partial fault tolerance in a stream processing application comprising a plurality of stream operators includes: defining a quality score function that expresses how well the application is performing quantitatively, injecting a fault into at least one of the plurality of operators, assessing an impact of the fault on the quality score function, and selecting at least one partial fault-tolerant technique for implementation in the application based on the quantitative metric-driven assessment.

First claim

What is claimed is:

1. A method for implementing partial fault tolerance in a stream processing application comprising a plurality of stream operators, the method comprising:
defining a quality score function that expresses how well the application is performing quantitatively;
injecting a fault into at least one of the plurality of operators;
assessing an impact of the fault on the quality score function based on an analysis of local sections of an output stream of the application including the fault, wherein each local section comprises a portion of the output stream of the application during a defined time interval; and
selecting at least one partial fault-tolerant technique for implementation in the application based on the assessing.

2. The method of claim 1 , wherein the injecting comprises:
modifying source code for the application to produce at least one instance of modified source code, where the at least one instance of modified source code incorporates a fault injection parameter for the at least one partial fault tolerance technique; and
compiling the at least one instance of modified source code into executable code.

3. The method of claim 1 , wherein the assessing comprises:
calculating the quality score function using the output stream of the application in a fault-free condition to generate a target quality score function;
calculating the quality score function using the local sections of the output stream of the application including the fault to generate a faulty quality score function; and
determining whether a difference between the faulty quality score function and the target quality score function is within a tolerance threshold.

4. The method of claim 1 , wherein the quality score function accounts for at least one metric specific to the at least one of the plurality of operators, and wherein the at least one metric characterizes the application in terms of effect on predictability and availability under a failure.

5. The method of claim 4 , wherein the at least one metric comprises an outage duration metric that correlates a duration of an outage of the at least one operator with a degradation in quality of the output stream of the application.

6. The method of claim 4 , wherein the at least one metric comprises a data dependency level metric that defines how the fault affects a quality of the output stream of the application when injected at different stream offsets.

7. The method of claim 4 , wherein the at least one metric comprises a quality impact metric that defines a deviation of a quality of the output stream of the application over the local sections of the output stream.

8. The method of claim 4 , wherein the at least one metric comprises a recovery time metric that estimates how long it takes the application to recover after occurrence of the fault.

9. The method of claim 1 , wherein the fault is injected at different stream offsets and with different outage durations.

10. The method of claim 1 , wherein the assessing comprises:
aggregating over a totality of the output stream measures of errors occurring in the local sections.

11. The method of claim 10 , wherein each of the measures comprises a square of the errors occurring in one of the local sections.

12. The method of claim 11 , wherein the aggregating comprises:
summing the measures.

13. The method of claim 1 , wherein each of the local sections comprises less than an entirety of the output stream.

14. An article of manufacture, comprising:
a non-transitory computer readable storage medium containing an executable program for implementing partial fault tolerance in a stream processing application comprising a plurality of stream operators, where the program performs steps comprising:
defining a quality score function that expresses how well the application is performing quantitatively;
injecting a fault into at least one of the plurality of operators;
assessing an impact of the fault on the quality score function based on an analysis of local sections of an output stream of the application including the fault, wherein each local section comprises a portion of the output stream of the application during a defined time interval; and
selecting at least one partial fault-tolerant technique for implementation in the application based on the assessing.

15. The article of manufacture of claim 14 , wherein the injecting comprises:
modifying source code for the application to produce at least one instance of modified source code, where the at least one instance of modified source code incorporates a fault injection parameter for the at least one partial fault tolerance technique; and
compiling the at least one instance of modified source code into executable code.

16. The article of manufacture of claim 14 , wherein the assessing comprises:
calculating the quality score function using the output stream of the application in a fault-free condition to generate a target quality score function;
calculating the quality score function using the local sections of the output stream of the application including the fault to generate a faulty quality score function; and
determining whether a difference between the faulty quality score function and the target quality score function is within a tolerance threshold.

17. The article of manufacture of claim 14 , wherein the quality score function accounts for at least one metric specific to the at least one of the plurality of operators, and wherein the at least one metric characterizes the application in terms of effect on predictability and availability under a failure.

18. The article of manufacture of claim 17 , wherein the at least one metric comprises an outage duration metric that correlates a duration of an outage of the at least one operator with a degradation in quality of the output stream of the application.

19. The article of manufacture of claim 17 , wherein the at least one metric comprises a data dependency level metric that defines how the fault affects a quality of the output stream of the application when injected at different stream offsets.

20. The article of manufacture of claim 17 , wherein the at least one metric comprises a quality impact metric that defines a deviation of a quality of the output stream of the application over the local sections of the output stream.

21. The article of manufacture of claim 17 , wherein the at least one metric comprises a recovery time metric that estimates how long it takes the application to recover after occurrence of the fault.

22. The article of manufacture of claim 14 , wherein the fault is injected at different stream offsets and with different outage durations.

23. A system for implementing partial fault tolerance in a stream processing application comprising a plurality of stream operators, the system comprising:
a hardware processor; and
a non-transitory computer-readable medium containing an executable program which, when executed by the hardware processor, cause the hardware processor to implement:
a pre-processor that receives as inputs: source code for the application, a target stream operator that is one of the plurality of stream operators and at which a fault is to be injected, a fault model corresponding to a partial fault tolerance technique, and a plurality of injection parameters for the partial fault tolerance technique, wherein the pre-processor produces as output a plurality of instances of modified source code for the application,

Assignees

• IBM

Inventors

• Andrade Henrique
• Gedik Bugra
• Jacques Da Silva Gabriela
• Wu Kun-Lung

Classifications

• G06F11/3616 Primary

  using software metrics · CPC title

• G06F11/3612

  by runtime analysis (performance monitoring G06F11/3466) · CPC title

• G06F11/28 Primary

  by checking the correct order of processing (G06F11/08 - G06F11/26 take precedence; monitoring patterns of pulse trains H03K5/19) · CPC title

• G06F16/2365

  Ensuring data consistency and integrity · CPC title