Early risk identification in DevOps environments

Patent metadata

Field	Value
Publication number	US-9921952-B2
Application number	US-201514727960-A
Country	US
Kind code	B2
Filing date	Jun 2, 2015
Priority date	Jun 2, 2015
Publication date	Mar 20, 2018
Grant date	Mar 20, 2018

Abstract

A computer executes a first version of a code module in a first test environment, collects a first set of execution measurements, and creates a first profile based on the first set of execution measurements. The computer executes the first version of the code module in a second test environment, collects a second set of execution measurements, and creates a second profile based on the second set of execution measurements. In response to a difference between the first profile and the second profile exceeding a threshold, the computer provides diagnostic data characterizing the difference, analyzes an impact of the difference on the execution of the code module, calculates a risk of code module failure value based on analyzing the impact of the difference, and provides a recommended action based on analyzing the impact of the difference and the calculated risk of code module failure value.

First claim

What is claimed is:

1. A method comprising:
executing, by a computer, a first version of a code module in a first test environment; collecting, by the computer, a first set of execution measurements, including code module execution times and code module control flows, for the first version of the code module executing in the first test environment;
creating, by the computer, a first profile of the first version of the code module based on the first set of execution measurements;
executing, by the computer, the first version of the code module in a second test environment;
collecting, by the computer, a second set of execution measurements, including code module execution times and control module control flows, for the first version of the code module executing in the second test environment;
creating, by the computer, a second profile of the first version of the code module based on the second set of execution measurements; and
in response to a difference between the first profile and the second profile providing, by the computer, diagnostic data characterizing the difference between the first profile and the second profile;
analyzing, by the computer, an impact of the difference between the first profile and the second profile on the execution of the code module, the impact comprising a value rating based on both of a number of other functions or other products dependent on the code module, and a frequency with which the code module is executed;
calculating, by the computer, a risk of code module failure value based on analyzing the impact of the difference between the first profile and the second profile;
providing, by the computer, a recommended action based on analyzing the impact of the difference and the calculated risk of code module failure value, based at least on a weighted value of an importance of an unstable function included in the first version of the code module based on a historical impact the unstable function has had on the first version of the code module and the number and criticality of the other functions and other products that depend on the unstable function, and
deploying corrective code to fix the first version of the code module, using the recommended action.

2. The method according to claim 1 , further comprising:
executing, by the computer, a second version of the code module in the first test environment, wherein the second version of the code module is the first version of the code module with one or more code changes;
collecting, by the computer, a third set of execution measurements, including code module execution times and code module control flows, for the second version of the code module executing in the first test environment;
creating, by the computer, a third profile of the second version of the code module based on the third set of execution measurements; and
in response to a difference between the first profile and the third profile exceeding a threshold:
providing, by the computer, diagnostic data characterizing the difference between the first profile and the third profile;
analyzing, by the computer, the impact of the difference between the first profile and the third profile on the execution of the code module;
calculating, by the computer, a risk of code module failure value based on analyzing the impact of the difference between the first profile and the third profile; and
providing, by the computer, a recommended action based on analyzing the impact of the difference and the calculated risk of code module failure value.

3. The method according to claim 2 , further comprising:
determining, by the computer, a set of code changes to the code module, based on the difference between profiles and the diagnostic data characterizing the difference between profiles.

4. The method according to claim 1 , wherein the diagnostic data diagnostic characterizing the difference between profiles includes at least one of:
code functions characterized as problematic;
execution times characterized as changed;
function invocation counts characterized as changed;
executed functions characterized as new;
executed functions characterized as unexpected; and
an identification, identifying a developer of the code functions characterized as problematic.

5. The method according to claim 1 , wherein the profile includes at least one of:
an aggregate of a number of times each function in the code module is invoked;
an aggregate execution time for each function in the code module;
an aggregate overall execution time for the code module;
an aggregate count of all function invocations in the code module;
a set of executed functions in the code module;
a set of unexecuted functions in the code module; and
an order of execution of the executed functions in the code module.

6. The method according to claim 1 , wherein analyzing, by the computer, the impact of the difference between profiles includes analyzing at least one of:
a ratio of stable to unstable functions;
a count of changed functions;
an identity of an unstable function;
an historical impact value of the unstable function;
a change in execution time of a function; and
a weighted value of importance of the unstable function.

7. The method according to claim 1 , wherein the code module failure risk value includes the risk of the executed code module experiencing at least one of:
a performance degradation;
a benchmark violation;
a code execution failure;
a code input/output failure;
an application failure; and
a system failure.

8. A computer program product, the computer program product comprising one or more computer readable storage medium and program instructions stored on at least one of the one or more computer readable storage medium, the program instructions comprising:
program instructions to execute, by a computer, a first version of a code module in a first test environment;
program instructions to collect, by the computer, a first set of execution measurements, including code module execution times and code module control flows, for the first version of the code module executing in the first test environment;
program instructions to create, by the computer, a first profile of the first version of the code module based on the first set of execution measurements;
program instruction to execute, by the computer, the first version of the code module in a second test environment;
program instructions to collect, by the computer, a second set of execution measurements, including code module execution times and control module control flows, for the first version of the code module executing in the second test environment;
program instructions to create, by the computer, a second profile of the first version of the code module based on the second set of execution measurements; and
in response to a difference between the first profile and the second profile exceeding a threshold:
program instructions to provide, by the computer, diagnostic data characterizing the difference between the first profile and the second profile;
program instructions to analyze, by the computer, an impact of the difference between the first profile and the second profile on the execution of the code module, the impact comprising a value rating based on both of a number of other functions and other products dependent on the code module, and a frequency with which the code module is executed;
program instructions to calculate, by the computer, a risk of code module failure value based analyzing the impact of the difference between the first profile and the second profile;
program instructions to provide, by the computer, a recommended action based on analyzing the impact of the difference and the calculated risk of code module fa

Assignees

• IBM

Inventors

• Dean Daniel J
• Kochut Andrzej
• Sailer Anca
• Schulz Charles O

Classifications

• G06F11/3692 Primary

  for test results analysis · CPC title

• G06F8/00

  Arrangements for software engineering (testing or debugging G06F11/36; administrative, planning or organisation aspects of software project management G06Q10/06) · CPC title

• G06Q10/00

  Administration; Management · CPC title