Performance regression framework

Patent metadata

Field	Value
Publication number	US-10783051-B2
Application number	US-201816123776-A
Country	US
Kind code	B2
Filing date	Sep 6, 2018
Priority date	Sep 6, 2018
Publication date	Sep 22, 2020
Grant date	Sep 22, 2020

Abstract

A performance regression test that monitors a resource based on a metric measured for each of a plurality of builds of a test instance is run. Measurement data of the metric triggered by an operation is logged in an operation log for each of a plurality of builds. The operation is logged in response to an incoming request for processing associated with the performance regression test for monitoring the resource. Based on the operation log and for each of the plurality of builds, the measurement data of the metric triggered by the operation associated with the performance regression test for monitoring the resource is collected. The collected measurement data of the metric for each of the plurality of builds is presented.

First claim

What is claimed is:

1. A computer system comprising:
a memory; and
one or more hardware processors communicatively coupled to the memory, wherein the memory comprises computer instructions that, when executed by the one or more hardware processors, cause the computer system to:
access a source code repository to obtain a version of a portion of source code, wherein the portion of source code is updated within the source code repository by a plurality of commits;
compile a build of a test instance for running the version of the portion of source code obtained from the source code repository;
create a file for a performance regression test that monitors a resource based on a metric measured for the build of the test instance, wherein the metric corresponds to one of a plurality of preexisting metrics associated with an operation log for the build, wherein the file identifies the resource for performance monitoring, and identifies, from among the plurality of preexisting metrics, the metric for monitoring the resource;
transmit the file to a continuous integration (CI) server to run the performance regression test on the build of the test instance;
run the performance regression test;
log, in the operation log for the build, measurement data of the plurality of preexisting metrics triggered by each of a plurality of operations, including measurement data of the metric triggered by a first operation, wherein the measurement data is logged in response to an incoming request for processing associated with the performance regression test for monitoring the resource;
collect, from the operation log and for the build, the measurement data of the metric generated by the operation associated with the performance regression test for monitoring the resource;
detect performance regression of the resource responsive to a determination that collected measurement data for the resource for the build meets a threshold value for the measurement data of the metric; and
present the collected measurement data and an indication of detected performance regression of the metric for the build.

2. The computer system according to claim 1 , wherein the plurality of preexisting metrics include a processing time, central processing unit percentage, a response time, a network time, structured query language (SQL) time, SQL count, enterprise rule time, enterprise rule count, client response time, client network time, total page load time, number of requests, browser time, client script time, and user interface policy time.

3. The computer system according to claim 1 , comprising the CI server, and
wherein the computer instructions that, when executed by the one or more hardware processors, cause the computer system to run the performance regression test on the build of the test instance comprise computer instructions that, when executed by the one or more hardware processors, cause the computer system to:
bundle the version of the portion of the source code into an artifact on the CI server, wherein the artifact is a production-ready build of the test instance;
run the performance regression test on the artifact; and
tear down the artifact after publishing the measurement data.

4. The computer system according to claim 3 , wherein the source code repository is accessed, and the version of the portion of the source code bundled into the artifact, on a periodic basis such that a plurality of builds of the test instance including the build of the test instance and corresponding artifacts including the artifact are compiled and bundled respectively.

5. The computer system according to claim 4 , wherein the memory comprises computer instructions that, when executed by the one or more hardware processors, cause the computer system to:
detect, for one of the plurality of builds of the test instance, performance regression of the resource responsive to a determination that the collected measurement data for the resource for the one of the plurality of builds meets a threshold difference value of the metric.

6. The computer system according to claim 1 , wherein the memory comprises computer instructions that, when executed by the one or more hardware processors, cause the computer system to, in response to detecting performance regression of the resource for the build of the test instance:
run the performance regression test on each of a plurality of bundled artifacts corresponding to the build of the test instance, the plurality of bundled artifacts respectively corresponding to the plurality of commits associated with the build of the test instance; and
detect, for one of the plurality of bundled artifacts, the performance regression of the resource responsive to a determination that collected measurement data for the resource for the one of the plurality of bundled artifacts meets a threshold value for the measurement data of the metric.

7. The computer system according to claim 1 , wherein the resource is accessible through a hypertext transfer protocol request to the test instance, and includes at least one of a given application programming interface, and a given uniform resource locator associated with the test instance.

8. The computer system according to claim 4 , comprising a display device, and wherein the computer instructions that, when executed by the one or more hardware processors, cause the computer system to present the collected measurement data of the metric comprise computer instructions that, when executed by the one or more hardware processors, cause the computer system to:
display, on the display device, a trendline indicating a value of the metric of the resource for the plurality of builds of the test instance including the build of the test instance over time.

9. A method comprising:
accessing, for each of a plurality of builds of a test instance, a source code repository implemented on a version control system to obtain an updated version of a source code, wherein the source code repository is continuously updated by making a plurality of commits to the source code as part of an ongoing development process associated with the test instance;
creating, for each of the plurality of builds a file for a performance regression test that monitors a resource based on a metric measured for the build of the test instance, wherein the metric corresponds to one of a plurality of preexisting metrics associated with an operation log for the build, wherein the file identifies the resource for performance monitoring, and identifies, from among the plurality of preexisting metrics, the metric for monitoring the resource;
transmitting, for each of the plurality of builds, the file to a continuous integration (CI) server to run the performance regression test on the build of the test instance;
running, on each of the plurality of builds, the performance regression test;
logging, for each of the plurality of builds, measurement data of the plurality of preexisting metrics triggered by each of a plurality of operations, including measurement data of the metric triggered by a first operation in an operation log for each of the plurality of builds, wherein the operation is logged in response to an incoming request for processing associated with the performance regression test for monitoring the resource;
collecting, based on the operation log and for each of the plurality of builds, the measurement data of the metric triggered by the operation associated with the performance regression test for monitoring the resource;
detecting, performance regression of the resource responsive to a determination that collected measurement data for the resource for one of the plurality of builds meets a threshold value for the measurement data of the metric; and
p

Assignees

• Servicenow Inc

Inventors

• Srinivasan Ravishankar
• Srungarapu Sravan

Classifications

• G06F2201/865

  Monitoring of software · CPC title

• G06F11/3476 Primary

  Data logging (G06F11/14, G06F11/2205 take precedence) · CPC title

• G06F8/71

  Version control (security arrangements therefor G06F21/57); Configuration management · CPC title

• G06F8/65

  Updates (security arrangements therefor G06F21/57) · CPC title

• G06F11/32

  with visual {or acoustical} indication of the functioning of the machine · CPC title