Performance regression framework

Patent metadata

Field	Value
Publication number	US-2021004310-A1
Application number	US-202017024500-A
Country	US
Kind code	A1
Filing date	Sep 17, 2020
Priority date	Sep 6, 2018
Publication date	Jan 7, 2021
Grant date	—

Abstract

A performance regression test that monitors a resource based on a metric measured for each of a plurality of builds of a test instance is run. Measurement data of the metric triggered by an operation is logged in an operation log for each of a plurality of builds. The operation is logged in response to an incoming request for processing associated with the performance regression test for monitoring the resource. Based on the operation log and for each of the plurality of builds, the measurement data of the metric triggered by the operation associated with the performance regression test for monitoring the resource is collected. The collected measurement data of the metric for each of the plurality of builds is presented.

First claim

What is claimed is:

1 . A method comprising:
installing, by one or more processors of a continuous integration (CI) server, a build of a test instance, wherein the build of the test instance is representative of a version of a portion of a source code updated by a plurality of commits; receiving, by the one or more processors, a file for a performance regression test that monitors a resource, the file identifying the resource and one or more metrics to monitor associated with accessing the resource; running, on the build of the test instance and by the one or more processors, the performance regression test based on the file, wherein the build of the test instance is configured to access the resource during the running of the performance regression test; recording, in an operation log, first measurement data of the one or more metrics triggered by a first operation of a plurality of operations, wherein the first measurement data is logged in response to an incoming request for processing associated with accessing the resource via the build of the test instance, and wherein the operation log comprises second measurement data from running the performance regression test on one or more other builds of the test instance; extracting, by the one or more processors, the measurement data from the operation log; detecting, by the one or more processors, a performance regression for the resource in the build of the test instance based on a comparison of the first measurement data from the build of the test instance to the second measurement data from the one or more other builds of the test instance; and generating, by the one or more processors, a notification for the detected performance regression, wherein the notification is configured to indicate the build of the test instance as a regressed build.

2 . The method of claim 1 , wherein the one or more metrics comprise a processing time, a central processing unit percentage, a response time, a network time, a structured query language (SQL) time, a SQL count, an enterprise rule time, an enterprise rule count, a client response time, a client network time, a total page load time, a number of requests, a browser time, a client script time, or a user interface policy time, or a combination thereof

3 . The method of claim 1 , comprising:
installing, by the one or more processors, a plurality of bundled artifacts corresponding to each of the plurality of commits on the version of the source code corresponding to the build of the test instance in response to detecting performance regression for the resource in the build of the test instance; running, by the one or more processors, the performance regression test on each of the plurality of bundled artifacts to determine a specific bundled artifact responsible for the performance regression in the build of the test instance; and generating an additional notification indicating the specific bundled artifact or specific commit of the plurality of commits responsible for the performance regression.

4 . The method of claim 1 , comprising, transmitting the notification to a computing device.

5 . The method of claim 1 , comprising running an additional performance regression test on the build of the test instance.

6 . The method of claim 1 , wherein the performance regression test is run on the one or more other builds of the test instance before the performance regression test is run on the build of the test instance.

7 . A system comprising:
a continuous integration (CI) server device configured to perform actions comprising:
installing a build of a test instance, wherein the build of the test instance is representative of a version of a portion of a source code updated by a plurality of commits;
receiving a file for a performance regression test that monitors a resource, the file identifying the resource and one or more metrics to monitor associated with accessing the resource;
running, on the build of the test instance, the performance regression test based on the file, wherein the build of the test instance is configured to access the resource during the running of the performance regression test; and
recording, in an operation log, first measurement data of the one or more metrics triggered by a first operation of a plurality of operations, wherein the first measurement data is logged in response to an incoming request for processing associated with accessing the resource via the build of the test instance, wherein the operation log comprises second measurement data from running the performance regression test on one or more other builds of the test instance, and wherein the first measurement data and the second measurement data can be compared to identify a performance regression.

8 . The system of claim 7 , wherein the one or more metrics comprise a processing time, a central processing unit percentage, a response time, a network time, a structured query language (SQL) time, a SQL count, an enterprise rule time, an enterprise rule count, a client response time, a client network time, a total page load time, a number of requests, a browser time, a client script time, or a user interface policy time, or a combination thereof.

9 . The system of claim 7 , wherein the CI server device is configured to perform actions comprising:
extracting the first measurement data and the second measurement data from the operation log; detecting the performance regression for the resource in the build of the test instance based on a comparison of the first measurement data from the build of the test instance to the second measurement data from the one or more other builds of the test instance; and generating a notification for the detected performance regression, wherein the notification is configured to indicate the build of the test instance as a regressed build.

10 . The system of claim 9 , wherein the CI server device is configured to perform actions comprising:
transmitting the notification to a computing device.

11 . The system of claim 10 , wherein the CI server device is configured to perform actions comprising:
installing a plurality of bundled artifacts corresponding to each of the plurality of commits on the version of the source code corresponding to the build of the test instance in response to detecting performance regression for the resource in the build of the test instance; running the performance regression test on each of the plurality of bundled artifacts to determine a specific bundled artifact of the plurality of bundled artifacts responsible for the performance regression in the build of the test instance; and generating a notification indicating the specific bundled artifact or specific commit of the plurality of commits responsible for the performance regression.

12 . The system of claim 7 , comprising an additional server device, wherein the additional server device is configured to perform actions comprising:
extracting the first measurement data and the second measurement data from the operation log; detecting the performance regression for the resource in the build of the test instance based on a comparison of the first measurement data from the build of the test instance to the second measurement data from the one or more other builds of the test instance; and generating a notification for the detected performance regression, wherein the notification is configured to indicate the build of the test instance as a regressed build.

13 . The system of claim 12 , wherein the additional server device comprises a remote client device.

14 . The system of claim 13 , wherein generating the notific

Assignees

• Servicenow Inc

Inventors

• Srinivasan Ravishankar
• Srungarapu Sravan

Classifications

• G06F8/71

  Version control (security arrangements therefor G06F21/57); Configuration management · CPC title

• G06F11/3476 Primary

  Data logging (G06F11/14, G06F11/2205 take precedence) · CPC title

• G06F2201/865

  Monitoring of software · CPC title

• G06F11/3452

  Performance evaluation by statistical analysis · CPC title

• G06F11/3688

  for test execution, e.g. scheduling of test suites · CPC title