Scalable continuous integration and delivery systems and methods

Patent metadata

Field	Value
Publication number	US-10025692-B2
Application number	US-201615199837-A
Country	US
Kind code	B2
Filing date	Jun 30, 2016
Priority date	Aug 6, 2015
Publication date	Jul 17, 2018
Grant date	Jul 17, 2018

Abstract

A scalable continuous integration (CI) system and/or method may be provided to manage and implement a plurality of code changes submitted in a code base. In particular, the scalable CI system may use a divide-and-conquer approach in which a set or a queue of code changes may be divided into smaller sets/queues of code changes. The division may continue recursively until the rate of defect introduction per change set, the rate of change set delivery per time, and the pipeline latency in the divided sets/queues satisfy a particular threshold. Change sets in the divided sets/queues that fail particular quality constraints may be removed from the divided sets/queues. After the failed change sets have been removed, the divided sets/queues may be recombined back into one set/queue.

First claim

What is claimed is:

1. A system comprising:
a non-transitory memory; and
one or more hardware processors coupled to the non-transitory memory and configured to read instructions from the non-transitory memory to cause the system to perform operations comprising:
receiving a plurality of change sets submitted by one or more users to update or modify a code base;
arranging the plurality of change sets in a queue;
determining an average rate of delivery over time for the queue, an average pipeline latency for the queue, and an average defect introduction rate for the queue;
computing, for the queue, a queue quality value based on the determined average rate of delivery over time for the queue, the determined average pipeline latency for the queue, and the determined average defect introduction rate for the queue;
determining whether the queue quality value of the queue exceeds a threshold;
dividing the plurality of change sets in the queue into sub-queues when it is determined that the queue quality value of the queue exceeds the threshold;
processing the sub-queues to remove one or more defective change sets in the sub-queues;
recombining the sub-queues back to the queue; and
integrating the plurality of change sets in the queue into the code base.

2. The system of claim 1 , wherein the queue quality value of the queue comprises a product of the average rate of delivery over time for the queue, the average pipeline latency for the queue, and the average defect introduction rate for the queue.

3. The system of claim 1 , wherein processing the sub-queues to remove the one or more defective change sets in the sub-queues comprises analyzing the sub-queues separately to identity the one or more defective change sets in the sub-queues.

4. The system of claim 1 , wherein the operations further comprise performing the computing, the determining, and the dividing on the sub-queues recursively until reaching sub-queues with a queue quality value that does not exceed the threshold.

5. The system of claim 4 , wherein the operations further comprise:
identifying the one or more defective change sets from the sub-queues;
removing the one or more defective change sets from the sub-queues; and
recursively recombining the sub-queues back to the queue.

6. The system of claim 5 , wherein identifying the one or more defective change sets from the sub-queues comprises using first quality constraints to identify a first defective change set in a first sub-queue based on a first recursive depth level of the first sub-queue and using second quality constraints to identify a second defective change set in a second sub-queue based on a second recursive depth level of the second sub-queue.

7. The system of claim 6 , wherein the first quality constraints are less than the second quality constraints when the first recursive depth level is lower than the second recursive depth level.

8. The system of claim 4 , wherein the operations further comprise changing the threshold for determining whether to further divide a sub-queue based on a recursive depth level of the sub-queue.

9. The system of claim 1 , wherein integrating the plurality of change sets in the queue into the code base comprises integrating the recombined sub-queues in the queue into the code base.

10. A method comprising:
receiving, by one or more hardware processors, a plurality of change sets submitted by one or more users to update or modify a code base;
arranging, by the one or more hardware processors, the plurality of change sets in a queue;
determining, by the one or more hardware processors, an average rate of delivery over time for the queue, an average pipeline latency for the queue, and an average defect introduction rate for the queue;
computing, by the one or more hardware processors for the queue, a queue quality value based on the determined average rate of delivery over time for the queue, the determined average pipeline latency for the queue, and the determined average defect introduction rate for the queue;
determining, by the one or more hardware processors, whether the queue quality value of the queue exceeds a threshold;
dividing, by the one or more hardware processors, the plurality of change sets in the queue into sub-queues when it is determined that the queue quality value exceeds the threshold;
processing, by the one or more hardware processors, the sub-queues to remove one or more defective change sets in the sub-queues;
recombining, by the one or more hardware processors, the sub-queues back to the queue; and
integrating, by the one or more hardware processors, the plurality of change sets in the queue into the code base.

11. The method of claim 10 , wherein the queue quality value of the queue comprises a product of the average rate of delivery over time for the queue, the average pipeline latency for the queue, and the average defect introduction rate for the queue.

12. The method of claim 11 , wherein processing the sub-queues to remove the one or more defective change sets in the sub-queues comprises analyzing the sub-queues separately to identity the one or more defective change sets in the sub-queues.

13. The method of claim 10 , further comprising performing the computing, the determining, and the dividing on the sub-queues recursively until reaching sub-queues with a queue quality value that does not exceed the threshold.

14. The method of claim 13 , further comprising:
identifying the one or more defective change sets from the sub-queues;
removing the one or more defective change sets from the sub-queues; and
recursively recombining the sub-queues back to the queue.

15. The method of claim 14 , wherein identifying the one or more defective change sets from the sub-queues comprises using first quality constraints to identify a first defective change set in a first sub-queue based on a first recursive depth level of the first sub-queue and using second quality constraints to identify a second defective change set in a second sub-queue based on a second recursive depth level of the second sub-queue.

16. The method of claim 13 , further comprising changing the threshold for determining whether to further divide a sub-queue based on a recursive depth level of the sub-queue.

17. The method of claim 10 , wherein integrating the plurality of change sets in the queue into the code base comprises integrating the recombined sub-queues in the queue into the code base.

18. A non-transitory machine-readable medium having stored thereon machine-readable instructions executable to cause a machine to perform operations comprising:
receiving a plurality of change sets submitted by one or more users to update or modify a code base;
arranging the plurality of change sets in a queue;
determining an average rate of delivery over time for the queue, an average pipeline latency for the queue, and an average defect introduction rate for the queue;
computing, for the queue, a queue quality value based on the determined average rate of delivery over time for the queue, the determined average pipeline latency for the queue, and the determined average defect introduction rate for the queue;
determining whether the queue quality value of the queue exceeds a threshold;
dividing the plurality of change sets in the queue into sub-queues when it is determined that the queue quality value of the queue exceeds the threshold;
processing the sub-queues to remove one or more defective change sets in the sub-queues;
recombining the sub-queues back to the queue; and
integrating the plur

Assignees

• Paypal Inc

Inventors

• Wood Zachary Paul
• Montoya Justin Vincent

Classifications

• G06F11/079

  Root cause analysis, i.e. error or fault diagnosis (in a hardware test environment G06F11/22; in a software test environment G06F11/36) · CPC title

• G06F8/33

  Intelligent editors · CPC title

• G06F8/71 Primary

  Version control (security arrangements therefor G06F21/57); Configuration management · CPC title

• G06F8/65

  Updates (security arrangements therefor G06F21/57) · CPC title

• G06F8/30

  Creation or generation of source code · CPC title