Methods and systems for identifying duplicate jobs in a continuous integration environment

Patent metadata

Field	Value
Publication number	US-10872028-B2
Application number	US-201916290170-A
Country	US
Kind code	B2
Filing date	Mar 1, 2019
Priority date	Mar 1, 2019
Publication date	Dec 22, 2020
Grant date	Dec 22, 2020

Abstract

Methods and systems are provided for comparing and identifying job definition files that are hard copies and soft copies of one another. In one embodiment, a method is provided that includes receiving and splitting a first job definition file into a first plurality of stages. The method further comprises comparing the first plurality of stages with a second plurality of stages from a second job definition file. Corresponding stages may be identified and used to determine whether the first and second job definition files are hard or soft copies of one another. If the first and second job definition files are hard copies, one of the job definition files may be deleted from a continuous integration environment. If the first and second job definition files are soft copies, a generic job definition file may be generated based on the first and second job definition files.

First claim

The invention claimed is:

1. A method comprising:
receiving a first job definition file for execution in a continuous integration environment;
splitting the first job definition file into a first plurality of stages;
comparing the first plurality of stages with a second plurality of stages from a second job definition file to determine one or more corresponding stages between the first and the second plurality of stages;
determining, based on the one or more corresponding stages, whether the first and the second job definition files are hard copies or soft copies of one another;
removing, responsive to determining that the first and the second job definition files are hard copies of one another, one of the first and the second job definition files from the continuous integration environment; and
generating, responsive to determining that the first and the second job definition files are soft copies of one another, a generic job definition file to replace the first and the second job definition files within the continuous integration environment.

2. The method of claim 1 , wherein comparing the first plurality of stages with the second plurality of stages further comprises:
determining a testing function performed by at least a subset of the first plurality of stages; and
identifying the corresponding stages as stages from the first and second plurality of stages that perform the same testing function.

3. The method of claim 2 , wherein the first plurality of stages comprises a first plurality of job commands and the second plurality of stages comprises a second plurality of job commands.

4. The method of claim 3 , wherein comparing the first plurality of stages with the second plurality of stages further comprises:
comparing the job commands of one of the first plurality of stages to the job commands of a corresponding stage from the second plurality of stages.

5. The method of claim 4 , wherein the generic job definition file includes a generic job command.

6. The method of claim 5 , wherein generating the generic job definition file further comprises:
identifying a first job command of the first job definition file and a second job command of the second job definition file;
identifying a common portion of the first and the second job commands;
identifying differing portions of the first and the second job commands; and
generating the generic job command to include the common portion and a generic portion representative of the differing portions of the first and the second job commands.

7. The method of claim 4 , further comprising:
identifying a third job command of the first job definition file and a fourth job command of the second job definition file, wherein the third and the fourth job commands do not share a common portion; and
generating the generic job command to include both the third and the fourth job commands.

8. The method of claim 1 , wherein a hard copy is determined if a similarity measure of the first and the second plurality of stages exceeds a first threshold.

9. The method of claim 8 , wherein a soft copy is determined if the similarity measure is less than the first threshold and exceeds a second threshold.

10. The method of claim 9 , wherein no copy is determined if the similarity measure is less than the second threshold.

11. The method of claim 1 , wherein the first job definition file is split into the first plurality of stages at locations identified based on at least one of a keyword and a pattern.

12. The method of claim 11 , wherein the keyword is received from an API of the continuous integration environment.

13. A system comprising:
a processor; and
a memory storing instructions which, when executed by the processor, cause the processor to:
receive a first job definition file for execution in a continuous integration environment;
split the first job definition file into a first plurality of stages;
compare the first plurality of stages with a second plurality of stages from a second job definition file to determine one or more corresponding stages between the first and the second plurality of stages;
determine, based on the one or more corresponding stages, whether the first and the second job definition files are hard copies or soft copies of one another;
remove, responsive to determining that the first and the second job definition files are hard copies of one another, one of the first and the second job definition files from the continuous integration environment; and
generate, responsive to determining that the first and the second job definition files are soft copies of one another, a generic job definition file to replace the first and the second job definition files within the continuous integration environment.

14. The system of claim 13 , wherein the memory contains further instructions which, when executed by the processor, cause the processor to:
determine a testing function performed by at least a subset of the first plurality of stages; and
identify the corresponding stages as stages from the first and second plurality of stages that perform the same testing function.

15. The system of claim 14 , wherein the first plurality of stages comprises a first plurality of job commands and the second plurality of stages comprises a second plurality of job commands, and
wherein the memory contains further instructions which, when executed by the processor, cause the processor to:
compare the job commands of one of the first plurality of stages to the job commands of a corresponding stage from the second plurality of stages.

16. The system of claim 15 , wherein the generic job definition file includes a generic job command, and
wherein the memory contains further instructions which, when executed by the processor, cause the processor to:
identify a first job command of the first job definition file and a second job command of the second job definition file;
identify a common portion of the first and the second job commands;
identify differing portions of the first and the second job commands; and
generate the generic job command to include the common portion and a generic portion representative of the differing portions of the first and the second job commands.

17. The system of claim 15 , wherein the memory contains further instructions which, when executed by the processor, cause the processor to:
identify a third job command of the first job definition file and a fourth job command of the second job definition file, wherein the third and the fourth job commands do not share a common portion; and
generate the generic job command to include both the third and the fourth job commands.

18. The system of claim 13 , wherein a hard copy is determined if a similarity measure of the first and the second plurality of stages exceeds a first threshold, and
wherein a soft copy is determined if the similarity measure is less than the first threshold and exceeds a second threshold.

19. The system of claim 13 , wherein the memory contains further instructions which, when executed by the processor, cause the processor to:
split the first job definition file into the first plurality of stages at locations identified based on at least one of a pattern and keyword, wherein the keyword is received from an API of the continuous integration environment.

20. A non-transitory, computer-readable medium storing instructions which, when executed by a processor, cause the processor to:
receive a first job definition file for execution in a continuous

Assignees

• Red Hat Israel Ltd

Inventors

• Kuris Eran
• Stafeyev Alexander
• Bregman Arie

Classifications

• G06F11/3698

  Environments for analysis, debugging or testing of software · CPC title

• G06F11/3688 Primary

  for test execution, e.g. scheduling of test suites · CPC title

• G06F11/3612 Primary

  by runtime analysis (performance monitoring G06F11/3466) · CPC title

• G06F11/3684

  for test design, e.g. generating new test cases · CPC title

• G06F11/3664

  Physics · mapped topic