Comparing software projects having been analyzed using different criteria

Patent metadata

Field	Value
Publication number	US-10346294-B2
Application number	US-201715485034-A
Country	US
Kind code	B2
Filing date	Apr 11, 2017
Priority date	Apr 11, 2017
Publication date	Jul 9, 2019
Grant date	Jul 9, 2019

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for comparing software projects having been analyzed using different criteria. One of the methods includes receiving, for each of a plurality of software projects, source code evaluation criteria that had been used to analyze source code of the respective software project. An overlapping set of source code evaluation criteria is determined. For each of the software projects, source code analysis results which resulted from the overlapping set of source code evaluation criteria are determined, and a respective value of a characteristic metric for the source code analysis results is computed. The respective values of the characteristic metric for each of the software projects are compared, and for at least one of the software projects, an assessment of the software project is output.

First claim

What is claimed is:

1. A computer-implemented method for ranking software projects according to a violation metric computed over violations identified by respective static analysis processes using different sets of violation rules over different respective time periods without reanalyzing the software projects, the method comprising:
performing a first static analysis process on a first software project during a first time period, including:
performing a first build process for the first software project and extracting source code of the first software project by intercepting calls to compilers invoked during the first build process, and
using a first set of violation rules to identify violations occurring in extracted source code belonging to respective snapshots of the first software project;

storing first static analysis result data representing the violations identified in the respective snapshots of the first software project by the first static analysis process;
receiving one or more updated violation rules and adding the one or more updated violation rules to the first set of violation rules to generate an updated set of violation rules;
performing a second static analysis process on a different second software project during a second time period subsequent to the first time period, including:
performing a second build process for the second software project and extracting source code of the second software project by intercepting calls to compilers invoked during the second build process, and
using the updated set of violation rules to identify violations occurring in extracted source code belonging to respective snapshots of the second software project;

storing second static analysis result data representing the violations identified in the respective snapshots of the second software project by the second static analysis process;
during a third time period subsequent to the second time period, ranking a plurality of software projects including the first software project and the second software project using respective static analysis result data previously generated for each software project including using the first static analysis result data for the first software project and the second static analysis result data for the second software project without reanalyzing any of the plurality of software projects, including:
receiving a request that identifies the plurality of software projects to be ranked according to a violation metric that is representative of the respective prevalence of violations occurring in each software project, wherein each software project of the plurality of software projects is a software project for which respective violations were previously identified by a static analysis process according to a respective set of violation rules;
receiving respective static analysis result data generated before the request was received, the static analysis result data representing respective violations identified as occurring in each of the plurality of software projects by the static analysis process using a respective violation rule to identify violations of a particular type among a plurality of different types of violations;
obtaining, for each software project of the plurality of software projects, a respective set of violation rules that the static analysis process used to identify violations occurring in the software project;
computing a set of overlapping violation rules for the plurality of software projects identified by the request, the set of overlapping violation rules comprising one or more violation rules that the static analysis process evaluated against each software project of the plurality of software projects identified by the request;
classifying each violation occurring in the plurality of software projects as being either (i) a violation that was identified by the static analysis process by evaluating a nonoverlapping violation rule that does not occur in the set of overlapping violation rules, or (ii) a violation that was identified by the static analysis process by evaluating an overlapping violation rule that occurs in the set of overlapping violation rules;
computing, for each software project of the plurality of software projects, a respective updated value of the violation metric by using only violations classified as being identified by overlapping violation rules for the plurality of software projects identified by the request; and
ranking the plurality of software projects according to the respective updated values of the violation metric computed using only violations identified by overlapping violation rules.

2. The method of claim 1 , further comprising:
computing an updated value of the violation metric for the first software project by using violations identified according to the set of overlapping violation rules instead of the first set of violation rules used by the first static analysis process when the first software project was analyzed.

3. The method of claim 1 , wherein computing, for each software project of the plurality of software projects, a respective updated value of the violation metric comprises computing the respected updated value by disregarding violations identified from nonoverlapping violation rules.

4. The method of claim 1 , wherein computing, for each software project of the plurality of software projects, a respective updated value of the violation metric comprises computing the respected updated value without identifying any new violations in the software project.

5. The method of claim 1 , wherein the violation metric is specific to a particular violation rule category of a plurality of violation rule categories, and further comprising providing, for each of the plurality of software projects, a respective category rating for the particular violation rule category.

6. The method of claim 5 , wherein the plurality of violation rule categories include one or more of a security category, a readability category, an efficiency category, or a concurrency category.

7. The method of claim 1 , wherein each of the plurality of software projects was analyzed using a different respective set of violation rules.

8. The method of claim 1 , further comprising:
partitioning the computed updated values of the violation metric for the overlapping violation rules into a plurality of percentile ranges; and
computing a respective rating of each software project based on a percentile range into which the software project falls according to the respective updated value of the violation metric computed using the overlapping violation rules.

9. The method of claim 1 , wherein the violation metric is a total number of violations or a violation density.

10. The method of claim 1 , wherein the violation metric is a net number of violations introduced over a particular time period.

11. A system comprising:
one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations for ranking software projects according to a violation metric identified by respective static analysis processes using different sets of violation rules over different respective time periods without reanalyzing the software projects, the operations comprising:
performing a first static analysis process on a first software project during a first time period, including:
performing a first build process for the first software project and extracting source code of the first software project by intercepting calls to compilers invoked during the first build process, and
using a first set of violation rules to identify violations occu

Assignees

• Semmle Ltd

Inventors

• Van Schaik Sebastiaan Johannes

Classifications

• G06F11/3692 Primary

  for test results analysis · CPC title

• G06F11/3616 Primary

  using software metrics · CPC title