Method and system for software application testing recommendations

Patent metadata

Field	Value
Publication number	US-10459828-B2
Application number	US-201715596706-A
Country	US
Kind code	B2
Filing date	May 16, 2017
Priority date	Apr 22, 2015
Publication date	Oct 29, 2019
Grant date	Oct 29, 2019

Abstract

A method for application testing recommendation includes deploying a software application to first testers having first system configurations, and receiving testing coverage data describing (i) a first subset of features of the software application being tested by the first testers and (ii) information describing the first system configurations. The method further includes deploying the software application to a marketplace, and receiving market coverage data describing (i) a second subset of features being used by market users and (ii) second system configurations of the market users. The method further includes comparing the market coverage data and the testing coverage data to identify a coverage discrepancy, selecting second testers to test the software application based on the coverage discrepancy, and deploying the software application to the second testers.

First claim

What is claimed is:

1. A computer-implemented method comprising:
receiving, through a software developer user interface, data associated with a software application to test;
determining that a criteria is satisfied by a difference between (i) testing coverage data that describes an amount of feature testing of the software application and system configurations of different systems executing the application software and (ii) market coverage data that describes a second amount of feature usage of the software application and a second set of system configurations of different systems executing the application software; and
in response to determining that the criteria is satisfied by the difference between (i) the testing coverage data that describes the amount of feature testing of the software application and the system configurations of different systems executing the application software and (ii) the market coverage data that describes the second amount of feature usage of the software application and the second set of system configurations of different systems executing the application software, providing, on the software developer user interface, one or more controls that are each associated with a respective recommended additional user, from among a set of additional users, to test the software application;
receiving, through the software developer user interface, a selection of a particular control that is associated with a particular recommended additional user; and
providing the software application to the particular recommended additional user to test in response to receiving the selection of the particular control.

2. The computer-implemented method of claim 1 , wherein the difference between (i) testing coverage data that describes an amount of feature testing of the software application and system configurations of different systems executing the application software and (ii) market coverage data that describes a second amount of feature usage of the software application and a second set of system configurations of different systems executing the application software, reflects a coverage discrepancy that indicates a portion of the market coverage data that is not in the testing coverage data.

3. The computer-implemented method of claim 2 , wherein the coverage discrepancy exists in response to determining that a portion of the market coverage data that is not in the testing coverage data satisfies a predetermined threshold.

4. The computer-implemented method of claim 2 , wherein the selection of the particular control that is associated with the particular recommended tester is in response to obtaining the particular recommended tester based on a type of difference in the coverage discrepancy.

5. The computer-implemented method of claim 2 , further comprising:
identifying defects during the test of an execution of the software application by the respective recommended tester; and
providing a recommendation to correct the coverage discrepancy based on the identified defects during the test of the execution of the software application.

6. The computer-implemented method of claim 2 , wherein the comparison of (i) the testing coverage data that describes the amount of feature testing on the different types of the system configures of the software application, and (ii) the market coverage data that describes the amount of the feature usage on the different types of the system configurations of the software application further comprises:
determining analytic data types associated with the testing coverage data and the market coverage data; and
determining a mapping to a numerical data format in response to determining at least one of the analytic data types is of a subjective data format.

7. The computer-implemented method of claim 1 , further comprising:
receiving real time analytics during the test of the execution of the software application by the particular recommended tester, wherein the real time analytics describe time-based data, geographic based data, defect based data, and input/output data associated with the execution of the software application.

8. A system comprising:
one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
receiving, through a software developer user interface, data associated with a software application to test;
determining that a criteria is satisfied by a difference between (i) testing coverage data that describes an amount of feature testing of the software, application and system configurations of different systems executing the application software and (ii) market coverage data that describes a second amount of feature usage of the software application and a second set of system configurations of different systems executing the application software; and
in response to determining that the criteria is satisfied by the difference between (i) the testing coverage data that describes the amount of feature testing of the software application and the system configurations of different systems executing the application software and (ii) the market coverage data that describes the second amount of feature usage of the software application and the second set of system configurations of different systems executing the application software, providing, on the software developer user interface, one or more controls that are each associated with a respective recommended additional user, from among a set of additional users, to test the software application;
receiving, through the software developer user interface, a selection of a particular control that is associated with a particular recommended additional user; and
providing the software application to the particular recommended additional user to test in response to receiving the selection of the particular control.

9. The system of claim 8 , wherein the difference between (i) testing coverage data that describes an amount of feature testing of the software application and system configurations of different systems executing the application software and (ii) market coverage data that describes a second amount of feature usage of the software application and a second set of system configurations of different systems executing the application software, reflects a coverage discrepancy that indicates a portion of the market coverage data that is not in the testing coverage data.

10. The system of claim 9 , wherein the coverage discrepancy exists in response to determining that a portion of the market coverage data that is not in the testing coverage data satisfies a predetermined threshold.

11. The system of claim 9 , wherein the selection of the particular control that is associated with the particular recommended tester is in response to obtaining the particular recommended tester based on a type of difference in the coverage discrepancy.

12. The system of claim 9 , wherein the operations further comprise:
identifying defects during the test of an execution of the software application by the respective recommended tester; and
providing a recommendation to correct the coverage discrepancy based on the identified defects during the test of the execution of the software application.

13. The system of claim 9 , wherein the comparison of (i) the testing coverage data that describes the amount of feature testing on the different types of the system configures of the software application, and (ii) the market coverage data that describes the amount of the feature usage on the different types of the system configurations of the

Assignees

• Google Llc

Inventors

• Seibert Jr Jeffrey
• Chang Wayne

Classifications

• G06F11/368

  for test version control, e.g. updating test cases to a new software version · CPC title

• G06F11/3676 Primary

  for coverage analysis · CPC title