Method and system for software application testing recommendations

Patent metadata

Field	Value
Publication number	US-9727448-B1
Application number	US-201514693438-A
Country	US
Kind code	B1
Filing date	Apr 22, 2015
Priority date	Apr 22, 2015
Publication date	Aug 8, 2017
Grant date	Aug 8, 2017

Abstract

A method for application testing recommendation includes deploying a software application to first testers having first system configurations, and receiving testing coverage data describing (i) a first subset of features of the software application being tested by the first testers and (ii) information describing the first system configurations. The method further includes deploying the software application to a marketplace, and receiving market coverage data describing (i) a second subset of features being used by market users and (ii) second system configurations of the market users. The method further includes comparing the market coverage data and the testing coverage data to identify a coverage discrepancy, selecting second testers to test the software application based on the coverage discrepancy, and deploying the software application to the second testers.

First claim

What is claimed is:

1. A method for application testing recommendation, comprising:
deploying a software application to a first plurality of testers having a first plurality of system configurations;
receiving testing coverage data describing (i) a first subset of a plurality of features of the software application being tested by the first plurality of testers and (ii) information describing the first plurality of system configurations;
deploying the software application to a marketplace;
receiving, from an analytics software development kit (SDK) embedded in the software application, market coverage data describing (i) a second subset of the plurality of features of the software application being used by a plurality of market users and (ii) a second plurality of system configurations of the plurality of market users;
comparing the market coverage data and the testing coverage data to identify a coverage discrepancy;
selecting a second plurality of testers to test the software application based on the coverage discrepancy; and
deploying the software application to the second plurality of testers.

2. The method of claim 1 , wherein selecting the second plurality of testers comprises obtaining a unique identifier of each tester in the second plurality of testers.

3. The method of claim 1 , wherein comparing the market coverage data and the testing coverage data comprises determining an overlap between the first plurality of system configurations and the second plurality of system configurations.

4. The method of claim 1 , wherein comparing the market coverage data and the testing coverage data comprises:
for each system configuration in the second plurality of system configurations:
determining a first proportion of the system configuration used by the plurality of market users;
determining a second proportion of the system configuration used by the plurality of testers; and
determining whether the first proportion is within a threshold difference to the second proportion.

5. The method of claim 1 , wherein comparing the market coverage data and the testing coverage data comprises determining whether the first subset of the plurality of features comprises the second subset of the plurality of features.

6. The method of claim 1 , wherein comparing the market coverage data and the testing coverage data comprises:
for each feature in the second subset of the plurality of features:
determining a first proportion of the feature used by the plurality of market users to a total usage of the second subset of the plurality of features;
determining a second proportion of the feature used by the plurality of testers to a total usage of the first subset of the plurality of features; and
determining whether the first proportion is within a threshold difference to the second proportion.

7. The method of claim 1 , further comprising:
generating a dashboard for displaying a comparison of the market coverage data and the testing coverage data; and
displaying the dashboard.

8. The method of claim 7 , further comprising:
generating a rating based on comparing the market coverage data and the testing coverage data; and
incorporating the rating into the dashboard.

9. A system for software application testing recommendation, comprising:
a computer processsor;
a data repository for storing information describing a plurality of system configurations and testing coverage data; and
a testing service configured to execute on the computer processor for:
deploying a software application to a first plurality of testers having a first plurality of system configurations,
an analytics service configured to execute on the computer processor for:
receiving testing coverage data describing (i) a first subset of a plurality of features of the software application being tested by the first plurality of testers and (ii) information describing the first plurality of system configurations, and
receiving, from an analytics software development kit (SDK) embedded in the software application, market coverage data describing (i) a second subset of the plurality of features of the software application being used by a plurality of market users and (ii) a second plurality of system configurations of the plurality of market users, and

a recommendation module configured to execute on the computer processor for:
comparing the market coverage data and the testing coverage data to identify a coverage discrepancy, and
selecting a second plurality of testers to test the software application based on the coverage discrepancy.

10. The system of claim 9 , further comprising:
a developer user interface configured to display a dashboard and the second plurality of testers.

11. The system of claim 9 , wherein comparing the market coverage data and the testing coverage data comprises determining whether the first plurality of system configurations comprises the second plurality of system configurations.

12. The system of claim 9 , wherein comparing the market coverage data and the testing coverage data comprises:
for each system configuration in the second plurality of system configurations:
determining a first proportion of the system configuration used by the plurality of market users;
determining a second proportion of the system configuration used by the plurality of testers; and
determining whether the first proportion is within a threshold difference to the second proportion.

13. The system of claim 9 , wherein the testing service is further for:
receiving defect information from a testing software development kit (SDK) embedded in the software application deployed to the plurality of testers, and
adding the defect information to the testing coverage data.

14. The system of claim 9 , wherein the testing coverage data received by the analytics service is received from the analytics SDK embedded in the software application.

15. A non-transitory computer readable medium for software application testing recommendation comprising computer readable program code, the computer readable program code for:
deploying a software application to a first plurality of testers having a first plurality of system configurations;
receiving testing coverage data describing (i) a first subset of a plurality of features of the software application being tested by the first plurality of testers and (ii) information describing the first plurality of system configurations;
deploying the software application to a marketplace;
receiving, from an analytics software development kit (SDK) embedded in the software application, market coverage data describing (i) a second subset of the plurality of features of the software application being used by a plurality of market users and (ii) a second plurality of system configurations of the plurality of market users;
comparing the market coverage data and the testing coverage data to identify a coverage discrepancy;
selecting a second plurality of testers to test the software application based on the coverage discrepancy; and
deploying the software application to the second plurality of testers.

16. The non-transitory computer readable medium of claim 15 , wherein comparing the market coverage data and the testing coverage data comprises determining whether the first subset of the plurality of features comprises the second subset of the plurality of features.

17. The non-transitory computer readable medium of claim 15 , wherein comparing the market coverage data and the testing coverage data comprises:
for each feature in the second subset of the plur

Assignees

• Google Inc

Inventors

• Seibert Jr Jeffrey
• Chang Wayne

Classifications

• G06F11/368

  for test version control, e.g. updating test cases to a new software version · CPC title

• G06F11/3676 Primary

  for coverage analysis · CPC title

• G06F11/3692

  for test results analysis · CPC title