Automated test case generation for applications

Patent metadata

Field	Value
Publication number	US-9268672-B1
Application number	US-201414287902-A
Country	US
Kind code	B1
Filing date	May 27, 2014
Priority date	May 27, 2014
Publication date	Feb 23, 2016
Grant date	Feb 23, 2016

Abstract

Some implementations include receiving an application binary file for an application to be tested. One or more static analysis operations may be performed on the application binary file to identify application parameters. In some cases, keywords may be associated with individual application parameters, and the keywords may be used to query a test case repository in order to identify test cases. The identified test cases may be used to generate a test plan, and at least a portion of the test plan may be automatically executed in some cases. A test report may be generated that includes a list of test case failures and potential solutions, and the test report may be sent e.g., to a third-party developer or an approval engineer for review.

First claim

What is claimed is:

1. A system comprising:
one or more processors;
one or more computer-readable media coupled to the one or more processors;
an analysis module maintained on the one or more computer-readable media and executed on the one or more processors to:
receive, for an application to be tested, an application binary file from an electronic device;
perform one or more static analysis operations on the application binary file to determine one or more application parameters, wherein the application parameters represent one or more features that are supported or used by the application, wherein the one or more features include at least one of a camera feature, a microphone feature, a maps feature, an accelerometer feature, a performance feature, or an in-application purchase feature;
determine one or more keywords that are associated with individual application parameters of the one or more application parameters; and
determine one or more test cases based on the one or more keywords; and

a test module maintained on the one or more computer-readable media and executed on the one or more processors to:
receive the one or more test cases;
generate a test plan based on the one or more test cases;
execute at least a portion of the test plan;
generate a test report that includes (i) a list of test cases that failed during execution and (ii) one or more potential solutions associated with individual test cases that failed; and
send the test report to the electronic device.

2. The system as recited in claim 1 , wherein performing the one or more static analysis operations on the application binary file includes determining at least one of: a version of an operating system of the electronic device, one or more software development kit (SDK) versions, one or more user permissions, one or more layout objects, or one or more application programming interfaces (APIs).

3. The system as recited in claim 1 , wherein the test module is further executed on the one or more processors to:
receive, via a test plan generation user interface, information associated with at least one of:
one or more devices upon which the application is to be tested;
individual features of the one or more features to be tested; or
one or more objects to be tested, wherein the one or more objects include at least one of a timer object, a button object, a zoom-in object, a zoom-out object, a scrolling object, a text box object, or an Internet object; and

receiving, via the test plan generation user interface, an indication to generate the test plan based at least in part on the information.

4. The system as recited in claim 1 , wherein generating the test report includes:
generating a summary of a failed test case;
generating a list of steps to reproduce the failed test case;
generating an observed result of the failed test case; and
generating an expected result of the failed test case.

5. The system as recited in claim 4 , wherein generating the test report further includes:
determining recommendation information that includes a possible cause of the failed test case;
determining software code that is usable by a developer to address the failed test case; and
sending the recommendation information and the sample code to the electronic device for display to the developer.

6. A method comprising:
under control of one or more processors,
receiving, for an application to be tested, an application binary file from an electronic device;
performing one or more static analysis operations on the application binary file to determine one or more application parameters;
determining one or more keywords that are associated with individual application parameters of the one or more application parameters;
determining one or more test cases based on the one or more keywords; and
generating a test plan based on the one or more test cases.

7. The method as recited in claim 6 , wherein the one or more application parameters include at least one of:
a user permission to be used by the application;
an application programming interface (API) to be used by the application;
an API version associated with the API to be used by the application;
a software development kit (SDK) version;
a hardware feature of the electronic device to be used by the application;
a software feature of the electronic device to be used by the application;
a layout object to be used by the application; or
style details to be used by the application.

8. The method as recited in claim 6 , further comprising:
executing at least one test case of the one or more test cases; and
generating a test report that includes results associated with execution of the at least one test case.

9. The method as recited in claim 8 , wherein the test report identifies a problem encountered during execution of the at least one test case and information regarding a potential solution to the problem.

10. The method as recited in claim 9 , wherein the information includes software code usable to address the problem.

11. The method as recited in claim 6 , wherein the one or more test cases include at least one test case that is to be executed manually on the electronic device.

12. The method as recited in claim 11 , further comprising:
receiving, via a network, information from the electronic device that includes a result of executing the at least one test case; and
storing the result in a test case repository.

13. The method of claim 6 , further comprising:
receiving, from the electronic device, information associated with an additional test case to be performed, wherein the electronic device includes a first type of electronic device and wherein the additional test case is to be performed on a second type of electronic device; and
updating the test plan to include the additional test case.

14. The method of claim 6 , further comprising sending the test plan to the electronic device, wherein the test plan identifies an expected behavior associated with execution of the one or more test cases.

15. One or more non-transitory computer-readable media maintaining instructions executable by one or more processors to perform operations comprising:
receiving, for an application to be tested, an application binary file;
performing one or more static analysis operations on the application binary file to determine one or more application parameters;
determining one or more keywords that are associated with individual application parameters of the one or more application parameters;
querying a test case repository to identify one or more test cases that are associated with the one or more keywords;
executing, without user input, at least one test case of the one or more test cases; and
generating a test report that includes results associated with execution of the at least one test case.

16. The one or more non-transitory computer-readable media as recited in claim 15 , wherein:
the test report includes information associated with a problem encountered during execution of the at least one test case; and
the test report includes information regarding one or more potential solutions to the problem.

17. The one or more non-transitory computer-readable media as recited in claim 16 , wherein the information includes software code usable to address the problem.

18. The one or more non-transitory computer-readable media as recited in claim 17 , wherein:
the software code is generated by a tester associated with an application store; and
the software code is usable by a third-party developer to

Assignees

• Amazon Tech Inc

Inventors

• Gupta Suresh Chandra

Classifications

• G06F11/3698

  Environments for analysis, debugging or testing of software · CPC title

• G06F11/3688

  for test execution, e.g. scheduling of test suites · CPC title

• G06F11/3684 Primary

  for test design, e.g. generating new test cases · CPC title

• G06F11/3664

  Physics · mapped topic