Automatically running tests against WEB APIs based on specifications

Patent metadata

Field	Value
Publication number	US-10409711-B2
Application number	US-201715620769-A
Country	US
Kind code	B2
Filing date	Jun 12, 2017
Priority date	Jun 12, 2017
Publication date	Sep 10, 2019
Grant date	Sep 10, 2019

Abstract

A method and system of determining whether a specification is an accurate representation of an application program interface (API) is provided. The specification is received electronically over a network. Service calls to be tested are identified based on the specification. A test case is created for each of the identified service calls. A sequence is created for the test cases. A test plan is generated based on the created sequence. The generated test plan is executed. Upon identifying an error in response to the executed test plan, a notification is generated, indicating that the specification is not an accurate representation of the API.

First claim

What is claimed is:

1. A computing device comprising:
a processor;
a network interface coupled to the processor to enable communication over a network;
a storage device coupled to the processor;
a compliance engine software stored in the storage device, wherein an execution of the software by the processor configures the computing device to perform acts comprising:
receiving an electronic specification defining an application program interface (API), over the network;
identifying service calls to be tested based on the specification;
creating a test case for each of the identified service calls;
creating a sequence for the created test cases, for an efficient correlation between the specification and the API, wherein the sequence for the created test cases comprises:
automatically identifying one or more Hypertext Transfer Protocol (HTTP) invocations comprising at least one of: POST, GET, PUT, PATCH, or DELETE invocation;
automatically ordering any identified POST invocation before any GET, PUT, PATCH, and DELETE invocation; and
automatically ordering any DELETE invocation after any POST, GET, PUT, and PATCH invocation;

generating a test plan based on the created sequence;
executing the generated test plan; and
upon identifying an error in response to the executed test plan, generating a notification indicating that the specification is not an accurate representation of the API.

2. The computing device of claim 1 , wherein identifying service calls to be tested comprises parsing a text of the specification to identify Web service calls and one or more input parameters associated with each Web service call.

3. The computing device of claim 1 , wherein creating a test case for each of the identified service calls comprises identifying one or more threshold criteria for an error.

4. The computing device of claim 3 , wherein the one or more threshold criteria comprises at least one of: (i) receiving a Hypertext Transfer Protocol (HTTP) error code over the network, or (ii) a discrepancy between an expected data and a received data, in response to the execution of the test case.

5. The computing device of claim 1 , wherein generating the test plan comprises creating an executable source code.

6. The computing device of claim 1 , wherein generating the test plan comprises translating the sequence of the created test cases in a format that a Web service testing application understands.

7. The computing device of claim 1 , wherein executing the generated test plan comprises, for each test case:
sending a Hypertext Transfer Protocol (HTTP) requests to the API;
receiving a response to the HTTP request; and
storing the response in the storage device.

8. The computing device of claim 1 , wherein execution of the compliance engine by the processor further configures the computing device to perform an act comprising: upon identifying no errors in response to the executed test plan, certifying that the specification is an accurate representation of the API.

9. The computing device of claim 1 , wherein execution of the compliance engine by the processor further configures the computing device to perform acts comprising, upon identifying an error in response to the executed test plan:
identifying a portion of the specification related to the error; and
indicating the portion of the specification in the generated notification.

10. The computing device of claim 9 , wherein execution of the compliance engine by the processor further configures the computing device to perform acts comprising, upon identifying an error in response to the executed test plan, sending the notification to a developer of the API.

11. A non-transitory computer readable storage medium tangibly embodying a computer readable program code having computer readable instructions that, when executed, causes a computer device to carry out a method of determining whether a specification is an accurate representation of an application program interface (API), the method comprising:
receiving the specification electronically, over a network;
identifying service calls to be tested based on the specification;
creating a test case for each of the identified service calls;
creating a sequence for the created test cases, for an efficient correlation between the specification and the API, wherein the sequence for the created test cases comprises:
automatically identifying one or more Hypertext Transfer Protocol (HTTP) invocations comprising at least one of: POST, GET, PUT, PATCH, or DELETE invocation;
automatically ordering any identified POST invocation before any GET, PUT, PATCH, and DELETE invocation; and
automatically ordering any DELETE invocation after any POST, GET, PUT, and PATCH invocation;

generating a test plan based on the created sequence;
executing the generated test plan; and
upon identifying an error in response to the executed test plan, generating a notification indicating that the specification is not an accurate representation of the API.

12. The non-transitory computer readable storage medium of claim 11 , wherein identifying service calls to be tested comprises parsing a text of the specification to identify Web service calls and one or more input parameters associated with each Web service call.

13. The non-transitory computer readable storage medium of claim 11 , wherein creating a test case for each of the identified service calls comprises identifying one or more threshold criteria for an error.

14. The non-transitory computer readable storage medium of claim 13 , wherein the one or more threshold criteria comprises at least one of: (i) receiving a Hypertext Transfer Protocol (HTTP) error code over the network, or (ii) a discrepancy between an expected data and a received data, in response to the execution of the test case.

15. The non-transitory computer readable storage medium of claim 11 , wherein generating the test plan comprises creating an executable source code.

16. The non-transitory computer readable storage medium of claim 11 , wherein executing the generated test plan comprises, for each test case:
sending a Hypertext Transfer Protocol (HTTP) requests to the API;
receiving a response to the HTTP request; and
storing the response in the storage device.

17. The non-transitory computer readable storage medium of claim 11 , further comprising, upon identifying no errors in response to the executed test plan, certifying that the specification is an accurate representation of the API.

18. The non-transitory computer readable storage medium of claim 11 , further comprising, upon identifying an error in response to the executed test plan:
identifying a portion of the specification related to the error; and
indicating the portion of the specification in the generated notification.

Assignees

• IBM

Inventors

• Dolby Julian Timothy
• Laredo Jim Alain
• Slominski Aleksander
• Wittern John Erik
• Ying Annie T
• Young Christopher
• Zheng Yunhui

Classifications

• G06F11/3692

  for test results analysis · CPC title

• G06F11/3684 Primary

  for test design, e.g. generating new test cases · CPC title

• G06F2201/81

  Threshold · CPC title

• G06F11/3688

  for test execution, e.g. scheduling of test suites · CPC title

• G06F11/3664

  Physics · mapped topic