Computing resource coverage

Patent metadata

Field	Value
Publication number	US-11366743-B2
Application number	US-201916268988-A
Country	US
Kind code	B2
Filing date	Feb 6, 2019
Priority date	Feb 6, 2019
Publication date	Jun 21, 2022
Grant date	Jun 21, 2022

Abstract

Computing resource coverage is disclosed. In an example, a computing resource (CR) pool includes a plurality of CR types hosted on a plurality of hosts, where the plurality of CR types are routinely updated. A test repository stores a plurality of test cases (TC). A processor is configured to execute a testing service to compile, from a plurality of test daemons, a CR manifest of the CR types included in the CR pool. A TC manifest is compiled, including CR types tested by the plurality of TCs. The CR types included in the CR manifest are compared with the CR types included in the TC manifest. A test coverage report of tested and untested CR types is generated. A TC is added to the test repository based on the test coverage report.

First claim

The invention is claimed as follows:

1. A method comprising:
compiling, from a plurality of test daemons each deployed to a respective virtual machine (VM), a computing resource (CR) manifest of CR types included in a CR pool hosted on a plurality of hosts, wherein a plurality of CR types include CR types that are unavailable on any individual host and are pooled together from multiple hosts, and the plurality of CR types are routinely updated;
compiling a test case (TC) manifest including CR types tested by a plurality of TCs stored in a test repository, wherein at least one TC is determined to be mislabeled by parsing source code of the at least one TC to determine whether an application programming interface (API) is invoked for testing allocation of at least one CR type that the TC is labeled for on the respective VM, and each TC of the plurality of TCs determined to be mislabeled is omitted from the TC manifest;
comparing the CR types included in the CR manifest with the CR types included in the TC manifest;
generating a test coverage report of tested and untested CR types; and
adding a TC to the test repository based on at least one untested CR type included in the test coverage report.

2. A system comprising:
a computing resource (CR) pool that includes a plurality of CR types hosted on a plurality of hosts, wherein the plurality of CR types include CR types that are unavailable on any individual host and are pooled together from multiple hosts, and the plurality of CR types are routinely updated;
a test repository storing a plurality of test cases (TCs); and
a processor configured to execute a testing service to:
compile, from a plurality of test daemons each deployed to a respective virtual machine (VM), a CR manifest of the CR types included in the CR pool;
compile a TC manifest including CR types tested by the plurality of TCs, wherein at least one TC is determined to be mislabeled by parsing source code of the at least one TC to determine whether an application programming interface (API) is invoked for testing allocation of at least one CR type that the TC is labeled for on the respective VM, and each TC of the plurality of TCs determined to be mislabeled is omitted from the TC manifest;
compare the CR types included in the CR manifest with the CR types included in the TC manifest;
generate a test coverage report of tested and untested CR types; and
add a TC to the test repository based on at least one untested CR type included in the test coverage report.

3. The system of claim 2 , wherein compiling the TC manifest includes determining respective CR types tested by each TC in the plurality of TCs.

4. The system of claim 3 , wherein the testing service is configured to:
parse a source code of a TC to determine a list of API calls invoked by the TC; and
determine the CR types tested by the TC by comparing the list of API calls invoked by the TC to a master list mapping associations between CR types and API calls.

5. The system of claim 4 , wherein the testing service identifies that the TC fails to invoke an API call corresponding to a CR type that the TC is labeled for testing, and the testing service is configured to:
generate an alert based on the lack of the API call; and
remove the CR type from the TC manifest in relation to the TC.

6. The system of claim 3 , wherein the testing service identifies a subset of TCs out of the plurality of TCs that test, in combination, each of the CR types represented in the TC manifest, executes each of the TCs in the subset of TCs, and reports results of executing the subset of TCs.

7. The system of claim 2 , wherein a CR type is one of a networking resource, a memory resource, a storage resource, a processor resource, or a graphical processor resource.

8. The system of claim 2 , wherein the plurality of test daemons is deployed to a plurality of endpoints associated with the CR pool.

9. The system of claim 2 , wherein the plurality of CR types is updated based on an update to a CR pool management system of the CR pool, and the testing service is configured to regenerate the CR manifest and the test coverage report based on the updated plurality of CR types.

10. The system of claim 2 , wherein updating the plurality of CR types includes at least one of adding a new CR type or removing an existing CR type.

11. The system of claim 10 , wherein a plurality of existing CR types are combined into the new CR type.

12. The system of claim 10 , wherein CRs represented by the existing CR type are recategorized as a plurality of new CR types.

13. A system comprising:
a computing resource (CR) pool that includes a plurality of CR types hosted on a plurality of hosts, wherein the plurality of CR types include CR types that are unavailable on any individual host and are pooled together from multiple hosts, and the plurality of CR types are routinely updated;
a plurality of test daemons each deployed to a respective virtual machine (VM);
a test repository storing a plurality of test cases (TCs); and
a processor configured to execute a testing service to:
compile a TC manifest including CR types tested by the plurality of TCs, wherein at least one TC is determined to be mislabeled by parsing source code of the at least one TC to determine whether an application programming interface (API) is invoked for testing allocation of at least one CR type that the TC is labeled for on the respective VM, and each TC of the plurality of TCs determined to be mislabeled is omitted from the TC manifest;
execute, against the CR pool, a subset of TCs of the plurality of TCs that in combination tests each of the CR types represented in the TC manifest;
record a test result report based on execution results of the subset of TCs;
determine at least one untested CR type of the CR pool that the subset of TCs fails to test;
generate a test coverage report including the at least one untested CR type; and
add a TC to the test repository that tests the at least one untested CR type.

14. The system of claim 13 , wherein compiling the TC manifest includes determining respective CR types tested by each TC in the plurality of TCs, and metadata associated with a TC of the plurality of TCs stores at least one CR type tested by the TC.

15. The system of claim 14 , wherein the testing service is configured to:
parse a source code of a TC to determine a list of API calls invoked by the TC;
determine the CR types tested by the TC by comparing the list of API calls invoked by the TC to a master list mapping associations between CR types and API calls; and
update metadata associated with the TC after evaluating the source code of the TC.

16. The system of claim 15 , wherein the testing service identifies that the TC fails to invoke an API call corresponding to a CR type that the TC is labeled for testing, and the testing service is configured to:
generate an alert based on the lack of the API call; and
remove the CR type from the TC manifest in relation to the TC.

17. The system of claim 13 , wherein a CR type is one of a networking resource, a memory resource, a storage resource, a processor resource, or a graphical processor resource.

18. The system of claim 13 , wherein the plurality of test daemons is deployed to a plurality of endpoints associated with the CR pool, and a test daemon of the plurality of test daemons executes a TC of the subset of TCs.

19. The system of claim 13 , wherein the plurality of CR types is updated based on an update to a CR pool management system of the CR pool, and t

Assignees

• Red Hat Israel Ltd

Inventors

• Bregman Arie
• Kuris Eran
• Stafeyev Alexander

Classifications

• G06F8/36 Primary

  Software reuse · CPC title

• G06F11/3676 Primary

  for coverage analysis · CPC title

• Y02D10/00

  Energy efficient computing, e.g. low power processors, power management or thermal management · CPC title

• G06F9/5027

  the resource being a machine, e.g. CPUs, Servers, Terminals · CPC title

• G06F11/3684

  for test design, e.g. generating new test cases · CPC title