System and methods for amalgamation of artificial intelligence (AI) and machine learning (ML) in test creation, execution, and prediction

Patent metadata

Field	Value
Publication number	US-11221942-B2
Application number	US-202016820615-A
Country	US
Kind code	B2
Filing date	Mar 16, 2020
Priority date	Apr 25, 2019
Publication date	Jan 11, 2022
Grant date	Jan 11, 2022

Abstract

Systems and methods provided for a software test automation platform including an artificial intelligence (AO/machine learning (ML) test analyzer. The AI/ML test analyzer can be a computer device operating with the software testing automation platform that is programmed to obtain run-time data from executing multiple software test cases for a software workload; obtain log data from the one or more test logs, relating to previous executions of the tests cases for the software workload. The AI/ML test analyzer can then analyze the log data and the run-time data by applying ML/AI tools, concurrently with executing the test cases, for recognizing patterns associated with the executed tests cases over a period of time, and automatically predicting an optimized set of test cases and analytics related to execution of the test cases for the software workload.

First claim

What is claimed is:

1. A system, comprising:
and
one or more processors; and
a computer-readable storage device coupled to the one or more processors and having instructions stored thereon which, when executed by the one or more processors cause the one or more processors to perform operations of:
obtaining run-time data during executing multiple test cases for a software workload;
obtaining test logs from the one or more storage devices, wherein test log data from each of the test logs is related to previous executions of the tests cases for the software workload;
automatically analyzing, before completing a software testing process, the test log data and the run-time data by applying AI/ML for recognizing patterns associated with the executed tests cases over a period of time;
automatically predicting, based on the AI/ML tools analyses, an optimized set of test cases and analytics related to execution of the test cases for the software workload;
communicating the predicted optimized set of test cases and analytics to the software test automation platform;
initiating execution of the predicted optimized set of test cases and analytics;
obtaining run-time data from execution of the predicted optimized set of test cases and analytics; and
applying AI/ML to update, prior to completion of the predicted optimized set of test cases, the predicted optimized set of test cases based on the run-time data obtained during execution of the predicted optimized set of test cases.

2. The system of claim 1 , wherein the computer-readable storage device further causes the one or more processors to perform:
determining automated recommended actions relating to a sequence of test cases within the optimized set of test cases.

3. The system of claim 2 , wherein the automated recommended actions comprise at least one of: ignoring a test case; adding a new test; and applying an updated sequence to test cases within the optimized set of test cases for the software workload.

4. The system of claim 1 , further comprising a user interface communicatively connected to the software test automation platform.

5. The system of claim 3 , wherein the user interface obtains input for approving or denying the automated recommended actions.

6. The system of claim 1 , wherein the software test automation platform executes the multiple test cases for the software workload such that the software run-time data is generated.

7. The system of claim 6 , wherein the AI/ML test analyzer automatically analyzes the test log data and the run-time data concurrently as the software test automation platform executes the multiple test cases for the software workload.

8. The system of claim 7 , wherein the AI/ML test analyzer automatically predicts the automated recommended actions concurrently as the software test automation platform executes the multiple test cases for the software workload; and communicates the predicted optimized set of test cases and analytics to the software test automation platform prior to completing execution of the multiple test cases for the software workload.

9. The system of claim 1 , wherein the computer-readable storage device further causes the one or more processors to perform:
automatically predicting one or more defect yielding test cases; and
automatically adding the one or more defect yielding test cases to the optimized set of test cases for the software workload.

10. The system of claim 1 , wherein the computer-readable storage device further causes the one or more processors to perform:
obtaining one or more defined patterns relating to logs of interest; and
analyzing the test log data from the one or more storage devices based on the defined patterns to selectively collect test logs data from the logs of interest, and prior to automatically analyzing the test log data by applying AI/ML; and
communicating the test log data only from the logs of interest for automatically analyzing by applying AI/ML.

11. A non-transitory computer-readable storage medium having executable instructions stored thereon that, when executed by a processor, causes the processor to perform operations of:
obtaining run-time data during an execution of multiple test cases for a software workload;
obtaining test log data from multiple test logs, wherein the log data from each of the test logs is related to previous executions of the tests cases for the software workload;
automatically analyzing, before completing a software testing process, the log data and the run-time data by applying AI/ML for recognizing patterns associated with the executed tests cases over a period of time;
automatically predicting, based on the AI/ML analyses, an optimized set of test cases and analytics related to execution of the test cases for the software workload;
communicating the predicted optimized set of test cases and analytics to a software test automation platform;
initiating execution of the predicted optimized set of test cases and analytics;
obtaining run-time data from execution of the predicted optimized set of test cases and analytics; and
applying AI/ML to update, prior to completion of the predicted optimized set of test cases, the predicted optimized set of test cases based on the run-time data obtained during execution of the predicted optimized set of test cases.

12. The non-transitory computer-readable storage medium of claim 11 , wherein the instructions, when executed by the processor, further cause the processor to perform:
determining automated recommended actions relating to a sequence of test cases within the optimized set of test cases.

13. The non-transitory computer-readable storage medium of claim 12 , wherein the automated recommended actions comprise at least one of: ignoring a test case; adding a new test; and applying an updated sequence to test cases within the optimized set of test cases for the software workload.

14. The non-transitory computer-readable storage medium of claim 11 , wherein the instructions, when executed by the processor, further cause the processor to perform:
presenting a user interface for interacting with the software test automation platform.

15. The non-transitory computer-readable storage medium of claim 13 , wherein the user interface obtains input for approving or denying the automated recommended actions.

16. The non-transitory computer-readable storage medium of claim 11 , wherein the run-time data is obtained from a software test automation platform executing the multiple test cases for the software workload.

17. The non-transitory computer-readable storage medium of claim 16 , wherein automatically analyzing the test log data and the run-time data occurs concurrently as the software test automation platform executes the multiple test cases for the software workload.

18. The non-transitory computer-readable storage medium of claim 17 , wherein automatically predicting the automated recommended actions occurs concurrently as the software test automation platform executes the multiple test cases for the software workload, and communicating the predicted optimized set of test cases and analytics to the software test automation platform occurs prior to completing execution of the multiple test cases for the software workload.

19. The non-transitory computer-readable storage medium of claim 18 , wherein the instructions, when executed by the processor, further cause the processor to perform:
automatically predicting one or more defect yielding test cases; and
automatically adding the one or more defect yi

Assignees

• Hewlett Packard Entpr Dev Lp

Inventors

• Sridhar Pavan
• Panambur Deepak
• Panambur Rohith

Classifications

• G06F11/3476

  Data logging (G06F11/14, G06F11/2205 take precedence) · CPC title

• G06N20/00 Primary

  Machine learning · CPC title

• G06F11/3688

  for test execution, e.g. scheduling of test suites · CPC title

• G06F11/3692 Primary

  for test results analysis · CPC title

• G06F11/3672

  Test management · CPC title