Using machine learning to optimize wireless carrier system testing

Patent metadata

Field	Value
Publication number	US-11812290-B2
Application number	US-202117555770-A
Country	US
Kind code	B2
Filing date	Dec 20, 2021
Priority date	Dec 20, 2021
Publication date	Nov 7, 2023
Grant date	Nov 7, 2023

Abstract

Described herein are techniques, devices, and systems for using a machine learning model(s) and/or artificial intelligence algorithm(s) to optimize testing of components of a system operated by a wireless carrier. For example, data generated as a result of executing a first test of a suite of tests may be provided as input to a trained machine learning model(s) to classify one or more tests of the suite of tests as having a particular characteristic. A to-be-executed test may be classified as likely to pass or likely to fail when executed, for example. An already-executed test may be classified as reliable or unreliable, as another example. Based on the classification of the test(s), the suite of tests may be modified to optimize testing of the wireless carrier's system.

First claim

We claim:

1. A computer-implemented method comprising:
executing a first test of a suite of tests that are to be executed for testing one or more components of a system operated by a wireless carrier;
providing, as input to a trained machine learning model, data generated as a result of executing the first test;
classifying, using the trained machine learning model, and based at least in part on the data, a second test of the suite of tests as a test that is likely to pass when executed;
and modifying the suite of tests by refraining from executing the second test;
wherein the data comprises performance data indicative of a performance of the one or more components; wherein the performance data indicates whether the one or more components interacted with a predefined set of other components or systems as a result of executing the first test.

2. The computer-implemented method of claim 1 , wherein the data comprises performance data indicative of a performance of the one or more components.

3. The computer-implemented method of claim 2 , wherein the performance data comprises a duration of the first test measured from a start of the first test to a completion of the first test.

4. The computer-implemented method of claim 1 , wherein the data comprises test result data indicative of the first test having passed or failed.

5. The computer-implemented method of claim 1 , wherein the one or more components comprise a device having a display, and wherein the data comprises image data representing a screenshot of a user interface displayed on the display of the device.

6. The computer-implemented method of claim 1 , wherein the modifying of the suite of tests by refraining from executing the second test comprises at least one of:
skipping the second test and generating a test result indicative of the second test having assumed to have been passed; or
removing the second test from the suite of tests.

7. The computer-implemented method of claim 1 , wherein each test of the suite of tests are automated tests that are executed by executing a script.

8. The computer-implemented method of claim 1 , wherein each test of the suite of tests is executed to test at least one of an availability of the one or more components or a performance of the one or more components.

9. A computer-implemented method comprising:
executing a first test of a suite of tests that are to be executed for testing one or more components of a system operated by a wireless carrier;
providing, as input to a trained machine learning model, data generated as a result of executing the first test;
classifying, using the trained machine learning model, and based at least in part on the data, the first test as unreliable; and
modifying the suite of tests based at least in part on the first test being classified as unreliable;
wherein the data comprises performance data indicative of a performance of the one or more components; wherein the performance data indicates whether the one or more components interacted with a predefined set of other components or systems as a result of executing the first test.

10. The computer-implemented method of claim 9 , wherein the data comprises performance data indicative of a performance of the one or more components.

11. The computer-implemented method of claim 10 , wherein the performance data comprises a response time of the one or more components.

12. The computer-implemented method of claim 10 , further comprising:
determining, based on the performance data, that a performance metric of the one or more components fails to satisfy a threshold; and
generating an alert based at least in part on the performance metric failing to satisfy the threshold.

13. The computer-implemented method of claim 9 , wherein the data comprises test result data indicative of the first test having passed or failed.

14. The computer-implemented method of claim 9 , wherein the one or more components comprise a device having a display, and wherein the data comprises image data representing a screenshot of a user interface displayed on the display of the device.

15. The computer-implemented method of claim 9 , wherein the modifying of the suite of tests comprises at least one of:
removing the first test from the suite of tests; or
increasing an execution frequency of a second test of the suite of tests.

16. A system comprising:
one or more processors; and
memory storing computer-executable instructions that, when executed by the processor, cause performance of operations comprising:
executing a first test of a suite of tests that are to be executed for testing one or more components of a system operated by a wireless carrier;
providing, as input to a trained machine learning model, data generated as a result of executing the first test;
classifying, using the trained machine learning model, and based at least in part on the data, a second test of the suite of tests as a test that is likely to pass when executed; and
modifying the suite of tests by refraining from executing the second test;
wherein the data comprises performance data indicative of a performance of the one or more components; wherein the performance data indicates whether the one or more components interacted with a predefined set of other components or systems as a result of executing the first test.

17. The system of claim 16 , wherein the modifying of the suite of tests by refraining from executing the second test comprises at least one of:
skipping the second test and generating a test result indicative of the second test having assumed to have been passed; or
removing the second test from the suite of tests.

18. The system of claim 16 , wherein the data comprises test result data indicative of the first test having passed or failed.

Assignees

• T Mobile Usa Inc

Inventors

• Myron Peter P
• Mitchell Michael J

Classifications

• H04W24/06 Primary

  Testing, {supervising or monitoring} using simulated traffic · CPC title