Risk-focused testing

Patent metadata

Field	Value
Publication number	US-11119895-B2
Application number	US-201916543635-A
Country	US
Kind code	B2
Filing date	Aug 19, 2019
Priority date	Aug 19, 2019
Publication date	Sep 14, 2021
Grant date	Sep 14, 2021

Abstract

A method, apparatus and product for testing a system under development. The method comprises obtaining information including a current version of a code, and predicting that a subset of code elements from the current version of the code are bug-prone elements. The method comprises determining one or more coverage events corresponding to the bug-prone elements. The method further comprises determining a testing policy based on the one or more coverage events that correspond to the bug-prone elements, wherein the testing policy is determined based on a statistical analysis of coverage likelihood of tests generated based on a test template for each coverage event of the one or more coverage events. The method further comprises implementing the testing policy.

First claim

What is claimed is:

1. A method comprising:
iteratively performing:
obtaining information comprising a current version of a code, the information comprising fault-related information indicating at least previously detected faults in a previous version of the code, the information comprising test-related information indicating at least previously performed tests of the previous version of the code;
based on a machine learning algorithm, predicting that a subset of code elements from the current version of the code are bug-prone elements;
determining one or more coverage events corresponding to the bug-prone elements;
determining a testing policy based on the one or more coverage events that correspond to the bug-prone elements, wherein said testing policy is determined based on a statistical analysis of coverage likelihood of tests generated based on a test template for each coverage event of the one or more coverage events, wherein the testing policy comprises a plurality of test templates and execution parameters thereof, wherein the testing policy is configured to statistically cover the one or more coverage events; and
implementing the testing policy, wherein said implementing comprises generating one or more tests based on the plurality of test templates, thereby covering the one or more coverage events,

wherein a first testing policy for a first set of bug-prone elements determined during a first iteration comprises different test templates than a second testing policy for a second set of bug-prone elements determined during a second iteration.

2. The method of claim 1 , wherein the information is obtained from at least one of a version control system, a bug tracking tool, and a test submission tool.

3. The method of claim 1 , wherein a new iteration is invoked every time a new coding element is committed.

4. The method of claim 1 , wherein a version control system is used to track versions of the code, wherein said iteratively performing comprises performing a new iteration in response to a commit event in the version control system, whereby each iteration is configured to test a committed version of the code.

5. The method of claim 1 , wherein a new iteration is invoked periodically, whereby performing adaptive regression testing.

6. The method of claim 1 , wherein the machine learning algorithm is configured to be trained on a training dataset comprising coding information, fault-related information, and test-related information obtained during a predefined period of time.

7. The method of claim 5 , wherein each element in the training dataset is labeled based on having at least one fault in a past period or not.

8. The method of claim 1 , wherein the bug-prone elements comprises at least one of a bug-prone file and a bug-prone function.

9. The method of claim 1 , wherein the one or more coverage events comprise one or more functional coverage events.

10. The method of claim 1 , wherein the fault-related information comprises a file having a previously detected fault and a timestamp of detecting the fault.

11. The method of claim 1 , wherein the test-related information comprises a count of performed tests and a count of failed tests.

12. A computer program product comprising a non-transitory computer readable storage medium retaining program instructions, which program instructions when read by a processor, cause the processor to iteratively perform:
obtaining information comprising a current version of a code, the information comprising fault-related information indicating at least previously detected faults in a previous version of the code, the information comprising test-related information indicating at least previously performed tests of the previous version of the code;
based on a machine learning algorithm, predicting that a subset of code elements from the current version of the code are bug-prone elements;
determining one or more coverage events corresponding to the bug-prone elements;
determining a testing policy based on the one or more coverage events that correspond to the bug-prone elements, wherein said testing policy is determined based on a statistical analysis of coverage likelihood of tests generated based on a test template for each coverage event of the one or more coverage events, wherein the testing policy comprises a plurality of test templates and execution parameters thereof, wherein the testing policy is configured to statistically cover the one or more coverage events; and
implementing the testing policy, wherein said implementing comprises generating one or more tests based on the plurality of test templates, thereby covering the one or more coverage events,
wherein a first testing policy for a first set of bug-prone elements determined during a first iteration comprises different test templates than a second testing policy for a second set of bug-prone elements determined during a second iteration.

13. The computer program product of claim 12 , wherein the information is obtained from at least one of a version control system, a bug tracking tool, and a test submission tool.

14. The computer program product of claim 12 , wherein a new iteration is invoked every time a new coding element is committed.

15. The computer program product of claim 12 , wherein a version control system is used to track versions of the code, wherein said iteratively performing comprises performing a new iteration in response to a commit event in the version control system, whereby each iteration is configured to test a committed version of the code.

16. The computer program product of claim 12 , wherein a new iteration is invoked periodically, whereby performing adaptive regression testing.

17. The computer program product of claim 12 , wherein the machine learning algorithm is configured to be trained on a training dataset comprising coding information, fault-related information, and test-related information obtained during a predefined period of time.

18. The computer program product of claim 12 , wherein the bug-prone elements comprises at least one of a bug-prone file and a bug-prone function.

19. The computer program product of claim 12 , wherein the one or more coverage events comprise one or more functional coverage events.

20. A computerized apparatus having a processor, the processor being adapted to iteratively perform:
obtaining information comprising a current version of a code, the information comprising fault-related information indicating at least previously detected faults in a previous version of the code, the information comprising test-related information indicating at least previously performed tests of the previous version of the code;
based on a machine learning algorithm, predicting that a subset of code elements from the current version of the code are bug-prone elements;
determining one or more coverage events corresponding to the bug-prone elements;
determining a testing policy based on the one or more coverage events that correspond to the bug-prone elements, wherein said testing policy is determined based on a statistical analysis of coverage likelihood of tests generated based on a test template for each coverage event of the one or more coverage events, wherein the testing policy comprises a plurality of test templates and execution parameters thereof, wherein the testing policy is configured to statistically cover the one or more coverage events; and
implementing the testing policy, wherein said implementing comprises generating one or more tests based on the plur

Assignees

• IBM

Inventors

• Gal Raviv
• Shurek Gil
• Simchoni Giora
• Ziv Avi

Classifications

• G06F8/71 Primary

  Version control (security arrangements therefor G06F21/57); Configuration management · CPC title

• G06N5/01

  Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound · CPC title

• G06F18/2148

  characterised by the process organisation or structure, e.g. boosting cascade · CPC title

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title