Testing a code using real time analysis

Patent metadata

Field	Value
Publication number	US-11288174-B2
Application number	US-202016795209-A
Country	US
Kind code	B2
Filing date	Feb 19, 2020
Priority date	Feb 19, 2020
Publication date	Mar 29, 2022
Grant date	Mar 29, 2022

Abstract

A method, non-transitory computer readable medium and a computerized system for testing a code using real time analysis. The method can include (i) executing a group of test cases while performing real time analysis to find a set of overlapping code segments (OCSs), input values that are fed, during the executing of the group, to each one of the set of OCSs, and output values that are outputted from each one of the OCSs during the execution of the group, (ii) generating, for at least some of the OCSs, at least one OCS test for testing each of the at least some OCSs, wherein the generating is based, at least in part, on the input values and the output values, (iii) determining an evaluation process of the code that includes executing one or more OCS tests for testing one or more OCSs, (iv) evaluating the code by executing the evaluation process.

First claim

What is claimed is:

1. A method for testing a code using real time analysis, the method comprising:
executing a group of test cases while performing real time analysis to find: (a) a set of overlapping code segments, (b) input values that are fed, during the executing of the group of test cases, to each one of the set of overlapping code segments, and (c) output values that are outputted from each one of the overlapping code segments during the execution of the group of test cases; wherein each overlapping code segment is independently testable and accessed during an execution of two or more test cases of the group of test cases;
generating, for at least some of the overlapping code segments, at least one overlapping code segment test for testing each of the at least some overlapping code segments; wherein the generating is based, at least in part, on: (i) the input values that are fed, during the executing of the group of test cases, to the each one of the overlapping code segments, and on (ii) the output values that are outputted from the each one of the overlapping code segments, during the execution of the group of test cases, wherein generating the at least one overlapping code segment test comprises feeding an overlapping code segment at least some of the input values that were fed to the group of test cases during the executing step;
determining an evaluation process of the code that comprises executing one or more overlapping code segment tests for testing one or more overlapping code segments;
evaluating the code by executing the evaluation process;
determining expected output values to be outputted during an execution of an overlapping code segment test, based at least in part, on output values that were outputted from the one or more overlapping code segments, during execution of the group of test cases in the executing step; and
identifying a test failure if, when the overlapping code segment test is executed, the test output values deviate substantially from the expected output values.

2. The method according to claim 1 further comprising evaluating the code by executing the one or more overlapping code segment tests instead of executing two or more test cases that include the one or more overlapping code segment tests.

3. The method according to claim 1 further comprising evaluating the code by executing the one or more overlapping code segment tests, and upon a failure of the one or more overlapping code segment tests avoiding from executing two or more test cases that include the one or more overlapping code segments.

4. The method according to claim 1 wherein a duration of the executing of one or more overlapping code segment tests is shorter that an aggregate duration of the executing of the two or more test cases.

5. The method according to claim 1 further comprising generating the one or more overlapping code segment tests to comprise feeding to the one or more overlapping code segments at least some of the input values that were fed, during the executing of the group of test cases, to the one or more overlapping code segments.

6. The method according to claim 5 further comprising determining expected output values to be outputted during an execution of the one or more overlapping code segment tests, based at least in part, on output values that were outputted from the one or more overlapping code segments, during the execution of the group of test cases.

7. The method according to claim 1 wherein the at least some of the overlapping code segments are all of the overlapping code segments.

8. The method according to claim 1 wherein the evaluating of the code comprises executing an overlapping code segment test for testing each one of the at least some of the overlapping code segments, wherein the at least some of the overlapping code segments comprise the code segment.

9. The method according to claim 1 wherein the overlapping code segment is a function.

10. The method according to claim 1 further comprising ranking at least some of the overlapping code segments based on numbers of test cases of the group of test cases that once executed accessed the overlapping code segments.

11. The method according to claim 10 wherein the ranking is also based on at least one parameter that differs from the numbers of test cases of the group of test cases that once executed accessed the overlapping code segments.

12. The method according to claim 11 wherein the at least one parameter is a number of bugs found in the test cases of the group.

13. The method according to claim 11 wherein the at least one parameter is priorities assigned to the test cases of the group.

14. The method according to claim 10 further comprising prioritizing an execution of at least some of the overlapping code segments tests based on the ranking of the at least some of the overlapping code segments tested by the at least some of the overlapping code segments tests.

15. The method according to claim 1 wherein the group of test cases comprises all tests cases associated with the code.

16. The method according to claim 1 further comprising executing multiple iterations of the evaluating of the code per each single executing of the group of test cases.

17. The method according to claim 1 wherein the evaluating of the code is included in regression testing of the code.

18. A non-transitory computer readable medium that stores instructions for:
executing a group of test cases while performing real time analysis to find: (a) a set of overlapping code segments, (b) input values that are fed, during the executing of the group of test cases, to each one of the set of overlapping code segments, and (c) output values that are outputted from each one of the overlapping code segments during the execution of the group of test cases; wherein each overlapping code segment is independently testable and accessed during an execution of two or more test cases of the group of test cases;
generating, for at least some of the overlapping code segments, at least one overlapping code segment test for testing each of the at least some overlapping code segments; wherein the generating is based, at least in part, on: (i) the input values that are fed, during the executing of the group of test cases, to the each one of the overlapping code segments, and on (ii) the output values that are outputted from the each one of the overlapping code segments, during the execution of the group of test cases, wherein generating the at least one overlapping code segment test comprises feeding an overlapping code segment at least some of the input values that were fed to the group of test cases during the executing step;
determining an evaluation process of the code that comprises executing one or more overlapping code segment tests for testing one or more overlapping code segments;
evaluating the code by executing the evaluation process;
determining expected output values to be outputted during an execution of an overlapping code segment test, based at least in part, on output values that were outputted from the one or more overlapping code segments, during execution of the group of test cases in the executing step; and
identifying a test failure if, when the overlapping code segment test is executed, the test output values deviate substantially from the expected output values.

19. A computerized system that comprises a test runner, a monitor, a test builder, and a code evaluator;
wherein the test runner is configured to execute a group of test cases;
wherein the monitor is configured

Assignees

• Applied Materials Israel Ltd

Inventors

• Levi Elad
• Herskovits Moshe

Classifications

• G06F11/368

  for test version control, e.g. updating test cases to a new software version · CPC title

• G06F11/3688 Primary

  for test execution, e.g. scheduling of test suites · CPC title

• G06F11/3692

  for test results analysis · CPC title

• G06F11/3684

  for test design, e.g. generating new test cases · CPC title

• G06F11/3676

  for coverage analysis · CPC title