Method to verify correctness of computer system software and hardware components and corresponding test environment

Patent metadata

Field	Value
Publication number	US-10241888-B2
Application number	US-201314082543-A
Country	US
Kind code	B2
Filing date	Nov 18, 2013
Priority date	Mar 18, 2013
Publication date	Mar 26, 2019
Grant date	Mar 26, 2019

Abstract

A method is presented to verify correctness of computer system software and hardware components. The method includes: operating a test environment with a verified system software and hardware version; monitoring and recording each hardware access during operation of the test environment with the verified system software and hardware version to generate a corresponding verified trace file; operating the test environment with a modified system software and/or hardware version; monitoring and recording each hardware access to generate a corresponding new trace file during operation of the test environment with the modified system software and/or hardware version; defining an arbitrary order for target chips in the verified and the modified hardware model or hardware system version; sorting sequences of entries in both trace files according to the target chip order; and comparing the sorted trace files by comparing their entries each by each and outputting a corresponding comparison result.

First claim

What is claimed is:

1. A method of facilitating computer system processing, the method comprising:
enhancing computer system test processing by establishing functional correctness of computer system components by comparing in operation a modified system version of a computer system with a verified system version of the computer system which functions correctly within a test environment, the test environment including an execution environment for running the modified system version and for running the verified system version, which are different system versions, each system version comprising computer software components connected to computer hardware components, the computer hardware components comprising multiple target chips, the multiple target chips comprising a number of target chips which is consistent between each version, the establishing functional correctness being performed within the test environment and comprising:
operating said test environment with said verified system version;
monitoring and recording each hardware access during operation of said test environment with said verified system version to generate a corresponding verified trace file;
operating said test environment with said modified system version;
monitoring and recording each hardware access to generate a corresponding new trace file during operation of said test environment with said modified system version;
defining an arbitrary chip order for evaluation of operations to the multiple target chips in said verified system version and the multiple target chips of the modified system version;
sorting sequences of entries in both trace files according to said arbitrary chip order, including splitting the trace files into separate chip-based trace files;
comparing said sorted, chip-based trace files by comparing, for each chip, their entries each by each to compare patterns of operations to the multiple target chips between the verified system version and the modified system version, and thus initialization sequences of the multiple target chips using the verified system version and the modified system version; and
identifying functional correctness of the computer system components based on initialization sequences of the multiple target chips in the verified system version and in the modified system version matching.

2. The method according to claim 1 , wherein a first type of execution environment runs verified system software of the verified system version and uses at least one hardware procedure to initialize at least one corresponding target chip in said verified system version during operation of said test environment with said verified system version.

3. The method according to claim 1 , wherein a second type of execution environment runs modified system software of the modified system version and uses at least one hardware procedure to initialize at least one corresponding target chip in said verified system version during operation of said test environment with said modified system version.

4. The method according to claim 1 , wherein a first type of execution environment runs verified system software of the verified system version and uses at least one hardware procedure to initialize at least one corresponding target chip in said modified system version during operation of said test environment with said verified system version.

5. The method according to claim 4 , wherein said verified trace file and said new trace file each comprise operation type, operation data, target chip identification and timing information as entries for each hardware access.

6. The method according to claim 1 , wherein said verified trace file and said new trace file or said verified separate trace files and said new separate trace files are sorted according to timing information.

7. The method according to claim 6 , wherein said timing information is removed from said verified trace file and said new trace file or said verified separate trace files and said new separate trace files after sorting.

8. The method of claim 1 , wherein:
said verified trace file and said new trace file each comprise timing information as entries for each hardware access;
said sorting further comprising sorting entries in the separate chip-based trace files according to said timing information; and
subsequent to the sorting, removing the timing information from the separate chip-based trace files.

9. A system of facilitating computer system processing, the system comprising:
a memory; and
a processor communicatively coupled to the memory, the system performing:
enhancing computer system test processing by establishing functional correctness of computer system components by comparing in operation a modified system version of a computer system with a verified system version of the computer system which functions correctly within a test environment, the test environment including an execution environment for running the modified system version and for running the verified system version, which are different system versions, each version comprising computer software components connected to computer hardware components, the computer hardware components comprising multiple target chips, the multiple target chips comprising a number of target chips which is consistent between each version, the establishing functional correctness being performed within the test environment and comprising:
operating said test environment with said verified system version;
monitoring and recording each hardware access during operation of said test environment with said verified system version to generate a corresponding verified trace file;
operating said test environment with said modified system version;
monitoring and recording each hardware access to generate a corresponding new trace file during operation of said test environment with said modified system version;
defining an arbitrary chip order for evaluation of operations to the multiple target chips in said verified system version and the multiple target chips of the modified system version;
sorting sequences of entries in both trace files according to said arbitrary chip order, including splitting the trace files into separate chip-based trace files;
comparing said sorted, chip-based trace files by comparing, for each chip, their entries each by each to compare patterns of operations to the multiple target chips between the verified system version and the modified system version, and thus initialization sequences of the multiple target chips using the verified system version and the modified system version; and
identifying functional correctness of the computer system components based on initialization sequences of the multiple target chips in the verified system version and in the modified system version matching.

10. The system according to claim 9 , wherein the system further performs writing the trace files to a data storage.

11. The system according to claim 10 , wherein a first type of execution environment runs verified system software of the verified system version and uses at least one hardware procedure to initialize at least one corresponding target chip of said multiple target chips.

12. The system according to claim 10 , wherein a second type of execution environment runs modified system software of the modified system version and uses at least one hardware procedure to initialize at least one corresponding target chip of said multiple target chips.

13. The system according to claim 12 , wherein said verified trace file and said new trace file each comprise the following entries: information to identify said affected target chip; information about a

Assignees

• IBM

Inventors

• Schaufler Ralf
• Senner Tobias

Classifications

• G01R31/318357

  Simulation (computer simulation of digital circuits G06F30/3308) · CPC title

• G06F11/261

  by simulating additional hardware, e.g. fault simulation · CPC title

• G06F11/3476

  Data logging (G06F11/14, G06F11/2205 take precedence) · CPC title

• G06F30/327 Primary

  Logic synthesis; Behaviour synthesis, e.g. mapping logic, HDL to netlist, high-level language to RTL or netlist · CPC title

• G01R31/3193

  with comparison between actual response and known fault free response {(receiver details G01R31/31924)} · CPC title