Binary program instrumentation and debugging

Patent metadata

Field	Value
Publication number	US-9817745-B2
Application number	US-201514595901-A
Country	US
Kind code	B2
Filing date	Jan 13, 2015
Priority date	Jan 13, 2014
Publication date	Nov 14, 2017
Grant date	Nov 14, 2017

Abstract

Disclosed is a system and method for setting breakpoints (BPs) that cannot be detected by code being debugged, e.g., to trap transparently at any instruction. The disclosed system and method permits execution tracing and instrumentation of binaries, including malware. The system and method can be used to analyze benign software also, e.g., software obfuscated for anti-reverse-engineering. An invisible breakpoint (IBP) is not detectable by the “guest,” the program being debugged. Hardware virtualization is used to split the code and data views of the program's text segment(s).

First claim

The invention claimed is:

1. A method of setting a breakpoint at an address in a guest software program in a memory of a virtualized data-processing system, the method comprising automatically performing the following steps using a processor:
a) locating a division of memory corresponding to the address;
b) creating a copy of at least part of the contents of the division;
c) modifying the contents of an instruction set of the division to set a breakpoint at the address;
d) configuring an environment of the guest software program to permit executable but not readable access to the division; and
e) installing an access violation handler so that:
i) when the guest software program attempts to read at least some of the contents of the modified division, the processor will provide a data view using the copy of the division and configure the environment to permit readable but not executable access to the data view in place of the division; and
ii) when the guest software program attempts to execute at least some of the contents of the division, the processor will configure the environment to provide a code view using the division having the modified contents which grants executable but not readable access.

2. The method according to claim 1 , wherein the permission settings of the division is not visible to the guest software program.

3. The method according to claim 2 , wherein a permission setting feature of the processor running the software is used to configure the permissions settings.

4. The method according to claim 1 , further comprising:
a) using the processor, intercepting at least one breakpoint event, said interception is invisible to the guest software program.

5. The method of claim 4 , further comprising:
a) using the processor, executing at least one breakpoint handler in the host, said executing invisible to the guest software program.

6. The method of claim 1 , wherein the guest software program is a malware program.

7. A system, comprising:
a) a computer processor;
b) a memory storage device coupled to the processor and comprising computer readable instructions for executing a method of setting a breakpoint at an address in a guest software program in the memory, the method comprising:
i) locating a division of memory corresponding to the address;
ii) creating a copy of at least part of the contents of the division;
iii) modifying the contents of an instruction set of the division to set a breakpoint at the address;
iv) configuring an environment of the guest software program to permit executable but not readable access to the division; and
v) installing an access violation handler so that:
A) when the guest software program attempts to read at least some of the contents of the modified division, the processor will provide a data view using the copy of the division and configure the environment to permit readable but not executable access to the data view in place of the division; and
B) when the guest software program attempts to execute at least some of the contents of the division, the processor will configure the environment to provide a code view using the division having the modified contents which grants executable but not readable access.

8. The system of claim 7 , wherein the permission settings of the division is not visible to the guest software program.

9. The system of claim 8 , wherein a permission setting feature of the processor running the software is used to configure the permissions settings.

10. The system of claim 7 , the method further comprising:
a) using the processor, intercepting at least one breakpoint event, said interception is invisible to the guest software program.

11. The system of claim 10 , the method further comprising:
a) using the processor, executing at least one breakpoint handler in the host, said executing invisible to the guest software program.

12. The system of claim 7 , wherein the guest software program is a malware program.

Assignees

• Purdue Research Foundation

Inventors

• Zhang Xiangyu
• Deng Zhui
• Xu Dongyan

Classifications

• G06F11/3636

  by tracing the execution of the program · CPC title

• G06F2221/033

  Test or assess software · CPC title

• G06F11/362 Primary

  Debugging of software · CPC title

• G06F21/566 Primary

  Dynamic detection, i.e. detection performed at run-time, e.g. emulation, suspicious activities · CPC title