Tuning sandbox behavior based on static characteristics of malware

Patent metadata

Field	Value
Publication number	US-9355246-B1
Application number	US-201314098488-A
Country	US
Kind code	B1
Filing date	Dec 5, 2013
Priority date	Dec 5, 2013
Publication date	May 31, 2016
Grant date	May 31, 2016

Abstract

An emulator on a host computer includes a static analysis module that analyzes executable code of a suspicious sample to determine whether the code identifies that a particular packing program (packer) has packed the sample. Once identified, a custom configuration file is generated that identifies particular API hooks or instructions that should be disabled (or enabled) so that the sample file cannot use these hooks or instructions to detect that it is executing within an emulator. The emulator (such as a virtual machine or sandbox) is configured using the configuration file. The suspicious sample is then executed and its behaviors are collected. The sample is prevented from detecting that it is operating within an emulator and thus prevented from terminating prematurely. Malicious behaviors are scored and a total score indicates whether or not the suspicious sample is malicious or not. Static analysis identifies signatures, instructions or strings.

First claim

We claim:

1. A method of collecting behaviors of a suspicious file, said method comprising:
receiving said suspicious file on a host computer, said suspicious file including executable code and suspected of being malicious, and wherein said suspicious file is a file in portable executable format;
analyzing said executable code before execution and identifying a packing program that has packed said executable code, and wherein said executable code has been compressed or encrypted by said packing program;
disabling a software hook in an emulator of said host computer based upon said identified packing program;
executing said suspicious file within said emulator of said host computer after said disabling; and
collecting behaviors of said suspicious file while said suspicious file is executing within said emulator, said executing suspicious file not prematurely terminating by virtue of not detecting use of said software hook in said emulator.

2. The method as recited in claim 1 wherein said executable code includes a routine to detect said emulator of said host computer utilizing said software hook when said software hook is enabled.

3. The method as recited in claim 1 further comprising:
identifying said packing program by identifying a signature, a string or an instruction in said executable code used by said packing program.

4. The method as recited in claim 1 further comprising:
modifying a configuration file to identify said software hook;
inputting said configuration file into said emulator.

5. The method as recited in claim 1 wherein said emulator is a virtual machine.

6. A method of collecting behaviors of a suspicious file, said method comprising:
receiving said suspicious file on a host computer, said suspicious file including executable code and suspected of being malicious, and wherein said suspicious file is a file in portable executable format;
analyzing said executable code before execution and identifying a packing program that has packed said executable code, and wherein said executable code has been compressed or encrypted by said packing program;
disabling an instruction in an emulator of said host computer based upon said identified packing program;
executing said suspicious file within said emulator of said host computer after said disabling; and
collecting behaviors of said suspicious file while said suspicious file is executing within said emulator, said executing suspicious file not prematurely terminating by virtue of not detecting use of said instruction in said emulator.

7. The method as recited in claim 6 wherein said executable code includes a routine to detect said emulator of said host computer utilizing said instruction when said instruction is enabled.

8. The method as recited in claim 6 further comprising:
identifying said packing program by identifying a signature, a string or an instruction in said executable code used by said packing program.

9. The method as recited in claim 6 further comprising:
modifying a configuration file to identify said instruction;
inputting said configuration file into said emulator.

10. The method as recited in claim 6 wherein said emulator is a virtual machine.

11. A method of collecting behaviors of a suspicious file, said method comprising:
receiving said suspicious file on a host computer, said suspicious file including executable code and suspected of being malicious, and wherein said suspicious file is a file in portable executable format;
analyzing said executable code before execution and identifying a packing program that has packed said executable code, and wherein said executable code has been compressed or encrypted by said packing program;
enabling a function in an emulator of said host computer based upon said identified packing program;
executing said suspicious file within said emulator of said host computer after said enabling; and
collecting behaviors of said suspicious file while said suspicious file is executing within said emulator, said executing suspicious file not prematurely terminating by virtue of detecting use of said function in said emulator.

12. The method as recited in claim 11 wherein said executable code includes a routine to detect said emulator of said host computer not utilizing said function when said function is disabled.

13. The method as recited in claim 11 further comprising:
identifying said packing program by identifying a signature, a string or an instruction in said executable code used by said packing program.

14. The method as recited in claim 11 further comprising:
modifying a configuration file to identify said instruction;
inputting said configuration file into said emulator.

15. The method as recited in claim 11 wherein said emulator is a virtual machine.

Assignees

• Wan Xiaochuan
• Huang Ben
• Chen Xuebin
• Huang Xiaodong
• Fan Hailiang
• Trend Micro Inc

Inventors

• Wan Xiaochuan
• Huang Ben
• Chen Xuebin
• Huang Xiaodong
• Fan Hailiang

Classifications

• G06F21/53 Primary

  by executing in a restricted environment, e.g. sandbox or secure virtual machine · CPC title

• H04L63/1408

  by monitoring network traffic (monitoring network traffic per se H04L43/00) · CPC title

• H04L63/1416

  Event detection, e.g. attack signature detection · CPC title

• H04L63/1441

  Countermeasures against malicious traffic (countermeasures against attacks on cryptographic mechanisms H04L9/002) · CPC title

• G06F21/566

  Dynamic detection, i.e. detection performed at run-time, e.g. emulation, suspicious activities · CPC title