Efficient program deobfuscation through system API instrumentation

Patent metadata

Field	Value
Publication number	US-10956570-B2
Application number	US-201916725748-A
Country	US
Kind code	B2
Filing date	Dec 23, 2019
Priority date	Sep 11, 2017
Publication date	Mar 23, 2021
Grant date	Mar 23, 2021

Abstract

Techniques for efficient program deobfuscation through system application program interface (API) instrumentation are disclosed. In some embodiments, a system/process/computer program product for efficient program deobfuscation through system API instrumentation includes monitoring changes in memory after a system call event during execution of a malware sample in a computing environment; and generating a signature based on an analysis of the monitored changes in memory after the system call event during execution of the malware sample in the computing environment.

First claim

What is claimed is:

1. A system, comprising:
a processor configured to:
perform dynamic analysis of a malware sample for program deobfuscation through system API instrumentation in a computing environment, wherein a plurality of pages in memory associated with a process launched by executing the malware sample are identified and monitored for changes after one or more system call events during execution of the malware sample for a predetermined period of time in the computing environment, and wherein an output of the monitored changes in memory after the system call event during execution of the malware sample for a predetermined period of time in the computing environment are reassembled and analyzed to identify a potential malware binary; and
generate an interface that includes a graphical visualization of a plurality of pages in memory associated with a process launched during execution of the malware sample in the computing environment, wherein the graphical visualization of the plurality of pages in memory indicates a subset of the plurality of pages in memory that were modified after a system call event that was intercepted through the system API instrumentation in the computing environment; and

a memory coupled to the processor and configured to provide the processor with instructions.

2. The system recited in claim 1 , wherein the computing environment comprises a virtual machine instance.

3. The system recited in claim 1 , wherein the processor is further configured to:
identify, in the interface, the system call event detected during execution of the malware sample in the computing environment.

4. The system recited in claim 1 , wherein the processor is further configured to:
monitor changes in memory after the system call event during execution of the malware sample in the computing environment, wherein monitoring changes in memory after the system call event during execution of the malware sample in the computing environment further comprises:
identify a plurality of pages in memory associated with a process launched by executing the malware sample in the computing environment;
perform an initial snapshot of all of the plurality of pages in memory associated with the process at initial time to and cache the initial snapshot of all of the plurality of pages in memory to provide a baseline for contents in memory while executing the malware sample in the computing environment;
perform another snapshot of all of the plurality of pages in memory associated with the process at subsequent time to after the system call event in response to detecting a return address in a call stack points to a memory address that has changed since the initial snapshot;
reassemble snapshots of the plurality of pages in memory associated with the process after executing the malware sample for a predetermined period of time in the computing environment; and
automatically detect unpacking of binaries in memory based on a static analysis of the reassembled snapshots of the plurality of pages in memory; and

a memory coupled to the processor and configured to provide the processor with instructions.

5. The system recited in claim 1 , wherein the potential malware binary is submitted for dynamic analysis and/or static analysis.

6. The system recited in claim 1 , wherein the processor is further configured to:
receive a plurality of malware samples; and
deduplicate the plurality of malware samples.

7. The system recited in claim 1 , wherein the processor is further configured to:
receive a plurality of malware samples;
deduplicate the plurality of malware samples to output a first malware sample; and
execute the first malware sample in the computing environment.

8. A method, comprising:
performing dynamic analysis of a malware sample for program deobfuscation through system API instrumentation in a computing environment, wherein a plurality of pages in memory associated with a process launched by executing the malware sample are identified and monitored for changes after one or more system call events during execution of the malware sample for a predetermined period of time in the computing environment, and wherein an output of the monitored changes in memory after the system call event during execution of the malware sample for a predetermined period of time in the computing environment are reassembled and analyzed to identify a potential malware binary; and
generating an interface that includes a graphical visualization of a plurality of pages in memory associated with a process launched during execution of the malware sample in the computing environment, wherein the graphical visualization of the plurality of pages in memory indicates a subset of the plurality of pages in memory that were modified after a system call event that was intercepted through the system API instrumentation in the computing environment.

9. The method of claim 8 , wherein the computing environment comprises a virtual machine instance.

10. The method of claim 8 , further comprising:
identifying, in the interface, the system call event detected during execution of the malware sample in the computing environment.

11. The method of claim 8 , further comprising:
monitoring changes in memory after a system call event during execution of a malware sample in a computing environment, wherein monitoring changes in memory after the system call event during execution of the malware sample in the computing environment further comprises:
identifying a plurality of pages in memory associated with a process launched by executing the malware sample in the computing environment;
performing an initial snapshot of all of the plurality of pages in memory associated with the process at initial time to and cache the initial snapshot of all of the plurality of pages in memory to provide a baseline for contents in memory while executing the malware sample in the computing environment;
performing another snapshot of all of the plurality of pages in memory associated with the process at subsequent time to after the system call event in response to detecting a return address in a call stack points to a memory address that has changed since the initial snapshot;
reassembling snapshots of the plurality of pages in memory associated with the process after executing the malware sample for a predetermined period of time in the computing environment; and
automatically detecting unpacking of binaries in memory based on a static analysis of the reassembled snapshots of the plurality of pages in memory.

12. The method of claim 11 , wherein the potential malware binary is submitted for dynamic analysis and/or static analysis.

13. The method of claim 8 , further comprising:
receiving a plurality of malware samples; and
deduplicating the plurality of malware samples.

14. The method of claim 8 , further comprising:
receiving a plurality of malware samples;
deduplicating the plurality of malware samples to output a first malware sample; and
executing the first malware sample in the computing environment.

15. A computer program product, the computer program product being embodied in a tangible non-transitory computer readable storage medium and comprising computer instructions for:
performing dynamic analysis of a malware sample for program deobfuscation through system API instrumentation in a computing environment, wherein a plurality of pages in memory associated with a process launched by executing the malware sample are identified and monitored for changes after one or more system call events during execution of the malware sample for a predetermined period

Assignees

• Palo Alto Networks Inc

Inventors

• Jung Robert

Classifications

• G06F11/3034

  where the computing system component is a storage system, e.g. DASD based or network based (digital input from or digital output to record carriers G06F3/06; digital recording or reproducing G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097) · CPC title

• H04L63/1416

  Event detection, e.g. attack signature detection · CPC title

• G06F21/56 Primary

  Computer malware detection or handling, e.g. anti-virus arrangements · CPC title

• H04L63/145

  the attack involving the propagation of malware through the network, e.g. viruses, trojans or worms · CPC title

• G06F11/3006

  where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems (multiprogramming arrangements G06F9/46; allocation of resources G06F9/50) · CPC title