Generic unpacking of program binaries

Patent metadata

Field	Value
Publication number	US-10311233-B2
Application number	US-201415038413-A
Country	US
Kind code	B2
Filing date	Dec 23, 2014
Priority date	Dec 26, 2013
Publication date	Jun 4, 2019
Grant date	Jun 4, 2019

Abstract

By hooking application programming interfaces in an execution environment, the return address for hooked application programming interface calls can be logged and used to determine when a packed binary has been unpacked. In one approach, memory allocations are detected and the return address is checked against the memory regions allocated. In another approach, the contents of memory at the return address in a pre-execution copy of the executable binary is compared with the contents of memory at the return address in the executing copy of the binary. This allows efficient detection of the completion of unpacking without knowledge of the unpacking technique. The unpacked binary may then be analyzed for possible malware.

First claim

What is claimed is:

1. A storage disk or storage device comprising instructions for detecting malware that, when executed, cause a processor to at least:
load a self-extracting first executable into memory, the self-extracting first executable including an unpacking second executable and a packed third executable; and
after the unpacking second executable unpacks the packed third executable into a fourth executable:
detect an application programming interface call;
determine a return address for the application programming interface call;
determine whether the fourth executable is being executed based on the return address not being in a region of memory previously allocated to the self-extracting first executable; and
when the fourth executable is being executed, scan the fourth executable for malware.

2. The storage disk or storage device of claim 1 , wherein the instructions, when executed cause the processor to detect the application programming interface call by detecting system related application programming interface calls.

3. The storage disk or storage device of claim 1 , wherein the instructions, when executed, cause the processor to detect the application programming interface call by detecting any of a set of application programming interface calls.

4. The storage disk or storage device of claim 1 , wherein the instructions, when executed, cause the processor to detect the application programming interface call by: detecting an allocation of the region of memory by the self-extracting first executable; and recording memory allocation data corresponding to the region of memory.

5. The storage disk or storage device of claim 4 , wherein the instructions, when executed, cause the processor to determine whether the fourth executable is being executed based on the recorded memory allocation data.

6. The storage disk or storage device of claim 1 , wherein the instructions, when executed, cause the processor to determine whether the fourth executable is being executed based on the return address not in the region of memory previously allocated by: comparing memory contents at the return address in an active binary image with memory contents at the return address in a passive binary image.

7. The storage disk or storage device of claim 6 , wherein the instructions, when executed, cause the processor to translate the return address in the active binary image to an address in the passive binary image.

8. A system for detecting malware, comprising:
a processor; and
a first memory including instructions that, when executed, cause the processor to at least:
load a self-extracting first executable into a second memory, the self-extracting first executable including an unpacking second executable and a packed third executable;
after the unpacking second executable has unpacked the packed third executable into a fourth executable:
determine a return address for an application programming interface call; and
in response to the return address not being in a region of memory previously allocated to the self-extracting first executable, scan the fourth executable for malware.

9. The system of claim 8 , wherein the instructions, when executed, cause the processor to: determine the return address for the application programming interface by detecting the application programming interface call by the self-extracting first executable.

10. The system of claim 8 , wherein the instructions, when executed, cause the processor to: detect the application programming interface call by detecting system related application programming interface calls.

11. The system of claim 8 , wherein the instructions, when executed, cause the processor to: detect the application programming interface call by detecting any of a predetermined set of application programming interface calls.

12. The system of claim 8 , wherein the instructions, that when executed, cause the processor to: detect the application programming interface call by:
detecting allocations of memory by the self-extracting first executable; and
recording memory allocation data corresponding to the allocations of memory.

13. The system of claim 12 , wherein the instructions, when executed, cause the processor to: determine whether the return address is in one of the allocations of memory based on the recorded memory allocation data.

14. The system of claim 8 , wherein the instructions, when executed, cause the processor to: determine whether the fourth executable is being executed based on the return address by: compare memory contents at the return address in an active binary image with memory contents at the return address in a passive binary image.

15. The system of claim 14 , wherein the instructions, that when executed, cause the processor to: determine whether the fourth executable is being executed based on the return address by translating the return address in the active binary image to an address in the passive binary image.

16. A method of unpacking a packed binary, comprising:
executing the packed binary in a programmable device to:
unpack a packed first executable of the packed binary into a second executable;
detecting an application programming interface call by the packed binary;
determining a return address for the application programming interface call; and
scanning the second executable for malware in response to the return address being in a region of memory previously allocated by the packed binary.

17. The method of claim 16 , further including capturing memory allocation data upon an allocation of memory by the packed binary.

18. The method of claim 16 , further including: wherein determining whether the third executable is being executed
mapping the packed binary into memory as a passive image and executing the packed binary as an active image; and
comparing memory contents at the return address in the passive image with memory contents at the return address in the active image.

19. The method of claim 18 , further including translating the return address in the active image into a return address in the passive image.

Assignees

• Mcafee Inc
• Mcafee Llc

Inventors

• Malik Amit
• Taneja Vikas
• Cruz Benjamin

Classifications

• G06F21/566

  Dynamic detection, i.e. detection performed at run-time, e.g. emulation, suspicious activities · CPC title

• G06F2221/034

  Test or assess a computer or a system · CPC title

• G06F21/577

  Assessing vulnerabilities and evaluating computer system security · CPC title

• G06F21/562 Primary

  Static detection · CPC title