Method for validating an untrusted native code module

Patent metadata

Field	Value
Publication number	US-10685123-B2
Application number	US-201715651150-A
Country	US
Kind code	B2
Filing date	Jul 17, 2017
Priority date	May 8, 2008
Publication date	Jun 16, 2020
Grant date	Jun 16, 2020

Abstract

A system that validates a native code module. During operation, the system receives a native code module comprised of untrusted native program code. The system validates the native code module by: (1) determining that code in the native code module does not include any restricted instructions and/or does not access restricted features of a computing device; and (2) determining that the instructions in the native code module are aligned along byte boundaries such that a specified set of byte boundaries always contain a valid instruction and control flow instructions have valid targets. The system allows successfully-validated native code modules to execute, and rejects native code modules that fail validation. By validating the native code module, the system facilitates safely executing the native code module in the secure runtime environment on the computing device, thereby achieving native code performance for untrusted program binaries without significant risk of unwanted side effects.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
receiving, by a computing device, a native code module that includes a set of program instructions that match an instruction set architecture of the computing device;
loading the native code module into a memory of the computing device;
identifying, by the computing device, a set of validation criteria that are to ensure safe execution of the native code module on the computing device, wherein the set of validation criteria is specific to the instruction set architecture of the computing device;
determining, by the computing device, after the native code module has been loaded into the memory of the computing device but before the native code module is executed on the computing device, (1) whether the native code module complies with the set of validation criteria that are to ensure safe execution of the native code module on the computing device, and (2) that a set of instructions in the native code module are aligned along byte boundaries such that a specified set of byte boundaries always contain a valid instruction and a set of control flow instructions in the native code module have valid targets; and
in response to determining that the native code module complies with the set of validation criteria and that the set of instructions in the native code module are aligned along the byte boundaries, executing the native code module on the computing device.

2. The computer-implemented method of claim 1 , wherein:
the instruction set architecture of the computing device is an x86 instruction set architecture, and
the set of validation criteria is specific to the x86 instruction set architecture.

3. The computer-implemented method of claim 2 , wherein determining whether the native code module complies with the set of validation criteria includes analyzing the native code module to determine whether the set of program instructions includes any disallowed program instructions from a set of disallowed program instructions.

4. The computer-implemented method of claim 3 , wherein the set of disallowed program instructions includes at least one of a syscall (system call) or int (interrupt) instruction.

5. The computer-implemented method of claim 3 , wherein the set of disallowed program instructions includes instructions that modify an x86 segment state.

6. The computer-implemented method of claim 3 , wherein the set of disallowed program instructions includes at least one of rdtsc (read time stamp counter) or redmsr (read from model specific register) instructions.

7. The computer-implemented method of claim 3 , wherein the set of disallowed program instructions includes a ret (return) instruction.

8. The computer-implemented method of claim 3 , wherein the set of disallowed program instructions includes an indirect control flow instruction that transfers execution of the native code module to an arbitrary location in memory.

9. The computer-implemented method of claim 1 , wherein the computing device is configured, in response to determining that the native code module does not comply with the set of validation criteria that are to ensure safe execution of the native code module on the computing device, to block execution of the native code module.

10. The computer-implemented method of claim 1 , wherein the native code module is executed on the computing device in a runtime environment by providing a limited interface between the native code module and other software entities and hardware resources on the computing device, the limited interface moderating external requests made by the native code module.

11. A non-transitory computer-readable storage medium storing instructions that when executed by a computing device cause the computing device to perform a method, the method comprising:
receiving, by the computing device, a native code module that includes a set of program instructions that match an instruction set architecture of the computing device;
loading the native code module into a memory of the computing device;
identifying, by the computing device, a set of validation criteria that are to ensure safe execution of the native code module on the computing device, wherein the set of validation criteria is specific to the instruction set architecture of the computing device;
determining, by the computing device, after the native code module has been loaded into the memory of the computing device but before the native code module is executed on the computing device, (1) whether the native code module complies with the set of validation criteria that are to ensure safe execution of the native code module on the computing device, and (2) that a set of instructions in the native code module are aligned along byte boundaries such that a specified set of byte boundaries always contain a valid instruction and a set of control flow instructions in the native code module have valid targets; and
in response to determining that the native code module complies with the set of validation criteria and that the set of instructions in the native code module are aligned along the byte boundaries, executing the native code module on the computing device.

12. The storage medium of claim 11 , wherein:
the instruction set architecture of the computing device is an x86 instruction set architecture, and
the set of validation criteria is specific to the x86 instruction set architecture.

13. The storage medium of claim 12 , wherein determining whether the native code module complies with the set of validation criteria includes analyzing the native code module to determine whether the set of program instructions includes any disallowed program instructions from a set of disallowed program instructions.

14. The storage medium of claim 13 , wherein the set of disallowed program instructions includes at least one of a syscall (system call) or int (interrupt) instruction.

15. The storage medium of claim 13 , wherein the set of disallowed program instructions includes instructions that modify an x86 segment state.

16. The storage medium of claim 13 , wherein the set of disallowed program instructions includes at least one of rdtsc (read time stamp counter) or redmsr (read from model specific register) instructions.

17. The storage medium of claim 13 , wherein the set of disallowed program instructions includes a ret (return) instruction.

18. The storage medium of claim 13 , wherein the set of disallowed program instructions includes an indirect control flow instruction that transfers execution of the native code module to an arbitrary location in memory.

19. The storage medium of claim 11 , wherein the computing device is configured, in response to determining that the native code module does not comply with the set of validation criteria that are to ensure safe execution of the native code module on the computing device, to block execution of the native code module.

20. A computing device, comprising:
a processor; and
a non-transitory computer-readable medium having instructions stored thereon that, when executed by the processor, cause the processor to perform operations comprising:
receiving, by the computing device, a native code module that includes a set of program instructions that match an instruction set architecture of the computing device;
loading the native code module into a memory of the computing device;
identifying, by the computing device, a set of validation criteria that are to ensure safe execution of the native code module on the computing device, w

Assignees

• Google Llc

Inventors

• Chen J Bradley
• Harren Matthew T
• Papakipos Matthew
• Sehr David C
• Yee Bennet S

Classifications

• G06F21/51 Primary

  at application loading time, e.g. accepting, rejecting, starting or inhibiting executable software based on integrity or source reliability · CPC title

• G06F21/577 Primary

  Assessing vulnerabilities and evaluating computer system security · CPC title