Buffer overflow trapping

Patent metadata

Field	Value
Publication number	US-11947465-B2
Application number	US-202017068915-A
Country	US
Kind code	B2
Filing date	Oct 13, 2020
Priority date	Oct 13, 2020
Publication date	Apr 2, 2024
Grant date	Apr 2, 2024

Abstract

Aspects of the invention include receiving, at an operating system executing on a processor, a write request from a program to write data to a memory. The write request includes a virtual memory address and the data. It is determined that the virtual memory address is not assigned to a physical memory address. Based on the determining, the unassigned virtual memory address is assigned to a physical memory address in an overflow memory. The data is written to the physical memory address in the overflow memory and an indication that the write data was successfully written is returned to the program. Future requests by the program to access the virtual memory address are directed to the physical memory address in the overflow memory.

First claim

What is claimed is:

1. A method comprising:
receiving, at an operating system executing on a processor, a write request from a program to write data to a memory, the write request comprising a virtual memory address and the data;
determining that the virtual memory address is not assigned to a physical memory address;
based on a determination that the virtual address is not assigned to a physical memory address, initiating recording of state information about the data and the program, wherein the state information includes an identifier of the program, addresses of memory being accessed by the program, and a loader associated with execution of the program;
based on the determining, assigning the virtual memory address to a physical memory address in an overflow memory;
writing the data to the physical memory address in the overflow memory; and
returning an indication to the program that the data was successfully written to the memory, wherein subsequent requests by the program to access the virtual memory address are directed to the physical memory address in the overflow memory.

2. The method of claim 1 , wherein the state information is utilized to identify a pattern associated with a malicious program or threat actor.

3. The method of claim 1 , wherein a security policy of the processor is updated based at least in part on the state information.

4. The method of claim 1 , wherein the data includes computer code and the overflow location is configured to prevent the computer code from executing from the overflow location.

5. The method of claim 1 , wherein a translation lookaside buffer (TLB) is used to translate virtual memory addresses into physical memory addresses, and the TLB is configured using sparse memory placement.

6. The method of claim 1 , further comprising:
determining a buffer overflow mode of the processor, wherein the assigning, writing, and returning are performed based on the buffer overflow mode of the processor being a first mode; and
based on the buffer overflow mode being a second mode, returning an indication to the program that the write was not successfully completed in response to determining that the virtual address is not assigned to a physical memory address in the memory.

7. A system comprising:
one or more processors for executing computer readable instructions, the computer readable instructions controlling the one or more processors to perform operations comprising:
receiving, at an operating system executing on a processor of the one or more processors, a write request from a program to write data to a memory, the write request comprising a virtual memory address and the data;
determining that the virtual memory address is not assigned to a physical memory address;
based on a determination that the virtual address is not assigned to a physical memory address, initiating recording of state information about the data and the program, wherein the state information includes an identifier of the program, addresses of memory being accessed by the program, and a loader associated with execution of the program;
based on the determining, assigning the virtual memory address to a physical memory address in an overflow memory;
writing the data to the physical memory address in the overflow memory; and
returning an indication to the program that the data was successfully written to the memory, wherein subsequent requests by the program to access the virtual memory address are directed to the physical memory address in the overflow memory.

8. The system of claim 7 , wherein the state information is utilized to identify a pattern associated with a malicious program or threat actor.

9. The system of claim 7 , wherein a security policy of the processor is updated based at least in part on the state information.

10. The system of claim 7 , wherein the data includes computer code and the overflow location is configured to prevent the computer code from executing from the overflow location.

11. The system of claim 7 , wherein a translation lookaside buffer (TLB) is used to translate virtual memory addresses into physical memory addresses, and the TLB is configured using sparse memory placement.

12. The system of claim 7 , wherein the operations further comprise:
determining a buffer overflow mode of the processor, wherein the assigning, writing, and returning are performed based on the buffer overflow mode of the processor being a first mode; and
based on the buffer overflow mode being a second mode, returning an indication to the program that the write was not successfully completed in response to determining that the virtual address is not assigned to a physical memory address in the memory.

13. A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform operations comprising:
receiving, at an operating system executing on the processor, a write request from a program to write data to a memory, the write request comprising a virtual memory address and the data;
determining that the virtual memory address is not assigned to a physical memory address;
based on a determination that the virtual address is not assigned to a physical memory address, initiating recording of state information about the data and the program, wherein the state information includes an identifier of the program, addresses of memory being accessed by the program, and a loader associated with execution of the program;
based on the determining, assigning the virtual memory address to a physical memory address in an overflow memory;
writing the data to the physical memory address in the overflow memory; and
returning an indication to the program that the data was successfully written to the memory, wherein subsequent requests by the program to access the virtual memory address are directed to the physical memory address in the overflow memory.

14. The computer program product of claim 13 , wherein a security policy of the processor is updated based at least in part on the state information.

15. The computer program product of claim 13 , wherein the data includes computer code and the overflow location is configured to prevent the computer code from executing from the overflow location.

16. The computer program product of claim 13 , wherein a translation lookaside buffer (TLB) is used to translate virtual memory addresses into physical memory addresses, and the TLB is configured using sparse memory placement.

17. The computer program product of claim 13 , wherein the operations further comprise:
determining a buffer overflow mode of the processor, wherein the assigning, writing, and returning are performed based on the buffer overflow mode of the processor being a first mode; and
based on the buffer overflow mode being a second mode, returning an indication to the program that the write was not successfully completed in response to determining that the virtual address is not assigned to a physical memory address in the memory.

Assignees

• IBM

Inventors

• Lyons Michael Peter
• Hicks Andrew C M
• Garrett Tynan J
• Pedrone Miles C

Classifications

• G06F12/1027 Primary

  using associative or pseudo-associative address translation means, e.g. translation look-aside buffer [TLB] · CPC title

• G06F9/45558

  Hypervisor-specific management and integration aspects · CPC title

• G06F21/52

  during program execution, e.g. stack integrity {; Preventing unwanted data erasure; Buffer overflow} · CPC title

• G06F21/566

  Dynamic detection, i.e. detection performed at run-time, e.g. emulation, suspicious activities · CPC title

• G06F2009/45583

  Memory management, e.g. access or allocation · CPC title