Migration of peer-mapped memory pages

Patent metadata

Field	Value
Publication number	US-9639474-B2
Application number	US-201314134148-A
Country	US
Kind code	B2
Filing date	Dec 19, 2013
Priority date	Mar 15, 2013
Publication date	May 2, 2017
Grant date	May 2, 2017

Abstract

Techniques are provided by which memory pages may be migrated among PPU memories in a multi-PPU system. According to the techniques, a UVM driver determines that a particular memory page should change ownership state and/or be migrated between one PPU memory and another PPU memory. In response to this determination, the UVM driver initiates a peer transition sequence to cause the ownership state and/or location of the memory page to change. Various peer transition sequences involve modifying mappings for one or more PPU, and copying a memory page from one PPU memory to another PPU memory. Several steps in peer transition sequences may be performed in parallel for increased processing speed.

First claim

What is claimed is:

1. A computer-implemented method for modifying memory page ownership in a virtual memory subsystem having two or more parallel processing units (PPUs), the method comprising:
determining a current ownership state for a memory page that indicates which PPU in the virtual memory subsystem is associated with a PPU memory in which the memory page is currently stored and which additional PPUs in the virtual memory subsystem have page tables that include a page table entry with a mapping to the memory page;
determining a new ownership state for the memory page that indicates which PPU in the virtual memory subsystem is associated with a PPU memory in which the memory page should be stored and which PPUs in the virtual memory subsystem should have page tables that include a page table entry corresponding to the memory page; and
modifying a page table entry included in a page table associated with at least one PPU in the virtual memory subsystem, based on the current ownership state and the new ownership state.

2. The method of claim 1 , wherein the current ownership state indicates that the memory page is stored in a first PPU memory associated with a first PPU, that a first PPU page table associated with the first PPU includes a page table entry corresponding to the memory page, and that a second PPU page table associated with a second PPU does not include a page table entry associated with the memory page.

3. The method of claim 2 , wherein:
the new ownership state indicates that the memory page should be transmitted to a second PPU memory associated with the second PPU, and that only the second PPU page table should include a page table entry corresponding to the memory page, and
further comprising:
removing a page table entry corresponding to the memory page from the first PPU page table;
transmitting the memory page from the first PPU memory to the second PPU memory; and
creating a page table entry in the second PPU page table corresponding to the memory page.

4. The method of claim 2 , wherein:
the new ownership state indicates that the memory page should be transmitted to the second PPU memory and that the first PPU page table and the second PPU page table should both include page table entries corresponding to the memory page, and
further comprising:
removing a page table entry corresponding to the memory page from the first PPU page table;
transmitting the memory page from the first PPU memory to the second PPU memory;
creating a page table entry in the first PPU page table corresponding to the memory page; and
creating a page table entry in the second PPU page table corresponding to the memory page.

5. The method of claim 2 , wherein:
the new ownership state indicates that the memory page should not be transmitted to a second PPU memory corresponding to the second PPU, and that the first PPU page table and the second PPU page table should include page table entries that are corresponding to the memory page, and
further comprising creating a page table entry in the second PPU page table that is corresponding to the memory page.

6. The method of claim 1 , wherein the current ownership state indicates that the memory page is stored in a first PPU memory corresponding to a first PPU, that a first PPU page table corresponding to the first PPU includes a page table entry corresponding to the memory page, and that a second PPU page table corresponding to a second PPU includes a page table entry corresponding to the memory page.

7. The method of claim 6 , wherein:
the new ownership state indicates that the page should not to be transmitted to a second PPU memory corresponding to the second PPU, and that a page table entry in the second PPU page table and corresponding to the memory page should be deleted, and
further comprising removing the page table entry in the second PPU page table that is corresponding to the memory page.

8. The method of claim 6 , wherein:
the new ownership state indicates that the page should be transmitted to a second PPU memory corresponding to the second PPU, that a page table entry should be deleted from the first PPU page table, and that the second PPU page table should include an entry corresponding to the memory page, and
further comprising:
removing a page table entry corresponding to the memory page from the first PPU page table;
removing a page table entry corresponding to the memory page from the second PPU page table;
transmitting the memory page from the first PPU memory to the second PPU memory; and
creating a page table entry in the second PPU page table that is corresponding to the memory page.

9. The method of claim 6 , wherein:
the new ownership state indicates that the page should be transmitted to a second PPU memory corresponding to the second PPU, that a page table entry corresponding to the memory page should be written to the first PPU page table, and that a page table entry corresponding to the memory page should be written to the second PPU page table, and
further comprising:
removing a page table entry corresponding to the memory page from the first PPU page table;
removing a page table entry corresponding to the memory page from the second PPU page table;
transmitting the memory page from the first PPU memory to the second PPU memory;
creating a page table entry in the first PPU page table that is corresponding to the memory page; and
creating a page table entry in the second PPU page table that is corresponding to the memory page.

10. The method of claim 1 , wherein the virtual memory system includes two PPUs.

11. The method of claim 1 , wherein the virtual memory system includes more than two PPUs.

12. A virtual memory subsystem for modifying memory page ownership, the virtual memory subsystem comprising:
two parallel processing units (PPUs); and
a unified virtual memory driver configured to:
determine a current ownership state for a memory page that indicates which PPU in the virtual memory subsystem is associated with a PPU memory in which the memory page is currently stored and which additional PPUs in the virtual memory subsystem have page tables that include a page table entry with a mapping to the memory page,
determine a new ownership state for the memory page that indicates which PPU in the virtual memory subsystem is associated with a PPU memory in which the memory page should be stored and which PPUs in the virtual memory subsystem should have page tables that include a page table entry corresponding to the memory page, and
modifying a page table entry included in a page table associated with at least one PPU in the virtual memory subsystem, based on the current ownership state and the new ownership state.

13. The virtual memory subsystem of claim 12 , wherein the current ownership state indicates that the memory page is stored in a first PPU memory associated with a first PPU, that a first PPU page table associated with the first PPU includes a page table entry corresponding to the memory page, and that a second PPU page table associated with a second PPU does not include a page table entry associated with the memory page.

14. The virtual memory subsystem of claim 13 , wherein:
the new ownership state indicates that the memory page should be transmitted to a second PPU memory associated with the second PPU, and that only the second PPU page table should include a page table entry corresponding to the memory page, and
further comprising:
removing a page table entry corresponding to the memory page from the first PPU page table;
transmitting the memory page from the first PPU memory to the second PPU m

Assignees

• Nvidia Corp

Inventors

• Duluk Jr Jerome F
• Mashey John
• Hairgrove Mark
• Jia Chenghuan
• Buschardt Cameron
• Dunning Lucien
• Fahs Brian

Classifications

• G06F2212/657

  Virtual address space management · CPC title

• G06F12/1009 Primary

  using page tables, e.g. page table structures · CPC title

• G06F3/0664

  at device level, e.g. emulation of a storage device or system · CPC title

• G06F13/4022

  using switching circuits, e.g. switching matrix, connection or expansion network (G06F13/4009 takes precedence) · CPC title

• G06F13/4282

  on a serial bus, e.g. I2C bus, SPI bus (on daisy chain buses G06F13/4247) · CPC title