Private memory table for reduced memory coherence traffic

Patent metadata

Field	Value
Publication number	US-9760490-B2
Application number	US-201615211064-A
Country	US
Kind code	B2
Filing date	Jul 15, 2016
Priority date	Apr 2, 2015
Publication date	Sep 12, 2017
Grant date	Sep 12, 2017

Abstract

A mechanism is provided for memory coherence in a multiple processor system. Responsive to a memory operation from a processing core of the multiple processor system resulting in a cache miss, the mechanism checks a private region table associated with the processing core. The memory operation attempts to access a memory region. Responsive to determining the memory region corresponds to an entry in the private region table, the mechanism performs a remote memory controller snoop of a remote memory controller without snooping the multiple processor system.

First claim

What is claimed is:

1. A method for memory coherence in a multiple processor system, the method comprising:
generating by a virtual machine manager of a first processing node in the multiple processor system, a private region table associated with a processing core of the first processing node, wherein:
the private region table stores an entry identifying a memory region, associated with a virtual machine executing on the first processing node, which is stored remotely in memory of a second processing node,
the entry includes a bitmask having a bit for each cache line of the memory region, and
the bit for a cache line of the memory region is set to a first value if the corresponding cache line is private to the virtual machine executing on the first processing node, or a second value if the cache line of the memory region is not private to the virtual machine;

responsive to a memory operation from the processing core in the first processing node of the multiple processor system resulting in a cache miss, checking the private region table associated with the processing core, wherein the memory operation attempts to access the memory region stored in memory of the second processing node of the multiple processor system;
responsive to determining the memory region corresponds to an entry in the private region table and the memory region is node-contained in the second processing node, performing the memory operation on the second processing node without snooping the entire multiple processor system;
responsive to another processing core accessing a cache line of the memory region, setting a value of a corresponding bit in the bitmask of the en to the second value; and
invalidating the entry in the private region table in response to all bits in the bitmask for the memory region being set to the second value.

2. The method of claim 1 , further comprising performing the memory operation using snoopy protocol responsive to determining the memory region does not correspond to an entry in the private region table.

3. The method of claim 1 , further comprising performing the memory operation using snoopy protocol responsive to determining the memory region is not node-contained in the second processing node.

4. The method of claim 1 , wherein each entry of the private region table comprises a tag, a size, and least recently used (LRU) information.

5. The method of claim 1 , wherein the virtual machine requests memory from the virtual machine manager, the virtual machine manager allocates a remote memory region to the virtual machine, and the virtual machine manager installs an entry in the private region table corresponding to the remote memory region.

6. The method of claim 1 , wherein the multiple processor system comprises a plurality of nodes, wherein each node comprises a plurality of chips, wherein each chip comprises a plurality of chiplets, and wherein each chiplet comprises a processing core and a cache associated with the processing core.

7. The method of claim 6 , wherein the private region table comprises a chiplet-level private region table and a chip-level private region table.

8. The method of claim 7 , further comprising:
responsive to the chiplet-level private region table running out of space, moving a selected entry in the chiplet-level private region table to the chip-level private region table and adding a chiplet identifier to the selected entry.

9. The method of claim 1 , further comprising responsive to migrating the virtual machine to a new processing core, migrating entries of the private region table to a private region table associated with the new processing core.

10. The method of claim 9 , wherein migrating entries of the private region table to the private region table associated with the new processing core comprises performing a data cache block flush of all cache lines in all regions for the virtual machine.

11. The method of claim 9 , wherein migrating entries of the private region table to the private region table associated with the new processing core comprises:
setting a flag on all private region table entries for the virtual machine;
installing entries in the private region table associated with the new processing core for the memory regions allocated to the virtual machine;
for each given memory region to migrate to the new processing core:
walk the given memory region and issue a data cache block move to cache inject into the new processing core;
invalidate a private region table entry for the given memory region in the private region table associated with the old processing core; and
clear the flag in the corresponding entry in the private region table associated with the new processing core.

Assignees

• IBM

Inventors

• Daly David M
• Srinivasan Vijayalakshmi

Classifications

• G06F2212/1016

  Performance improvement · CPC title

• G06F12/0831 Primary

  using a bus scheme, e.g. with bus monitoring or watching means · CPC title

• G06F12/0817

  using directory methods · CPC title

• G06F12/0815 Primary

  Cache consistency protocols · CPC title

• G06F3/06

  Digital input from, or digital output to, record carriers {, e.g. RAID, emulated record carriers or networked record carriers} · CPC title