Caching streams of memory requests

Patent metadata

Field	Value
Publication number	US-11188472-B2
Application number	US-202016774595-A
Country	US
Kind code	B2
Filing date	Jan 28, 2020
Priority date	Feb 13, 2019
Publication date	Nov 30, 2021
Grant date	Nov 30, 2021

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for allocating cache resources according to page-level attribute values. In one implementation, the system includes one or more integrated client devices and a cache. Each client device is configured to generate at least a memory request. Each memory request has a respective physical address and a respective page descriptor of a page to which the physical address belongs. The cache is configured to cache memory requests for each of the one or more integrated client devices. The cache comprises a cache memory having multiple ways. The cache is configured to distinguish different memory requests using page-level attributes of respective page descriptors of the memory requests, and to allocate different portions of the cache memory to different respective memory requests.

First claim

What is claimed is:

1. A system comprising:
one or more integrated client devices, each client device being configured to generate memory requests, each memory request having a respective physical address and a respective page-level attribute of a respective page of a first memory to which the physical address belongs; and
a cache configured to cache memory requests to the first memory for each of the one or more integrated client devices,
wherein the cache comprises a cache memory having multiple ways, the cache memory being separate from the first memory, and
wherein the cache is configured to distinguish different memory requests using page-level attributes of physical pages in the first memory of the memory requests and to allocate different portions of the cache memory to different respective memory requests.

2. The system of claim 1 , wherein each client device has a respective address translation module that is configured to convert a virtual address received from a software driver to a respective memory request having a respective physical address and a respective page descriptor, and wherein each client device is configured to update a page table to assign a particular page-level attribute value to a particular page.

3. The system of claim 2 , wherein the cache is configured to distinguish different memory requests using particular page-level attributes of page descriptors generated by the respective address translation modules of the client devices.

4. The system of claim 3 , wherein the address translation module is a memory management unit that is configured to perform a hardware walk of a page table in the first memory in order to perform address translation.

5. The system of claim 1 , wherein the cache is configured to identify physical addresses occurring on different pages as being part of the same memory requests.

6. The system of claim 1 , wherein a first page-level attribute is included in memory requests that are part of a first stream of instructions executed by a client device, and a second page-level attribute different from the first page-level attribute is included in memory requests that are part of a second stream of data used by the client device, and
wherein the cache is configured to distinguish the first stream of instructions from the second stream of data based on the first and the second page-level attributes to allocate a first portion of the cache memory to the first stream of instructions and a second portion of the cache memory to the second stream of data.

7. The system of claim 1 , wherein a first page-level attribute is included in memory requests that are part of a first stream of instructions executed by a first client device, and a second page-level attribute different from the first page-level attribute is included in memory requests that are part of a second stream of instructions executed by the first client device or a different second client device, and
wherein the cache is configured to distinguish the first stream of instructions from the second stream of instructions based on the first and the second page-level attributes to allocate a first portion of the cache memory to the first stream of instructions and a second portion of the cache memory to the second stream of instructions.

8. The system of claim 1 , wherein a first page-level attribute is included in memory requests that are part of a first stream of data written to a first data buffer, and a second page-level attribute different from the first page-level attribute is included in memory requests that are part of a second stream of data written to a second data buffer, and
wherein the cache is configured to distinguish the first stream of data from the second stream of data based on the first and the second page-level attributes to allocate a first portion of the cache memory to the first stream of data and a second portion of the cache memory to the second stream of data based on respective first and second page-level attributes included in the respective first and second streams.

9. The system of claim 8 , wherein the cache is configured to use the page-level attributes to allocate more cache memory to data buffers storing page table data than to data buffers storing non-page-table data.

10. The system of claim 1 , wherein the cache is configured to assign different replacement policies to different memory requests based on the respective page-level attributes of the memory requests.

11. The system of claim 10 , wherein the cache is configured to identify a first stream of data written by a producer process executing on one of the one or more client devices, wherein the first stream of data written by the producer process is consumed by a consumer process executing on one of the one or more of client devices,
wherein the cache is configured to allocate a first portion of the cache memory to the first stream of data written by the producer process.

12. The system of claim 11 , wherein the cache is configured to invalidate cache entries in the first portion of cache memory whenever a cache entry is read by the consumer process.

13. The system of claim 1 , wherein the cache is configured to:
determine that a page-level attribute of a read request indicates that a particular page uses compressed data, and
in response, reading less than all of a full cache line from memory in order to fulfill the read request.

14. The system of claim 1 , wherein the cache is configured to map a page-level attribute value to a particular partition identifier, and thereby associating the particular partition identifier with multiple different pages in the first memory.

15. A computer-implemented method executed by a computing system, the method comprising:
receiving, by the system, one or more memory requests generated by respective ones of one or more client devices, each memory request having a respective physical address and a respective page-level attribute of a respective page of a first memory to which the physical address belongs; and
caching, by a cache of the system, the memory requests to the first memory in a cache memory by:
distinguishing different memory requests using page-level attributes of physical pages in the first memory of the memory requests, and
allocating different portions of the cache memory to different respective memory requests.

16. The method of claim 15 , wherein a first page-level attribute is included in memory requests that are part of a first stream of instructions executed by a client device, and a second page-level attribute is included in memory requests that are part of a second stream of data used by the client device,
wherein distinguishing different memory requests comprises distinguishing the first stream of instructions from the second stream of data based on the first and the second page-level attributes to allocate, and
wherein a first portion of the cache memory is allocated to the first stream of instructions and a second portion of the cache memory to the second stream of data.

17. The method of claim 15 , wherein a first page-level attribute is included in memory requests that are part of a first stream of instructions executed by a first client device, and a second page-level attribute different from the first page-level attribute is included in memory requests that are part of a second stream of instructions executed by the first client device or a different second client device,
wherein distinguishing different memory requests comprises distinguishing the first stream of instructions from the second stream of instructions ba

Assignees

• Google Llc

Inventors

• Chamarty Vinod
• Dias Joao

Classifications

• G06F9/544

  Buffers; Shared memory; Pipes · CPC title

• G06F12/0292

  using tables or multilevel address translation means (G06F12/023 takes precedence; address translation in virtual memory systems G06F12/10) · CPC title

• G06F12/1054

  the data cache being concurrently physically addressed · CPC title

• G06F12/1009

  using page tables, e.g. page table structures · CPC title

• G06F12/0891

  using clearing, invalidating or resetting means · CPC title