Prefetcher based speculative dynamic random-access memory read request technique

Patent metadata

Field	Value
Publication number	US-10613983-B2
Application number	US-201815925859-A
Country	US
Kind code	B2
Filing date	Mar 20, 2018
Priority date	Mar 20, 2018
Publication date	Apr 7, 2020
Grant date	Apr 7, 2020

Abstract

A method includes monitoring a request rate of speculative memory read requests from a penultimate-level cache to a main memory. The speculative memory read requests correspond to data read requests that missed in the penultimate-level cache. A hit rate of searches of a last-level cache for data requested by the data read requests is monitored. Core demand speculative memory read requests to the main memory are selectively enabled in parallel with searching of the last-level cache for data of a corresponding core demand data read request based on the request rate and the hit rate. Prefetch speculative memory read requests to the main memory are selectively enabled in parallel with searching of the last-level cache for data of a corresponding prefetch data read request based on the request rate and the hit rate.

First claim

What is claimed is:

1. A method for reducing memory access latency of a memory system comprising:
monitoring a request rate of speculative memory read requests from a penultimate-level cache to a main memory, the speculative memory read requests corresponding to data read requests that missed in the penultimate-level cache, each of the speculative memory read requests being a core demand speculative memory read request or a prefetch speculative memory read request, and each of the data read requests being a core demand data read request or a prefetch data read request;
monitoring a hit rate of searches of a last-level cache for data requested by the data read requests, each of the speculative memory read requests being issued in parallel with a search of the last-level cache for corresponding data;
selectively enabling core demand speculative memory read requests to the main memory in parallel with searching of the last-level cache for data of a corresponding core demand data read request based on the request rate and the hit rate; and
selectively enabling prefetch speculative memory read requests to the main memory in parallel with searching of the last-level cache for data of a corresponding prefetch data read request based on the request rate and the hit rate.

2. The method, as recited in claim 1 ,
wherein the core demand speculative memory read requests are enabled and the prefetch speculative memory read requests are enabled, and
wherein in response to the request rate being greater than a predetermined request rate threshold value or the hit rate being greater than a predetermined hit rate threshold value, the selectively enabling of the core demand speculative memory read requests comprises enabling the core demand speculative memory read requests and the selectively enabling of the prefetch speculative memory read requests comprises disabling the prefetch speculative memory read requests.

3. The method, as recited in claim 2 ,
wherein the core demand speculative memory read requests are enabled and the prefetch speculative memory read requests are disabled,
wherein in response to the request rate being less than a second predetermined request rate threshold value and the hit rate being less than a second predetermined hit rate threshold value, the selectively enabling of the core demand speculative memory read requests comprises enabling the core demand speculative memory read requests and the selectively enabling of the prefetch speculative memory read requests comprises enabling the prefetch speculative memory read requests, and
wherein the second predetermined request rate threshold value is less than the predetermined request rate threshold value and the second predetermined hit rate threshold value is less than the predetermined hit rate threshold value.

4. The method, as recited in claim 1 ,
wherein the core demand speculative memory read requests are enabled and the prefetch speculative memory read requests are disabled, and
wherein in response to the request rate being greater than a predetermined request rate threshold value or the hit rate being greater than a predetermined hit rate threshold value, the selectively enabling of the core demand speculative memory read requests comprises disabling the core demand speculative memory read requests and the selectively enabling of the prefetch speculative memory read requests comprises disabling the prefetch speculative memory read requests.

5. The method, as recited in claim 4 ,
wherein the core demand speculative memory read requests are disabled and the prefetch speculative memory read requests are disabled, and
wherein in response to the request rate being less than a second predetermined request rate threshold value and the hit rate being less than a second predetermined hit rate threshold value, the selectively enabling of the core demand speculative memory read requests comprises enabling the core demand speculative memory read requests and the selectively enabling of the prefetch speculative memory read requests comprises disabling the prefetch speculative memory read requests, and
wherein the second predetermined request rate threshold value is less than the predetermined request rate threshold value and the second predetermined hit rate threshold value is less than the predetermined hit rate threshold value.

6. The method, as recited in claim 1 ,
wherein the core demand speculative memory read requests are enabled and the prefetch speculative memory read requests are enabled, and
wherein in response to the request rate being less than a predetermined request rate threshold value and the hit rate being less than a predetermined hit rate threshold value, the selectively enabling of the core demand speculative memory read requests comprises enabling the core demand speculative memory read requests and the selectively enabling of the prefetch speculative memory read requests comprises enabling the prefetch speculative memory read requests.

7. The method, as recited in claim 1 ,
wherein core demand speculative memory read requests are enabled and prefetch speculative memory read requests are disabled, and
wherein in response to the request rate being less than a predetermined request rate threshold value and the hit rate being less than a predetermined hit rate threshold value, the selectively enabling of core demand speculative memory read requests comprises enabling core demand speculative memory read requests and the selectively enabling of prefetch speculative memory read requests comprises disabling prefetch speculative memory read requests.

8. The method, as recited in claim 1 ,
wherein the core demand speculative memory read requests are disabled and the prefetch speculative memory read requests are disabled, and
wherein in response to the request rate being less than a predetermined request rate threshold value and the hit rate being less than a predetermined hit rate threshold value, the selectively enabling of the core demand speculative memory read requests comprises disabling the core demand speculative memory read requests and the selectively enabling of the prefetch speculative memory read requests comprises disabling the prefetch speculative memory read requests.

9. The method, as recited in claim 1 ,
monitoring a cluster-level request rate for a plurality of penultimate-level caches coupled to the main memory; and
monitoring a cluster-level hit rate of searching of a shared last-level cache for data requested by data read requests that missed in the plurality of penultimate-level caches,
wherein the selectively enabling of the core demand speculative memory read requests is further based on the cluster-level request rate and the cluster-level hit rate, and
wherein the selectively enabling of the prefetch speculative memory read requests is further based on the cluster-level request rate and the cluster-level hit rate.

10. The method, as recited in claim 9 , further comprising:
adjusting a predetermined request rate threshold value or a predetermined hit rate threshold value based on the cluster-level request rate or the cluster-level hit rate, respectively,
wherein the selectively enabling of the core demand speculative memory read requests is further based on the predetermined request rate threshold value and the predetermined hit rate threshold value, and
wherein the selectively enabling of the prefetch speculative memory read requests is further based on the predetermined request rate threshold value and the predetermined hit rate threshold value.

11. The method, as recited in claim 1 , wherein each of the speculative memory read requests is an actual speculative memory read reques

Assignees

• Advanced Micro Devices Inc

Inventors

• Agarwal Tanuj Kumar
• Bhowmik Anasua
• Hunt Douglas Benson

Classifications

• G06F2212/1024

  Latency reduction · CPC title

• G06F3/0653

  Monitoring storage devices or systems · CPC title

• G06F12/0846

  Cache with multiple tag or data arrays being simultaneously accessible · CPC title

• G06F12/0859

  with reload from main memory · CPC title

• G06F12/0897

  with two or more cache hierarchy levels (with multilevel cache hierarchies G06F12/0811) · CPC title