Cache grouping for increasing performance and fairness in shared caches

Patent metadata

Field	Value
Publication number	US-12487933-B2
Application number	US-202318319016-A
Country	US
Kind code	B2
Filing date	May 17, 2023
Priority date	Dec 11, 2020
Publication date	Dec 2, 2025
Grant date	Dec 2, 2025

Abstract

A method includes monitoring one or more metrics for each of a plurality of cache users sharing a cache, and assigning each of the plurality of cache users to one of a plurality of groups based on the monitored one or more metrics.

First claim

I claim:

1 . A method, comprising:
operating a cache according to a first policy for a first group of similar a cache users and according to a second policy for a second group of cache users, wherein the first group of the similar cache users are assigned to the first group based on a degree of similarity of performance metrics between the cache users; for one or more cache users of the first group of the similar cache users, operating the cache according to a test policy for a subset of cache entries associated with the one or more cache users; replacing the first policy with the test policy based on the test policy resulting in an improvement in the performance metrics for the first group of the similar cache users as compared to the first policy; and after the replacing, operating the cache according to the test policy for the first group of the similar cache users.

2 . The method of claim 1 , wherein the improvement in the performance metrics comprises an increase in a hit rate using the test policy.

3 . The method of claim 1 , wherein the improvement in the performance metrics comprises an increase in throughput using the test policy.

4 . The method of claim 1 , wherein the improvement in the performance metrics comprises a decrease in latency using the test policy.

5 . The method of claim 1 , further comprising operating the cache according to one or more additional test policies, and wherein the replacing comprises replacing the first policy with the test or policy or one of the one or more additional test policies based on which test policy results in a greatest improvement in the performance metrics.

6 . The method of claim 1 , wherein the subset of cache entries are within a test region.

7 . The method of claim 1 , wherein each of the similar cache users comprises one of a plurality of processing cores.

8 . The method of claim 1 , further comprising determining the first group of similar cache users by:
monitoring the cache users that access the cache and recording the performance metrics for each cache user that access the cache; and for each cache user that accesses the cache, calculating the degree of similarity by comparing the performance metrics for the cache user to the performance metrics of other cache users that access the cache.

9 . The method of claim 8 , wherein the degree of similarity is calculated as a sum of weighted differences of the respective performance metrics of the cache users.

10 . The method of claim 1 , wherein cache users are similar if:
the respective performance metrics fall within a same range; or the respective performance metrics differ from each other by less than a threshold amount.

11 . The method of claim 1 , wherein the improvement in the performance metrics comprises a decrease in cache misses using the test policy.

12 . The method of claim 1 , wherein the performance metrics include at least one of a hit rate for data associated with each cache user, a traffic volume of the data, or a cache miss penalty for the data.

13 . A method, comprising:
operating a cache according to a policy for a group of a plurality of cache users; for one or more cache users of the plurality of cache users, operating the cache according to a test policy for a subset of cache entries associated with the one or more cache users; replacing the policy with the test policy for at least a subset of users of the plurality of cache users based on the test policy resulting in an improvement in performance metrics for the subset of users of the plurality of cache users; and after the replacing, operating the cache according to the test policy for the subset of users of the plurality of cache users.

14 . The method of claim 13 , wherein the improvement in performance metrics comprises an increase in hit rate using the test policy.

15 . The method of claim 13 , wherein the improvement in performance metrics comprises an increase in throughput using the test policy.

16 . The method of claim 13 , wherein the improvement in performance metrics comprises a decrease in latency using the test policy.

17 . The method of claim 13 , further comprising operating the cache according to one or more additional test policies, and wherein the replacing comprises replacing the policy with the test or policy or one of the one or more additional test policies based on which test policy resulted in a greatest improvement in the performance metrics.

18 . A computing system, comprising:
a plurality of processing cores; a shared cache configured to cache data for each of the plurality of processing cores; and a cache controller coupled with the shared cache and configured to:
operate the shared cache according to a first policy for the plurality of processing cores;
operate the shared cache according to a test policy for a subset of the processing cores;
replace the first policy with the test policy for the plurality of processing cores based on the test policy resulting in an improvement in performance metrics as compared to the first policy; and
after the replacing, operate the shared cache according to the test policy for the plurality of processing cores.

19 . The computing system of claim 18 , wherein the improvement in the performance metrics comprises an increase in hit rate using the test policy.

20 . The computing system of claim 18 , wherein the improvement in the performance metrics comprises an increase in throughput using the test policy.

Assignees

• Advanced Micro Devices Inc

Inventors

• Kelley John

Classifications

• G06F12/128 Primary

  adapted to multidimensional cache systems, e.g. set-associative, multicache, multiset or multilevel · CPC title

• G06F12/126

  with special data handling, e.g. priority of data or instructions, handling errors or pinning · CPC title

• G06F11/3037

  where the computing system component is a memory, e.g. virtual memory, cache (accessing, addressing or allocating within memory systems or architectures G06F12/00; checking stores for correct operation G11C29/00) · CPC title

• G06F2201/885

  Monitoring specific for caches · CPC title

• G06F11/3466

  Performance evaluation by tracing or monitoring · CPC title