Dynamic memory bandwidth shaping

Patent metadata

Field	Value
Publication number	US-12519730-B2
Application number	US-202418668765-A
Country	US
Kind code	B2
Filing date	May 20, 2024
Priority date	May 20, 2024
Publication date	Jan 6, 2026
Grant date	Jan 6, 2026

Abstract

Bandwidth shaping mechanisms in a memory hierarchy operate a first bandwidth shaper on a first memory, such as a cache memory, to shape bandwidth to a second memory, such as a Dynamic Random Access Memory, and replenish the first bandwidth shaper from a second bandwidth shaper based on a hit bandwidth on the first memory.

First claim

What is claimed is:

1 . A process comprising:
operating a first bandwidth shaper on a first memory to shape bandwidth to a second memory; and replenishing the first bandwidth shaper from a second bandwidth shaper based on a hit bandwidth on the first memory.

2 . The process of claim 1 , wherein the first memory operates as a cache for the second memory.

3 . The process of claim 2 , wherein the second memory is a dynamic random access memory.

4 . The process of claim 1 , further comprising:
associating a hit bandwidth limit with a traffic class; and on condition that the traffic class violates the hit bandwidth limit, operating the second bandwidth shaper to suppress replenishments to the first bandwidth shaper for bandwidth to the second memory.

5 . The process of claim 1 , further comprising:
associating with a traffic class a limit on bandwidth to the second memory; and on condition that the traffic class violates the bandwidth limit, operating the first bandwidth shaper to suppress bandwidth requests from the traffic class to an input arbiter of the first memory.

6 . The process of claim 1 , further comprising:
the second bandwidth shaper replenishing the first bandwidth shaper with bandwidth to the second memory for all requests that hit the first memory for traffic classes that do not have a configured limit of the hit bandwidth.

7 . The process of claim 1 , further comprising:
accumulating credit for the bandwidth for the second memory in the first bandwidth shaper at a first rate; and accumulating credit for the hit bandwidth in the second bandwidth shaper at a second rate different than the first rate.

8 . The process of claim 1 , wherein the first bandwidth shaper is configured to assume that every grant of bandwidth to the first memory results in bandwidth to the second memory.

9 . A memory system comprising:
a first memory and a second memory; the first memory configured as a cache for the second memory; a first bandwidth shaper configured to shape bandwidth for the second memory; a second bandwidth shaper configured to track hit bandwidth on the first memory; and the second bandwidth shaper configured to provide replenishments of bandwidth to the second memory to the first bandwidth shaper based on the hit bandwidth.

10 . The memory system of claim 9 , the second bandwidth shaper further configured to provide the replenishments to the first bandwidth shaper unconditionally for traffic that hits the first memory, on condition that the traffic lacks a configured hit bandwidth limit.

11 . The memory system of claim 9 , the second bandwidth shaper further configured to suppress the replenishments for traffic classes that have violated a configured limit on the hit bandwidth.

12 . The memory system of claim 9 , the first bandwidth shaper configured to mask bandwidth requests to an input bandwidth arbiter of the first memory.

13 . The memory system of claim 9 , the second bandwidth shaper comprising:
logic to refresh credit for the hit bandwidth for each of the traffic classes at a first rate.

14 . The memory system of claim 13 , the first bandwidth shaper comprising:
logic to refresh credit for the bandwidth to the second memory for each of the traffic classes at a second rate different than the first rate.

15 . The memory system of claim 9 , the first bandwidth shaper configured to decrement bandwidth credit for the second memory pessimistically in response to grants of bandwidth to the first memory by an arbiter.

16 . A system comprising:
an upper memory of a memory hierarchy; a lower memory of the memory hierarchy; a bandwidth shaper comprising:
an accumulator for bandwidth credit for a traffic class to access the lower memory;
logic configured to:
increment the accumulator at a rate based on a cycle time of the upper memory; decrement the accumulator by a set amount in response to a grant of bandwidth for the traffic class to the upper memory; apply a lower memory bandwidth replenishment signal for the traffic class to the accumulator, the replenishment signal based on a hit bandwidth of the traffic class on the upper memory; and generate, based on a value of the accumulator, a mask setting for the traffic class in a bandwidth arbiter of the upper memory.

17 . The system of claim 16 , wherein the upper memory is a cache memory of the lower memory.

18 . The system of claim 17 , wherein the lower memory is a system memory.

19 . A system comprising:
an upper memory of a memory hierarchy; a lower memory of the memory hierarchy; a bandwidth shaper comprising:
an accumulator for hit bandwidth credit for a traffic class on the upper memory;
logic configured to:
increment the accumulator at a rate based on a cycle time of the upper memory; decrement the accumulator in response to a hit on the upper memory by the traffic class; and in response to the hit on the memory by the traffic class, and on condition that the accumulator has not reached a value indicating a violation of a configured limit on the hit bandwidth for the traffic class, generate a replenishment signal for the traffic class to a bandwidth shaper of traffic between the upper memory and the lower memory.

20 . The system of claim 19 , wherein the lower memory is a dynamic random access memory and the upper memory is a cache memory of the lower memory.

Assignees

• Nvidia Corp

Inventors

• Pandey Ashutosh
• Gupta Jay Kishora
• Chaudhary Anurag
• Molnar Steven E
• Kelley John

Classifications

• H04L47/521

  Static queue service slot or fixed bandwidth allocation · CPC title

• H04L47/39 Primary

  Credit based · CPC title

• H04L47/22 Primary

  Traffic shaping · CPC title