Methods and systems for memory bandwidth control

Patent metadata

Field	Value
Publication number	US-11829637-B2
Application number	US-202318166561-A
Country	US
Kind code	B2
Filing date	Feb 9, 2023
Priority date	Sep 1, 2021
Publication date	Nov 28, 2023
Grant date	Nov 28, 2023

Abstract

Resources of an electronic device are partitioned into a plurality of resource portions to be utilized by a plurality of clients. Each resource portion is assigned to a respective client, has a respective partition identifier (ID), and corresponds to a plurality of memory bandwidth usage states tracked for a plurality of memory blocks. For each resource portion, each of the memory bandwidth usage states is associated with a respective memory block and indicates at least how much of a memory access bandwidth assigned to the respective partition ID to access the respective memory block is used. A usage level is determined for each resource partition based on the memory bandwidth usage states, and applied to adjust a credit count. When the credit count is adjusted beyond a request issue threshold, a next data access request is issued from a memory access request queue for the respective partition ID.

First claim

What is claimed is:

1. A method for managing memory access, comprising, at a first memory coupled to one or more processing clusters and a plurality of memory blocks in an electronic device:
forwarding a plurality of data access requests associated with a plurality of clients to the plurality of memory blocks, wherein resources of the electronic device are partitioned to a plurality of resource portions to be utilized by the plurality of clients, each resource portion being assigned to a respective client and having a respective partition identifier (ID);
for each resource portion having the respective partition ID:
identifying a subset of data access requests associated with the respective partition ID for accessing the plurality of memory blocks;

tracking a plurality of memory bandwidth usage states corresponding to the plurality of memory blocks, wherein each memory bandwidth usage state is associated with a respective memory block and indicates at least how much of a memory access bandwidth assigned to the respective partition ID to access the respective memory block is used; and
in response to each data access request of the subset of data access requests:
determining the respective data access request is to access a corresponding memory block, receiving the memory bandwidth usage state of the corresponding memory block, and reporting the memory bandwidth usage state of the corresponding memory block to the one or more processing clusters.

2. The method of claim 1 , further comprising:
monitoring a first total number of data access requests that are waiting in a first request queue of the first memory, and determining a first current congestion level indicating whether the first total number of data access requests exceeds a first predefined portion of a system cache capacity; and
in response to each data access request of the subset of data access requests, reporting the first current congestion level jointly with the memory bandwidth usage state of the corresponding memory block to the one or more processing clusters.

3. The method of claim 2 , further comprising, at the one or more processing clusters, in accordance with a determination that the first current congestion level satisfies a throttling condition, throttling a plurality of prefetch requests from the plurality of resource portions.

4. The method of claim 2 , further comprising, in response to each data access request of the subset of data access requests:
updating, from the corresponding memory block, a second current congestion level indicating whether a second total number of data access requests that are waiting in a second request queue of the plurality of memory blocks exceeds a second predefined portion of an external memory capacity; and
reporting the second current congestion level to the one or more processing clusters jointly with the memory bandwidth usage state of the corresponding memory block and the first current congestion level.

5. The method of claim 4 , further comprising, at the one or more processing clusters:
in accordance with a determination that the first and second current congestion levels satisfy a prefetch control condition, selecting a first subset of prefetch requests having first qualities that exceed a threshold quality corresponding to the prefetch control condition, including a subset of prefetch requests in a memory access request queue, and excluding a second subset of prefetch requests having second qualities that does not exceed the threshold quality from the memory access request queue.

6. The method of claim 1 , wherein each memory bandwidth usage state associated with a respective memory block includes a respective flag that is configured to be enabled by the respective memory block in accordance with (a) a determination that an average data access level to the respective memory block has exceeded a predefined threshold portion of a predefined memory access bandwidth and (b) a determination that the predefined memory access bandwidth is enforced or that an alternative congestion level of the plurality of memory blocks is high.

7. An electronic device, comprising:
one or more processing clusters;
a plurality of memory blocks; and
a cache memory coupled to the one or more processing clusters and the plurality of memory blocks, the cache memory configured to:
forward a plurality of data access requests associated with a plurality of clients to the plurality of memory blocks, wherein resources of the electronic device are partitioned to a plurality of resource portions to be utilized by the plurality of clients, each resource portion being assigned to a respective client and having a respective partition identifier (ID);

for each resource portion having the respective partition ID, the cache memory configured to:
identify a subset of data access requests associated with the respective partition ID for accessing the plurality of memory blocks;
track a plurality of memory bandwidth usage states corresponding to the plurality of memory blocks, wherein each memory bandwidth usage state is associated with a respective memory block and indicates at least how much of a memory access bandwidth assigned to the respective partition ID to access the respective memory block is used; and

in response to each data access request of the subset of data access requests, the cache memory configured to:
determine the respective data access request is to access a corresponding memory block, receive the memory bandwidth usage state of the corresponding memory block, and report the memory bandwidth usage state of the corresponding memory block to the one or more processing clusters.

8. The electronic device of claim 7 , wherein the electronic device is configured to:
monitor a first total number of data access requests that are waiting in a first request queue of the cache memory, and determine a first current congestion level indicating whether the first total number of data access requests exceeds a first predefined portion of a system cache capacity; and
in response to each data access request of the subset of data access requests, report the first current congestion level jointly with the memory bandwidth usage state of the corresponding memory block to the one or more processing clusters.

9. The electronic device of claim 8 , further comprising, at the one or more processing clusters, in accordance with a determination that the first current congestion level satisfies a throttling condition, throttling a plurality of prefetch requests from the plurality of resource portions.

10. The electronic device of claim 8 , wherein the cache memory is further configured to, in response to each data access request of the subset of data access requests;
update, from the corresponding memory block, a second current congestion level indicating whether a second total number of data access requests that are waiting in a second request queue of the plurality of memory blocks exceeds a second predefined portion of an external memory capacity; and
report the second current congestion level to the one or more processing clusters jointly with the memory bandwidth usage state of the corresponding memory block and the first current congestion level.

11. The electronic device of claim 10 , wherein the one or more processing clusters are configured to, in accordance with a determination that the first current congestion level and the second current congestion level satisfy a prefetch control condition:
select a first subset of prefetch requests having first qualities that exceed a threshold quality corresponding to the prefetch control condition, including a subset of prefetch requests in a memory access re

Assignees

• Qualcomm Inc

Inventors

• Srinivasan Ramkumar
• Kumar Amit
• Geetha Vedaraman
• Pflederer Keith Robert
• Sinha Vikas Kumar

Classifications

• G06F2212/152

  Virtualized environment, e.g. logically partitioned system · CPC title

• G06F3/0679

  Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP] · CPC title

• G06F3/0658

  Controller construction arrangements · CPC title

• G06F3/0604

  Improving or facilitating administration, e.g. storage management · CPC title

• G06F12/0848 Primary

  Partitioned cache, e.g. separate instruction and operand caches · CPC title