Managing processing system efficiency

Patent metadata

Field	Value
Publication number	US-10908964-B2
Application number	US-201816198583-A
Country	US
Kind code	B2
Filing date	Nov 21, 2018
Priority date	Nov 21, 2017
Publication date	Feb 2, 2021
Grant date	Feb 2, 2021

Abstract

Methods, systems, and computer storage media storing instructions for managing processing system efficiency. One of the methods includes obtaining data splitting a plurality of general-purpose processing units in a processing system into a high-priority domain and a low-priority domain, wherein the general-purpose processing units in the high-priority domain are assigned to perform one or more tasks comprising one or more high-priority tasks, and the general-purpose processing units in the low-priority domain are assigned to perform one or more low-priority tasks; and during runtime of the processing system, obtaining memory usage measurements that characterize usage of system memory by the high-priority domain and the low-priority domain; and adjusting, based on the memory usage measurements, a configuration of (i) the high-priority domain, (ii) the low-priority domain, or (iii) both to adjust utilization of the system memory by the general-purpose processing units.

First claim

What is claimed is:

1. A method comprising:
obtaining data splitting a plurality of general-purpose processing units in a processing system into a high-priority domain and a low-priority domain, wherein
the general-purpose processing units in the high-priority domain are assigned to perform one or more tasks comprising one or more high-priority tasks, and
the general-purpose processing units in the low-priority domain are assigned to perform one or more low-priority tasks; and

during runtime of the processing system, obtaining memory usage measurements that characterize usage of system memory by the high-priority domain and the low-priority domain; and
adjusting, based on the memory usage measurements, a configuration of (i) the high-priority domain, (ii) the low-priority domain, or (iii) both to adjust utilization of the system memory by the general-purpose processing units.

2. The method of claim 1 , wherein the processing system further comprises a plurality of special-purpose hardware accelerators assigned to perform an accelerated workload and wherein the high-priority tasks are tasks associated with the accelerated workload.

3. The method of claim 2 , wherein the accelerated workload is a machine learning workload.

4. The method of claim 1 , wherein the one or more tasks assigned to the high-priority domain further comprise one or more low-priority tasks.

5. The method of claim 1 , wherein the memory usage measurements comprise a measurement of system memory bandwidth.

6. The method of claim 5 , wherein adjusting, based on the memory usage measurements, a configuration of (i) the high-priority domain, (ii) the low-priority domain, or (iii) both to adjust utilization of the system memory by the general-purpose processing units comprises:
reducing the number of active general purpose processors in the low-priority domain when the system memory bandwidth is high.

7. The method of claim 5 , wherein adjusting, based on the memory usage measurements, a configuration of (i) the high-priority domain, (ii) the low-priority domain, or (iii) both to adjust utilization of the system memory by the general-purpose processing units comprises:
increasing the number of active general purpose processors in the low-priority domain when the system memory bandwidth is low.

8. The method of claim 1 , wherein the memory usage measurements comprise a measurement of system memory latency.

9. The method of claim 8 , wherein adjusting, based on the memory usage measurements, a configuration of (i) the high-priority domain, (ii) the low-priority domain, or (iii) both to adjust utilization of the system memory by the general-purpose processing units comprises:
reducing the number of active general purpose processors in the low-priority domain when the system latency is high.

10. The method of claim 8 , wherein adjusting, based on the memory usage measurements, a configuration of (i) the high-priority domain, (ii) the low-priority domain, or (iii) both to adjust utilization of the system memory by the general-purpose processing units comprises:
increasing the number of active general purpose processors in the low-priority domain when the system memory bandwidth is low.

11. The method of claim 1 , wherein the memory usage measurements comprise a measurement of system memory saturation.

12. The method of claim 11 , wherein adjusting, based on the memory usage measurements, a configuration of (i) the high-priority domain, (ii) the low-priority domain, or (iii) both to adjust utilization of the system memory by the general-purpose processing units comprises:
reducing the number of active general purpose processors in the low-priority domain when the system memory saturation is high.

13. The method of claim 11 , wherein adjusting, based on the memory usage measurements, a configuration of (i) the high-priority domain, (ii) the low-priority domain, or (iii) both to adjust utilization of the system memory by the general-purpose processing units comprises:
increasing the number of active general purpose processors in the low-priority domain when the system memory saturation is low.

14. The method of claim 1 , wherein the memory usage measurements comprise a measurement of high-priority domain memory bandwidth.

15. The method of claim 14 , wherein adjusting, based on the memory usage measurements, a configuration of (i) the high-priority domain, (ii) the low-priority domain, or (iii) both to adjust utilization of the system memory by the general-purpose processing units comprises:
reducing the number of active general purpose processors in the high-priority domain when the high-priority domain memory bandwidth is high.

16. The method of claim 14 , wherein adjusting, based on the memory usage measurements, a configuration of (i) the high-priority domain, (ii) the low-priority domain, or (iii) both to adjust utilization of the system memory by the general-purpose processing units comprises:
increasing the number of active general purpose processors in the high-priority domain when the high-priority domain memory bandwidth is low.

17. The method of claim 1 , wherein the adjusting comprises:
reducing the number of active general purpose processors in the low-priority domain when measurements of system memory bandwidth, system memory latency, or system memory saturation are high, and
increasing the number of active general purpose processors in the low-priority domain when the measurements of system memory bandwidth, system memory latency, and system memory saturation are low.

18. The method of claim 1 , wherein the adjusting comprises:
reducing the number of general purpose processors utilizing cache prefetching in the low-priority domain when the measurements of system memory bandwidth, system memory latency, or system memory saturation are high, and
increasing the number of general purpose processors utilizing cache prefetching in the low-priority domain when the measurements of system memory bandwidth, system memory latency, and the system memory saturation are low.

19. The method of claim 1 , wherein the adjusting comprises:
reducing the number of active general purpose processors in the high-priority domain when the measurements of high-priority domain memory bandwidth or system memory latency are high, and
increasing the number of active general purpose processors in the high-priority domain when high-priority domain memory bandwidth and system memory latency are low.

20. A system comprising:
one or more computers; and
one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to perform operations comprising:
obtaining data splitting a plurality of general-purpose processing units in a processing system into a high-priority domain and a low-priority domain
the general-purpose processing units in the high-priority domain are assigned to perform one or more tasks comprising one or more high-priority tasks, and
the general-purpose processing units in the low-priority domain are assigned to perform one or more low-priority tasks; and

during runtime of the processing system, obtaining memory usage measurements that characterize usage of system memory by the high-priority domain and the low-priority domain; and
adjusting, based on the memory usage measurements, a configuration of (i) the high-priority domain, (ii) the low-priority domain, or (iii) both to adjust utilization of the system memory by the general-purpose processing units.

Assignees

• Google Llc

Inventors

• Cheng Liqun
• Govindaraju Rama Krishna
• Zhu Haishan
• Lo David
• Ranganathan Parthasarathy
• Patil Nishant

Classifications

• G06F9/5061 Primary

  Partitioning or combining of resources · CPC title

• G06F2209/5021

  Priority · CPC title

• G06N20/00

  Machine learning · CPC title

• G06N3/063

  using electronic means · CPC title

• G06F9/4881

  Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues · CPC title