Software solution for cooperative memory-side and processor-side data prefetching

Patent metadata

Field	Value
Publication number	US-9798528-B2
Application number	US-53131306-A
Country	US
Kind code	B2
Filing date	Sep 13, 2006
Priority date	Sep 13, 2006
Publication date	Oct 24, 2017
Grant date	Oct 24, 2017

Abstract

A solution for cooperative data prefetching that enables software control of a memory-side data prefetch and/or a processor-side data prefetch is provided. In one embodiment, the invention provides a solution for generating an application, in which access to application data for the application is improved (e.g., optimized) in program code for the application. In particular, a push request, for performing a memory-side data prefetch of the application data, and a prefetch request, for performing a processor-side data prefetch, are added to the program code. The memory-side data prefetch results in the application data being copied from a first data store to a second data store that is faster than the first data store while the processor-side data prefetch results in the application data being copied from the second data store to a third data store that is faster than the second data store.

First claim

What is claimed is:

1. A method of generating an application, the method comprising:
loading source code for an application into a memory of a computing system,
compiling the source code into program code for the application, and during the compilation of the source code:
for each reference to application data determined as likely to generate a memory miss, adding at a position in the program code that is several operations prior to the reference a push request into the program code for a memory-side data prefetch to the program code for the referenced application data, wherein the memory-side data prefetch causes a near memory processor (NMP) to copy referenced application data for the application from random access memory to an L2 cache; and
thereafter adding a prefetch request for a processor-side data prefetch to the program code at a position in the program code that is a number operations prior to the reference equivalent to a number of operations required to complete the prefetch, wherein the processor-side data prefetch causes a main processor that is different than the NMP to copy the application data from the L2 cache to an L1 cache.

2. The method of claim 1 , further comprising analyzing a memory access pattern for the application to identify at least one of:
the application data, a location in the program code for the push request, or a location in the program code for the prefetch request.

3. The method of claim 2 , wherein the analyzing includes determining that a request to access the application data is likely to incur a memory miss.

4. The method of claim 2 , wherein the analyzing includes analyzing at least one of:
application data dependencies or application data structure types at compile-time.

5. The method of claim 2 , wherein the analyzing includes analyzing application data access patterns of the application during runtime.

6. The method of claim 1 , further comprising translating source code for the application into the program code.

7. The method of claim 1 , wherein the push request causes a program to be executed by an execution environment separately from the application.

8. The method of claim 7 , wherein the improving further includes defining a custom program for the program.

9. The method of claim 1 , further comprising:
performing a set of high level optimizations on the program code prior to the improving; and
performing a set of low level optimizations on the program code after the improving.

10. A computer hardware system for generating an application, the system comprising:
at least one processor, wherein the at least one processor is configured to
load source code for an application into a memory of the computing hardware system,
compiling the source code into program code for the application, and during the compilation of the source code:
for each reference to application data determined as likely to generate a memory miss, add at a position in the program code that is several operations prior to the reference a push request into the program code for a memory-side data prefetch to program code of the application for the referenced application data, wherein the memory-side data prefetch causes a near memory processor (NMP) to copy referenced application data for the application from random access memory to an L2 cache; and
thereafter add a prefetch request for a processor-side data prefetch to the program code at a position in the program code that is a number operations prior to the reference equivalent to a number of operations required to complete the prefetch, wherein the processor-side data prefetch causes a main processor that is different than the NMP to copy the application data from the L2 cache to an L1 cache.

11. The system of claim 10 , wherein the at least one processor is further configured to
analyze a memory access pattern for the application to identify at least one of:
the application data,
a location in the program code for the push request, and
a location in the program code for the prefetch request.

12. The system of claim 11 , wherein
the analyzing includes determining that a request to access the application data is likely to incur a memory miss.

13. The system of claim 11 , wherein
the analyzing includes analyzing at least one of:
application data dependencies and
application data structure types at compile-time.

14. The system of claim 11 , wherein
the system for analyzing includes a system for analyzing application data access patterns of the application during runtime.

15. The system of claim 10 , wherein the at least one processor is further configured to
translate source code for the application into the program code.

16. The system of claim 10 , wherein
the push request causes a program to be executed by the execution environment separately from the application.

17. The system of claim 10 , wherein the at least one processor is further configured to:
perform a set of high level optimizations on the program code prior improving access to the application data; and
perform a set of low level optimizations on the program code after the improving access to the application data.

18. A computer program product comprising at least one non-transitory computer-readable storage medium having stored therein computer usable program code for generating an application, which when executed by a computer hardware system, causes the computer hardware system to perform:
loading source code for an application into a memory of a computing system,
compiling the source code into program code for the application, and during the compilation of the source code:
for each reference to application data determined as likely to generate a memory miss, adding at a position in the program code that is several operations prior to the reference a push request into the program code for a memory-side data prefetch to the program code for the referenced application data, wherein the memory-side data prefetch causes a near memory processor (NMP) to copy referenced application data for the application from random access memory to an L2 cache; and
thereafter adding a prefetch request for a processor-side data prefetch to the program code at a position in the program code that is a number operations prior to the reference equivalent to a number of operations required to complete the prefetch, wherein the processor-side data prefetch causes a main processor that is different than the NMP to copy the application data from the L2 cache to an L1 cache.

19. The computer program product of claim 18 , wherein the computer hardware system is further configured to perform
analyzing a memory access pattern for the application to identify at least one of:
the application data,
a location in the program code for the push request, and
a location in the program code for the prefetch request.

20. The computer program product of claim 18 , wherein the computer hardware system is further configured to perform
translating source code for the application into the program code.

21. The computer program product of claim 18 , wherein
the push request causes a program to be executed by the execution environment separately from the application.

22. The computer program product of claim 21 , wherein the computer hardware system is further configured to perform
defining a custom program for the program.

Assignees

• Gao Yaoqing
• Cascaval Gheorghe C
• Kielstra Allan H
• Tremaine Robert B
• Wazlowski Michael E
• Zhang Lixin
• IBM

Inventors

• Gao Yaoqing
• Cascaval Gheorghe C
• Kielstra Allan H
• Tremaine Robert B
• Wazlowski Michael E
• Zhang Lixin

Classifications

• G06F8/4442 Primary

  Reducing the number of cache misses; Data prefetching (cache prefetching G06F12/0862) · CPC title

• G06F2212/6028

  Prefetching based on hints or prefetch instructions · CPC title

• G06F12/0811

  with multilevel cache hierarchies · CPC title

• G06F8/40

  Transformation of program code · CPC title

• G06F12/0862

  with prefetch · CPC title