Distributed hardware tracing

Patent metadata

Field	Value
Publication number	US-9875167-B1
Application number	US-201715473101-A
Country	US
Kind code	B1
Filing date	Mar 29, 2017
Priority date	Mar 29, 2017
Publication date	Jan 23, 2018
Grant date	Jan 23, 2018

Abstract

A computer-implemented method executed by one or more processors, the method includes monitoring execution of program code executed by a first processor component; and monitoring execution of program code executed by a second processor component. A computing system stores data identifying hardware events in a memory buffer. The stored events occur across processor units that include at least the first and second processor components. The hardware events each include an event time stamp and metadata characterizing the event. The system generates a data structure identifying the hardware events. The data structure arranges the events in a time ordered sequence and associates events with at least the first or second processor components. The system stores the data structure in a memory bank of a host device and uses the data structure to analyze performance of the program code executed by the first or second processor components.

First claim

What is claimed is:

1. A computer-implemented method performed by a distributed hardware system, the method comprising:
monitoring execution of program code by a first multi-core processor component in the distributed hardware system, the first multi-core processor component being configured to execute at least a first portion of the program code, and wherein the distributed hardware system includes multiple component nodes, each component node corresponding to a particular hardware component of the distributed hardware system;
monitoring execution of the program code by a second multi-core processor component in the distributed hardware system, the second multi-core processor component being configured to execute at least a second portion of the program code;
storing, by the distributed hardware system, data identifying one or more hardware events occurring across the distributed hardware system that includes the first multi-core processor component and the second multi-core processor component, each hardware event representing at least: i) data communications about a memory access operation where data is routed at least between respective component nodes of the distributed hardware system during execution of the program code or ii) a timing status and an execution status of an instruction included in the program code, wherein the data identifying each of the one or more hardware events comprises a hardware event time stamp and metadata characterizing the hardware event;
generating, by the distributed hardware system, a data structure that identifies the one or more hardware events, the data structure being configured to arrange the one or more hardware events in a time ordered sequence of events that occurred during execution of program code by at least the first multi-core processor component and the second multi-core processor component; and
storing, by the distributed hardware system, the generated data structure in a memory bank of a host device.

2. The method of claim 1 , further comprising:
detecting, by the distributed hardware system, a trigger function associated with portions of program code being executed by at least one of the first multi-core processor component or the second multi-core processor component; and
responsive to detecting the trigger function, initiating, by the distributed hardware system, at least one trace event that causes data associated with the one or more hardware events to be stored in at least one memory buffer.

3. The method of claim 2 , wherein:
the trigger function corresponds to a particular sequence step in the program code or a particular time parameter indicated by a global time clock used by the distributed hardware system;
initiating the at least one trace event comprises determining that a trace bit is set to a particular value, the at least one trace event being associated with a memory access operation including multiple intermediate operations that occur across the distributed hardware system; and
data associated with the multiple intermediate operations is stored in one or more memory buffers in response to determining that the trace bit is set to the particular value.

4. The method of claim 1 , wherein storing data identifying the one or more hardware events further comprises:
storing, in a first memory buffer of the first multi-core processor component, a first subset of data identifying hardware events of the one or more hardware events, wherein storing occurs in response to the first multi-core processor component executing a hardware trace instruction associated with at least the first portion of the program code.

5. The method of claim 4 , wherein storing data identifying the one or more hardware events further comprises:
storing, in a second memory buffer of the second multi-core processor component, a second subset of data identifying hardware events of the one or more of hardware events, wherein storing occurs in response to the second multi-core processor component executing a hardware trace instruction associated with at least the second portion of the program code.

6. The method of claim 5 , wherein generating the data structure further comprises:
comparing, by the distributed hardware system, at least hardware event time stamps of respective events in the first subset of data identifying hardware events with at least hardware event time stamps of respective events in the second subset of data identifying hardware events; and
providing, by the distributed hardware system and for presentation in the data structure, a correlated set of hardware events based, in part, on the comparison between the respective events in the first subset and the respective events in the second subset.

7. The method of claim 1 , wherein the generated data structure identifies at least one parameter that indicates a latency attribute of a particular hardware event or the timing status of the instruction in the program code, the latency attribute indicating at least a duration of the particular hardware event.

8. The method of claim 1 , wherein at least one processor of the distributed hardware system is a multi-core multi-node processor having one or more multi-core processor components, and the one or more hardware events correspond, in part, to data transfers that occur between at least the first multi-core processor component of a first node and the second multi-core processor component of a second node.

9. The method of claim 1 , wherein:
each of the first multi-core processor component and the second multi-core processor component is one of: a multi-core processor, a multi-node processor, a memory access engine, or a hardware component of the distributed hardware system, and wherein the one or more hardware events correspond, in part, to movement of data packets between a source and a destination; and
metadata characterizing the hardware event corresponds to at least one of a source memory address, a destination memory address, a unique trace identification number, or a size parameter associated with a direct memory access (DMA) trace.

10. A distributed hardware system, comprising:
one or more multi-core processor components;
one or more non-transitory machine-readable storage units for storing instructions that are executable by the one or more multi-core processor components to cause performance of operations comprising:
monitoring execution of program code by a first multi-core processor component in the distributed hardware system, the first multi-core processor component being configured to execute at least a first portion of the program code, and wherein the distributed hardware system includes multiple component nodes, each component node corresponding to a particular hardware component of the distributed hardware system;
monitoring execution of the program code by a second multi-core processor component in the distributed hardware system, the second multi-core processor component being configured to execute at least a second portion of the program code;
storing, by a computing system that includes the distributed hardware system, data identifying one or more hardware events occurring across the distributed hardware system that includes the first multi-core processor component and the second multi-core processor component, each hardware event representing at least: i) data communications about a memory access operation where data is routed at least between respective component nodes of the distributed hardware system during execution of the program code or ii) a timing status and an execution status of an instruction included in the program code, wherein the data identifying each of the one or more hardware events comprises a hardware event time stamp an

Assignees

• Google Inc

Inventors

• Norrie Thomas
• Kumar Naveen

Classifications

• G06F16/489

  using time information · CPC title

• G06F11/3476 Primary

  Data logging (G06F11/14, G06F11/2205 take precedence) · CPC title

• G06F9/542

  Event management; Broadcasting; Multicasting; Notifications · CPC title

• G06F2201/865

  Monitoring of software · CPC title

• G06F11/3072

  where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting · CPC title