Coordination and increased utilization of graphics processors during inference

Patent metadata

Field	Value
Publication number	US-12462328-B2
Application number	US-202318351898-A
Country	US
Kind code	B2
Filing date	Jul 13, 2023
Priority date	Apr 24, 2017
Publication date	Nov 4, 2025
Grant date	Nov 4, 2025

Abstract

A mechanism is described for detecting, at training time, information related to one or more tasks to be performed by the one or more processors according to a training dataset for a neural network, analyzing the information to determine one or more portions of hardware of a processor of the one or more processors that is configurable to support the one or more tasks, configuring the hardware to pre-select the one or more portions to perform the one or more tasks, while other portions of the hardware remain available for other tasks, and monitoring utilization of the hardware via a hardware unit of the graphics processor and, via a scheduler of the graphics processor, adjusting allocation of the one or more tasks to the one or more portions of the hardware based on the utilization.

First claim

What is claimed is:

1 . An apparatus comprising:
one or more processors, including a graphics processor, the one or more processors to:
detect, at training time, information related to one or more tasks to be performed by the one or more processors according to a training dataset for a neural network;
analyze the information to determine one or more portions of hardware of a processor of the one or more processors that is configurable to support the one or more tasks;
configure the hardware to pre-select the one or more portions to perform the one or more tasks, while other portions of the hardware remain available for other tasks; and
monitor utilization of the hardware via a hardware unit of the graphics processor and, via a scheduler of the graphics processor, adjust allocation of the one or more tasks to the one or more portions of the hardware based on the utilization.

2 . The apparatus of claim 1 , wherein the one or more processors are to monitor utilization of the hardware by existing contexts and adjust allocation of the hardware to the existing contexts or new contexts based on the utilization.

3 . The apparatus of claim 2 , wherein to adjust allocation of the one or more tasks includes to reallocate a compute element from the neural network to another neural network or to adjust one or more of memory and cache for one or more neural networks.

4 . The apparatus of claim 1 , wherein the one or more processors are further to facilitate communication of hints between a first camera model obtained from one or more cameras and a second camera model obtained from one or more databases, wherein the hints include early hints to enable early fusion to facilitate predictions, path planning, and decision making.

5 . The apparatus of claim 1 , wherein the one or more processors are further to establish coordination between one or more sensors and inference output associated with inference operations related to the training dataset.

6 . The apparatus of claim 5 , wherein to establish coordination includes to facilitate application of one or more filters to sensor data to alter the inference output to match a threshold of normalcy prior to completing the inference operations.

7 . The apparatus of claim 6 , wherein the one or more sensors include one or more cameras and the sensor data includes one or more images of a scene.

8 . The apparatus of claim 1 , wherein the graphics processor is co-located with an application processor of the one or more processors on a common semiconductor package.

9 . A method comprising:
detecting, at training time, information relating to one or more tasks to be performed by one or more processors including a graphics processor according to a training dataset; analyzing the information to determine one or more portions of hardware of a processor of the one or more processors that is configurable to support the one or more tasks; configuring the hardware to pre-select the one or more portions to perform the one or more tasks, while other portions of the hardware remain available for other tasks; and monitoring utilization of the hardware via a hardware unit of the graphics processor and, via a scheduler of the graphics processor, adjusting allocation of the one or more tasks to the one or more portions of the hardware based on the utilization.

10 . The method of claim 9 , comprising monitoring utilization of the hardware by existing contexts and adjusting allocation of the hardware to the existing contexts or new contexts based on the utilization.

11 . The method of claim 10 , wherein adjusting allocation of the one or more tasks includes reallocating a compute element from one neural network to another neural network or adjusting one or more of memory and cache for one or more neural networks.

12 . The method of claim 9 , further comprising facilitating communication of hints between a first camera model obtained from one or more cameras and a second camera model obtained from one or more databases, wherein the hints include early hints to enable early fusion to facilitate predictions, path planning, and decision making.

13 . The method of claim 9 , wherein the graphics processor is co-located with an application processor of the one or more processors on a common semiconductor package.

14 . The method of claim 9 , further comprising establishing coordination between one or more sensors and inference output associated with inference operations related to the training dataset, including facilitating application of one or more filters to sensor data to alter the inference output to match a threshold of normalcy prior to completing the inference operations, wherein the one or more sensors include one or more cameras and the sensor data includes one or more images of a scene.

15 . A data processing system comprising:
one or more processors, including a graphics processor, the one or more processors to:
detect, at training time, information related to one or more tasks to be performed by the one or more processors according to a training dataset for a neural network;
analyze the information to determine one or more portions of hardware of a processor of the one or more processors that is configurable to support the one or more tasks;
configure the hardware to pre-select the one or more portions to perform the one or more tasks, while other portions of the hardware remain available for other tasks; and
monitor utilization of the hardware via a hardware unit of the graphics processor and, via a scheduler of the graphics processor, adjust allocation of the one or more tasks to the one or more portions of the hardware based on the utilization.

16 . The data processing system of claim 15 , wherein the one or more processors are to monitor utilization of the hardware by existing contexts and adjust allocation of the hardware to the existing contexts or new contexts based on the utilization.

17 . The data processing system of claim 16 , wherein to adjust allocation of the one or more tasks includes to reallocate a compute element from the neural network to another neural network or to adjust one or more of memory and cache for one or more neural networks.

18 . The data processing system of claim 15 , wherein the one or more processors are further to facilitate communication of hints between a first camera model obtained from one or more cameras and a second camera model obtained from one or more databases, wherein the hints include early hints to enable early fusion to facilitate predictions, path planning, and decision making.

19 . The data processing system of claim 15 , wherein the one or more processors are further to establish coordination between one or more sensors and inference output associated with inference operations related to the training dataset, to establish coordination includes to facilitate application of one or more filters to sensor data to alter the inference output to match a threshold of normalcy prior to completing the inference operations, and the one or more sensors include one or more cameras and the sensor data includes one or more images of a scene.

20 . The data processing system of claim 15 , wherein the graphics processor is co-located with an application processor of the one or more processors on a common semiconductor package.

Assignees

• Intel Corp

Inventors

• Appu Abhishek R
• Koker Altug
• Weast John C
• Macpherson Mike B
• Hurd Linda L
• Baghsorkhi Sara S
• Gottschlich Justin E
• Surti Prasoonkumar
• Sakthivel Chandrasekaran
• Ma Liwei
• Ould-Ahmed-Vall Elmoustapha
• Sinha Kamal
• Ray Joydeep
• Vembu Balaji
• Jahagirdar Sanjeev
• Ranganathan Vasanth
• Kim Dukhwan

Classifications

• G06N5/041

  Abduction · CPC title

• G06F9/3867

  using instruction pipelines · CPC title

• G06T1/20 Primary

  Processor architectures; Processor configuration, e.g. pipelining · CPC title

• G06N3/084

  Backpropagation, e.g. using gradient descent · CPC title

• G06N3/063

  using electronic means · CPC title