Multi-processor graphics rendering

Patent metadata

Field	Value
Publication number	US-9697579-B2
Application number	US-201314055782-A
Country	US
Kind code	B2
Filing date	Oct 16, 2013
Priority date	Jun 7, 2013
Publication date	Jul 4, 2017
Grant date	Jul 4, 2017

Abstract

An operating system that includes an image processing framework as well as a job management layer is provided. The image processing framework is for performing image processing operations and the job management layer is for assigning the image processing operations to multiple concurrent computing resources. The computing resources include several processing units and one or more direct memory access (DMA) channels for concurrently rendering image data and transferring image data between the processing units.

First claim

What is claimed is:

1. A non-transitory computer readable medium storing an operating system which when executed by at least one processor of a computing device allows media editing applications to interact with a plurality of computing resources of the computing device in order to generate media presentations, the operating system comprising sets of instructions for:
performing image processing operations on media content received from the media editing applications, each image processing operation comprising a plurality of image processing jobs;
assigning said image processing operations to be concurrently performed by the plurality of computing resources, wherein each image processing job comprises a resource specification that identifies one of the computing resources for performing the image processing job;
creating a data transfer job for an image processing job when an output of the performed image processing job is not stored at a particular memory location, wherein the particular memory location is specified by a media editing application to which the image processing job belongs; and
generating different media presentations for the media editing applications based on the performed image processing operations.

2. The non-transitory computer readable medium of claim 1 , wherein the resource specification for a particular image processing job is specified by a media editing application, wherein the operating system further comprises a set of instructions for providing an application programming interface (API) for the media editing application to specify the resource specification through the API.

3. The non-transitory computer readable medium of claim 1 , wherein the operating system further comprises a set of instructions for creating a plurality of storage queues for queuing the plurality of image processing jobs.

4. The non-transitory computer readable medium of claim 1 , wherein each image processing job further comprises a priority specification of the image processing job.

5. The non-transitory computer readable medium of claim 1 , wherein the plurality of concurrent computing resources comprises at least one processing unit and one direct memory access (DMA) channel.

6. A system for performing a plurality of sets of image processing instructions, the system comprising:
a storage queue for queuing image processing jobs comprising sets of image processing instructions, wherein each image processing job comprises a resource specification that identifies a particular renderer for performing the image processing job, a priority specification, and an output destination specification that identifies a particular memory location for storing an output of the performed image processing job;
a first renderer for executing a first set of image processing instructions in a first image processing job that specifies the first renderer in the resource specification;
a second renderer for executing a second set of image processing instructions in a second image processing job; and
a direct memory access (DMA) controller for performing a data transfer operation to transfer a block of data from a first memory location to a second memory location while the first and second renderers are concurrently executing the first and second sets of image processing instructions, wherein the block of data is produced by the second renderer and the second memory location is specified by the output destination specification of the second image processing job that is processed by the second renderer, wherein the first memory location is closely coupled with the second renderer.

7. The system of claim 6 , wherein the first renderer is associated with a graphics processing unit (“GPU”), while the second renderer is associated with a central processing unit (“CPU”).

8. The system of claim 6 , wherein the first renderer is associated with a first graphics processing unit (“GPU”), while the second renderer is associated with a second GPU.

9. The system of claim 8 , wherein the first memory location comprises a video memory (VRAM) of the second GPU, while the second image processing job specifies the second memory location to be the output location of the performed second image processing job.

10. The system of claim 6 , wherein the first and second image processing jobs are received from a media editing application, wherein the first and second renderers concurrently execute the first and second sets of image processing instructions in order to generate a real-time playback for the media editing application.

11. The system of claim 6 , wherein the block of data is a pixel buffer object (PBO).

12. The system claim of 6 , wherein the block of data is a first block of data, wherein the DMA controller is further for transferring a second block of data from a third memory location to a fourth memory location, wherein the second block of data is an output produced by the first renderer based on the first set of image processing instructions and the third memory location is closely coupled with the first renderer.

13. A non-transitory computer readable medium storing a computer program which when executed by at least one processor assigns a plurality of image processing instruction sets to a plurality of renderers for execution, the computer program comprising sets of instructions for:
storing a set of image processing jobs in a rendering queue, each image processing job comprising a set of image processing instructions, a resource specification that identifies a particular renderer of the plurality of renderers for performing the image processing job, and an output destination specification that identifies a particular memory location for storing an output of the performed image processing job;
assigning a first set of image processing instructions of a first image processing job of the set of image processing jobs to a first renderer for execution, wherein the resource specification of the first image processing job specifies the first renderer;
assigning a second set of image processing instructions of a second image processing job of the set of image processing jobs to a second renderer for execution; and
assigning a direct memory access operation to a direct memory access (DMA) controller for transferring a block of data from a first memory location to a second memory location while the first and second renderers concurrently execute the first and second sets of image processing instructions, wherein the block of data is produced by the second renderer and the second memory location is specified by the output destination specification of the second image processing job, wherein the first memory location is closely coupled with the second renderer.

14. The non-transitory computer readable medium of claim 13 , wherein the block of data is a pixel buffer object (PBO).

15. The non-transitory computer readable medium of claim 13 , wherein the resource specification for the second image processing job specifies a preference for the second renderer.

16. The non-transitory computer readable medium of claim 13 , wherein the DMA operation is specified by a data transfer job stored in a queue that is different from the rendering queue.

17. The non-transitory computer readable medium of claim 16 , wherein the data transfer job is a first data transfer job, wherein the computer program further comprises a set of instructions for generating a second data transfer job for copying an output of the first set of image processing instructions from a third memory location to a fourth memory location, wherein the third memory l

Assignees

• Apple Inc

Inventors

• Taggart Angus M
• Graves Eric J
• Dumais Jean-Francois N

Classifications

• G06T15/005

  General purpose rendering architectures · CPC title

• G06T1/20 Primary

  Processor architectures; Processor configuration, e.g. pipelining · CPC title