Graphics engine and environment for efficient real time rendering of graphics that are not pre-known

Patent metadata

Field	Value
Publication number	US-9679346-B2
Application number	US-201615154471-A
Country	US
Kind code	B2
Filing date	May 13, 2016
Priority date	Jun 7, 2015
Publication date	Jun 13, 2017
Grant date	Jun 13, 2017

Abstract

This disclosure pertains to the operation of graphics systems and to a variety of architectures for design and/or operation of a graphics system spanning from the output of an application program and extending to the presentation of visual content in the form of pixels or otherwise. In general, many embodiments of the invention contemplate a high level graphics framework to receive graphic requests from an application. The graphics request is analyzed by the high-level framework and sorted into groups of command statements for execution. The command statements are sorted to cause the most efficient processing by the underlying hardware and the groups are submitted separately to a GPU using a low-level standard library that facilitates close control of the hardware functionality.

First claim

The invention claimed is:

1. A method comprising:
receiving, by a high-level framework, a logical representation of a requested graphic for immediate display on a display element of a computer system, wherein the requested graphic is not pre-known to the high level framework;
evaluating, using the high-level framework, the logical representation of the requested graphic and to derive one or more statements representing commands to a target hardware, wherein the one or more statements are expressed in a program interface for a standard low-level graphics library;
sorting the one or more statements into two or more groups of statements;
determining if a first group of statements is ready for rendering;
if the first group of statements is ready for rendering, committing the first group of statements using the low-level graphics library and rendering to a first pre-rendering buffer;
determining if a second group of statements is ready for rendering;
if the second group of statements is ready for rendering, committing the second group of statements using the low-level graphics library and rendering to a second pre-rendering buffer;
compositing the contents of the first pre-rendering buffer and the second pre-rendering buffer into a frame buffer for display on the display element.

2. The method of claim 1 , wherein the logical representation of the requested graphic is received from an application program.

3. The method of claim 2 , wherein the requested graphic is not pre-known to the application program.

4. The method of claim 1 , wherein the logical representation of a requested graphic is represented as a scene graph.

5. The method of claim 1 , wherein the first pre-rendering buffer and the second pre-rendering buffer are a same physical memory.

6. The method of claim 1 , wherein the first group of statements resides in a first command buffer and the second group of statements resides in a second command buffer.

7. The method of claim 1 , wherein determining if the first group of statements is ready for rendering comprises one or more of:
determining if further statements will be added to the first group of statements;
determining if computer system memory is constrained;
determining if evaluation the logical representation of the requested graphic is complete; and,
determining if all aspects of the logical representation of the requested graphic that require pre-rendering have been pre-rendered.

8. The method of claim 1 , wherein sorting the one or more statements into two or more groups of statements comprises one or more of:
sorting the one or more statements according to required GPU state;
sorting the one or more statements according to target hardware; and,
sorting the one or more statements according to a graphic surface represented by each statement.

9. A non-transitory program storage device, readable by a processor and comprising instructions stored thereon to cause one or more processors to:
receive, by a high-level framework, a logical representation of a requested graphic for immediate display on a display element of a computer system, wherein the requested graphic is not pre-known to the high-level framework;
evaluate, using the high-level framework, the logical representation of the requested graphic and to derive one or more statements representing commands to a target hardware;
sort the one or more statements into two or more groups of statements;
determine if a first group of statements is ready for rendering;
if the first group of statements is ready for rendering, commit the first group of statements and render to a first pre-rendering buffer;
determine if a second group of statements is ready for rendering;
if the second group of statements is ready for rendering, commit the second group of statements and render to a second pre-rendering buffer;
composite the contents of the first pre-rendering buffer and the second pre-rendering buffer into a frame buffer for display on the display element.

10. The non-transitory program storage device of claim 9 , wherein the logical representation of the requested graphic is received from an application program and the requested graphic is not pre-known to the application program.

11. The non-transitory program storage device of claim 9 , wherein the logical representation of the requested graphic is represented as a scene graph.

12. The non-transitory program storage device of claim 9 , wherein to instructions to cause one or more processors to determine if the first group of statements is ready for rendering comprise instructions to cause the one or more processors to determine one or more of the following:
if further statements will be added to the first group of statements;
if computer system memory is constrained;
if evaluation of the logical representation of the requested graphic is complete; and,
if all aspects of the logical representation of the requested graphic that require pre-rendering have been pre-rendered.

13. The non-transitory program storage device of claim 9 , wherein the one or more statements are expressed in a program interface for a standard low-level graphics library.

14. The non-transitory program storage device of claim 13 , wherein the first group of statements and the second group of statements are committed using the standard low-level graphics library.

15. The non-transitory program storage device of claim 9 , wherein the instructions to cause one or more processors to sort the one or more statements into two or more groups of statements comprise instructions to cause the one or more processors to sort the one or more statements according to one or more of the following:
required GPU state;
target hardware; and,
a graphic surface represented by each statement.

16. A system comprising:
one or more CPUs;
one or more GPUs for processing a graphics request;
a memory for storing program instructions for the one or more CPUs, where certain instructions, when executed, cause the one or more CPUs to:
receive, by a high-level framework, a logical representation of a requested graphic, wherein the requested graphic is for immediate display on a display element of a computer system, and wherein the requested graphic is not pre-known to the high-level framework;
evaluate, using the high-level framework, the logical representation of the requested graphic;
derive, using the high-level framework, one or more statements representing commands to a target hardware, wherein the one or more statements are expressed in a program interface for a standard low-level graphics library;
sort the one or more statements into two or more groups of statements;
determine if a first group of statements is ready for rendering;
if the first group of statements is ready for rendering, commit the first group of statements using the low-level graphics library and render to a first pre-rendering buffer;
determine if a second group of statements is ready for rendering;
if the second group of statements is ready for rendering, commit the second group of statements using the low-level graphics library and render to a second pre-rendering buffer;
composite the contents of the first pre-rendering buffer and the second pre-rendering buffer into a frame buffer for display on the display element.

17. The system of claim 16 , wherein the logical representation of the requested graphic is received from an application program and the requested graphic is not pre-known to the application program.

18. The system of claim 16 , wherein the instructions to cause t

Assignees

• Apple Inc

Inventors

• Begeman Nathaniel C
• Gies Sean M
• Pangborn Andrew M

Classifications

• G06T15/005

  General purpose rendering architectures · CPC title

• G06T2200/28

  involving image processing hardware · CPC title

• G06T15/00

  Three-dimensional [3D] image rendering · CPC title

• G06T17/005

  Tree description, e.g. octree, quadtree · CPC title

• G06T15/503

  Blending, e.g. for anti-aliasing · CPC title