Controlling display performance using display statistics and feedback

Patent metadata

Field	Value
Publication number	US-10726604-B2
Application number	US-201615274658-A
Country	US
Kind code	B2
Filing date	Sep 23, 2016
Priority date	Jun 3, 2016
Publication date	Jul 28, 2020
Grant date	Jul 28, 2020

Abstract

One embodiment provides for a display system to generate and display data on a display device, the display system comprising one or more graphics processors to generate one or more frames of data for display on the display device; a window manager to submit a request to display the one or more frames of data; a display engine to present the one or more frames of data to the display device for display; and display logic to receive the request to display the one or more frames of data and generate one or more display events for the display engine based on the request to display the one or more frames of data, wherein the display logic is to manage a set of statistics associated with the request.

First claim

What is claimed is:

1. A display system to generate and display data on a display device, the display system comprising:
one or more graphics processors to generate one or more frames of data for display on the display device;
a window manager to submit a request to display the one or more frames of data;
a display engine to present the one or more frames of data to the display device for display; and
display logic to receive the request to display the one or more frames of data and generate one or more display events for the display engine based on the request to display the one or more frames of data;
the display system configured to adjust a refresh rate of the display device in response to the one or more display events being processed, to enable the one or more frames of data to be displayed at a first target frame rate;
wherein the display logic is to manage a set of statistics associated with the request and periodically submit the set of statistics to the window manager, the set of statistics including a difference between a requested presentation time and an actual display time on the display device for each of the one or more frames of data;
wherein the display system, in response to an indication in the set of statistics that the one or more frames of data have not been presented at the first target frame rate, causes a reduction in graphical complexity or an animation rate of one or more successive frames of data generated by the one or more graphics processors to display the one or more successive frames of data at the first target frame rate.

2. The display system as in claim 1 , wherein the set of statistics additionally includes a count of frames in which the actual display time is greater than the requested presentation time.

3. The display system as in claim 2 , wherein the window manager is to submit the request to display the one or more frames of data on behalf of one or more applications, the one or more applications are to generate a set of frames via the one or more graphics processors at a first target animation frame rate, and the window manager is to present at least a subset of the set of statistics to the one or more applications.

4. The display system as in claim 3 , wherein the one or more applications are to adjust a complexity of the set of frames in response to an indication in the subset of the set of statistics that one or more frames have not been presented at the first target animation frame rate.

5. The display system as in claim 3 , wherein the one or more applications are to adjust the first target animation frame rate to a second target animation frame rate in response to an indication in the subset of the set of statistics that one or more frames of data have not been presented at the first target animation frame rate.

6. The display system as in claim 1 , wherein the window manager is to submit the request to display the one or more frames of data on behalf of one or more applications, the one or more applications to generate one or more surfaces via the one or more graphics processors and the window manager is to configure generation of the one or more frames of data based on the one or more surfaces.

7. The display system as in claim 6 , wherein the window manager is to generate a set of frames at a first target animation frame rate based on the one or more surfaces and to adjust the first target animation frame rate to a second target animation frame rate in response to an indication that the set of frames have not been presented at the first target animation frame rate.

8. The display system as in claim 1 , the display logic further to insert the one or more display events for the display engine into a memory and associate the one or more display events with a target presentation time in which the display event is to be processed.

9. The display system as in claim 8 , wherein the memory is to store a data structure defining multiple display events that include display engine state to display a frame, the target presentation time in which the display event is to be processed, and a submit time in which the display event for the frame was inserted into the data structure, wherein the set of statistics additionally includes the submit time for a display event associated with the request.

10. The display system as in claim 9 , wherein the target presentation time for a frame is derived based on the requested presentation time for a frame adjusted by a programming time for the display engine, the display engine to adjust a refresh rate of the display device at a time based on the target presentation time and explicitly refresh the display to display the frame at the requested presentation time.

11. A non-transitory machine readable medium storing instructions which, when executed by one or more processors of an electronic device, cause the one or more processors to perform operations comprising:
generating one or more frames of data at a first frame rate;
submitting a request to display logic to display the one or more frames of data; receiving the request at the display logic and generating one or more display events for a display engine based on the request;
processing the display events at the display engine to display the one or more frames of data;
adjusting a refresh rate of the display device in response to the one or more display events being processed, to enable the one or more frames of data to be displayed at a first target frame rate;
generating a set of statistics associated with processing the display events, the set of statistics including an indication that the one or more frames of data have not been presented at the first target frame rate and the set of statistics including a difference between a requested presentation time and an actual display time on the display device for each of the one or more frames of data; and
causing, in response to the indication, a reduction in graphical complexity or an animation rate of one or more successive frames of data generated by one or more graphics processors to display the one or more successive frames of data at the first target frame rate.

12. The non-transitory machine readable medium as in claim 11 , wherein submitting a request to display logic to display the one or more frames of data is performed by a window manager on behalf of one or more applications and the operations additionally comprise submitting the set of statistics to the window manager, wherein the window manager submits at least a subset of the set of statistics to the one or more applications.

13. The non-transitory machine readable medium as in claim 12 , the operations additionally adjusting an animation frame rate at the one or more applications based on the subset of the set of statistics, the adjusting to reduce a difference between a subsequent requested presentation time and actual display time.

14. The non-transitory machine readable medium as in claim 12 , the operations additionally comprising generating one or more surfaces at the one or more applications and configuring the generation of the one or more frames of data based on the one or more surfaces via the window manager.

15. The non-transitory machine readable medium as in claim 14 , wherein configuring the generation of the one or more frames of data based on the one or more surfaces includes generating an animation at the window manager using the one or more surfaces, the animation having a first target animation frame rate, and in based on the set of statistics, adjusting the first target animation frame rate to a second target animation frame rate to reduce a difference between

Assignees

• Apple Inc

Inventors

• Spence Arthur L
• Sandmel Jeremy T
• Chan David M
• Zhang Chendi
• Tsoi Peter C

Classifications

• G06F3/0412

  Digitisers structurally integrated in a display · CPC title

• G09G2320/10

  Special adaptations of display systems for operation with variable images · CPC title

• G06F3/14

  Digital output to display device {; Cooperation and interconnection of the display device with other functional units} · CPC title

• G09G2360/12

  Frame memory handling · CPC title

• G06F3/041

  Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means · CPC title