Predictive viewport renderer and foveated color compressor

Patent metadata

Field	Value
Publication number	US-11756247-B2
Application number	US-202117194819-A
Country	US
Kind code	B2
Filing date	Mar 8, 2021
Priority date	Apr 1, 2017
Publication date	Sep 12, 2023
Grant date	Sep 12, 2023

Abstract

An embodiment of a graphics apparatus may include a focus identifier to identify a focus area, and a color compressor to selectively compress color data based on the identified focus area. Another embodiment of a graphics apparatus may include a motion detector to detect motion of a real object, a motion predictor to predict a motion of the real object, and an object placer to place a virtual object relative to the real object based on the predicted motion of the real object. Another embodiment of a graphics apparatus may include a frame divider to divide a frame into viewports, a viewport prioritizer to prioritize the viewports, a renderer to render a viewport of the frame in order in accordance with the viewport priorities, and a viewport transmitter to transmit a completed rendered viewport. Other embodiments are disclosed and claimed.

First claim

What is claimed is:

1. A computing system comprising:
a graphics processor;
a central processing unit; and
a memory including a set of instructions, which when executed by one or more of the graphics processor or the central processing unit, cause the computing system to:
identify a distortion map corresponding to one or more optical characteristics of a head-mounted display (HMD), wherein the one or more optical characteristics correspond to one or more lens distortions of a lens of the HMD, wherein the one or more lens distortions are to distort an image;
divide a frame into viewports based on the distortion map;
assign the viewports one or more of different priorities, different resolutions or different frame rates; and
adjust a total number of the viewports based on one or more of a first area of focus, first gaze information, first motion information, first motion prediction information or first content information.

2. The system of claim 1 , wherein the instructions, when executed, cause the computing system to:
assign the viewports the different priorities; and
adjust the different priorities based on one or more of a second area of focus, second gaze information, second motion information, second motion prediction information or second content information.

3. The system of claim 1 , wherein the instructions, when executed, cause the computing system to:
adjust sizes of the viewports based on one or more of a second area of focus, second gaze information, second motion information, second motion prediction information or second content information.

4. The system of claim 3 , wherein the instructions, when executed, cause the computing system to:
enlarge a size of a first viewport of the viewports that is to include the second area of focus; and
increase a priority of the priorities associated with the first viewport.

5. The system of claim 1 , wherein the instructions, when executed, cause the computing system to:
determine a render order to render the viewports based on the different priorities;
determine resolutions to render the viewports based on the different priorities; and
determine frame rates of the viewports based on the different priorities.

6. The system of claim 1 , wherein the instructions, when executed, cause the computing system to:
load the distortion map.

7. An apparatus comprising:
a memory; and
logic communicatively coupled to the memory, wherein the logic is implemented at least partly in one or more of configurable logic or fixed-functionality logic hardware, the logic communicatively coupled to the memory to:
identify a distortion map corresponding to one or more optical characteristics of a head-mounted display (HMD), wherein the one or more optical characteristics correspond to one or more lens distortions of a lens of the HMD, wherein the one or more lens distortions are to distort an image;
divide a frame into viewports based on the distortion map;
assign the viewports one or more of different priorities, different resolutions or different frame rates; and
adjust a total number of the viewports based on one or more of a first area of focus, first gaze information, first motion information, first motion prediction information or first content information.

8. The apparatus of claim 7 , wherein the logic coupled to the memory is to:
assign the viewports the different priorities; and
adjust the different priorities based on one or more of a second area of focus, second gaze information, second motion information, second motion prediction information or second content information.

9. The apparatus of claim 7 , wherein the logic coupled to the memory is to:
adjust sizes of the viewports based on one or more of a second area of focus, second gaze information, second motion information, second motion prediction information or second content information.

10. The apparatus of claim 9 , wherein the logic coupled to the memory is to:
enlarge a size of a first viewport of the viewports that is to include the second area of focus; and
increase a priority of the priorities associated with the first viewport.

11. The apparatus of claim 7 , wherein the logic coupled to the memory is to:
determine a render order to render the viewports based on the different priorities;
determine resolutions to render the viewports based on the different priorities; and
determine frame rates of the viewports based on the different priorities.

12. The apparatus of claim 7 , wherein the logic coupled to the memory is to:
load the distortion map.

13. At least one non-transitory computer readable storage medium comprising a set of instructions, which when executed by a computing device, cause the computing device to:
identify a distortion map corresponding to one or more optical characteristics of a head-mounted display (HMD), wherein the one or more optical characteristics correspond to one or more lens distortions of a lens of the HMD, wherein the one or more lens distortions are to distort an image;
divide a frame into viewports based on the distortion map;
assign the viewports one or more of different priorities, different resolutions or different frame rates; and
adjust a total number of the viewports based on one or more of a first area of focus, first gaze information, first motion information, first motion prediction information or first content information.

14. The at least one non-transitory computer readable storage medium of claim 13 , wherein the instructions, when executed, cause the computing device to:
assign the viewports the different priorities; and
adjust the different priorities based on one or more of a second area of focus, second gaze information, second motion information, second motion prediction information or second content information.

15. The at least one non-transitory computer readable storage medium of claim 13 , wherein the instructions, when executed, cause the computing device to:
adjust sizes of the viewports based on one or more of a second area of focus, second gaze information, second motion information, second motion prediction information or second content information.

16. The at least one non-transitory computer readable storage medium of claim 15 , wherein the instructions, when executed, cause the computing device to:
enlarge a size of a first viewport of the viewports that is to include the second area of focus; and
increase a priority of the priorities associated with the first viewport.

17. The at least one non-transitory computer readable storage medium of claim 13 , wherein the instructions, when executed, cause the computing device to:
determine a render order to render the viewports based on the different priorities;
determine resolutions to render the viewports based on the different priorities; and
determine frame rates of the viewports based on the different priorities.

18. The at least one non-transitory computer readable storage medium of claim 13 , wherein the instructions, when executed, cause the computing device to:
load the distortion map.

19. A method comprising:
identifying a distortion map corresponding to one or more optical characteristics of a head-mounted display (HMD), wherein the one or more optical characteristics correspond to one or more lens distortions of a lens of the HMD, wherein the one or more lens distortions are to distort an image;
dividing a frame into viewports based on the distortion map;
assigning the viewports one or more of different priorities, different resolutions or diff

Assignees

• Intel Corp

Inventors

• Vembar Deepak S
• Kuwahara Atsuo
• Sakthivel Chandrasekaran
• Venkataraman Radhakrishnan
• Insko Brent E
• Kalra Anupreet S
• Labbe Hugues
• Appu Abhishek R
• Shah Ankur N
• Ray Joydeep
• Ould-Ahmed-Vall Elmoustapha
• Holland James M

Classifications

• G06T9/00 Primary

  Image coding (bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00) · CPC title

• G06T11/60 Primary

  Creating or editing images; Combining images with text · CPC title

• H04N19/503

  involving temporal prediction (adaptive coding with adaptive selection between spatial and temporal predictive coding H04N19/107; adaptive coding with adaptive selection among a plurality of temporal predictive coding modes H04N19/109) · CPC title

• H04N19/124

  Quantisation · CPC title

• H04N19/17

  the unit being an image region, e.g. an object · CPC title