Rendering composite content on a head-mounted display including a high resolution inset

Patent metadata

Field	Value
Publication number	US-9779478-B1
Application number	US-201615284898-A
Country	US
Kind code	B1
Filing date	Oct 4, 2016
Priority date	Oct 4, 2016
Publication date	Oct 3, 2017
Grant date	Oct 3, 2017

Abstract

A head-mounted display (HMD) divides an image into a high resolution (HR) inset portion at a first resolution, a peripheral portion, and a transitional portion. The peripheral portion is downsampled to a second resolution that is less than the first resolution. The transitional portion is blended such that there is a smooth change in resolution that corresponds to a change in resolution between a fovea region and a non-fovea region of a retina. An inset region is generated using the HR inset portion and the blended transitional portion, and a background region is generated using the downsampled peripheral portion. The inset region is provided to a HR inset display, and the background region is provided to a peripheral display. An optics block combines the displayed inset region with the displayed background region to generate composite content.

First claim

What is claimed is:

1. A method comprising:
dividing an image into a high resolution (HR) inset portion, a peripheral portion, and a transitional portion between the HR inset portion and the peripheral portion, wherein the image is at a first resolution corresponding to a resolution of a fovea region of a human eye, and the transitional portion has an inner boundary between the transitional portion and the HR inset portion and an outer boundary between the transitional portion and the peripheral portion;
downsampling the peripheral portion to a second resolution that is less than the first resolution, the second resolution corresponding to a resolution of a non-fovea region of the human eye;
blending the transitional portion of the image such that there is a smooth change in resolution from the inner boundary at the first resolution to the outer boundary at the second resolution, the smooth change corresponding to a change in resolution between the fovea region and the non-fovea region;
generating an inset region using the HR inset portion of the image and the blended transitional portion of the image;
generating a background region using the downsampled peripheral portion;
providing, for display, the inset region to a HR inset display on a head-mounted display (HMD);
providing, for display, the background region to a peripheral display on the HMD, and
wherein an optics block combines the displayed inset region with the displayed background region to generate composite content.

2. The method of claim 1 , wherein dividing an image into a high resolution (HR) inset portion, a peripheral portion, and a transitional portion between the HR inset portion and the peripheral portion, further comprises:
applying a first fading function to the image that causes the inset portion and the transitional portion to fade to black, the faded image being the peripheral portion; and
applying a second fading function to some of the transitional portion of the image that surrounds the HR inset.

3. The method of claim 1 , wherein dividing an image into a high resolution (HR) inset portion, a peripheral portion, and a transitional portion between the HR inset portion and the peripheral portion is based in part on an eye orientation of a user of the HMD.

4. The method of claim 1 , wherein blending the transitional portion of the image such that there is a smooth change in resolution from the inner boundary at the first resolution to the outer boundary at the second resolution, the smooth change corresponding to a change in resolution between the fovea region and the non-fovea region, comprises:
applying a blurring function that blurs a portion of the transitional portion of the image.

5. The method of claim 1 , wherein the background region that is displayed using the peripheral display passes through an optical anti-aliasing filter before being combined with the inset portion that is displayed using the HR inset display, the optical anti-aliasing filter optically blurring some of the background region.

6. The method of claim 1 , further comprising:
calibrating color for the peripheral display and the inset display such that they both are within a threshold range of color values.

7. The method of claim 1 , further comprising:
calibrating luminance for the peripheral display and the inset display such that they both are within a threshold range of luminance values.

8. The method of claim 1 , wherein the composite image has variable resolution that matches a resolution of a retina of a human eye.

9. A non-transitory computer-readable storage medium storing executable computer program instructions, the instructions executable to perform steps comprising:
dividing an image into a high resolution (HR) inset portion, a peripheral portion, and a transitional portion between the HR inset portion and the peripheral portion, wherein the image is at a first resolution corresponding to a resolution of a fovea region of a human eye, and the transitional portion has an inner boundary between the transitional portion and the HR inset portion and an outer boundary between the transitional portion and the peripheral portion;
downsampling the peripheral portion to a second resolution that is less than the first resolution, the second resolution corresponding to a resolution of a non-fovea region of the human eye;
blending the transitional portion of the image such that there is a smooth change in resolution from the inner boundary at the first resolution to the outer boundary at the second resolution, the smooth change corresponding to a change in resolution between the fovea region and the non-fovea region;
generating an inset region using the HR inset portion of the image and the blended transitional portion of the image;
generating a background region using the downsampled peripheral portion;
providing, for display, the inset region to a HR inset display on a head-mounted display (HMD);
providing, for display, the background region to a peripheral display on the HMD, and
wherein an optics block combines the displayed inset region with the displayed background region to generate composite content.

10. The computer readable medium of claim 9 , wherein dividing an image into a high resolution (HR) inset portion, a peripheral portion, and a transitional portion between the HR inset portion and the peripheral portion, further comprises:
applying a first fading function to the image that causes the inset portion and the transitional portion to fade to black, the faded image being the peripheral portion; and
applying a second fading function to some of the transitional portion of the image that surrounds the HR inset.

11. The computer readable medium of claim 9 , wherein dividing an image into a high resolution (HR) inset portion, a peripheral portion, and a transitional portion between the HR inset portion and the peripheral portion is based in part on an eye orientation of a user of the HMD.

12. The computer readable medium of claim 9 , wherein blending the transitional portion of the image such that there is a smooth change in resolution from the inner boundary at the first resolution to the outer boundary at the second resolution, the smooth change corresponding to a change in resolution between the fovea region and the non-fovea region, comprises:
applying a blurring function that blurs a portion of the transitional portion of the image.

13. The computer readable medium of claim 9 , wherein the background region that is displayed using the peripheral display passes through an optical anti-aliasing filter before being combined with the inset portion that is displayed using the HR inset display, the optical anti-aliasing filter optically blurring some of the background region.

14. The computer readable medium of claim 9 , further comprising:
calibrating color for the peripheral display and the inset display such that they both are within a threshold range of color values.

15. The computer readable medium of claim 9 , further comprising:
calibrating luminance for the peripheral display and the inset display such that they both are within a threshold range of luminance values.

16. The computer readable medium of claim 9 , wherein the composite image has variable resolution that matches a resolution of a retina of a human eye.

17. A head-mounted display (HMD) comprising:
a controller configured to:
divide an image into a high resolution (HR) inset portion, a peripheral portion, and a transitional portion between the HR inset portion and the peripheral portion, wherein the image is at a fir

Assignees

• Oculus Vr Llc

Inventors

• Wilson Austin
• Lanman Douglas Robert
• Trail Nicholas Daniel
• McEldowney Scott Charles
• Mcnally Stephen James
• Sulai Yusufu Njoni Bamaxam

Classifications

• G06T11/00

  Two-dimensional [2D] image generation · CPC title

• G09G3/002

  to project the image of a two-dimensional display, such as an array of light emitting or modulating elements or a CRT · CPC title

• G02B27/0172

  characterised by optical features · CPC title

• G06T7/12

  Edge-based segmentation · CPC title

• G06T3/4092 Primary

  Image resolution transcoding, e.g. by using client-server architectures · CPC title