Device and method for foveated rendering

Patent metadata

Field	Value
Publication number	US-11748956-B2
Application number	US-202117500281-A
Country	US
Kind code	B2
Filing date	Oct 13, 2021
Priority date	Oct 13, 2021
Publication date	Sep 5, 2023
Grant date	Sep 5, 2023

Abstract

A display driver includes interface circuitry, image processing circuitry, and drive circuitry. The interface circuitry is configured to receive a full frame image and a foveal image from a source external to the display driver. The image processing circuitry is configured to: upscale the full frame image; render a foveated image from the upscaled full frame image and the foveal image. The foveated image includes a foveal area based on the foveal image, a peripheral are based on the upscaled full frame image, and a border area based on the foveal image and the upscaled full frame image. The border area being located between the foveal area and the peripheral area. The drive circuitry is configured to drive a display panel using the foveated image.

First claim

What is claimed is:

1. A display driver, comprising:
interface circuitry configured to receive a full frame image and a foveal image from a source external to the display driver;
image processing circuitry configured to:
upscale the full frame image;
render a foveated image from the upscaled full frame image and the foveal image, wherein the foveated image comprises:
a foveal area based on the foveal image;
a peripheral area based on the upscaled full frame image; and
a border area based on the foveal image and the upscaled full frame image, the border area being located between the foveal area and the peripheral area; and

drive circuitry configured to drive a display panel using the foveated image.

2. The display driver of claim 1 , wherein the rendering of the foveated image comprises determining a graylevel of a target pixel in the border area based on a weighted sum of a graylevel of a first pixel in the foveal image and a graylevel of a second pixel in the upscaled full frame image.

3. The display driver of claim 2 , wherein a weight is assigned to the first pixel in the weighted sum based on a position of the target pixel in the border area.

4. The display driver of claim 3 , wherein the weight assigned to the first pixel increases as a distance of the target pixel from the foveal area decreases.

5. The display driver of claim 1 , wherein the border area comprises:
a first section that surrounds the foveal area; and
a second section that surrounds the first section,
wherein rendering the foveated image comprises:
determining a graylevel of a first target pixel in the first section based on a first weighted sum of a graylevel of a first pixel in the foveal image and a graylevel of a second pixel in the upscaled full frame image; and
determining a graylevel of a second target pixel in the second section based on a second weighted sum of a graylevel of a third pixel in the foveal image and a graylevel of a fourth pixel in the upscaled full frame image, and

wherein a first weight is assigned to the first pixel in the first weighted sum and a second weight is assigned to the third pixel in the second weighted sum, the first weight being larger than the second weight.

6. The display driver of claim 5 , wherein the border area further comprises a third section that surrounds the second section, and
wherein the rendering of the border area further comprises:
determining a graylevel of a third target pixel in the third section based on a third weighted sum of a graylevel of a fifth pixel in the foveal image and a graylevel of a sixth pixel in the upscaled full frame image,
wherein a third weight is assigned to the fifth pixel in the third weighted sum, the second weight being larger than the third weight.

7. The display driver of claim 1 , further comprising:
a register configured to store a border area dimension data set that indicates one or more dimensions of the border area,
wherein the border area is rendered based on the border area dimension data set.

8. The display driver of claim 7 , wherein the border area dimension data set is rewritable by the source.

9. The display driver of claim 1 , wherein the image processing circuitry is further configured to:
receive, via the interface circuitry, control data from the source; and
modify at least one dimension of the border area based on the control data received from the source.

10. The display driver of claim 1 , further comprising:
register circuitry configured to store a plurality of border area dimension data sets each indicating one or more dimensions of the border area,
wherein the image processing circuitry is further configured to:
receive, via the interface circuitry, control data from the source; and
select one of the plurality of border area dimension data sets based on the control data, wherein the border area is rendered based on the selected one of the plurality of border area dimension data sets.

11. The display driver of claim 1 , wherein the full frame image has the same resolution as the foveal image and encompasses a larger display area than the foveal image.

12. The display driver of claim 1 , wherein the rendering the foveated image comprises rendering the border area by alpha blending respective portions of the upscaled full frame image and the foveal image.

13. The display driver of claim 1 , wherein the interface circuitry, the image processing circuitry, and the drive circuitry are monolithically integrated in a single semiconductor chip.

14. A display system, comprising:
a source configured to render a full frame image and a foveal image;
a display panel; and
a display driver configured to:
receive the full frame image and the foveal image from the source;
upscale the full frame image;
render a foveated image from the upscaled full frame image and the foveal image, wherein the foveated image includes:
a foveal area based on the foveal image;
a peripheral area based on the upscaled full frame image; and
a border area based on the foveal image and the upscaled full frame image, the border area being located between the foveal area and the peripheral area; and
drive the display panel using the foveated image.

15. The display system of claim 14 , wherein the rendering of the foveated image comprises determining a graylevel of a target pixel in the border area based on a weighted sum of a graylevel of a first pixel in the foveal image and a graylevel of a second pixel in the upscaled full frame image.

16. The display system of claim 15 , wherein a weight is assigned to the first pixel in the weighted sum based on a position of the target pixel in the border area.

17. The display system of claim 14 , wherein the border area comprises:
a first section that surrounds the foveal area; and
a second section that surrounds the first section,
wherein rendering the foveated image comprises:
determining a graylevel of a first target pixel in the first section based on a first weighted sum of a graylevel of a first pixel in the foveal image and a graylevel of a second pixel in the upscaled full frame image; and
determining a graylevel of a second target pixel in the second section based on a second weighted sum of a graylevel of a third pixel in the foveal image and a graylevel of a fourth pixel in the upscaled full frame image,

wherein a first weight is assigned to the first pixel in the first weighted sum and a second weight is assigned to the third pixel in the second weighted sum, the first weight being larger than the second weight.

18. The display system of claim 14 , wherein the source is configured as a separate integrated circuit from the display driver.

19. A method of operating a display driver, comprising:
receiving a full frame image and a foveal image from a source external to the display driver;
upscaling the full frame image; and
rendering a foveated image from the upscaled full frame image and the foveal image, wherein the foveated image comprises:
a foveal area based on the foveal image;
a peripheral area based on the upscaled full frame image; and
a border area based on the foveal image and the upscaled full frame image, the border area being located between the foveal area and the peripheral area; and

driving a display panel using the foveated image.

20. The method of claim 19 , wherein the rendering of the foveated image comprises determining a graylevel of a target pixel in the border area based on a weighted sum of a graylevel of a first pixel in the fov

Assignees

• Synaptics Inc

Inventors

• Kitamura Kota
• Sakamaki Goro
• MINAKI Tomoo

Classifications

• G06T5/70 Primary

  Denoising; Smoothing · CPC title

• G06T3/04

  Context-preserving transformations, e.g. by using an importance map (panospheric to cylindrical image transformations G06T3/12) · CPC title

• G06T3/10

  Selection of transformation methods according to the characteristics of the input images · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06T3/0056

  Physics · mapped topic