Foveated compression of display streams

Patent metadata

Field	Value
Publication number	US-10319114-B2
Application number	US-201715649047-A
Country	US
Kind code	B2
Filing date	Jul 13, 2017
Priority date	Jul 13, 2017
Publication date	Jun 11, 2019
Grant date	Jun 11, 2019

Abstract

A processing unit is configured to render first pixels representative of a high-acuity region in the image and second pixels representative of a low-acuity region in the image. A shaper is configured to reorganize the first pixels based on at least one dimension of the low-acuity region. A multiplexer is configured to multiplex the reorganized first pixels and the second pixels to form a display stream. An encoder is configured to compress the display stream for transmission to a display. A decoder configured to decompress the display stream. A demultiplexer is configured to demultiplex the first pixels and the second pixels. Another processing unit is configured to blend the first pixels and the second pixels to form blended pixel values representative of the image for presentation on a screen.

First claim

What is claimed is:

1. A method comprising:
accessing first pixels from a first buffer of an electronic device, wherein values of the first pixels represent a high-acuity region in an image and wherein the first pixels are organized as a set of first pixel lines in the first buffer;
accessing second pixels from a second buffer of the electronic device, wherein values of the second pixels represent a low-acuity region in the image and wherein the second pixels are organized as a set of second pixel lines in the second buffer;
reorganizing the first pixels based on at least one dimension of the low-acuity region, wherein reorganizing the first pixels comprises concatenating subsets of the set of first pixel lines to form modified first pixel lines including a number of first pixels that is equal to a number of second pixels in the second pixel lines;
multiplexing the reorganized first pixels and the second pixels to form a display stream; and
compressing the display stream for transmission to a display.

2. The method of claim 1 , wherein multiplexing the reorganized first pixels and the second pixels comprises combining the modified first pixel lines and the second pixel lines on a line-by-line basis.

3. The method of claim 2 , wherein combining the modified first pixel lines and the second pixel lines on the line-by-line basis comprises concatenating the modified first pixel lines and the second pixel lines.

4. The method of claim 3 , wherein concatenating the modified first pixel lines and the second pixel lines comprises appending the modified first pixel lines above, below, to the left, or to the right of the second pixel lines.

5. The method of claim 2 , wherein combining the modified first pixel lines and the second pixel lines on the line-by-line basis comprises interlacing the modified first pixel lines and the second pixel lines on a line-by-line basis.

6. The method of claim 5 , wherein interlacing the modified first pixel lines and the second pixel lines comprises partitioning the modified first pixel lines and the second pixel lines into slices having a height that corresponds to a slice height used to compress the display stream.

7. The method of claim 6 , wherein interlacing the modified first pixel lines and the second pixel lines comprises interlacing a number s 2 of slices of the modified first pixel lines with each slice of the second pixel lines, wherein s represents an upscaling factor that is applied to the low-acuity region.

8. The method of claim 5 , wherein interlacing the modified first pixel lines and the second pixel lines comprises statically interlacing the modified first pixel lines at a fixed position relative to the second pixel lines.

9. The method of claim 5 , wherein interlacing the modified first pixel lines and the second pixel lines comprises dynamically interlacing the modified first pixel lines at a variable position relative to the second pixel lines, wherein the variable position is determined based on relative positions of the high-acuity region and the low-acuity region in the image.

10. The method of claim 1 , further comprising:
decompressing the display stream;
demultiplexing the display stream to generate the reorganized first pixels and the second pixels; and
reconstructing an original configuration of the first pixels from the reorganized first pixels.

11. A method comprising:
accessing first pixels from a first buffer of an electronic device, wherein values of the first pixels represent a high-acuity region in an image;
accessing second pixels from a second buffer of the electronic device, wherein values of the second pixels represent a low-acuity region in the image;
reorganizing the first pixels based on at least one dimension of the low-acuity region;
multiplexing the reorganized first pixels and the second pixels to form a display stream; and
compressing the display stream for transmission to a display, wherein compressing the display stream comprises compressing slices of the display stream on a line-by-line basis, wherein each slice of the display stream includes a number of lines that is an integer fraction of a total number of the modified first pixel lines.

12. The method of claim 10 , further comprising:
blending the original configuration of the first pixels and the second pixels to form blended pixel values representative of the image; and
displaying the blended pixel values on a screen.

13. An apparatus comprising:
a processing unit configured to render first pixels representative of a high-acuity region in an image and second pixels representative of a low-acuity region in the image;
a first buffer configured to store values of the first pixels rendered by the processing unit, wherein the first pixels are organized as a set of first pixel lines in the first buffer;
a second buffer configured to store values of the second pixels rendered by the processing unit, wherein the second pixels are organized as a set of second pixel lines in the second buffer;
a shaper configured to reorganize the first pixels based on at least one dimension of the low-acuity region, wherein the shaper is configured to access the values of the first pixels from the first buffer and to access the values of the second pixels from the second buffer and configured to reorganized the first pixels by concatenating subsets of the set of first pixel lines to form modified first pixel lines including a number of first pixels that is equal to a number of second pixels in the second pixel lines;
a multiplexer configured to multiplex the reorganized first pixels and the second pixels to form a display stream; and
an encoder configured to compress the display stream for transmission to a display.

14. The apparatus of claim 13 , wherein the multiplexer is configured to combine the modified first pixel lines and the second pixel lines on a line-by-line basis.

15. The apparatus of claim 14 , wherein the multiplexer is configured to concatenate the modified first pixel lines and the second pixel lines.

16. The apparatus of claim 15 , wherein the multiplexer is configured to append the modified first pixel lines above, below, to the left, or to the right of the second pixel lines.

17. The apparatus of claim 14 , wherein the multiplexer is configured to interlace the modified first pixel lines and the second pixel lines on a line-by-line basis.

18. The apparatus of claim 17 , wherein the multiplexer is configured to interlace the modified first pixel lines and the second pixel lines into slices having a height that corresponds to a slice height used to compress the display stream.

19. The apparatus of claim 18 , wherein the multiplexer is configured to interlace a number s 2 of slices of the modified first pixel lines with each slice of the second pixel lines, wherein s represents an upscaling factor that is applied to the low-acuity region.

20. The apparatus of claim 17 , wherein the multiplexer is configured to statically interlace the modified first pixel lines at a fixed position relative to the second pixel lines.

21. The apparatus of claim 17 , wherein the multiplexer is configured to dynamically interlace the modified first pixel lines at a variable position relative to the second pixel lines, wherein the variable position is determined based on relative positions of the high-acuity region and the low-acuity region in the image.

22. An apparatus comprising:
a processing unit configured to render first pixels representative of a hig

Assignees

• Google Inc
• Google Llc

Inventors

• Bastani Behnam
• Vieri Carlin
• Jiang Haomiao

Classifications

• G09G2340/10

  Mixing of images, i.e. displayed pixel being the result of an operation, e.g. adding, on the corresponding input pixels · CPC title

• G09G2310/0297

  Special arrangements with multiplexing or demultiplexing of display data in the drivers for data electrodes, in a pre-processing circuitry delivering display data to said drivers or in the matrix panel, e.g. multiplexing plural data signals to one D/A converter or demultiplexing the D/A converter output to multiple columns · CPC title

• G09G2340/0407

  Resolution change, inclusive of the use of different resolutions for different screen areas · CPC title

• G06T9/00 Primary

  Image coding (bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00) · CPC title

• G06F3/147

  using display panels · CPC title