Method of compressing image data for network transport

Patent metadata

Field	Value
Publication number	US-11830168-B2
Application number	US-202217704638-A
Country	US
Kind code	B2
Filing date	Mar 25, 2022
Priority date	Mar 31, 2021
Publication date	Nov 28, 2023
Grant date	Nov 28, 2023

Abstract

A method for improved compression and filtering of images for foveated transport applications. The improvements including calculating along at least one axis of the full-resolution image data set, the distance from the foveal point to edges of full-resolution image data set, calculating the distance from each edge of the full resolution data set to the closest point on a foveal region surrounding the foveal point, calculating the distribution of available space in the adjacent peripheral regions of the image using a compression parameter, and calculating a compression curve wherein the compression of both adjacent peripheral regions is such that neither side of the adjacent peripheral regions are compressed more than the other.

First claim

The invention claimed is:

1. A method for providing images in an image transmission system comprising an image source server and an image display client, the image source server having a full-resolution image data set, and the image display client having a user display, the method comprising:
determining a viewer's point of interest in a display device by the client;
separating the full-resolution image data set into a full-resolution foveal section data set and a full-resolution peripheral section data set by the server by:
determining position of a foveal point on the display device and a full-resolution foveal section data set, using a point of interest,
calculating along at least one axis of the full-resolution image data set, a distance from the foveal point to edges of full-resolution image data set,
calculating a distance from each edge of the full resolution data set to a foveal region surrounding the foveal point, and determining the full-resolution peripheral section data set which is made up of adjacent peripheral regions from the full resolution data set;

compressing the full-resolution peripheral data set to create a reduced resolution peripheral data set by reducing resolution of an image it represents by:
calculating a distribution of available space in the adjacent peripheral regions of the image represented by the full-resolution peripheral data set using a compression parameter,
calculating a compression curve wherein compression of both adjacent peripheral regions is such that neither side of the adjacent peripheral regions are compressed more than the other, for a compression of the image,
mapping the image when thus compressed to the image when uncompressed using a mapping function,
filtering, using the compression curve and its derivative, the full-resolution peripheral data set using a gaussian filter so to remove potential visual artifacts occurring due to the compression of the image;

transmitting the full-resolution foveal section data set and the reduced resolution peripheral data set to the client; and
displaying the image to a user by the client comprising a full-resolution foveal area at a current eye position surrounded by a reduced resolution peripheral area according to the transmitted data sets.

2. The method of claim 1 , wherein the distance from the foveal point to edges of full-resolution image data set is calculated along two or more axes of the full-resolution image data set.

3. The method of claim 1 , wherein a length of foveal region is inputted as a parameter.

4. The method of claim 1 , wherein based on a calculation that the foveal point is closer to an edge of image than half of the length of the foveal region, then the foveal point is adjusted so that the foveal point is half of the length of the foveal region away from the edge of the image.

5. The method of claim 1 , wherein the calculating a distribution of available space in the adjacent peripheral regions of the image comprises calculating a length of output image by dividing the full resolution data set size by a function of a compression level, wherein compression level is inputted as parameter, and subtracting the length of the foveal region from the length of the output image to obtain the available space to compress the peripheral area into.

6. The method of claim 5 , wherein the length of output image is calculated by dividing the full resolution data set divided by the square root of compression level.

7. The method of claim 5 , wherein the mapping function comprises mapping a center of a discrete pixel of the compressed image to a point on the uncompressed image, and wherein the mapping function has a calculable derivative which describes for any point on the output image, one or several points on an original image to which it has been compressed from.

8. The method of claim 5 , wherein the mapping function has a calculable derivative which describes for any point on the output image, one or several points on an original image to which it has been compressed from.

9. The method of claim 1 , wherein a kernel size is controlled by the derivative of the compression curve.

10. The method of claim 1 further comprising filtering, using a gaussian filter applied by using a kernel size determined by a derivative of the mapping function to further reduce visual artifacts caused by compression.

11. The method of claim 1 wherein a step of determining a viewer's point of interest in a display device comprises providing an eye movement tracking system associated with the image display client.

12. The method of claim 1 , wherein the display device is a HMD.

13. A non-transitory computer readable medium encoded with software for providing images in an image transmission system comprising an image source server and an image display client, the image source server having a full-resolution image data set, and the image display client having a user display, the software causing the server and client to perform steps of:
determining a viewer's point of interest in a display device by the client;
separating a full-resolution image data set into a full-resolution foveal section data set and a full-resolution peripheral section data set by the server by:
determining position of a foveal point on the display device and a full-resolution foveal section data set, using a point of interest,
calculating along at least one axis of the full-resolution image data set, a distance from the foveal point to edges of full-resolution image data set,
calculating a distance from each edge of the full resolution data set to a foveal region surrounding the foveal point, and determining the full-resolution peripheral section data set which is made up of adjacent peripheral regions from the full resolution data set;

compressing the full-resolution peripheral data set to create a reduced resolution peripheral data set by reducing resolution of an image it represents by:
calculating a distribution of available space in the adjacent peripheral regions of the image represented by the full-resolution peripheral data set using a compression parameter,
calculating a compression curve wherein compression of both adjacent peripheral regions is such that neither side of the adjacent peripheral regions are compressed more than the other, for a compression of the image,
mapping the image when thus compressed to the image when uncompressed using a mapping function,
filtering, using the compression curve and its derivative, the full-resolution peripheral data set using a gaussian filter so to remove potential visual artifacts occurring due to the compression of the image;

transmitting the full-resolution foveal section data set and the reduced resolution peripheral data set to the client; and
displaying the image to a user by the client comprising a full-resolution foveal area at a current eye position surrounded by a reduced resolution peripheral area according to the transmitted data sets.

14. The non-transitory computer readable medium of claim 13 , wherein the distance from the foveal point to edges of full-resolution image data set is calculated along two or more axes of the full-resolution image data set.

15. The non-transitory computer readable medium of claim 13 , wherein a length of foveal region is inputted as a parameter.

16. The non-transitory computer readable medium of claim 13 , wherein based on the calculation that the foveal point is closer to the edge of image than half of the length of the foveal region, then the foveal point is adjusted so that the foveal point is half of the length of the foveal regi

Assignees

• Tobii Ab

Inventors

• Bezugly Alexey
• Rådell Dennis
• Brannan Ritchie

Classifications

• G06T3/4092 Primary

  Image resolution transcoding, e.g. by using client-server architectures · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06T5/002

  Physics · mapped topic

• G06T5/20

  using local operators · CPC title

• G09G3/002

  to project the image of a two-dimensional display, such as an array of light emitting or modulating elements or a CRT · CPC title