Using contextual and spatial awareness to improve remote desktop imaging fidelity

Patent metadata

Field	Value
Publication number	US-10360046-B2
Application number	US-201715436097-A
Country	US
Kind code	B2
Filing date	Feb 17, 2017
Priority date	Dec 26, 2012
Publication date	Jul 23, 2019
Grant date	Jul 23, 2019

Abstract

Image data representing a desktop image for a client device that is accessing the desktop remotely is compressed according to a method that preserves image fidelity in selected non-text regions. The method, which is carried out in a remote server, includes the steps of generating image data for the remote desktop image and analyzing different regions of the remote desktop image, identifying those regions of the remote desktop image that are text regions, selecting non-text regions of the remote desktop image for lossless compression based on a spatial relationship between the non-text regions and the text regions, compressing the image data using a lossless compression protocol for a portion of the image data corresponding to the selected non-text regions, and transmitting the compressed image data to the client device.

First claim

What is claimed is:

1. A method of refining an image, comprising:
transmitting a version of the image to a client device; and
responsive to determining that the image has remained static,
selecting one or more regions in the image based, at least in part, on a spatial relationship between the one or more regions and at least one feature in the image, and
transmitting, to the client device, image data for improving a quality of the version of the image transmitted to the client device, wherein image data corresponding to the one or more selected regions is transmitted prior to image data corresponding to at least one region that is not selected.

2. The method of claim 1 , wherein the image is an image of a remote desktop.

3. The method of claim 2 , wherein:
the spatial relationship specifies that non-text regions located higher in the image of the remote desktop are given higher weighting factors than non-text regions located lower in the image; and
non-text regions given higher weighting factors are selected prior to non-text regions given lower weighting factors.

4. The method of claim 2 , wherein:
the at least one feature includes a location of a mouse cursor; and
the spatial relationship is a proximity to the mouse cursor.

5. The method of claim 1 , wherein the at least one feature includes a text region in the image.

6. The method of claim 5 , wherein the spatial relationship is one of being directly above, directly below, directly to the right of, or directly to the left of the text region.

7. The method of claim 5 , wherein non-text regions that surround the text region are selected.

8. The method of claim 5 , wherein the spatial relationship is a predefined distance from the text region.

9. The method of claim 1 , wherein the selecting of the one or more regions is further based on an available network bandwidth.

10. The method of claim 9 , wherein:
the spatial relationship specifies an area around the at least one feature from which non-text regions are to be selected; and
a size of the area is inversely proportional to network bandwidth constraints.

11. A non-transitory computer-readable storage medium comprising instructions for causing a computer system to carry out a method of refining an image, said method comprising the steps of:
transmitting a version of the image to a client device; and
responsive to determining that the image has remained static:
selecting one or more regions in the image based, at least in part, on a spatial relationship between the one or more regions and at least one feature in the image, and
transmitting, to the client device, image data for improving a quality of the version of the image transmitted to the client device, wherein image data corresponding to the one or more selected regions is transmitted prior to image data corresponding to at least one region that is not selected.

12. The computer-readable storage medium of claim 11 , wherein the image is an image of a remote desktop.

13. The computer-readable storage medium of claim 12 , wherein:
the spatial relationship specifies that non-text regions located higher in the image of the remote desktop are given higher weighting factors than non-text regions located lower in the image; and
non-text regions given higher weighting factors are selected prior to non-text regions given lower weighting factors.

14. The computer-readable storage medium of claim 12 , wherein:
the at least one feature includes a location of a mouse cursor; and
the spatial relationship is a proximity to the mouse cursor.

15. The computer-readable storage medium of claim 11 , wherein the at least one feature includes a text region in the image.

16. The computer-readable storage medium of claim 15 , wherein the spatial relationship is one of being directly above, directly below, directly to the right of, or directly to the left of the text region.

17. The computer-readable storage medium of claim 15 , wherein non-text regions that surround the text region are selected.

18. The computer-readable storage medium of claim 15 , wherein the spatial relationship is a predefined distance from the text region.

19. The computer-readable storage medium of claim 11 , wherein the selecting of the one or more regions is further based on an available network bandwidth.

20. A system, comprising:
a processor; and
a memory, wherein the memory includes a program configured to perform operations for refining an image, the operations comprising:
transmitting a version of the image to a client device, and
responsive to determining that the image has remained static:
selecting one or more regions in the image based, at least in part, on a spatial relationship between the one or more regions and at least one feature in the image; and
transmitting, to the client device, image data for improving a quality of the version of the image transmitted to the client device, wherein image data corresponding to the one or more selected regions is transmitted prior to image data corresponding to at least one region that is not selected.

Assignees

• Vmware Inc

Inventors

• Spracklen Lawrence
• Agrawal Banit
• Bidarkar Rishi

Classifications

• H04N1/4115

  involving the recognition of specific patterns, e.g. by symbol matching · CPC title

• G06F9/452 Primary

  Remote windowing, e.g. X-Window System, desktop virtualisation (protocols for virtual reality H04L67/131) · CPC title

• G06T3/4092

  Image resolution transcoding, e.g. by using client-server architectures · CPC title

• G06K9/344

  Physics · mapped topic

• H04N1/415

  in which the picture-elements are subdivided or grouped into fixed one-dimensional [1D] or two-dimensional [2D] blocks · CPC title