Dynamic remote display resource management

Patent metadata

Field	Value
Publication number	US-11561809-B2
Application number	US-202117367581-A
Country	US
Kind code	B2
Filing date	Jul 5, 2021
Priority date	Jul 20, 2020
Publication date	Jan 24, 2023
Grant date	Jan 24, 2023

Abstract

Systems and methods for remote display resource management can allow a remote display to be dynamically modified according to user eye movements. A user device can receive graphical data for a remote display and present the remote display on a local display. A remote display module (RD module) can detect dynamic content in the graphical data. The RD module can monitor eye movements of a user using an eye tracker device to determine when a user is not looking at the dynamic content. When a user looks away from the dynamic content for more than an allowed amount of time, the RD module can send instructions to a server to reduce the quality of the dynamic content in the remote display. The RD module can send instructions to restore the quality of the dynamic content after determining that the user is looking at the dynamic content.

First claim

What is claimed is:

1. A method for dynamic remote display resource management, comprising:
using eye tracking data received from an eye tracking device, determining that a user is not looking at a screen;
sending, to a server, instructions to downgrade graphical data that displays on the screen,
wherein the downgrade includes reducing a quality of the graphical data;

receiving, from the server, downgraded graphical data; and
displaying the downgraded graphical data on the screen.

2. The method of claim 1 , further comprising detecting dynamic content in the graphical data displayed on the screen prior to sending the instructions to downgrade the graphical data.

3. The method of claim 1 , further comprising:
determining that the user is looking at the downgraded graphical data;
sending, to the server, additional instructions to upgrade the graphical data;
receiving, from the server, upgraded graphical data for the remote display; and
displaying the upgraded graphical data.

4. The method of claim 1 , wherein reducing the quality of the graphical data includes at least one of: reducing a frame rate, reducing an image quality, and reducing a maximum bandwidth available for sending the graphical data.

5. The method of claim 1 , wherein determining that the user is not looking at the screen includes determining that the user is not looking at any portion of a first display from among multiple displays, and the downgrade includes freezing the graphical data of the first display while not freezing graphical data for at least one other of the multiple displays.

6. The method of claim 1 , further comprising:
in an instance where the user is not looking at dynamic content displayed on a second screen, starting a no-gazing timer for the dynamic content; and
determining whether to send second instructions to downgrade graphical data of the second display based on whether the user looks back at the dynamic content before the no-gazing timer reaches a threshold.

7. The method of claim 1 ,
wherein a remote display agent executes as part of a virtual machine that runs on the server, wherein the virtual machine provides a desktop for the user,
wherein a virtual desktop client executes on a user device and performs at least the displaying and receiving, and
wherein the virtual desktop client and remote display agent communicate using a remoting protocol.

8. A non-transitory, computer-readable medium containing instructions that, when executed by a hardware-based processor, performs stages for dynamic remote display resource management, the stages comprising:
using eye tracking data received from an eye tracking device, determining that a user is not looking at a screen;
sending, to a server, instructions to downgrade graphical data that displays on the screen,
wherein the downgrade includes reducing a quality of the graphical data;

receiving, from the server, downgraded graphical data; and
displaying the downgraded graphical data on the screen.

9. The non-transitory, computer-readable medium of claim 8 , the stages further comprising detecting dynamic content in the graphical data displayed on the screen prior to sending the instructions to downgrade the graphical data.

10. The non-transitory, computer-readable medium of claim 8 , the stages further comprising:
determining that the user is looking at the downgraded graphical data;
sending, to the server, additional instructions to upgrade the graphical data;
receiving, from the server, upgraded graphical data for the remote display; and
displaying the upgraded graphical data.

11. The non-transitory, computer-readable medium of claim 8 , wherein reducing the quality of the graphical data includes at least one of: reducing a frame rate, reducing an image quality, and reducing a maximum bandwidth available for sending the graphical data.

12. The non-transitory, computer-readable medium of claim 8 , wherein determining that the user is not looking at the screen includes determining that the user is not looking at any portion of a first display from among multiple displays, and the downgrade includes freezing the graphical data of the first display while not freezing graphical data for at least one other of the multiple displays.

13. The non-transitory, computer-readable medium of claim 8 , the stages further comprising:
in an instance where the user is not looking at dynamic content displayed on a second screen, starting a no-gazing timer for the dynamic content; and
determining whether to send second instructions to downgrade graphical data of the second display based on whether the user looks back at the dynamic content before the no-gazing timer reaches a threshold.

14. The non-transitory, computer-readable medium of claim 8 ,
wherein a remote display agent executes as part of a virtual machine that runs on the server, wherein the virtual machine provides a desktop for the user,
wherein a virtual desktop client executes on a user device and performs at least the displaying and receiving, and
wherein the virtual desktop client and remote display agent communicate using a remoting protocol.

15. A system for dynamic remote display resource management, comprising:
a hardware memory that stores instructions; and
a processor that executes the instructions to perform stages comprising:
using eye tracking data received from an eye tracking device, determining that a user is not looking at a screen;
sending, to a server, instructions to downgrade graphical data that displays on the screen, wherein the downgrade includes reducing a quality of the graphical data;
receiving, from the server, downgraded graphical data; and
displaying the downgraded graphical data on the screen.

16. The system of claim 15 , the stages further comprising detecting dynamic content in the graphical data displayed on the screen prior to sending the instructions to downgrade the graphical data.

17. The system of claim 15 , the stages further comprising:
determining that the user is looking at the downgraded graphical data;
sending, to the server, additional instructions to upgrade the graphical data;
receiving, from the server, upgraded graphical data for the remote display; and
displaying the upgraded graphical data.

18. The system of claim 15 , wherein reducing the quality of the graphical data includes at least one of: reducing a frame rate, reducing an image quality, and reducing a maximum bandwidth available for sending the graphical data.

19. The system of claim 15 , wherein determining that the user is not looking at the screen includes determining that the user is not looking at any portion of a first display from among multiple displays, and the downgrade includes freezing the graphical data of the first display while not freezing graphical data for at least one other of the multiple displays.

20. The non-transitory, computer-readable medium of claim 8 , the stages further comprising:
in an instance where the user is not looking at dynamic content displayed on a second screen, starting a no-gazing timer for the dynamic content; and
determining whether to send second instructions to downgrade graphical data of the second display based on whether the user looks back at the dynamic content before the no-gazing timer reaches a threshold.

Assignees

• Vmware Inc

Inventors

• Song Jian Ken
• Lv Lin
• Liu Jack
• Sun Yue
• Cheng Sarah

Classifications

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06F9/5044

  considering hardware capabilities · CPC title

• G06F2009/45575

  Starting, stopping, suspending or resuming virtual machine instances · CPC title

• G06F16/9035

  Filtering based on additional data, e.g. user or group profiles · CPC title

• G06F9/452 Primary

  Remote windowing, e.g. X-Window System, desktop virtualisation (protocols for virtual reality H04L67/131) · CPC title