Locally implemented terminal latency mitigation

Patent metadata

Field	Value
Publication number	US-11054915-B2
Application number	US-201916576859-A
Country	US
Kind code	B2
Filing date	Sep 20, 2019
Priority date	Sep 20, 2019
Publication date	Jul 6, 2021
Grant date	Jul 6, 2021

Abstract

Terminal latency mitigation improves performance of networked consoles by embedding latency mitigation functionality in a terminal emulator. An enhanced terminal reduces perceived and actual delays by accurately predicting responses a remote service will provide to key press data. Latency mitigation may receive key press data, transmit it toward the remote service, locate a prediction, obtain a prediction string from the prediction, and write the prediction string to a display. A prediction indication may visually indicate the presence of a prediction string as opposed to an actual response from the remote service. After writing the prediction string, this terminal compares the response to the prediction string. When the prediction is accurate the terminal shows the prediction string without the prediction indication. When the prediction is inaccurate, the terminal replaces the prediction string by the response. The enhanced terminal may be server kernel independent, shell independent, and independent of proprietary communication protocols.

First claim

What is claimed is:

1. A latency-mitigated terminal in operable communication over a network with a remote service, the latency-mitigated terminal having local components comprising:
a display;
a keyboard;
a memory;
a processor in operable communication with the display, the keyboard, and the memory, the processor configured by terminal emulator software which upon execution by the processor performs terminal emulator latency mitigation steps which include:
receiving first key press data which identifies a first pressed key;
transmitting the first pressed key data toward the remote service;
locating a first prediction that is associated with the first key press data, obtaining a first prediction string from the first prediction, and writing the first prediction string to the display together with a prediction indication that visually indicates the first prediction string is based on a response prediction as opposed to being based on a response from the remote service;
after writing the first prediction string to the display, getting from the remote service a first response to the first pressed key;
comparing the first response to the first prediction string;
when the first prediction string is determined to be an accurate prediction because the first response matches the first prediction string, then modifying the display to show the first prediction string without the prediction indication; and
when the first prediction string is determined to not be an accurate prediction because the first response does not match the first prediction string, then modifying the display to replace the first prediction string and the prediction indication by the first response without the prediction indication; and
repeating the foregoing steps with additional pressed keys and additional responses, whereby latency of remote service responses to key press data received by the terminal is mitigated by at least one accurate prediction.

2. The terminal of claim 1 , further comprising a prediction queue residing in the memory, the prediction queue configured to hold a prediction sequence designating prediction strings which have been displayed and have not yet been compared with respective responses from the remote service.

3. The terminal of claim 1 , wherein the terminal has a normal screen buffer and an alternate screen buffer, the normal screen buffer and the alternate screen buffer have different scrollback functionality from one another, and wherein the terminal emulator software mitigates latency for both buffers in that the terminal emulator software performs the terminal emulator latency mitigation steps with the alternate screen buffer and also performs the terminal emulator latency mitigation steps with the normal screen buffer.

4. The terminal of claim 1 , wherein the predictions reside entirely local to the terminal.

5. The terminal of claim 1 , wherein a latency mitigating capability and a display writing capability each operate together in a terminal emulation process as opposed to the latency mitigating capability residing in a separate process from the display writing capability.

6. The terminal of claim 1 , further comprising a cache of pressed keys and their respective responses.

7. A method of mitigating latency during operation of a terminal emulator, the method comprising the terminal emulator:
receiving first key press data which identifies a first pressed key;
transmitting the first pressed key data toward a remote service;
locating a first prediction that is associated with the first key press data;
obtaining a first prediction string from the first prediction;
writing the first prediction string to the display;
after writing the first prediction string to the display, getting from the remote service a first response to the first pressed key;
determining that the first prediction string is not an accurate prediction because the first response does not match the first prediction string;
modifying the display to replace the first prediction string based on the first response;
receiving second key press data which identifies a second pressed key;
transmitting the second pressed key data toward the remote service;
locating a second prediction that is associated with the second key press data;
obtaining a second prediction string from the second prediction, the second prediction string having second prediction string character content;
writing the second prediction string to the display;
after writing the second prediction string to the display, getting from the remote service a second response to the second pressed key, the second response having second response character content;
determining that the second prediction string is an accurate prediction because the second response matches the second prediction string; and
avoiding modifying the display to replace the second prediction string character content based on the second response character content;
whereby latency is mitigated by the accurate prediction.

8. The method of claim 7 , further comprising disabling or pausing a class of predictions without ending or suspending execution of the terminal emulator.

9. The method of claim 7 , further comprising inferring an input boundary.

10. The method of claim 7 , further comprising undoing respective display modifications of multiple predictions of a sequence of predictions as a unit.

11. The method of claim 7 , further comprising:
writing to the display a prediction indicator which indicates that the second prediction string is based on a prediction of how the remote service will respond; and
removing the prediction indicator from the display after determining that the second prediction string is an accurate prediction.

12. The method of claim 7 , wherein the method is characterized in that the terminal emulator mitigates latency of remote service responses to key presses in a single-ended manner by operating entirely locally.

13. The method of claim 7 , wherein the method is characterized in that the terminal emulator mitigates latency of remote service responses to key presses when the terminal emulator and the remote service communicate over a network connection utilizing a secure shell protocol for transmitting pressed keys data toward the remote service and for getting responses from the remote service.

14. The method of claim 7 , wherein the method is characterized in that the terminal emulator mitigates latency of remote service responses to key presses without reliance on any particular kind of kernel supporting the remote service.

15. The method of claim 7 , wherein the method is characterized as follows:
the terminal emulator mitigates latency of remote service responses to key presses without creating or storing any of the predictions on a server running the remote service; and
the terminal emulator mitigates latency of remote service responses to key presses without creating or storing any of the predictions within a shell program.

16. A computer-readable storage hardware device configured with data and instructions which upon execution by at least one processor cause one or more devices to perform a terminal emulator latency mitigation method in a terminal emulator, the method comprising:
receiving first key press data which identifies a first pressed key;
transmitting the first pressed key data toward the remote service;
locating a first prediction that is associated with the first key press data, obtaining a first prediction string from the first prediction, and writing the first prediction string to the display together with a prediction

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Imms Daniel John

Classifications

• H04L67/5681

  Pre-fetching or pre-delivering data based on network characteristics · CPC title

• H04L67/568

  Storing data temporarily at an intermediate stage, e.g. caching · CPC title

• G06F3/0237 Primary

  using prediction or retrieval techniques · CPC title

• H04L67/08

  specially adapted for terminal emulation, e.g. Telnet · CPC title

• G06F13/107

  Terminal emulation · CPC title