View rendering from multiple server-side renderings

Patent metadata

Field	Value
Publication number	US-9569812-B1
Application number	US-201614990623-A
Country	US
Kind code	B1
Filing date	Jan 7, 2016
Priority date	Jan 7, 2016
Publication date	Feb 14, 2017
Grant date	Feb 14, 2017

Abstract

A first user input is received when a client program executed by a client computing device is in a first state. The first user input is sent to a server computing device to render a view of a virtual scene. A state change from the first state in the client program due to a second user input or a program event is identified. One or more gaps in a server-rendered current view due to the state change are determined. A rendering of the one or more gaps is selected from among the server-rendered current view, a server-rendered predicted view and one or more prior-rendered views. A current view is rendered using a simplified model of the virtual scene by rendering the one or more gaps from the selected rendering. The current rendered view is visually presented via a display of the client computing device.

First claim

The invention claimed is:

1. A client computing device comprising:
a processor configured to:
receive a first user input when the client program is in a first state;
send the first user input to a server computing device to render a view of a virtual scene;
receive from the server computing device a server-rendered current view of the virtual scene that is based on the first input;
receive from the server computing device a server-rendered predicted view of the virtual scene that is based on the first input;
receive from the server computing device a simplified model of geometry of the virtual scene;
retrieve one or more prior-rendered views of the virtual scene from memory of the client computing device;
identify a state change from the first state in the client program due to a second user input or a program event;
determine one or more gaps in the server-rendered current view due to the state change;
select a rendering of the one or more gaps from among the server-rendered current view, the server-rendered predicted view and the one or more prior-rendered views; and
render from the simplified model a current view by rendering the one or more gaps from the rendering; and

a display configured to visually present the current view;
wherein the server-rendered current view, the server-rendered predicted view and the one or more prior-rendered views are candidate views, and
wherein selecting the rendering of the one or more gaps includes:
selecting a candidate view in which the one or more gaps are visible as the highest quality rendering; and
if the one or more gaps are visible in more than one candidate view, assigning a quality score to each candidate view, and selecting a candidate view having a highest quality score as the rendering, wherein the quality score is derived at least in part from one or more of an angle of a perspective of the candidate view relative to the one or more gaps and a distance between a surface corresponding to the one or more gaps and the perspective of the candidate view.

2. The client computing device of claim 1 , wherein the one or more gaps includes a plurality of gaps, and wherein rendering the current view includes if a subset of gaps of the plurality of gaps are not visible in the rendering, selecting a different rendering that includes the subset of gaps, and rendering the subset of gaps from the different rendering.

3. The client computing device of claim 2 , wherein rendering the current view includes rendering gaps that are not visible in any of the candidate views using a smoothing or blurring algorithm.

4. The client computing device of claim 1 , wherein the simplified model of geometry of the virtual scene includes a subset of geometry of the virtual scene that is selected for inclusion in the simplified model based the first state of the program and the first user input.

5. The client computing device of claim 1 , wherein the server-rendered current view and the server-rendered predicted view include a plurality of pixels each having color data and depth data corresponding to a surface in the virtual scene.

6. The client computing device of claim 1 , wherein the server-rendered predicted view has a field of view that is wider than a field of view of the server-rendered current view.

7. The client computing device of claim 1 , wherein the server-rendered predicted view has a perspective of the virtual scene that is higher than a perspective of the server-rendered current view.

8. A method for predicting and rendering content, executable on a client computing device, the method comprising:
receiving a first user input when a client program executed by the client computing device is in a first state;
sending the first user input to a server computing device to render a view of a virtual scene;
receiving from the server computing device a server-rendered current view of the virtual scene that is based on the first input;
receiving from the server computing device a server-rendered predicted view of the virtual scene that is based on the first input;
receiving from the server computing device a simplified model of geometry of the virtual scene;
retrieving one or more prior-rendered views of the virtual scene from memory of the client computing device;
identifying a state change from the first state in the client program due to a second user input or a program event;
determining one or more gaps in the server-rendered current view due to the state change;
selecting a rendering of the one or more gaps from among the server-rendered current view, the server-rendered predicted view and the one or more prior-rendered views;
rendering from the simplified model a current view by rendering the one or more gaps from the rendering; and
visually presenting the rendered view via a display of the client computing device;
wherein the server-rendered current view, the server-rendered predicted view and the one or more prior-rendered views are candidate views, and
wherein selecting the rendering of the one or more gaps includes:
selecting a candidate view in which the one or more gaps are visible as the highest quality rendering; and

if the one or more gaps are visible in more than one candidate view, assigning a quality score to each candidate view, and selecting a candidate view having a highest quality score as the rendering, wherein the quality score is derived at least in part from one or more of an angle of a perspective of the candidate view relative to the one or more gaps and a distance between a surface corresponding to the one or more gaps and the perspective of the candidate view.

9. The method of claim 8 , wherein the one or more gaps includes a plurality of gaps, and wherein rendering the current view includes if a subset of gaps of the plurality of gaps are not visible in the rendering, selecting a different rendering that includes the subset of gaps, and rendering the subset of gaps from the different rendering.

10. The method of claim 9 , wherein rendering the current view includes rendering gaps that are not visible in any of the candidate views using a smoothing or blurring algorithm.

11. The method of claim 8 , wherein the simplified model of geometry of the virtual scene includes a subset of geometry of the virtual scene that is selected for inclusion in the simplified model based on the first state of the program, and the first user input.

12. A client computing device comprising:
a processor configured to:
receive a first user input when the client program is in a first state;
send the first user input to a server computing device to render a view of a virtual scene;
receive from the server computing device a server-rendered current view of the virtual scene that is based on the first input;
receive from the server computing device a server-rendered predicted view of the virtual scene that is based on the first input;
receive from the server computing device a simplified model of geometry of the virtual scene;
retrieve one or more prior-rendered views of the virtual scene from memory of the client computing device;
identify a state change from the first state in the client program due to a second user input or a program event;
determine one or more gaps in the server-rendered current view due to the state change;
identify one or more candidate views in which the one or more gaps are visible from among the server-rendered current view, the server-rendered predicted view and the one or more prior-rendered views;
if only one candidate view is identified, select the candidate view as a rendering of the one or more gaps;
if more than one candidate view is identified, 1) assign a quali

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Kopf Johannes
• Cuervo Laffaye Eduardo Alberto
• Chu David Chiyuan
• Reinert Bernhard

Classifications

• G09G2370/022

  Centralised management of display operation, e.g. in a server instead of locally · CPC title

• G06T17/00

  Three-dimensional [3D] modelling for computer graphics · CPC title

• G06T1/20 Primary

  Processor architectures; Processor configuration, e.g. pipelining · CPC title

• G09G2360/06

  Use of more than one graphics processor to process data before displaying to one or more screens · CPC title

• G06T5/002

  Physics · mapped topic