Client rendering of latency sensitive game features

Patent metadata

Field	Value
Publication number	US-9717982-B2
Application number	US-201213723498-A
Country	US
Kind code	B2
Filing date	Dec 21, 2012
Priority date	Dec 21, 2012
Publication date	Aug 1, 2017
Grant date	Aug 1, 2017

Abstract

Embodiments of the present invention split game processing and rendering between a client and a game server. A rendered video game image is received from a game server and combined with a rendered image generated by the game client to form a single video game image that is presented to a user. Game play may be controlled using a rich sensory input, such as three-dimensional image data and audio data. The three-dimensional image data describes the shape, size and orientation of objects present in a play space. The rich sensory input is communicated to a game server, potentially with some preprocessing, and is also consumed locally on the client, at least in part. In one embodiment, latency sensitive features are the only features processed on the client and rendered on the client.

First claim

The invention claimed is:

1. A computing system comprising:
a processor; and
computer storage memory having computer-executable instructions stored thereon which, based on execution executed by the processor, configure the processor to play a video game using three-dimensional image data as a control input in a server based gaming environment by being configured to:
receive, at a game client over a computer network, three-dimensional image data as a video game control input;
process, at the game client, the three-dimensional image data to generate a basic control input;
communicate through a network interface the basic control input from the game client over the computer network to a game server on which the video game is running;
process, at the game client, part of the three-dimensional image data to generate a client-rendered video game feature;
receive, through the network interface over the computer network at the game client, a server-rendered video game image from the game server on which the video game is running, the server-rendered video game image being generated in response to a change in game state caused by the basic control input;
composite the client-rendered video game feature with the server-rendered video game image to form a rendered composite video; and
output the rendered composite video to a display device.

2. The system of claim 1 , wherein the client-rendered video game feature is a latency sensitive game feature.

3. The system of claim 2 , wherein the latency sensitive game feature is one where more than 80 ms roundtrip latency causes degradation of the game experience for a player.

4. The system of claim 1 , wherein the client-rendered video game feature is an avatar for a player using the game client.

5. The system of claim 1 , wherein the processor is further configured to time synchronize the server-rendered video game feature with the client-rendered video game image.

6. The system of claim 1 , wherein the game server does not render an avatar for a player using the game client.

7. The system of claim 1 , wherein the client-rendered video game feature is generated by game code that is downloaded from the game server upon initiation of a game session.

8. A method for playing a video game using three-dimensional image data as a control input, the method comprising;
receiving, at a game client over a computer network, the three-dimensional image data as a video game control input;
processing, at the game client, part of the three-dimensional image data locally to generate a client-rendered video game feature;
receiving over the computer network, through a network interface on the game client, a server-rendered video game image from a game server on which the video game is running;
compositing the client-rendered video game feature with the server-rendered video game image to form a rendered composite video; and
outputting the rendered composite video to a display device.

9. The method of claim 8 , wherein the processing is performed by game-specific code.

10. The method of claim 9 , wherein the game-specific code is downloaded from the game server.

11. The method of claim 8 , wherein the client-rendered video game feature is an avatar image responding to a user action detected in the three-dimensional image data.

12. The method of claim 8 , wherein the client-rendered video game feature is a perspective change initiated in response to detecting a user action in the three-dimensional image data that is mapped to a perspective change instruction.

13. The method of claim 12 , wherein the perspective change is a scene rotation.

14. The method of claim 8 , wherein the client-rendered video game feature is a video of the player shown as a picture-in-picture in combination with game scenes.

15. A computing system comprising:
a processor; and
computer storage memory having computer-executable instructions stored thereon which, based on executed by the processor, configure the processor to play a video game using three-dimensional image data as a control input by the processor being configured to:
receive, at a game client over a computer network, the three-dimensional image data as a video game control input;
process, at the game client, part of the three-dimensional image data locally to generate a client-rendered video game feature;
receive, at the game client over the computer network, a server-rendered video game image from a game server on which the video game is running;
composite the client-rendered video game feature with the server-rendered video game image to form a rendered composite video; and
output the rendered composite video to a display device.

16. The computing system of claim 15 , wherein game-specific code is used to configure the processor to process part of the three-dimensional image data locally.

17. The computing system of claim 15 , wherein the client-rendered video game feature is an avatar image responding to a user action detected in the three-dimensional image data.

18. The computing system of claim 15 , wherein the client-rendered video game feature is a perspective change initiated in response to detecting a user action in the three-dimensional image data that is mapped to a perspective change instruction.

19. The computing system of claim 18 , wherein the perspective change is a scene rotation.

20. The computing system of claim 15 , wherein the client-rendered video game feature is a video of the player shown as a picture-in-picture in combination with game scenes.

Assignees

• Microsoft Corp
• Microsoft Technology Licensing Llc

Inventors

• Quinn David James
• Barsoum Emad
• Marais Charles Claudius
• Justice John Raymond
• Karamfilov Krassimir Emilov
• Toll Roderick Michael

Classifications

• A63F2300/534

  for network load management, e.g. bandwidth optimization, latency reduction · CPC title

• A63F2300/69

  Involving elements of the real world in the game world, e.g. measurement in live races, real video · CPC title

• A63F2300/1087

  comprising photodetecting means, e.g. a camera · CPC title

• A63F13/358

  Adapting the game course according to the network or server load, e.g. for reducing latency due to different connection speeds between clients · CPC title

• A63F2300/538

  for performing operations on behalf of the game client, e.g. rendering · CPC title