Client side processing of game controller input

Patent metadata

Field	Value
Publication number	US-9526980-B2
Application number	US-201213723652-A
Country	US
Kind code	B2
Filing date	Dec 21, 2012
Priority date	Dec 21, 2012
Publication date	Dec 27, 2016
Grant date	Dec 27, 2016

Abstract

Embodiments of the present invention enable rich control input data to control video games that are remotely executed. Rich control input includes three-dimensional image data, color video, audio, device orientation data, and touch input. A remotely-executed video game is one executed on a server or other computing device that is networked to a client device receiving the rich control input. Rich control input includes more data than can be uploaded to a game server without degrading game performance. Embodiments of the present invention preprocess the rich control data on the client and into data that may be uploaded to the game server. The rich input stream may be processed in a general way or in a game-specific way.

First claim

The invention claimed is:

1. A method of enabling game play over a remote connection using a data input device, the method comprising:
receiving, at a client device from a camera, initial controller input for a video game running on a game server that renders at least part of the video game video image, the initial controller input comprising three-dimensional image data;
processing the initial controller input on the client device to generate a reduced controller input, wherein the reduced controller input is an environmental delta for objects in an environment described by the three-dimensional image data within the initial controller input, the environmental delta describing changes to an inanimate object in the environment, wherein the initial control input comprises more data than the reduced controller input;
communicating over a wide area network the reduced controller input to the game server; and
receiving, from the game server, a rendered video that reflects game progress manipulated according to the reduced controller input.

2. The method of claim 1 , wherein the processing is performed with code specific to the video game being played.

3. The method of claim 1 , wherein the method further comprises not communicating the entire initial controller input to the server.

4. The method of claim 1 , wherein the reduced controller input further comprises skeletal geometry for a person described by three-dimensional image data within the initial controller input.

5. The method of claim 1 , wherein the reduced controller input further comprises a game command derived by recognizing a gesture described by three-dimensional image data within the initial controller input.

6. The method of claim 1 , wherein the initial controller input further comprises an audio signal and the method further comprises performing speech to text translation on the audio signal and communicating the text to the game server.

7. The method of claim 1 , wherein the reduced controller input further comprises a motion blob for an object described by three-dimensional image data within the initial controller input, the motion blob describing one or more of an outline or carve out of moving objects in the environment.

8. The method of claim 1 , wherein the reduced controller input further comprises a subset of data describing only a portion of a scene that is described by the three-dimensional image data within the initial controller input, the subset describing an object of interest to be analyzed by the video game running on the game server.

9. The method of claim 1 , wherein the initial controller input is a point cloud.

10. A computing system comprising:
a processor; and
computer storage memory having computer-executable instructions stored thereon which, when executed by the processor, configure the computing system to:
receive from a camera initial controller input for a video game running on a game server that renders at least part of the video game video image, the initial controller input comprising a three-dimensional image data having pixels associated with a physical location in three-dimensional space viewed by the camera;
process the initial controller input using code specific to the video game to generate a game-specific reduced controller input, wherein the initial control input comprises more data than the game-specific reduced controller input;
communicate over a wide area network the game-specific reduced controller input to the game server;
process the initial controller input using system code to generate a generic reduced controller input, wherein the generic reduced controller input is an environmental delta for objects in an environment described by three-dimensional image data within the initial controller input, the environmental delta describing changes to an inanimate object in the environment wherein the initial control input comprises more data than the generic reduced controller input; and
communicate over the wide area network the generic reduced controller input to the game server; and
receive over the wide area network from the game server, a rendered video that reflects game progress manipulated according to the game-specific reduced controller input and the generic reduced controller input.

11. The computing system of claim 10 , wherein the reduced controller input is a game command derived by recognizing a gesture described by three-dimensional image data within the initial controller input.

12. The computing system of claim 11 , wherein the game-specific reduced controller input is a subset of the three-dimensional image data describing only a portion of a scene that is described by the three-dimensional image data, the subset describing an object of interest to be analyzed by the video game running on the game server.

13. The computing system of claim 12 , wherein the subset is identified on the client using a skeletal geometry for a person described by the three-dimensional image data.

14. The computing system of claim 10 , wherein the initial controller input further comprises an audio signal and the method further comprises performing speech to text translation on the audio signal and communicating the text to the game server.

15. The computing system of claim 10 , wherein the generic reduced controller input is a player identification for a person described by the three-dimensional image data.

16. The method of claim 10 , wherein the initial controller input is a point cloud.

17. The computing system of claim 10 , further configured to:
process the initial controller input on the client device using system code to generate a generic reduced controller input, wherein the initial control input comprises more data than the generic reduced controller input;
communicate over the wide area network the generic reduced controller input to the game server; and
receive, from the game server, a rendered video that reflects game progress manipulated according to the game-specific reduced controller input and the generic reduced controller input.

18. The computing system of claim 17 , wherein the generic reduced controller input is a skeletal geometry for a person described by the three-dimensional image data and wherein the game-specific reduced controller input is a subset of the three-dimensional image data describing only a portion of a scene that is described by the three-dimensional image data, the subset describing an object of interest to be analyzed by the video game running on the game server.

19. The computing system of claim 18 , wherein the subset is identified on the client using the skeletal geometry for a person described by the three-dimensional image data.

20. A computing system comprising:
a processor; and
computer storage memory having computer-executable instructions stored thereon which, when executed by the processor, configure the computing system to:
receive at a client device from a camera, initial controller input for a video game running on a game server that renders at least part of the video game video image, the initial controller input comprising a three-dimensional image data having pixels associated with a physical location in three-dimensional space viewed by the camera;
process the initial controller input on the client device using code specific to the video game to generate a game-specific reduced controller input, wherein the initial control input comprises more data than the game-specific reduced controller input;
communicate over a wide area network the game-spe

Assignees

• Microsoft Corp
• Microsoft Technology Licensing Llc

Inventors

• Karamfilov Krassimir Emilov
• Barsoum Emad
• Marais Charles Claudius
• Justice John Raymond
• Quinn David James
• Toll Roderick Michael

Classifications

• A63F2300/6072

  of an input signal, e.g. pitch and rhythm extraction, voice recognition · CPC title

• A63F2300/6045

  for mapping control signals received from the input arrangement into game commands · CPC title

• A63F2300/538

  for performing operations on behalf of the game client, e.g. rendering · CPC title

• A63F2300/1087

  comprising photodetecting means, e.g. a camera · CPC title

• A63F2300/1081

  Input via voice recognition · CPC title