Contextual user interface based on video game playback

Patent metadata

Field	Value
Publication number	US-10166465-B2
Application number	US-201715629459-A
Country	US
Kind code	B2
Filing date	Jun 21, 2017
Priority date	Jan 20, 2017
Publication date	Jan 1, 2019
Grant date	Jan 1, 2019

Abstract

A contextual user interface based on video game playback is described. An assistant device can receive video game state information indicating a state of a playback of a video game on a video game console. Using that information, the assistant device can determine characteristics of the playback and generate a button on a graphical user interface (GUI) based on the characteristics. A user can then select the button and cause the assistant device to instruct the video game console to perform a corresponding action.

First claim

We claim:

1. A method for providing a graphical user interface (GUI) on a touchscreen of a home assistant device with artificial intelligence (AI) capabilities, the GUI providing content related to playback of a video game within an environment of the home assistant device, comprising:
receiving video game state information from a video game console, the video game state information indicating a state of playback of a video game within the environment;
determining that the video game state information indicates that the state of the playback of the video game includes a player character controlled by a user having a health status approaching a level that results in termination of a current iteration of the playback of the video game, and that the video game state information indicates that an item controlled by the player character has an inventory level approaching a level that results in the item not being functional until the inventory level is replenished;
determining characteristics of the user while playing the video game;
generating a first hot button for display on the GUI of the touchscreen of the home assistant device based on the determination that the state of the playback of the video game includes a player character controlled by a user having a health status approaching a level that results in termination of a current iteration of the playback of the video game, the first hot button configured to instruct the video game console to use an item or ability corresponding to the player character to modify the health status such that it adjusted to a modified level away from the level that results in termination of the current iteration of the playback of the video game;
providing an audio response indicating that the first hot button was generated based on the characteristics of the user while playing the video game;
generating a second hot button for display on the GUI of the touchscreen of the home assistant device based on the determination that the video game state information indicates that an item controlled by the player character has an inventory level approaching a level that results in the item not being functional until it is replenished, the second hot button configured to instruct the video game console to use an item or ability corresponding to the player character to modify the inventory level such that it is adjusted to a modified inventory level away from the level that results in the item not being functional;
determining that the first hot button or the second hot button was selected via a touch on the touchscreen of the home assistant device or via a response to the audio response; and
instructing the video game console to modify the health status or modify the inventory level based on the selection of the first hot button or the second hot button.

2. A method, comprising:
receiving video game state information representing a state of playback of a video game on a video game console;
determining, by a processor of an assistant device, that the state of the playback of the video game includes a player character controlled by a user, the player character having characteristics based on the playback of the video game;
determining, by the processor, characteristics of the user while playing the video game;
generating, by the processor, a first button on a graphical user interface (GUI) displayed upon a display of the assistant device, the first button providing an indication of a first action that can be performed by the video game to modify the characteristics of the player character;
providing an audio response indicating that the first button was generated, the audio response provided based on the characteristics of the user while playing the video game;
determining, by the processor, that the first button was selected in response to the audio response; and
instructing, by the processor, the video game console to modify the characteristics of the player character corresponding to the first action upon selection of the first button.

3. The method of claim 2 , wherein the characteristics of the user include visual characteristics of the user while playing the video game.

4. The method of claim 3 , wherein the visual characteristics include how the user is manipulating a controller used to play back the video game.

5. The method of claim 4 , wherein the visual characteristics include how fast the user is manipulating the controller used to play back the video game.

6. The method of claim 2 , wherein the characteristics of the user include audio characteristics of the user while playing the video game.

7. The method of claim 2 , further comprising:
receiving graphical content related to similar subject matter of the video game being played back on the video game console; and
displaying the graphical content on the GUI with the first button.

8. The method of claim 2 ,
generating, by the processor, a second button on the GUI, the second button providing an indication of a second action that can be performed by the video game to modify the characteristics of the player character, the characteristics modified by the second action being different than the characteristics modified by the first action;
determining, by the processor, that the second button was selected; and
instructing, by the processor, the video game console to modify the characteristics of the player character upon selection of the second button.

9. The method of claim 8 , further comprising:
determining that the video game state information indicates that a change in the playback of the video game is about to occur, wherein characteristics of the first button and the second button are based on the change in the playback of the video game that is about to occur.

10. The method of claim 2 , further comprising:
determining that the video game state information indicates that a change in the playback of the video game is about to occur, wherein the first button is generated based on the change in the playback of the video game that is about to occur.

11. A computer program product, comprising one or more non-transitory computer-readable media having computer program instructions stored therein, the computer program instructions being configured such that, when executed by one or more computing devices, the computer program instructions cause the one or more computing devices to:
receive video game state information representing a state of playback of a video game on a video game console;
determine that the state of the playback of the video game includes a player character controlled by a user, the player character having characteristics based on the playback of the video game;
determine characteristics of the user while playing the video game;
generate a first button on a graphical user interface (GUI), the first button providing an indication of a first action that can be performed by the video game to modify the characteristics of the player character;
provide an audio response indicating that the first button was generated, the audio response provided based on the characteristics of the user while playing the video game;
determine that the first button was selected in response to the audio response; and
instruct the video game console to modify the characteristics of the player character corresponding to the first action upon selection of the first button.

12. The computer program product of claim 11 , wherein the characteristics of the user include visual characteristics of the user while playing the video game.

13. The computer program product of claim 11 , wherein the characteristics of the user include audio characteristics of the use

Assignees

• Essential Products Inc

Inventors

• Roman Manuel
• Segal Mara Clair
• Desai Dwipal
• Rubin Andrew E

Classifications

• G06F3/04842

  Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) · CPC title

• A63F13/23 Primary

  for interfacing with the game device, e.g. specific interfaces between game controller and console · CPC title

• A63F2300/308

  Details of the user interface · CPC title

• A63F13/65 Primary

  automatically by game devices or servers from real world data, e.g. measurement in live racing competition · CPC title

• A63F13/58

  by computing conditions of game characters, e.g. stamina, strength, motivation or energy level · CPC title