Natural language configuration and operation for tangible games

Patent metadata

Field	Value
Publication number	US-11600263-B1
Application number	US-202016914603-A
Country	US
Kind code	B1
Filing date	Jun 29, 2020
Priority date	Jun 29, 2020
Publication date	Mar 7, 2023
Grant date	Mar 7, 2023

Abstract

This disclosure describes a tabletop game assistant system configured to ingest and guide tangible games (such as board games, card games, etc.) using natural language interaction and image capture/visual display components. The system can include features enabling a game developer to “teach” the system the rules of a game using natural language, such as written instructions, to reduce or eliminate the need for writing dedicated code. The system may process images of a game board and/or tokens such as game pieces and/or cards to further generate game data in the form of a logical game model. The system can use the game data to guide human players of the game and, in some cases, participate as a player itself. The system may further be configured to observe a game and detect invalid actions, answer questions regarding the rules, and suggest moves. The system may provide additional utilities such as generating a random output (e.g., rolling virtual dice) and learning to recognize new game pieces.

First claim

What is claimed is:

1. A computer-implemented method comprising:
receiving first data representing printed instructions for playing a game;
performing natural language understanding (NLU) processing on the first data to generate first NLU results data corresponding to the printed instructions;
identifying a first portion of the first NLU results data related to game setup;
processing the first portion to determine first state data representing an initial state of the game;
identifying a second portion of the first NLU results data related to a game objective;
processing the second portion to determine first condition data representing at least a first condition that results in ending the game;
identifying a third portion of the first NLU results data related to at least one player of the game;
processing the third portion to determine first player data representing at least one player of the game;
identifying a fourth portion of the first NLU results data related to at least one action to be performed during the game;
processing the fourth portion to determine first event data representing the at least one action available during the game;
determining first game data including at least the first state data, first condition data, first player data and first event data; and
processing at least a portion of the first game data to configure a first speech processing model for use with a future natural language input corresponding to the game.

2. The computer-implemented method of claim 1 , further comprising:
processing the first game data to simulate at least a first hypothetical game without human player involvement;
detecting, based at least in part on the first hypothetical game, a first game state in which a computerized player has no valid action available;
determining that the first game data does not address the first game state;
outputting first output data indicating a first potential flaw in the first game data;
receiving second data representing a natural language response to the first output data;
performing NLU processing on the second data to generate second NLU results data;
determining at least one modification of the first game data based on the second NLU results data;
determining that the at least one modification resolves the first potential flaw in the first game data; and
generating second game data based on the first game data and the at least one modification.

3. The computer-implemented method of claim 1 , further comprising:
receiving, from an image capture component, image data representing a game board corresponding to the game;
performing image processing on the image data to determine:
a first node associated with a first game space,
a second node associated with a second game space, and
an edge associating the first game space with the second game space;

determining, based at least on the first node, the second node, the edge, and the first game data, first graph data representing the game board;
receiving first input audio data corresponding to a first utterance;
performing speech processing on the first input audio data to generate second NLU results data;
determining, based on the second NLU results data, that the first utterance describes at least a first additional connection between a first game space and a second game space, the first additional connection not determined based on the image processing;
generating second graph data based on the first graph data and at least a first additional edge corresponding to the first additional connection; and
generating second game data based on the first game data and the second graph data.

4. The computer-implemented method of claim 1 , further comprising:
receiving second data representing a request to initiate a first instance of a game;
receiving the first game data, the first game data corresponding to the game;
receiving third data corresponding to a first utterance;
performing speech processing on the third data to generate second NLU results data;
determining, based on the second NLU results data, that the first utterance describes at least a first rule modification to use for the first instance of the game; and
generating second game data based on the first game data and the first rule modification.

5. A computer-implemented method comprising:
receiving first data representing a natural language representation of instructions for operations corresponding to a first physical token;
performing natural language understanding (NLU) processing on the first data to generate first NLU results data corresponding to the instructions;
processing the first NLU results data to determine:
first state data representing an initial state of the operations,
first condition data representing at least a first condition that results in ending the operations,
first operator data representing at least one operator corresponding to the first physical token, and
first event data representing at least one action available corresponding to the first physical token;

determining second data including at least the first state data, the first condition data, the first operator data and the first event data; and
processing at least a portion of the second data to configure a first speech processing model for use with a future natural language input.

6. The computer-implemented method of claim 5 , further comprising:
detecting at least a first potential flaw in the second data, wherein the first potential flaw is one of an ambiguity, a conflict, or an undefined state;
outputting first output data corresponding to the first potential flaw;
receiving third data representing a natural language response to the first output data;
performing NLU processing on the third data to generate second NLU results data;
determining at least one modification of the second data based on the second NLU results data;
determining that the at least one modification resolves the first potential flaw in the second data; and
generating fourth data based on the second data and the at least one modification.

7. The computer-implemented method of claim 5 , further comprising:
receiving image data representing an area corresponding to the operations corresponding to a first physical token;
performing image processing on the image data to determine:
a first node associated with a first space in the area,
a second node associated with a second space in the area, and
an edge associating the first space with the second space;

determining, based at least on the first node, the second node, the edge, and the second data, graph data representing the area; and
generating third data based on the second data and the graph data.

8. The computer-implemented method of claim 7 , further comprising:
receiving fourth data indicating that a portion of the image data corresponds to a portion of the area other than a space associated with potential operations of the first physical token;
determining at least one modification to the graph data based on the fourth data; and
generating fifth data based on the third data and the at least one modification.

9. The computer-implemented method of claim 5 , further comprising:
processing the first NLU results data to determine first token data representing the first physical token;
outputting first output data requesting the first physical token be positioned within a field of view of an image capture component;
receiving, from the image capture component, first image data representing a first image of the first physical token;
performing image processing on the first image data to generate first representational data; and
generating third data based on the second data and an ass

Assignees

• Amazon Tech Inc

Inventors

• Blair James Robert
• Chien Mark
• Houston Farah Lynn
• Rabuchin Steven T

Classifications

• G10L15/26

  Speech to text systems (G10L15/08 takes precedence) · CPC title

• G09B19/22

  Games, e.g. card games · CPC title

• G06F40/30

  Semantic analysis · CPC title

• A63F2011/0086

  Rules · CPC title

• A63F2009/2476

  Speech or voice synthesisers, e.g. using a speech chip · CPC title