Gesture translation with modification based on game context

Patent metadata

Field	Value
Publication number	US-12569752-B2
Application number	US-202318296091-A
Country	US
Kind code	B2
Filing date	Apr 5, 2023
Priority date	Apr 5, 2023
Publication date	Mar 10, 2026
Grant date	Mar 10, 2026

Abstract

A method is provided for facilitating communication between players in a video game, including the following operations: receiving video of a first player during gameplay of the video game; analyzing the video to identify gestures made by the first player; responsive to identifying the gestures, then translating the gestures into a text communication, wherein translating the gestures uses a context of the video game to determine content of the text communication; presenting the text communication through an in-game chat of the video game, for viewing by a second player during the gameplay of the video game.

First claim

The invention claimed is:

1 . A computer-implemented method comprising:
receiving video of a first player during gameplay of a video game; analyzing the video to identify one or more gestures made by the first player during the gameplay of the video game; generating a base textual transcription of the one or more gestures made by the first player during the gameplay of the video game; determining a current game state of the video game, wherein determining the current game state comprises analyzing in-game chat text using a natural language processor; selecting a communication modification template that is associated with the current game state of the video game; generating a modified textual transcription of the one or more gestures made by the first player during the gameplay of the video game using the selected communication modification template that is associated with the current game state; and providing the modified textual transcription to an in-game chat logic of the video game.

2 . The method of claim 1 , wherein the base textual transcription is generated by one or more machine learning models that are trained using a dataset of known gestures and corresponding text transcriptions.

3 . The method of claim 1 , wherein determining the current game state comprises determining that the current game state reflects a tense in-game situation.

4 . The method of claim 1 , wherein determining the current game state comprises determining that the current game state reflects an in-game combat situation.

5 . The method of claim 1 , wherein determining the current game state comprises determining that the current game state reflects a relaxing in-game situation.

6 . The method of claim 1 , wherein determining the current game state comprises determining current game scores for different players.

7 . The method of claim 1 , wherein determining the current game state comprises determining whether the current game state reflects adversarial or cooperative in-game states.

8 . The method of claim 1 , wherein the communication modification template is selected from among multiple candidate communication modification templates that a developer of the video game has customized for different game states of the video game.

9 . The method of claim 1 , wherein the communication modification template is selected from among multiple candidate communication modification templates for different game states that have been customized by the first player.

10 . The method of claim 1 , wherein the communication modification template is implemented as part of a software development kit (SDK) of a game engine platform associated with the video game.

11 . The method of claim 1 , wherein the communication modification template specifies different modifications that are associated with different current game states.

12 . The method of claim 1 , wherein the current game state comprises a characteristic of the video game that varies based on game intensity throughout the gameplay.

13 . One or more non-transitory computer-readable medium that store instructions which, when executed by one or more computer processors, cause the one or more computer processors to perform operations comprising:
receiving video of a first player during gameplay of a video game; analyzing the video to identify one or more gestures made by the first player during the gameplay of the video game; generating a base textual transcription of the one or more gestures made by the first player during the gameplay of the video game; determining a current game state of the video game, wherein determining the current game state comprises analyzing in-game chat text using a natural language processor; selecting a communication modification template that is associated with the current game state of the video game; generating a modified textual transcription of the one or more gestures made by the first player during the gameplay of the video game using the selected communication modification template that is associated with the current game state; and providing the modified textual transcription to an in-game chat logic of the video game.

14 . The medium of claim 13 , wherein the base textual transcription is generated by one or more machine learning models that are trained using a dataset of known gestures and corresponding text transcriptions.

15 . The medium of claim 13 , wherein determining the current game state comprises determining that the current game state reflects a tense in-game situation.

16 . The medium of claim 13 , wherein determining the current game state comprises determining that the current game state reflects an in-game combat situation.

17 . The medium of claim 13 , wherein determining the current game state comprises determining that the current game state reflects a relaxing in-game situation.

18 . A system comprising:
one or more computer processors; and one or more non-transitory computer-readable medium that store instructions which, when executed by the one or more computer processors, cause the one or more computer processors to perform operations comprising:
receiving video of a first player during gameplay of a video game;
analyzing the video to identify one or more gestures made by the first player during the gameplay of the video game;
generating a base textual transcription of the one or more gestures made by the first player during the gameplay of the video game;
determining a current game state of the video game, wherein determining the current game state comprises analyzing in-game chat text using a natural language processor;
selecting a communication modification template that is associated with the current game state of the video game;
generating a modified textual transcription of the one or more gestures made by the first player during the gameplay of the video game using the selected communication modification template that is associated with the current game state; and
providing the modified textual transcription to an in-game chat logic of the video game.

Assignees

• Sony Interactive Entertainment Inc

Inventors

• Bean Celeste
• Black Glenn
• Dorn Victoria

Classifications

• A63F13/67

  adaptively or by learning from player actions, e.g. skill level adjustment or by storing successful combat sequences for re-use · CPC title

• A63F13/87 Primary

  Communicating with other players during game play, e.g. by e-mail or chat · CPC title

• A63F13/245

  specially adapted to a particular type of game, e.g. steering wheels · CPC title

• A63F13/212 Primary

  using sensors worn by the player, e.g. for measuring heart beat or leg activity · CPC title