Systems and methods for artificial intelligence (AI)-assisted temporal context processing in support of hand-gesture communication in video gaming

Patent metadata

Field	Value
Publication number	US-12465859-B2
Application number	US-202318314479-A
Country	US
Kind code	B2
Filing date	May 9, 2023
Priority date	May 9, 2023
Publication date	Nov 11, 2025
Grant date	Nov 11, 2025

Abstract

A hand-gesture tracking engine includes a first artificial intelligence (AI) model component configured and trained to automatically identify a hand-gesture made by a source player within a video game. A hand-gesture intent engine includes a second AI model component configured and trained to automatically determine a message for communication to a target player within the video game as conveyed by the hand-gesture identified by the hand-gesture tracking engine. A hand-gesture translation engine having a third AI model component configured and trained to automatically generate a communication to the target player that conveys the message as determined by the hand-gesture intent engine. The third AI model component is configured and trained to automatically control sending of the communication to the target player.

First claim

What is claimed is:

1 . A computer-implemented method comprising:
automatically identifying a hand-gesture made by a source player in a video game, through execution of a first artificial intelligence model component; automatically determining a message intent for a communication to a target player as conveyed by the hand-gesture, through execution of a second artificial intelligence model component; after automatically determining the message intent, automatically (i) determining a timing for sending the communication to the target player based on game state data that is temporally correlated to the source player making the hand-gesture, and (ii) generating the communication to the target player that conveys the message intent, through execution of a third artificial intelligence model component; and automatically controlling sending of the communication to the target player according to the determined timing, through execution of the third artificial intelligence model component.

2 . The method of claim 1 , wherein determining the timing for the communication comprises determining to block sending the communication.

3 . The method of claim 1 , wherein determining the timing for the communication comprises determining to immediately send the communication.

4 . The method of claim 1 , wherein the timing for sending the communication is determined based on game state data that indicates that the communication is not relevant to a current game state of the video game.

5 . The method of claim 1 , wherein the timing for sending the communication is determined based on game state data that indicates that the target player is looking at the source player in the video game.

6 . The method of claim 1 , wherein the timing for sending the communication is determined based on game state data that indicates that the source player is not within a field of view of the target player in the video game.

7 . The method of claim 1 , wherein the communication to the target player comprises an enlarged representation of the hand-gesture in the video game.

8 . The method of claim 1 , wherein the communication to the target player comprises a highlighted representation of the hand-gesture in the video game.

9 . The method of claim 1 , comprising determining an identify of the target player based at least on the hand-gesture.

10 . The method of claim 1 , wherein the message intent is automatically determined based on game state data of other players who have previously played the video game.

11 . A system comprising:
one or more computer processors; and one or more non-transitory computer-readable media that store instructions which, when executed by the one or more computer processors, cause the one or more computer processors to perform operations comprising:
automatically identifying a hand-gesture made by a source player in a video game, through execution of a first artificial intelligence model component;
automatically determining a message intent for a communication to a target player as conveyed by the hand-gesture, through execution of a second artificial intelligence model component;
after automatically determining the message intent, automatically (i) determining a timing for sending the communication to the target player based on game state data that is temporally correlated to the source player making the hand-gesture, and (ii) generating the communication to the target player that conveys the message intent, through execution of a third artificial intelligence model component; and
automatically controlling sending of the communication to the target player according to the determined timing, through execution of the third artificial intelligence model component.

12 . The system of claim 11 , wherein determining the timing for the communication comprises determining to block sending the communication.

13 . The system of claim 11 , wherein determining the timing for the communication comprises determining to immediately send the communication.

14 . The system of claim 11 , wherein the timing for sending the communication is determined based on game state data that indicates that the communication is not relevant to a current game state of the video game.

15 . The system of claim 11 , wherein the timing for sending the communication is determined based on game state data that indicates that the target player is looking at the source player in the video game.

16 . The system of claim 11 , wherein the timing for sending the communication is determined based on game state data that indicates that the source player is not within a field of view of the target player in the video game.

17 . The system of claim 11 , wherein the communication to the target player comprises an enlarged representation of the hand-gesture in the video game.

18 . The system of claim 11 , wherein the communication to the target player comprises a highlighted representation of the hand-gesture in the video game.

19 . The system of claim 11 , comprising determining an identify of the target player based at least on the hand-gesture.

20 . The system of claim 11 , wherein the message intent is automatically determined based on game state data of other players who have previously played the video game.

21 . One or more non-transitory computer-readable media that store instructions which, when executed by the one or more computer processors, cause the one or more computer processors to perform operations comprising:
automatically identifying a hand-gesture made by a source player in a video game, through execution of a first artificial intelligence model component; automatically determining a message intent for a communication to a target player as conveyed by the hand-gesture, through execution of a second artificial intelligence model component; after automatically determining the message intent, automatically (i) determining a timing for sending the communication to the target player based on game state data that is temporally correlated to the source player making the hand-gesture, and (ii) generating the communication to the target player that conveys the message intent, through execution of a third artificial intelligence model component; and automatically controlling sending of the communication to the target player according to the determined timing, through execution of the third artificial intelligence model component.

22 . The media of claim 21 , wherein determining the timing for the communication comprises determining to block sending the communication.

23 . The media of claim 21 , wherein determining the timing for the communication comprises determining to immediately send the communication.

24 . The media of claim 21 , wherein the timing for sending the communication is determined based on game state data that indicates that the communication is not relevant to a current game state of the video game.

25 . The media of claim 21 , wherein the timing for sending the communication is determined based on game state data that indicates that the target player is looking at the source player in the video game.

26 . The media of claim 21 , wherein the timing for sending the communication is determined based on game state data that indicates that the source player is not within a field of view of the target player in the video game.

27 . The media of claim 21 ,

Assignees

• Sony Interactive Entertainment Inc

Inventors

• Black Glenn
• Stafford Jeffrey
• Young Andrew

Classifications

• G06V20/41

  Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49) · CPC title

• G06V40/28

  Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113) · CPC title

• G06V10/70

  using pattern recognition or machine learning (optical pattern recognition or electronic computations therefor G06V10/88) · CPC title

• A63F13/87

  Communicating with other players during game play, e.g. by e-mail or chat · CPC title

• A63F13/215

  comprising means for detecting acoustic signals, e.g. using a microphone · CPC title