System and method for predicting fine-grained adversarial multi-agent motion

Patent metadata

Field	Value
Publication number	US-12437211-B2
Application number	US-202318313050-A
Country	US
Kind code	B2
Filing date	May 5, 2023
Priority date	Jan 21, 2018
Publication date	Oct 7, 2025
Grant date	Oct 7, 2025

Abstract

A system and method for predicting multi-agent locations is disclosed herein. A computing system retrieves tracking data from a data store. The computing system generates a predictive model using a conditional variational autoencoder. The conditional variational autoencoder learns one or more paths a subset of agents of the plurality of agents are likely to take. The computing system receives tracking data from a tracking system positioned remotely in a venue hosting a candidate sporting event. The computing system identifies one or more candidate agents for which to predict locations. The computing system infers, via the predictive model, one or more locations of the one or more candidate agents. The computing system generates a graphical representation of the one or more locations of the one or more candidate agents.

First claim

What is claimed:

1. A method of predicting multi-player locations, comprising:
receiving, by a computing system, tracking data from a tracking system positioned remotely in a venue hosting a sporting event, the tracking data comprising coordinate data for a plurality of sequences of movements for a first plurality of players and a second plurality of players on a playing surface during the sporting event;
accessing, by the computing system, identity information for each player of the first plurality of players and the second plurality of players, the identity information comprising at least one of a player name, a player team, or a player position;
pre-processing, by the computing system, the tracking data to generate aligned tracking data by aligning at least one path of each of the first plurality of players to reduce a number of tracking data permutations;
modifying, by the computing system, an autoencoder based on the aligned tracking data, the autoencoder including at least a first encoder model, a second encoder model, and a third encoder model, wherein the first encoder model is configurable to at least encode historical tracking data of the first plurality of players, wherein the second encoder model is configurable to encode future tracking data of a second plurality of players, wherein the third encoder is configurable to encode the identity information for each of the first plurality of players, wherein the first encoder model and the second encoder model of the autoencoder are modified using the aligned tracking data by encoding the aligned tracking data as an input to the autoencoder and modifying one or more weights of the first encoder model and the second encoder model to reduce an error between an output of the autoencoder and the aligned tracking data;
projecting, via the modified autoencoder of the computing system, a future location of each player of the first plurality of players based on an output of the first, second, and third encoder models, each player's sequence of movements, the aligned tracking data, and the second plurality of players co-located with the first plurality of players on the playing surface;
generating, by the computing system, a graphical representation of the future location of each player on the playing surface based on an output of the modified autoencoder; and
displaying, on a display device, the graphical representation as an overlay on a visual representation of the playing surface, wherein the overlay includes predicted trajectories of the players based on the output of the modified autoencoder.

2. The method of claim 1 , wherein projecting, via the modified autoencoder of the computing system, the future location of each player of the first plurality of players based on each player's sequence of movements and the second plurality of players co-located with the first plurality of players on the playing surface further comprises:
projecting, via the modified autoencoder, the future location of each player of the first plurality of players based on learned trajectories of the first plurality of players.

3. The method of claim 1 , further comprising:
encoding, by the modified autoencoder, each player's sequence of movements using an encoder of the modified autoencoder.

4. The method of claim 1 , wherein projecting, via the modified autoencoder of the computing system, the future location of each player of the first plurality of players based on each player's sequence of movements and the second plurality of players co-located with the first plurality of players on the playing surface further comprises:
projecting, via the modified autoencoder, the future location of each player of the first plurality of players based an identity of a team comprising the first plurality of players.

5. The method of claim 1 , wherein projecting, via the modified autoencoder of the computing system, the future location of each player of the first plurality of players based on each player's sequence of movements and the second plurality of players co-located with the first plurality of players on the playing surface further comprises:
projecting, via the modified autoencoder, the future location of each player of the first plurality of players based on identities of each player of the first plurality of players.

6. The method of claim 1 , wherein projecting, via the modified autoencoder of the computing system, the future location of each player of the first plurality of players based on each player's sequence of movements and the second plurality of players co-located with the first plurality of players on the playing surface further comprises:
projecting, via the modified autoencoder, the future location of each player of the first plurality of players based on a current context of the sporting event.

7. The method of claim 1 , wherein generating, by the computing system, the graphical representation of the future location of each player on the playing surface comprises:
generating a first graphical representation of the plurality of sequences of movements for the first plurality of players;
generating a second graphical representation of a second plurality of future sequences of movements for the first plurality of players based on the projected future location of each player; and
appending the first graphical representation with the second graphical representation corresponding to each player.

8. A non-transitory computer readable medium comprising one or more sequences of instructions, which, when executed by a processor, causes a computing system to perform operations comprising:
receiving, by the computing system, tracking data from a tracking system positioned remotely in a venue hosting a sporting event, the tracking data comprising coordinate data for a plurality of sequences of movements for a first plurality of players and a second plurality of players on a playing surface during the sporting event;
accessing, by the computing system, identity information for each player of the first plurality of players and the second plurality of players, the identity information comprising at least one of a player name, a player team, or a player position;
pre-processing, by the computing system, the tracking data to generate aligned tracking data by aligning at least one path of each of the first plurality of players to reduce a number of tracking data permutations;
modifying, by the computing system, an autoencoder based on the aligned tracking data, the autoencoder including at least a first encoder model, a second encoder model, and a third encoder model, wherein the first encoder model is configurable to at least encode historical tracking data of the first plurality of players, wherein the second encoder model is configurable to encode future tracking data of a second plurality of players, wherein the third encoder is configurable to encode the identity information for each of the first plurality of players, wherein the first encoder model and the second encoder model of the autoencoder are modified using the aligned tracking data by encoding the aligned tracking data as an input to the autoencoder and modifying one or more weights of the first encoder model and the second encoder model to reduce an error between an output of the autoencoder and the aligned tracking data;
projecting, via the modified autoencoder of the computing system, a future location of each player of the first plurality of players based on an output of the first, second, and third encoder models, each player's sequence of movements, the aligned tracking data, and the second plurality of players co-located with the first plurality of players on the playing surface;
generating, by the computing system, a graphical representation of the future l

Assignees

• Stats Llc

Inventors

• Felsen Panna
• Ganguly Sujoy
• Lucey Patrick

Classifications

• G06N20/00

  Machine learning · CPC title

• G06N5/04

  Inference or reasoning models · CPC title

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title