System and method for predicting fine-grained adversarial multi-agent motion

Patent metadata

Field	Value
Publication number	US-11645546-B2
Application number	US-201916254037-A
Country	US
Kind code	B2
Filing date	Jan 22, 2019
Priority date	Jan 21, 2018
Publication date	May 9, 2023
Grant date	May 9, 2023

Abstract

A system and method for predicting multi-agent locations is disclosed herein. A computing system retrieves tracking data from a data store. The computing system generates a predictive model using a conditional variational autoencoder. The conditional variational autoencoder learns one or more paths a subset of agents of the plurality of agents are likely to take. The computing system receives tracking data from a tracking system positioned remotely in a venue hosting a candidate sporting event. The computing system identifies one or more candidate agents for which to predict locations. The computing system infers, via the predictive model, one or more locations of the one or more candidate agents. The computing system generates a graphical representation of the one or more locations of the one or more candidate agents.

First claim

What is claimed:

1. A method of predicting multi-agent location, comprising:
receiving, by a computing system, tracking data from a tracking system positioned remotely in a venue hosting a sporting event, the tracking data comprising agent coordinate data for a plurality of sequences of agent movement for a plurality of agents on a playing surface during the sporting event;
accessing, by the computing system, identity information for each agent of the plurality of agents, the identity information comprising at least one of agent name, agent team, and agent position;
projecting, by the computing system, a future location of each agent of the plurality of agents on the playing surface by:
encoding, by a first encoder of a conditional variational autoencoder, the agent coordinate data for the plurality of sequences to generate a first encoded data set,
encoding, by a second encoder of the conditional variational autoencoder, the identity information for each agent to generate a second encoded data set,
encoding, by a third encoder of the conditional variational autoencoder, future trajectory information of a second plurality of agents to generate a third encoded data set, wherein the second plurality of agents are on the playing surface with the plurality of agents, and
predicting, by the conditional variational autoencoder, one or more paths each agent of the plurality of agents is likely to take based at least on the first encoded data set, the second encoded data set, and the third encoded data set, the conditional variational autoencoder trained using historical location data of each agent for a plurality of sporting events; and
generating, by the computing system, a graphical representation of the future location of each agent on the playing surface.

2. The method of claim 1 , further comprising:
pre-processing, by the computing system, the tracking data by aligning agent tracks using a tree-based role alignment.

3. The method of claim 1 , further comprising:
generating, by a variation module of the conditional variational autoencoder, a sample of a random latent variable based on the second encoded data set.

4. The method of claim 3 , wherein the variation module predicts a mean and standard deviation of a latent variable distribution based on the second encoded data set.

5. The method of claim 3 , further comprising:
inferring, by a decoder, a location of each agent based on the sample of the random latent variable, the first encoded data set, and the second encoded data set.

6. A system, comprising:
a processor; and
a memory having programming instructions stored thereon, which, when executed by the processor, performs one or more operations, comprising:
receiving tracking data from a tracking system positioned remotely in a venue hosting a sporting event, the tracking data comprising agent coordinate data for a plurality of sequences of agent movement for a plurality of agents on a playing surface during the sporting event;

accessing identity information for each agent of the plurality of agents, the identity information comprising at least one of agent name, agent team, and agent position;
projecting a future location of each agent of the plurality of agents on the playing surface by:
encoding, by a first encoder of a conditional variational autoencoder, the agent coordinate data for the plurality of sequences to generate a first encoded data set,
encoding, by a second encoder of the conditional variational autoencoder, the identity information for each agent to generate a second encoded data set,
encoding, by a third encoder of the conditional variational autoencoder, future trajectory information of a second plurality of agents to generate a third encoded data set, wherein the second plurality of agents are on the playing surface with the plurality of agents, and
predicting, by the conditional variational autoencoder, one or more paths a subset of agents of the plurality of agents is likely to take based at least on the first encoded data set, the second encoded data set, and the third encoded data set, the conditional variational autoencoder trained using historical location data of each agent for a plurality of sporting events; and
generating a graphical representation of the future location of each agent on the playing surface.

7. The system of claim 6 , wherein the one or more operations further comprises:
pre-processing the tracking data by aligning agent tracks using a tree-based role alignment.

8. The system of claim 6 , wherein the one or more operations further comprise:
generating, by a variation module of the conditional variational autoencoder, a sample of a random latent variable based on the second encoded data set.

9. The system of claim 8 , wherein the variation module predicts a mean and standard deviation of a latent variable distribution based on the second encoded data set.

10. The system of claim 8 , wherein the one or more operations further comprise:
inferring, by a decoder, a location of each agent based on the sample of the random latent variable, the first encoded data set, and the second encoded data set.

11. A non-transitory computer readable medium including one or more sequences of instructions that, when executed by one or more processors causes a computing system to perform operations comprising:
receiving, by the computing system, tracking data from a tracking system positioned remotely in a venue hosting a sporting event, the tracking data comprising agent coordinate data for a plurality of sequences of agent movement for a plurality of agents on a playing surface during the sporting event;
accessing, by the computing system, identity information for each agent of the plurality of agents, the identity information comprising at least one of agent name, agent team, and agent position;
projecting, by the computing system, a future location of each agent of the plurality of agents on the playing surface by:
encoding, by a first encoder of a conditional variational autoencoder, the agent coordinate data for the plurality of sequences to generate a first encoded data set,
encoding, by a second encoder of the conditional variational autoencoder, the identity information for each agent to generate a second encoded data set,
encoding, by a third encoder of the conditional variational autoencoder, future trajectory information of a second plurality of agents to generate a third encoded data set, wherein the second plurality of agents are on the playing surface with the plurality of agents, and
predicting, by the conditional variational autoencoder, one or more paths each agent of the plurality of agents is likely to take based at least on the first encoded data set, the second encoded data set, and the third encoded data set, the conditional variational autoencoder trained using historical location data of each agent for a plurality of sporting events; and
generating, by the computing system, a graphical representation of the future location of each agent on the playing surface.

12. The non-transitory computer readable medium of claim 11 , further comprising:
pre-processing, by the computing system, the tracking data by aligning agent tracks using a tree-based role alignment.

13. The non-transitory computer readable medium of claim 11 , further comprising:
generating, by a variation module of the conditional variational autoencoder, a sample of a random latent variable based on the second encoded data set.

14. The non-transitory computer readable medium of claim 13 , wherein the variation module predicts a mean and standard deviation of a latent variab

Assignees

• Stats Llc

Inventors

• Felsen Panna
• Ganguly Sujoy
• Lucey Patrick

Classifications

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06T2207/30196

  Human being; Person · CPC title