System and method for player reidentification in broadcast video

Patent metadata

Field	Value
Publication number	US-2025148615-A1
Application number	US-202519014617-A
Country	US
Kind code	A1
Filing date	Jan 9, 2025
Priority date	Feb 28, 2019
Publication date	May 8, 2025
Grant date	—

Abstract

A system and method of re-identifying players in a broadcast video feed are provided herein. A computing system retrieves a broadcast video feed for a sporting event. The broadcast video feed includes a plurality of video frames. The computing system generates a plurality of tracks based on the plurality of video frames. Each track includes a plurality of image patches associated with at least one player. Each image patch of the plurality of image patches is a subset of the corresponding frame of the plurality of video frames. For each track, the computing system generates a gallery of image patches. A jersey number of each player is visible in each image patch of the gallery. The computing system matches, via a convolutional autoencoder, tracks across galleries. The computing system measures, via a neural network, a similarity score for each matched track and associates two tracks based on the measured similarity.

First claim

What is claimed:

1 . A method of re-identifying players in a broadcast video feed, comprising:
retrieving, by a computing system, a broadcast video feed for a sporting event, the broadcast video feed comprising a plurality of video frames; generating, by the computing system, a plurality of tracks based on the plurality of video frames, wherein each track comprises a plurality of image patches associated with at least one player including a visible jersey number of the at least one player, each image patch of the plurality of image patches being a subset of a corresponding frame of the plurality of video frames; matching, by the computing system, the plurality of tracks including the visible jersey number of the at least one player; measuring, by the computing system via a neural network, a similarity score for each matched track; and associating, by the computing system, two tracks based on the measured similarity.

2 . The method of claim 1 , wherein each track further comprises a player identity label associated with each image patch of the plurality of image patches.

3 . The method of claim 1 , wherein matching, by the computing system, the plurality of tracks including the visible jersey number of the at least one player comprises:
determining an orientation of a player in each image patch of the plurality of image patches associated with a track; identifying a subset of image patches in which the player's orientation in the track is at least greater than a threshold value; and adding the subset of tracks to the gallery of image patches associated with the track.

4 . The method of claim 3 , wherein identifying the subset of image patches in which the player's orientation in the track is at least greater than the threshold value comprises:
calculating, for each image patch, a shoulder width of the player to determine the player; and normalizing the shoulder width by a length of a torso of the player.

5 . The method of claim 1 , wherein matching, by the computing system, the plurality of tracks including the visible jersey number of the at least one player, comprises:
identifying a first track corresponding to a first player; learning, by an autoencoder, a first set of jersey features associated with the first player in a first set of image patches associated with the first track; identifying a second track corresponding to a second player; and learning, by the autoencoder, a second set of jersey features associated with the second player in a second set of image patches associated with the second track.

6 . The method of claim 5 , wherein measuring, by the computing system via the neural network, the similarity score for each matched track comprises:
computing, via the neural network, the similarity score between the first set of jersey features and the second set of jersey features by comparing every pair of image patches across the first set of image patches and the second set of image patches; and averaging, via the neural network, the similarity score of all pairs of image patches.

7 . The method of claim 6 , further comprising:
determining that the average similarity score is at least higher than a threshold value; and based on the average similarity score being at least higher than the threshold value, determining that the first player and the second player are the same player.

8 . A system for re-identifying players in a broadcast video feed, comprising:
a processor; and a memory having programming instructions stored thereon, which, when executed by the processor, performs one or more operations, comprising: retrieving a broadcast video feed for a sporting event, the broadcast video feed comprising a plurality of video frames; generating a plurality of tracks based on the plurality of video frames, wherein each track comprises a plurality of image patches associated with at least one player including a visible jersey number of the at least one player, each image patch of the plurality of image patches being a subset of a corresponding frame of the plurality of video frames; matching, via an autoencoder, the plurality of tracks including the visible jersey number of the at least one player; measuring, via a neural network, a similarity score for each matched track; and associating two tracks based on the measured similarity.

9 . The system of claim 8 , wherein each track further comprises a player identity label associated with each image patch of the plurality of image patches.

10 . The system of claim 8 , wherein matching the plurality of tracks including the visible jersey number of the at least one player comprises:
determining an orientation of a player in each image patch of the plurality of image patches associated with a track; identifying a subset of image patches in which the player's orientation in the track is at least greater than a threshold value; and adding the subset of tracks to the gallery of image patches associated with the track.

11 . The system of claim 10 , wherein identifying the subset of image patches in which the player's orientation in the track is at least greater than the threshold value comprises:
calculating, for each image patch, a shoulder width of the player to determine the player; and normalizing the shoulder width by a length of a torso of the player.

12 . The system of claim 8 , wherein matching, via the autoencoder, the plurality of tracks including the visible jersey number of the at least one player, comprises:
identifying a first track corresponding to a first player; learning, by the autoencoder, a first set of jersey features associated with the first player in a first set of image patches associated with the first track; identifying a second track corresponding to a second player; and learning, by the autoencoder, a second set of jersey features associated with the second player in a second set of image patches associated with the second track.

13 . The system of claim 12 , wherein measuring, via the neural network, the similarity score for each matched track comprises:
computing, via the neural network, the similarity score between the first set of jersey features and the second set of jersey features by comparing every pair of image patches across the first set of image patches and the second set of image patches; and averaging, via the neural network, the similarity score of all pairs of image patches.

14 . The system of claim 13 , further comprising:
determining that the average similarity score is at least higher than a threshold value; and based on the average similarity score being at least higher than the threshold value, determining that the first player and the second player are the same player.

15 . A non-transitory computer readable medium including one or more sequences of instructions that, when executed by one or more processors, perform operations comprising:
retrieving, by a computing system, a broadcast video feed for a sporting event, the broadcast video feed comprising a plurality of video frames; generating, by the computing system, a plurality of tracks based on the plurality of video frames, wherein each track comprises a plurality of image patches associated with at least one player including a visible jersey number of the at least one player, each image patch of the plurality of image patches being a subset of a corresponding frame of the plurality of video frames; matching, by the computing system, the plurality of tracks including the visible jersey number of the at least one player; measuring, by the computing system via a neural network, a s

Assignees

• Stats Llc

Inventors

• Sha Long
• Ganguly Sujoy
• Wei Xinyu
• Lucey Patrick Joseph
• Cherukumudi Aditya

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06V10/454

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06V10/82

  using neural networks · CPC title