System and method for player reidentification in broadcast video

Patent metadata

Field	Value
Publication number	US-11861850-B2
Application number	US-202318171066-A
Country	US
Kind code	B2
Filing date	Feb 17, 2023
Priority date	Feb 28, 2019
Publication date	Jan 2, 2024
Grant date	Jan 2, 2024

Abstract

A system and method of re-identifying players in a broadcast video feed are provided herein. A computing system retrieves a broadcast video feed for a sporting event. The broadcast video feed includes a plurality of video frames. The computing system generates a plurality of tracks based on the plurality of video frames. Each track includes a plurality of image patches associated with at least one player. Each image patch of the plurality of image patches is a subset of the corresponding frame of the plurality of video frames. For each track, the computing system generates a gallery of image patches. A jersey number of each player is visible in each image patch of the gallery. The computing system matches, via a convolutional autoencoder, tracks across galleries. The computing system measures, via a neural network, a similarity score for each matched track and associates two tracks based on the measured similarity.

First claim

What is claimed:

1. A method comprising:
identifying, by a computing system, broadcast video feed of a game, the broadcast video feed captured by a broadcast camera;
identifying, by the computing system from the broadcast video feed, a first track corresponding to a first player, the first track comprising trajectory information corresponding to the first player up to a first time, wherein after the first time, the first player is not a field of view of the broadcast camera;
identifying, by the computing system from the broadcast video feed, a second track corresponding to a second player, the second track comprising second trajectory information corresponding to the second player up to a second time, the second time after the first time;
learning, by the computing system, a first set of visual attributes corresponding to the first player in the first track;
learning, by the computing system, a second set of visual attributes corresponding to the second player in the second track;
determining, by the computing system, that the first player and the second player are the same player by measuring a similarity between the first set of visual attributes and the second set of visual attributes; and
inferring, by the computing system, movement of the first player between the first time and the second time, wherein the first player is not in the field of view of the broadcast camera between the first time and the second time.

2. The method of claim 1 , wherein learning, by the computing system, the first set of visual attributes corresponding to the first player in the first track comprises:
generating a plurality of player patches corresponding to the first player based on body-pose information of the first player and an appearance of the first player.

3. The method of claim 2 , wherein learning, by the computing system, the second set of visual attributes corresponding to the second player in the second track comprises:
generating a second plurality of player patches corresponding to the second player based on second body-pose information of the second player and a second appearance of the second player.

4. The method of claim 3 , wherein determining, by the computing system, that the first player and the second player are the same player by measuring the similarity between the first set of visual attributes and the second set of visual attributes comprises:
comparing the plurality of player patches corresponding to the first player to the second plurality of player patches corresponding to the second player.

5. The method of claim 3 , wherein determining, by the computing system, that the first player and the second player are the same player by measuring the similarity between the first set of visual attributes and the second set of visual attributes comprises:
measuring a further similarity between the plurality of player patches and the second plurality of player patches based on their feature representations using a Siamese neural network.

6. The method of claim 1 , further comprising:
based on the inferring, generating, by the computing system, a tracklet of player movement based on the first track and the second track.

7. The method of claim 1 , wherein the first track and the second track do not include any overlapping time.

8. A non-transitory computer readable medium comprising one or more sequences of instructions, which, when executed by a processor, causes a computing system to perform operations comprising:
identifying, by the computing system, broadcast video feed of a game, the broadcast video feed captured by a broadcast camera;
identifying, by the computing system from the broadcast video feed, a first track corresponding to a first player, the first track comprising trajectory information corresponding to the first player up to a first time, wherein after the first time, the first player is not a field of view of the broadcast camera;
identifying, by the computing system from the broadcast video feed, a second track corresponding to a second player, the second track comprising second trajectory information corresponding to the second player up to a second time, the second time after the first time;
learning, by the computing system, a first set of visual attributes corresponding to the first player in the first track;
learning, by the computing system, a second set of visual attributes corresponding to the second player in the second track;
determining, by the computing system, that the first player and the second player are the same player by measuring a similarity between the first set of visual attributes and the second set of visual attributes; and
inferring, by the computing system, movement of the first player between the first time and the second time, wherein the first player is not in the field of view of the broadcast camera between the first time and the second time.

9. The non-transitory computer readable medium of claim 8 , wherein learning, by the computing system, the first set of visual attributes corresponding to the first player in the first track comprises:
generating a plurality of player patches corresponding to the first player based on body-pose information of the first player and an appearance of the first player.

10. The non-transitory computer readable medium of claim 9 , wherein learning, by the computing system, the second set of visual attributes corresponding to the second player in the second track comprises:
generating a second plurality of player patches corresponding to the second player based on second body-pose information of the second player and a second appearance of the second player.

11. The non-transitory computer readable medium of claim 10 , wherein determining, by the computing system, that the first player and the second player are the same player by measuring the similarity between the first set of visual attributes and the second set of visual attributes comprises:
comparing the plurality of player patches corresponding to the first player to the second plurality of player patches corresponding to the second player.

12. The non-transitory computer readable medium of claim 10 , wherein determining, by the computing system, that the first player and the second player are the same player by measuring the similarity between the first set of visual attributes and the second set of visual attributes comprises:
measuring a further similarity between the plurality of player patches and the second plurality of player patches based on their feature representations using a Siamese neural network.

13. The non-transitory computer readable medium of claim 8 , further comprising:
based on the inferring, generating, by the computing system, a tracklet of player movement based on the first track and the second track.

14. The non-transitory computer readable medium of claim 8 , wherein the first track and the second track do not include any overlapping time.

15. A system, comprising:
a processor; and
a memory having programming instructions stored thereon, which, when executed by the processor, causes the system to perform operations comprising:
identifying broadcast video feed of a game, the broadcast video feed captured by a broadcast camera;
identifying, from the broadcast video feed, a first track corresponding to a first player, the first track comprising trajectory information corresponding to the first player up to a first time, wherein after the first time, the first player is not a field of view of the broadcast camera;
identifying, from the broadcast video feed, a second track corresponding to a second player, the second track c

Assignees

• Stats Llc

Inventors

• Sha Long
• Ganguly Sujoy
• Wei Xinyu
• Lucey Patrick Joseph
• Cherukumudi Aditya

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06V10/761

  Proximity, similarity or dissimilarity measures · CPC title

• H04N21/44008

  involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59) · CPC title