System and method for generating player tracking data from broadcast video

Patent metadata

Field	Value
Publication number	US-11830202-B2
Application number	US-202117532707-A
Country	US
Kind code	B2
Filing date	Nov 22, 2021
Priority date	Feb 28, 2019
Publication date	Nov 28, 2023
Grant date	Nov 28, 2023

Abstract

A system and method of generating a player tracking prediction are described herein. A computing system retrieves a broadcast video feed for a sporting event. The computing system segments the broadcast video feed into a unified view. The computing system generates a plurality of data sets based on the plurality of trackable frames. The computing system calibrates a camera associated with each trackable frame based on the body pose information. The computing system generates a plurality of sets of short tracklets based on the plurality of trackable frames and the body pose information. The computing system connects each set of short tracklets by generating a motion field vector for each player in the plurality of trackable frames. The computing system predicts a future motion of a player based on the player's motion field vector using a neural network.

First claim

What is claimed:

1. A method, comprising:
identifying, by a computing system, a broadcast video feed for a sporting event, the broadcast video feed comprising a plurality of video frames;
segmenting, by the computing system, the broadcast video feed into a unified view, wherein the unified view comprises a plurality of trackable frames, the plurality of trackable frames is a subset of the plurality of video frames;
generating, by the computing system, body pose information for each player in each trackable frame of the plurality of trackable frames; and
constructing, by the computing system, future motion of a player based on the plurality of trackable frames and the body pose information, the constructing comprising:
projecting motion of the player when the player has left a field of view and is not visible in the broadcast video feed, the projecting comprising:
identifying a first set of frames in which the player is present,
identifying a second set of frames following the first set of frames in which the player is not present, and
predicting a trajectory of the player based on prior trajectories of the player in the first set of frames.

2. The method of claim 1 , wherein segmenting, by the computing system, the broadcast video feed into the unified view comprises:
parsing the broadcast video feed to identify a first subset of video frames corresponding to a same view of the sporting event; and
discarding a second subset of video frames corresponding to a different view of the sporting event.

3. The method of claim 1 , further comprising:
identifying, by the computing system, a pattern of motion between two successive trackable frames by identifying players in each frame using the body pose information.

4. The method of claim 3 , further comprising:
generating, by the computing system, a motion field vector for each player in the plurality of trackable frames.

5. The method of claim 4 , wherein constructing, by the computing system, the future motion of the player based on the plurality of trackable frames and the body pose information comprises:
generating, via a neural network, the future motion of the player based on the motion field vector generated for the player.

6. A system for generating a player tracking prediction, comprising:
a processor; and
a memory having programming instructions stored thereon, which, when executed by the processor, causes the system to perform one or more operations comprising:
identifying a broadcast video feed for a sporting event, the broadcast video feed comprising a plurality of video frames;
segmenting the broadcast video feed into a unified view, wherein the unified view comprises a plurality of trackable frames, the plurality of trackable frames is a subset of the plurality of video frames;
generating body pose information for each player in each trackable frame of the plurality of trackable frames; and
constructing future motion of a player based on the plurality of trackable frames and the body pose information, the constructing comprising:
projecting motion of the player when the player has left a field of view and is not visible in the broadcast video feed, the projecting comprising:
identifying a first set of frames in which the player is present,
identifying a second set of frames following the first set of frames in which the player is not present, and
predicting a trajectory of the player based on prior trajectories of the player in the first set of frames.

7. The system of claim 6 , wherein segmenting the broadcast video feed into the unified view comprises:
parsing the broadcast video feed to identify a first subset of video frames corresponding to a same view of the sporting event; and
discarding a second subset of video frames corresponding to a different view of the sporting event.

8. The system of claim 6 , wherein the one or more operations further comprise:
identifying a pattern of motion between two successive trackable frames by identifying players in each frame using the body pose information.

9. The system of claim 8 , wherein the one or more operations further comprise:
generating a motion field vector for each player in the plurality of trackable frames.

10. The system of claim 9 , wherein constructing the future motion of the player based on the plurality of trackable frames and the body pose information comprises:
generating, via a neural network, the future motion of the player based on the motion field vector generated for the player.

11. A non-transitory computer readable medium including one or more sequences of instructions that, when executed by one or more processors, causes a computing system to perform one or more operations comprising:
identifying, by the computing system, a broadcast video feed for a sporting event, the broadcast video feed comprising a plurality of video frames;
segmenting, by the computing system, the broadcast video feed into a unified view, wherein the unified view comprises a plurality of trackable frames, the plurality of trackable frames is a subset of the plurality of video frames;
generating, by the computing system, body pose information for each player in each trackable frame of the plurality of trackable frames; and
constructing, by the computing system, future motion of a player based on the plurality of trackable frames and the body pose information, the constructing comprising:
projecting motion of the player when the player has left a field of view and is not visible in the broadcast video feed, the projecting comprising:
identifying a first set of frames in which the player is present,
identifying a second set of frames following the first set of frames in which the player is not present, and
predicting a trajectory of the player based on prior trajectories of the player in the first set of frames.

12. The non-transitory computer readable medium of claim 11 , wherein segmenting, by the computing system, the broadcast video feed into the unified view comprises:
parsing the broadcast video feed to identify a first subset of video frames corresponding to a same view of the sporting event; and
discarding a second subset of video frames corresponding to a different view of the sporting event.

13. The non-transitory computer readable medium of claim 11 , further comprising:
identifying, by the computing system, a pattern of motion between two successive trackable frames by identifying players in each frame using the body pose information.

14. The non-transitory computer readable medium of claim 13 , further comprising:
generating, by the computing system, a motion field vector for each player in the plurality of trackable frames.

15. The non-transitory computer readable medium of claim 14 , wherein constructing, by the computing system, the future motion of the player based on the plurality of trackable frames and the body pose information comprises:
generating, via a neural network, the future motion of the player based on the motion field vector generated for the player.

Assignees

• Stats Llc

Inventors

• Sha Long
• Ganguly Sujoy
• Wei Xinyu
• Lucey Patrick Joseph
• Cherukumudi Aditya

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06V10/761

  Proximity, similarity or dissimilarity measures · CPC title

• H04N21/44008

  involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59) · CPC title