Method and apparatus of game status determination

Patent metadata

Field	Value
Publication number	US-11514704-B2
Application number	US-201817057020-A
Country	US
Kind code	B2
Filing date	Oct 2, 2018
Priority date	Oct 2, 2018
Publication date	Nov 29, 2022
Grant date	Nov 29, 2022

Abstract

A video capture and processing system includes a memory configured to store a pose database. The pose database includes poses that indicate a start or stoppage in an event. The system also includes a processor operatively coupled to the memory. The processor is configured to generate a pose of an individual in a video frame of captured video of the event. The pose can be three-dimensional pose or a two-dimensional pose. The processor is also configured to determine, based on the pose database, whether the pose of the individual indicates a start or a stoppage in the event. The processor is further configured to control an upload of video of the event based on the determination of whether the pose indicates the start or the stoppage in the event.

First claim

What is claimed is:

1. A video capture and processing system comprising:
a memory configured to store a pose database, wherein the pose database includes a first pose that indicates a resumption of an ongoing event and a second pose that indicates a pause in the ongoing event; and
a processor operatively coupled to the memory, wherein the processor is configured to:
generate a pose of an individual in a video frame of captured video of the event;
identify, from the pose database, whether the pose of the individual matches the first pose or the second pose; and
determine, based on the identified pose, whether the individual has indicated the resumption or the pause of the ongoing event,

wherein the processor is configured to stop an upload of video of the event to a remote location responsive to a determination that the pose indicates a pause in the event, and wherein the processor is configured to commence the upload of the video to the remote location responsive to a determination that the pose indicates a resumption of the event.

2. The video capture and processing system of claim 1 , wherein the processor is configured to apply a mask to the video frame, wherein the mask identifies a portion of the video frame in which the individual is supposed to be located.

3. The video capture and processing system of claim 1 , wherein the video frame comprises a first video frame captured by a first video camera, and wherein the pose of the individual is generated based on the first video frame and a second video frame captured by a second video camera.

4. The video capture and processing system of claim 3 , wherein the processor is configured to detect a first plurality of individuals in the first video frame and a second plurality of individuals in the second video frame.

5. The video capture and processing system of claim 4 , wherein the processor is configured to:
generate a bounding box for each of the first plurality of individuals detected in the first video frame and each of the second plurality of individuals detected in the second video frame; and
analyze the generated bounding boxes for the first video frame and the second video frame to identify one or more bounding boxes corresponding to one or more referees.

6. The video capture and processing system of claim 5 , wherein the processor analyzes the generated bounding boxes with a pre-trained classifier that distinguishes referees from non-referees.

7. The video capture and processing system of claim 5 , wherein the processor is configured to generate a two dimensional (2D) pose for each of the one or more bounding boxes corresponding to the one or more referees.

8. The video capture and processing system of claim 7 , wherein the 2D pose for a given referee includes a plurality of points corresponding to a plurality of joints of the given referee.

9. The video capture and processing system of claim 8 , wherein the processor is configured to generate a vertical line for each of the one or more bounding boxes corresponding to the one or more referees, wherein each vertical line has a starting location that is based on the 2D pose associated with each of the one or more bounding boxes.

10. The video capture and processing system of claim 9 , wherein the processor is configured to convert each vertical line into a principal line, wherein the principal line maps the vertical line onto a ground plane of a venue in which the event is held.

11. The video capture and processing system of claim 10 , wherein the processor uses a first homography matrix associated with the first video camera to convert vertical lines in bounding boxes generated from the first video frame, and wherein the processor uses a second homography matrix associated with the second video camera to convert vertical lines in bounding boxes generated from the second video frame.

12. The video capture and processing system of claim 10 , wherein the processor is configured to:
identify an intersection of principal lines in the ground plane, wherein the principal lines that form the intersection correspond to vertical lines of bounding boxes that include different views of the given referee; and
identify, based on the intersection, a group of the bounding boxes that include the different views of the given referee, wherein the different views of the given referee include a first view captured by the first video camera in the first video frame and a second view captured by the second video camera in the second video frame.

13. The video capture and processing system of claim 12 , wherein the processor is configured to reconstruct each of the plurality of joints of the given referee in a three dimensional (3D) grid based on the 2D pose generated for each of the bounding boxes in the group of bounding boxes that include the different views of the given referee.

14. The video capture and processing system of claim 13 , wherein the individual is the given referee, wherein the pose comprises a three dimensional (3D) pose, and wherein the processor generates the 3D pose based on the reconstruction of each of the plurality of joints in the 3D grid.

15. A method of capturing and processing video, the method comprising:
analyzing, by a processor of a computing system, a plurality of video frames depicting a plurality of views of an ongoing event, wherein each video frame in the plurality of video frames is captured by a distinct video camera;
generating, by the processor, a pose of a referee that is captured in one or more of the plurality of video frames;
determining, by the processor and based on a pose database stored in a memory of the computing system, whether the pose of the referee indicates a resumption or a pause in the ongoing event; and
controlling an upload of video of the event by stopping the upload of the video to a remote location responsive to a determination that the pose indicates the pause in the ongoing event, and commencing the upload of the video to the remote location responsive to a determination that the pose indicates the resumption of the ongoing event.

16. The method of claim 15 , wherein the referee is captured in a first video frame and a second video frame of the plurality of video frames, and further comprising:
generating, by the processor, a first bounding box for the referee in the first video frame and a second bounding box for the referee in the second video frame;
generating, by the processor, a first two dimensional (2D) pose associated with a first view of the referee in the first bounding box and a second 2D pose associated with a second view of the referee in the second bounding box; and
wherein the pose comprises a three-dimensional (3D) pose that is generated based on the first 2D pose and the second 2D pose.

17. A non-transitory computer-readable storage medium having computer-readable instructions stored thereon that, upon execution by one or more processors in a video capturing and processing system, result in operations comprising:
analyzing a plurality of video frames depicting a plurality of views of an ongoing event, wherein each video frame in the plurality of video frames is captured by a distinct video camera;
generating a pose of a referee that is captured in one or more of the plurality of video frames;
determining, based on a pose database, whether the pose of the referee indicates a resumption or a pause in the ongoing event; and
controlling an upload of video of the event by stopping the upload of the video to a remote location responsive to a determination that the pose indicates the pause in the ongoi

Assignees

• Intel Corp

Inventors

• Ling Chen
• Lu Ming
• Li Qiang
• Li Wenlong
• Tong Xiaofeng
• Fang Yikai
• Wang Yumeng

Classifications

• G06V40/103 Primary

  Static body considered as a whole, e.g. static pedestrian or occupant recognition · CPC title

• G06V20/46

  Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames · CPC title

• G06V20/41

  Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49) · CPC title

• G06V20/42

  of sport video content · CPC title

• G06V40/23

  Recognition of whole body movements, e.g. for sport training · CPC title