Automated or assisted umpiring of baseball game using computer vision

Patent metadata

Field	Value
Publication number	US-10076698-B2
Application number	US-201615157222-A
Country	US
Kind code	B2
Filing date	May 17, 2016
Priority date	May 17, 2016
Publication date	Sep 18, 2018
Grant date	Sep 18, 2018

Abstract

Methods and systems for use in automating or assisting umpiring of a baseball or softball game are described herein. A location of a strike zone is determined based on video images of a batter standing next to home plate captured by a camera. Locations of a ball traveling towards the batter, and locations of the bat being held by the batter, are autonomously tracked using computer vision based on video images captured by at least two cameras having different positions. Additionally, there are autonomous determinations of whether a location of the ball intersects with the strike zone, and whether the batter made a genuine attempt to swing the bat at the ball, and based one at least one of these determinations, there is an autonomous determination of whether a “strike” or a “ball” occurred. Additionally, an indication of whether a “strike” or a “ball” occurred is autonomously output.

First claim

What is claimed is:

1. A method for use in automating or assisting umpiring of a baseball or softball game, comprising:
receiving video images, captured using at least one camera, of a batter standing next to home plate;
determining, based on the video images of the batter standing next to home plate, a location of a strike zone;
receiving video images, captured using at least two different cameras having different positions, of a ball traveling towards the batter for which the strike zone was determined; wherein the video images of the ball traveling towards the batter are in two-dimensional (2D) space;
receiving video images, captured using at least two different cameras having different positions, of a bat being held by the batter as the ball travels towards the batter, wherein at least some of the video images of the bat being held by the batter may or may not be the same as at least some of the video images of the ball traveling towards the batter; wherein the video images of the bat being held by the batter as the ball travels towards the batter are in 2D space;
autonomously tracking locations of the ball traveling towards the batter, using computer vision, based on the video images of the ball traveling towards the batter; wherein the autonomously tracking locations of the ball traveling towards the batter includes using transformations associated with the at least two different cameras used to capture the video images of the ball traveling towards the batter to determine locations of the ball traveling towards the batter in three-dimensional (3D) space;
autonomously tracking locations of the bat being held by the batter as the ball travels towards the batter, using computer vision, based on the video images of the bat being held by the batter as the ball travels towards the batter; wherein the autonomously tracking locations of the bat being held by the batter as the ball travels towards the batter includes using transformation associated with the at least two different cameras used to capture the video images of the bat being held by the batter to determine locations of the bat in 3D space;
autonomously determining whether a said location of the bat in 3D space is the same as a said location of the ball in 3D space at a same point in time;
autonomously determining whether at least one location of the ball traveling towards the batter intersects with the strike zone, based on the determined location of the strike zone and the tracked locations of the ball traveling towards the batter;
autonomously determining whether the batter made a genuine attempt to swing the bat at the ball, based on the tracked locations of the bat being held by the batter as the ball travels towards the batter;
autonomously determining that the bat made contact with the ball if there was an autonomous determination that a said location of the bat in 3D space was the same as a said location of the ball in 3D space at a same point in time;
autonomously determining that the bat did not make contact with the ball if there was an autonomous determination that a said location of the bat in 3D space was not the same as a said location of the ball in 3D space at a same point in time;
autonomously determining whether a “strike” or a “ball” occurred, based on at least one of the determination of whether at least one location of the ball intersected with the strike zone, or the determination of whether the batter made a genuine attempt to swing at the ball; and
autonomously outputting an indication of whether a “strike” or a “ball” occurred.

2. The method of claim 1 , wherein the autonomously determining whether a “strike” or a “ball” occurred, includes determining that a “strike” occurred if there was an autonomous determination that the bat did not make contact with the ball, and there was also an autonomous determination that the batter made a genuine attempt to swing the bat at the ball.

3. The method of claim 2 , wherein the autonomously determining whether a “strike” or a “ball” occurred, includes determining that a “ball” occurred if there was an autonomous determination that at least one location of the ball traveling towards the batter did not intersect with the strike zone, and there was an autonomous determination that the batter did not make a genuine attempt to swing the bat at the ball.

4. The method of claim 1 , further comprising:
determining a vertical plane in 3D space that is aligned with a front of the home plate; and
autonomously determining whether a said location of the bat in 3D space intersects with the vertical plane in 3D space that is aligned with the front of home plate as the ball travels towards the batter;
wherein the autonomously determining whether the batter made a genuine attempt to swing the bat at the ball is based on the autonomous determination of whether a said location of the bat in 3D space intersected with the vertical plane in 3D space that is aligned with the front of home plate as the ball traveled towards the batter.

5. The method of claim 4 , wherein the autonomously determining whether the batter made a genuine attempt to swing the bat at the ball includes:
determining that the batter made a genuine attempt to swing the bat at the ball if a said location of the bat in 3D space intersected with the plane in 3D space that corresponds to the front of home plate; and
determining that the batter did not make a genuine attempt to swing the bat at the ball if a said location of the bat in 3D space did not intersect with the plane in 3D space that corresponds to the front of home plate.

6. The method of claim 1 , further comprising:
autonomously determining a rotation angle of the bat as the ball travels towards the batter; and
comparing the rotation angle of the bat to a threshold angle;
wherein the autonomously determining whether the batter made a genuine attempt to swing the bat at the ball is based on whether the rotation angle of the bat exceeded the threshold angle.

7. The method of claim 1 , further comprising autonomously:
determining a trajectory of the ball in 3D space as the ball travels towards the batter;
determining whether a said location of the bat in 3D space is within a specified vicinity of a said location of the ball in 3D space at a same point in time;
determining whether the trajectory of the ball in 3D space changed beyond a specified threshold while a said location of the bat in 3D space was within the specified vicinity of a said location of the ball in 3D space at a same point in time;
determining whether the bat made contact with the ball, based on the determination of whether the trajectory of the ball in 3D space changed beyond the specified threshold while a said location of the bat in 3D space was within the specified vicinity of a said location of the ball in 3D space at a same point in time; and
outputting an indication of whether the ball made contact with the bat, which indication can be used to determine whether a foul tip occurred.

8. The method of claim 1 , further comprising:
receiving audio associated with the batter holding the bat as the ball travels towards the batter; and
autonomously determining whether the bat made contact with the ball based on the audio associated with the batter holding the bat as the ball travels towards the batter; and
outputting an indication of whether the ball made contact with the bat, which indication can be used to determine whether a foul tip occurred.

9. The method of claim 1 , further comprising:
autonomously determining whether the bat made contact with the ball;
receiving video images, captured by at least two different cameras having different positions, of the ball, after the bat made contact with the ball, if t

Assignees

• Sportsmedia Tech Corp

Inventors

• Spivak Josh D
• Goldbeck Graham W
• Zander Ryan A
• Chen Ted C
• Amyx Robert C
• Painter James G

Classifications

• G06T7/246 Primary

  using feature-based methods, e.g. the tracking of corners or segments · CPC title

• G06T2207/30241

  Trajectory · CPC title

• A63B2220/806

  Video cameras · CPC title

• G06T2207/30224

  Ball; Puck · CPC title

• G06T7/254

  involving subtraction of images · CPC title