Determining x,y,z,t biomechanics of moving actor with multiple cameras

Patent metadata

Field	Value
Publication number	US-10019806-B2
Application number	US-201514687791-A
Country	US
Kind code	B2
Filing date	Apr 15, 2015
Priority date	Apr 15, 2015
Publication date	Jul 10, 2018
Grant date	Jul 10, 2018

Abstract

A plurality of high speed tracking cameras are pointed towards a routine hovering area of an in-the-field sports participant who routinely hovers about that area. Spots within the hovering area are registered relative to a predetermined multi-dimensional coordinates reference frame (e.g., Xw, Yw, Zw, Tw) such that two-dimensional coordinates of 2D images captured by the high speed tracking cameras can be converted to multi-dimensional coordinates of the reference frame. A body part recognizing unit recognizes 2D locations of a specific body part in the 2D captured images and a mapping unit maps them into the multi-dimensional coordinates of the reference frame. A multi-dimensional curve generator then generates a multi-dimensional motion curve describing motion of the body part based on the mapped coordinates (e.g., Xw, Yw, Zw, Tw). The generated multi-dimensional motion curve is used to discover cross correlations between play action motions of the in-the-field sports participant and real-world sports results.

First claim

What is claimed is:

1. A method of determining multi-dimensional (mD) performance attributes of an in-the-field and moving sports participant who, at least at certain times routinely hovers about a predetermined hovering area while participating in a live in-the-field sports event, where the mD performance attributes are ones mapped along a time line as well as over a plurality of spatial reference coordinates and where the moving sports participant has a plurality of identifiable body parts, the method comprising:
providing at least three high speed cameras, wherein each of the at least three high speed cameras operates at more than 30 frames per second and the at least three high speed cameras are respectively pointed to have different points of view of the predetermined hovering area;
identifying uniquely identifiable spots within the predetermined hovering area of the moving sports participant from at least two of the at least three high speed cameras;
determining respective mD coordinates of the identified uniquely identifiable spots relative to a predetermined mD frame of reference;
receiving and recording at least two 2 dimensional (2D) images of the moving sports participant from the at least two of the at least three high speed cameras;
determining at least one of when a play action activity of the moving sports participant has been entered into or has just ended;
determining temporal segment lengths of the play action activity in the at least two 2D images received from the at least two of the at least three high speed cameras;
discarding parts of the at least two 2D images temporally adjacent to the temporal segment lengths of the play action activity;
determining a common timing reference along which the at least two 2D images of the play action activity can be approximately placed;
intertwining the at least two 2D images of the play action activity such that the at least two 2D images are approximately disposed along the common timing reference;
determining from the at least two 2D images a mD smooth and continuous motion curve of a selected one of the identifiable body parts of the moving sports participant, the mD smooth and continuous motion curve covering a continuous segment of time in which the moving sports participant was performing the play action activity;
revising the intertwining of the at least two 2D images of the play action activity; wherein the revised intertwining conforms to respective smooth and continuous motion curves developed for two or more of the identifiable body parts of the moving sports participant.

2. The method of claim 1 , wherein:
each of the at least three high speed cameras operates at 120 or more frames per second and independently of the other high speed cameras.

3. The method of claim 1 , wherein:
the mD smooth and continuous motion curve is mapped within a mD space having at least time (Tw) as well three spatial and orthogonal coordinate axes (Xw, Yw and Zw) as its multiple dimensions.

4. The method of claim 1 , and further comprising:
after the respective smooth and continuous motion curves are developed for the two or more of the identifiable body parts of the moving sports participant, identifying camera frames or camera scan lines of the at least two of the at least three high speed cameras that are closest in time with respect to spatial points along the respective smooth and continuous motion curves;
where the identifying of the camera frames or camera scan lines provides a logical linkage between any spatial point along the respective smooth and continuous motion curves and its respective closest in time camera frames such that the closest in time camera frames can be called up for viewing when a spatial point along the respective smooth and continuous motion curves is identified.

5. The method of claim 1 , and further comprising:
storing in a database, one or more definitions of the determined mD smooth and continuous motion curve of the selected one of the identifiable body parts of the moving sports participant where the stored definitions input at least one of: a first mathematical expression defining all spatial points versus time of the determined mD smooth and continuous motion curve; coefficients of the first mathematical expression; one or more graphic curves defining all spatial points versus time of the determined mD smooth and continuous motion curve; sample points in tabular form identifying spatial points and their respective timings along the determined mD smooth and continuous motion curve; a second mathematical expression defining first derivatives versus time of spatial points of the determined mD smooth and continuous motion curve; a third mathematical expression defining second derivatives versus time of spatial points of the determined mD smooth and continuous motion curve; sample points in tabular form identifying points of potential interest of the determined mD smooth and continuous motion curve, the points of potential interest including at least one of maximums, minimums, means and medians of the spatial points or velocities or accelerations associated therewith.

6. The method of claim 1 wherein the moving sports participant is selected from the group consisting of:
a baseball pitcher; a baseball batter; a baseball catcher; an on-deck and swing practicing next baseball batter; a football placekicker; a goalie; a player shooting a penalty shot from a predetermined penalty shot location; and a player shooting a shot from a predetermined favorite shooting location of that player.

7. The method of claim 1 and further comprising:
after the play action activity, using the determined mD smooth and continuous motion curve of the selected one of the identifiable body parts of the moving sports participant or data derived from the curve to modify a behavior of the moving sports participant.

8. The method of claim 2 , wherein:
each of the at least three high speed cameras operates at 240 or more frames per second.

9. The method of claim 5 , and further comprising:
storing in the database, and as logically linked to the stored definitions of the determined mD smooth and continuous motion curve of the selected one of the identifiable body parts, biometric data obtained from the moving sports participant during or at a time substantially close to the play action activity such that the obtained biometric data can be said to be potentially relevant to the performance of the moving sports participant during the play action.

10. The method of claim 5 , and further comprising:
storing in the database, and as logically linked to the stored definitions of the determined mD smooth and continuous motion curve of the selected one of the identifiable body parts, field effects data obtained from surroundings of the moving sports participant during or at a time substantially close to the play action activity such that the obtained field effects data can be said to be potentially relevant to the performance of the moving sports participant during the play action activity.

11. The method of claim 5 , and further comprising:
storing in the database, and as logically linked to the stored definitions of the determined mD smooth and continuous motion curve of the selected one of the identifiable body parts, play action results data that follows the play action motions of the moving sports participant.

12. The method of claim 9 wherein:
the obtained biometric data is indicative of at least one of: a heart rate of the moving sports participant around the time of the play action activity; a breathing rate of the moving sports participant around the time of the play action activity; a perspiration leve

Assignees

• Sportsmedia Tech Corp

Inventors

• Perry Mark
• Spivak Joshua
• Zander Ryan
• Goldbeck Graham Wei-Feng
• Painter James G

Classifications

• H04N23/90

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title

• G06T2207/30241

  Trajectory · CPC title

• G06T2200/04

  involving 3D image data · CPC title

• H04N5/23229

  Electricity · mapped topic

• G06T2207/30221

  Sports video; Sports image · CPC title