System and method for inter-sensor calibration

Patent metadata

Field	Value
Publication number	US-11619708-B2
Application number	US-202016947312-A
Country	US
Kind code	B2
Filing date	Jul 28, 2020
Priority date	Jul 28, 2020
Publication date	Apr 4, 2023
Grant date	Apr 4, 2023

Abstract

A method includes generating an initial radar track using radar data and initial radar parameters and an initial camera track using image angular position data, initial camera parameters, and the radar range data in combination with calculating correction parameters to be applied to one of the radar data and the image data by comparing positions for the object from the initial radar track and the initial camera track, the first correction parameters being selected so that, when applied to the data from the other of the radar and the camera, to generate a first corrected track, a degree of correspondence between the first corrected track and the track of the other of the radar and camera is higher than a degree of correspondence between the initial radar track and the initial camera track.

First claim

The invention claimed is:

1. A method, comprising:
capturing radar data with a radar and image data with a camera in a multi-sensor tracking system, the radar and image data corresponding to a path of an object, the radar data including radar range data and radar velocity data for the object and the image data including image angular position data for the object;
generating an initial radar track using the radar data and initial radar parameters associated therewith stored to memory;
generating an initial camera track using the image angular position data, initial camera parameters associated therewith stored to the memory, and the radar range data;
calculating one of:
first correction parameters to be applied to the radar data by comparing positions for the object from the initial radar track and the initial camera track wherein, the first correction parameters are selected so that, when applied to the radar data to generate a first corrected track, a degree of correspondence between the first corrected track and the initial camera track is higher than a degree of correspondence between the initial radar track and the initial camera track; and
second correction parameters to be applied to the image angular position data by comparing positions for the object from the initial radar track and the initial camera track wherein, the second correction parameters are selected so that, when applied to the image angular position data to generate a second corrected track, a degree of correspondence between the second corrected track and the initial radar track is higher than a degree of correspondence between the initial radar track and the initial camera track; and

storing the first or second correction parameters to the memory for generating further initial radar tracks or initial camera tracks using further radar data or further camera data and the first or second correction parameters without manual intervention to the radar or camera.

2. The method of claim 1 , further comprising calibrating either the radar with the first correction parameters or the camera with the second correction parameters.

3. The method of claim 1 , wherein the first correction parameters comprise an optimized orientation of the radar and wherein the second correction parameters comprise an optimized orientation of the camera, the method further comprising:
determining one of a tilt adjustment and a pan adjustment to the initial radar parameters to calibrate the radar to a corrected orientation based on the first correction parameters and a tilt adjustment and a pan adjustment to the initial camera parameters to calibrate the camera to a corrected orientation based on the second correction parameters.

4. The method of claim 3 , further comprising:
fitting a polynomial to the initial radar track; and
evaluating the polynomial at a series of time points corresponding to times at which image data was acquired to determine a radar position corresponding to each time point in the series.

5. The method of claim 4 , further comprising:
determining a series of tilt and pan adjustments to one of the radar and the camera by comparing the polynomial and the camera track at each of the time points in the series;
determining a median for the series of tilt and pan adjustments as median tilt and pan adjustments; and
low pass filtering the median tilt and pan adjustments.

6. The method of claim 3 , wherein the initial radar track is represented by a state vector and a dynamical model, the state vector representing path parameters of the object and the dynamical model accounting for lift and drag on the object.

7. The method of claim 1 , wherein the radar range data is used to expand the image angular position data to 3D coordinates.

8. The method of claim 1 , wherein the multi-sensor tracking system is implemented at a baseball field and the object is a baseball.

9. A system, comprising:
a central processing arrangement in communication with a radar and a camera in a multi-sensor tracking system, the central processing arrangement receiving radar data from the radar and camera data from the camera corresponding to a path of an object, the radar data including radar range data and radar velocity data for the object and the camera data including image angular position data for the object, the central processing arrangement generating an initial radar track using the radar data and initial radar parameters associated therewith stored to memory, the central processing arrangement generating an initial camera track using the image angular position data, initial camera parameters associated therewith stored to the memory, and the radar range data, and the central processing arrangement calculating one of:
first correction parameters to be applied to the radar data by comparing positions for the object from the initial radar track and the initial camera track wherein, the first correction parameters are selected so that, when applied to the radar data to generate a first corrected track, a degree of correspondence between the first corrected track and the initial camera track is higher than a degree of correspondence between the initial radar track and the initial camera track; and
second correction parameters to be applied to the image angular position data by comparing positions for the object from the initial radar track and the initial camera track wherein, the second correction parameters are selected so that, when applied to the image angular position data to generate a second corrected track, a degree of correspondence between the second corrected track and the initial radar track is higher than a degree of correspondence between the initial radar track and the initial camera track; and

the central processing arrangement storing the first or second correction parameters to the memory for generating further initial radar tracks or initial camera tracks using further radar data or further camera data and the first or second correction parameters without manual intervention to the radar or camera.

10. The system of claim 9 , wherein the first correction parameters comprise an optimized orientation of the radar and wherein the second correction parameters comprise an optimized orientation of the camera, wherein the central processing arrangement determines one of a tilt adjustment and a pan adjustment to the initial radar parameters to calibrate the radar to a corrected orientation based on the first correction parameters and a tilt adjustment and a pan adjustment to the initial camera parameters to calibrate the camera to a corrected orientation based on the second correction parameters.

11. The system of claim 10 , wherein the central processing arrangement fits a polynomial to the initial radar track and evaluates the polynomial at a series of time points corresponding to times at which image data was acquired to determine a radar position corresponding to each time point in the series.

12. The system of claim 11 , wherein the central processing arrangement determines a series of tilt and pan adjustments to one of the radar and the camera by comparing the polynomial and the camera track at each of the time points in the series and determines a median for the series of tilt and pan adjustments as median tilt and pan adjustments, the system further comprising a low pass filtering arrangement filtering the median tilt and pan adjustments.

13. The system of claim 10 , wherein the central processing arrangement represents the initial radar track as a state vector and a dynamical model, the state vector representing path parameters of the object and the dynamical model accounting for lift and drag on

Assignees

• Trackman As

Inventors

• Søndergaard Thomas
• Tuxen Fredrik
• Ungstrup Michael

Classifications

• A63B2220/806

  Video cameras · CPC title

• G01S7/40

  Means for monitoring or calibrating · CPC title

• G01S13/867 Primary

  Combination of radar systems with cameras · CPC title

• G01S7/4021

  of receivers · CPC title

• A63B2220/89

  Field sensors, e.g. radar systems · CPC title