Method and system for dynamically calibrating vehicular cameras

Patent metadata

Field	Value
Publication number	US-9357208-B2
Application number	US-201214113414-A
Country	US
Kind code	B2
Filing date	Jan 20, 2012
Priority date	Apr 25, 2011
Publication date	May 31, 2016
Grant date	May 31, 2016

Abstract

A method of dynamically ascertaining the alignment of a vehicular camera. The method involves acquiring a sequence of images provided by the camera whilst the vehicle is in motion. For each range of steering angles, the method (i) selects a plurality of feature points in the images, (ii) tracks a motion trajectory for each selected feature point, and (iii) determines a vanishing point in the image plane based on the tracked motion trajectories. The method determines a vanishing line in the image plane based on a locus of these vanishing points and determines the alignment of the camera based on the position of a central vanishing point (corresponding to the zero degree angle) and the vanishing line.

First claim

The invention claimed is:

1. A method for dynamically ascertaining the position of a vehicular camera relative to a vehicle to which the camera is attached, the method comprising:
(a) establishing a plurality of vehicular steering angle ranges;
(b) acquiring a set of image frames in a video stream provided by the camera whilst the vehicle is in motion, the image frames defining an image plane;
(c) measuring the steering angle of the vehicle and, for each steering angle range:
(i) selecting a plurality of feature points in the image frames;
(ii) tracking a motion trajectory of each selected feature point in the set of image frames, and
(iii) determining a vanishing point in the image plane for the plurality of tracked motion trajectories;

(d) determining a vanishing line in the image plane provided by the camera based on a locus of said vanishing points;
(e) determining the position or alignment of the camera based on the position of the vanishing line and the position of a central vanishing point, the central vanishing point being the vanishing point that is determined for the steering angle range that encompasses zero degrees;
removing distortion in the motion trajectories caused by use of a wide angle camera lens;
linearly fitting the undistorted motion trajectories to generate straight lines;
wherein determining a vanishing point comprises determining the vanishing point in a given steering range by finding a substantial intersection point for the distortion free, straight line tracked motion trajectories in the given steering range;
wherein the camera is a front facing camera and rotational angles α and β of the camera are determined as a function of d 1 and d 2 , where:
α is the rotation of the camera around an X-axis in a camera coordinate system,
β is the rotation of the camera around a Y-axis in the camera coordinate system,
d 1 is a distance in the image plane from a principal point of the camera to a right angled projection point on the vanishing line, and
d 2 is a distance in the image plane along the vanishing line from the projection point to the central vanishing point; and
wherein a rotational angle γ of the camera is determined by solving for γ as follows:
A cos γ+ B sin γ= C

where,
A−f sin α sin β

B=f cos α

C−(cvpX−axisX) sin α cos β

where γ is the rotation angle around the Z-axis of a vehicle coordinate system, f is the focal length of the camera, cvpX is an image row coordinate of the central vanishing point in the image plane, and axisX is an image column location of the principal point.

2. A method according to claim 1 , wherein α and β are determined from a table correlating unique values of d 1 and d 2 with unique values of α and β, said table created by setting the α and β positions of the front camera to known quantities and recording the resultant d 1 and d 2 values.

3. A method according to claim 1 , wherein rotations Rx, Ry and Rz around X-, Y- and Z-axes, respectively, of the vehicle coordinate system are determined based on the rotations α, β and y around the X-, Y- and Z- axes of the camera coordinate system.

4. A method for dynamically ascertaining the position of a vehicular camera relative to a vehicle to which the camera is attached, the method comprising:
(a) establishing a plurality of vehicular steering angle ranges;
(b) acquiring a set of image frames in a video stream provided by the camera whilst the vehicle is in motion, the image frames defining an image plane;
(c) measuring the steering angle of the vehicle and, for each steering angle range:
(i) selecting a plurality of feature points in the image frames;
(ii) tracking a motion trajectory of each selected feature point in the set of image frames, and
(iii) determining a vanishing point in the image plane for the plurality of tracked motion trajectories;

(d) determining a vanishing line in the image plane provided by the camera based on a locus of said vanishing points; and
(e) determining the position or alignment of the camera based on the position of the vanishing line and the position of a central vanishing point, the central vanishing point being the vanishing point that is determined for the steering angle range that encompasses zero degrees;
removing distortion in the motion trajectories caused by use of a wide angle camera lens;
linearly fitting the undistorted motion trajectories to generate straight lines;
wherein determining a vanishing point comprises determining the vanishing point in a given steering range by finding a substantial intersection point for the distortion free, straight line tracked motion trajectories in the given steering range;
wherein the camera is a side facing camera and rotational angles α and β of the side facing camera are determined as a function of d 1 and d 2 , where
α is a rotation of the side facing camera around an X-axis of a side facing camera coordinate system,
β is a rotation of the side facing camera around a Y-axis of the side camera coordinate system,
d 1 is a distance in the image plane from a principal point of the camera to a right angled projection point on the vanishing line, and
d 2 is a distance in the image plane along the vanishing line from the projection point to the central vanishing point;
wherein a rotational angle v is determined by measuring an angle projected by the central vanishing point on a horizontal axis of the image plane; and
wherein rotations Rx, Ry and Rz, for the side camera around the X-, Y- and Z-axes, respectively, of the vehicle coordinate system are determined from the rotations α, β and γ around the X-, Y- and Z-axes of the camera coordinate system.

5. A method according to claim 4 , wherein α and β are determined from a table correlating unique values of d 1 and d 2 with unique values of Rx and Rz, said table created by setting the α and β positions of the side camera to known quantities and recording the resultant d 1 and d 2 values.

6. A system for dynamically ascertaining the position of a vehicular camera relative to a vehicle to which the camera is attached, the system comprising a controller which:
(a) establishes a plurality of vehicular steering angle ranges;
(b) acquires a set of image frames in a video stream provided by the camera whilst the vehicle is in motion, wherein the image frames define an image plane;
(c) measures the steering angle of the vehicle and, for each steering angle range:
(i) selects a plurality of feature points in the image frames;
(ii) tracks a motion trajectory of each selected feature point in the set of image frames, and
(iii) determines a vanishing point in the image plane for the plurality of tracked motion trajectories;

(d) determines a vanishing line in the image plane provided by the camera based on a locus of said vanishing points; and
(e) determines the position or alignment of the camera based on the position of the vanishing line and the position of a central vanishing point, wherein the central vanishing point is the vanishing point that is determined for the steering angle range that encompasses zero degrees;
wherein the controller removes distortion in the motion trajectories caused by use of a wide angle camera lens, linearly fits the undistorted motion trajectories to generate straight lines, and determines the vanishing point in a given steering range by finding a substantial intersection point for the distortion free, straight line tracked motion trajectories in the given steering range;
wherein the camera is a front facing camera and rotational angles α and β of the camera are determined as a function of d 1 and d 2 , where
α is the rotation of the camera around an X-axis in a camera coordinate system,
β is the rotation of the camera around a Y-axis in the camer

Assignees

• Gupta Nikhil
• Faraji Hilda
• He Daan
• Rathi Ghanshyam
• Magna Electronics Inc

Inventors

• Gupta Nikhil
• Faraji Hilda
• He Daan
• Rathi Ghanshyam

Classifications

• G06T2207/30252

  Vehicle exterior; Vicinity of vehicle · CPC title

• B60R2300/8093

  for obstacle warning · CPC title

• B60R2300/105

  using multiple cameras · CPC title

• H04N17/00 Primary

  Diagnosis, testing or measuring for television systems or their details · CPC title

• B60R2300/402

  Image calibration · CPC title