Camera calibration device, camera calibration method, and camera calibration program

Patent metadata

Field	Value
Publication number	US-9875542-B2
Application number	US-201214373726-A
Country	US
Kind code	B2
Filing date	Nov 29, 2012
Priority date	Jan 23, 2012
Publication date	Jan 23, 2018
Grant date	Jan 23, 2018

Abstract

There is provided a camera calibration device capable of highly precisely and easily estimating camera parameters without installing a piece of special calibration equipment or measuring 3D coordinates of reference points employed for calibration. A normal vector acquisition unit acquires normal vectors perpendicular to a reference horizontal plane from an image of a camera to be calibrated. A rotation matrix estimation unit projects the acquired normal vectors on a projection virtual plane perpendicular to the reference horizontal plane and evaluates that the projected normal vectors are perpendicular to the reference horizontal plane thereby to estimate a rotation matrix employed for calibrating the camera.

First claim

The invention claimed is:

1. A camera calibration device comprising:
a normal vector acquisition unit which acquires normal vectors perpendicular to a reference horizontal plane as a line segment on an image from the image of a camera to be calibrated; and
a rotation matrix estimation unit which projects the acquired normal vectors on a projection virtual plane perpendicular to the reference horizontal plane and evaluates that the projected normal vectors are perpendicular to the reference horizontal plane thereby to estimate a rotation matrix employed for calibrating the camera;
a translation matrix estimation unit which estimates a translation matrix employed for calibrating a camera,
wherein the normal vector acquisition unit acquires height information on the heights of the end points of at least one normal vector from among the acquired normal vectors, and
the translation matrix estimation unit projects one end point of a normal vector the height information of which is acquired in the real space by use of a rotation matrix estimated by the rotation matrix estimation unit thereby to calculate a 3D coordinate in the real space, calculates a 3D coordinate of the other end point paired with the one end point by use of the calculated 3D coordinate and the height information, re-projects the calculated 3D coordinate of the other end point on the image by use of the rotation matrix thereby to calculate a coordinate of the other end point on the image, and estimates the translation matrix such that an error between the coordinate of the other end point re-projected on the image and the coordinate of the other end point directly acquired from the image is minimum.

2. A camera calibration device comprising:
a normal vector acquisition unit which acquires normal vectors perpendicular to a reference horizontal plane as a line segment on an image from the image of a camera to be calibrated;
a rotation matrix estimation unit which projects the acquired normal vectors on a projection virtual plane perpendicular to the reference horizontal plane and evaluates that the projected normal vectors are perpendicular to the reference horizontal plane thereby to estimate a rotation matrix employed for calibrating the camera;
a translation matrix estimation unit which estimates a translation matrix employed for calibrating a camera,
wherein the normal vector acquisition unit acquires height information on the heights of the end points of at least one normal vector among the acquired normal vectors, and
the translation matrix estimation unit projects both of the end points of a normal vector the height information of which is acquired in the real space by use of a rotation matrix estimated by the rotation matrix estimation unit thereby to calculate 3D coordinates in the real space, calculates 2D coordinates in which the height information is excluded from the calculated 3D coordinates of both of the end points, respectively, and estimates the translation matrix such that an error between the calculated 2D coordinates of both of the end points is minimum.

3. The camera calibration device according to claim 1 , comprising:
an image supply unit which supplies, to the normal vector acquisition unit, an animation shooting therein a motion state in which a moving medium expressed in a normal vector is moving in a shooting space where a measurement medium employed for measuring height information is arranged, and a passage state in which the moving medium passes at a position where height information can be acquired by the measurement medium,
wherein the normal vector acquisition unit traces the moving medium thereby to acquire a plurality of normal vectors with known height information from an animation supplied from the image supply unit.

4. The camera calibration device according to claim 1 ,
wherein the rotation matrix estimation unit calculates a tilt of a normal vector projected on a projection virtual plane defined as a plane parallel to the Z axis direction of a world coordinate system relative to the Z axis of the world coordinate system thereby to evaluate that the normal vector is perpendicular to a reference horizontal plane, and estimates a rotation matrix such that an error indicated by the tilt is minimum.

5. The camera calibration device according to claim 1 ,
wherein the rotation matrix estimation unit calculates an error in the X axis direction between both of the end points of a normal vector projected on a projection virtual plane defined as a plane parallel to the XZ plane of a world coordinate system thereby to evaluate that the normal vector is perpendicular to a reference horizontal plane, and estimates a rotation matrix such that the error is minimum.

6. The camera calibration device according to claim 1 ,
wherein the rotation matrix estimation unit restricts a rotation of a camera for shooting an image by 2D parameters of only roll rotation and pitch rotation thereby to estimate a rotation matrix, and
the translation matrix estimation unit assumes that the camera is on the Z axis of a world coordinate system, and restricts a position of the camera by one 1D parameter indicating a position on the Z axis of the world coordinate system thereby to estimate a translation matrix.

7. The camera calibration device according to claim 1 , comprising:
a normal vector selection unit which compares tilts between all of normal vectors acquired by the normal vector acquisition unit and user-selected correct normal vectors from among all the normal vectors, and selects a normal vector whose tilt relative to any of the selected normal vector is within a threshold as a normal vector effective for calibrating the camera.

8. A camera calibration method comprising:
acquiring normal vectors perpendicular to a reference horizontal plane as a line segment on an image from the image of a camera to be calibrated;
projecting the acquired normal vectors on a projection virtual plane perpendicular to the reference horizontal plane;
evaluating that the projected normal vectors are perpendicular to the reference horizontal plane thereby to estimate a rotation matrix employed for calibrating the camera;
estimating a translation matrix employed for calibrating a camera;
acquiring height information on the heights of the end points of at least one normal vector from among the acquired normal vectors;
projecting one end point of a normal vector the height information of which is acquired in the real space by use of a rotation matrix estimated by the rotation matrix estimation unit thereby to calculate a 3D coordinate in the real space;
calculating a 3D coordinate of the other end point paired with the one end point by use of the calculated 3D coordinate and the height information;
re-projecting the calculated 3D coordinate of the other end point on the image by use of the rotation matrix thereby to calculate a coordinate of the other end point on the image; and
estimating the translation matrix such that an error between the coordinate of the other end point re-projected on the image and the coordinate of the other end point directly acquired from the image is minimum.

9. A non-transitory computer readable information recording medium storing a camera calibration program that, when executed by a processor, performs a method for:
acquiring normal vectors perpendicular to a reference horizontal plane as a line segment on an image from the image of a camera to be calibrated;
projecting the acquired normal vectors on a projection virtual plane perpendicular to the reference horizontal plane and evaluating that the projected normal vectors are perpendicular to the reference horizontal plane thereby to estimate a rotation matrix employed for calibratin

Assignees

• Ikeda Hiroo
• Nec Corp

Inventors

• Ikeda Hiroo

Classifications

• G06T7/80 Primary

  Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration · CPC title

• G06T2207/10004

  Still image; Photographic image · CPC title

• G06T2207/30244

  Camera pose · CPC title

• H04N17/002

  for television cameras · CPC title

• G06T7/0018 Primary

  Physics · mapped topic