Systems and methods to improve camera intrinsic parameter calibration

Patent metadata

Field	Value
Publication number	US-10504244-B2
Application number	US-201816102644-A
Country	US
Kind code	B2
Filing date	Aug 13, 2018
Priority date	Sep 28, 2017
Publication date	Dec 10, 2019
Grant date	Dec 10, 2019

Abstract

Described herein are systems and methods that provide easy and effective camera calibration. In one or more embodiments, a set of calibration patterns, such as array of unique markers, are used as a calibration target. The unique calibration markers resolve ambiguity when only partial views of the calibration target are captured. Embodiments disclosed herein also allow for an interactive calibration process that can direct users to specific locations within the camera image that require additional calibration image captures. Also, in one or more embodiments, the calibration process may be checked at one or more stages to help insure that the final camera intrinsic parameters will be sufficiently accuracy. One of the verifications may include addressing overfitting by, for example, using a first subset of the captured images to compute the intrinsic parameters and using a second set of the captured images to verify those intrinsic parameters.

First claim

What is claimed is:

1. A non-transitory computer-readable medium or media comprising one or more sequences of instructions which, when executed by one or more processors, causes steps for performing intrinsic parameter calibration of a camera comprising:
capturing a set of captured images of a calibration target, the calibration target comprising a set of unique calibration markers and is positioned so that the camera captures a view of at least a portion of the calibration target in an image frame of the camera;
dividing the image frame of the camera into regions;
for each captured image of the set of captured images, detecting one or more features of one or more calibration markers of the calibration target that are viewable in the captured image;
counting a number of detected features for each region of the image frame; and
responsive to the regions containing a number of detected features above a detected features threshold:
selecting images from the set of captured images to form a set of selected images;
responsive to the set of selected images having detected features at or above a detected features threshold for each region of the image frame, selecting a subset of detected features to determine an estimate of intrinsic calibration parameters of the camera;
using the subset of detected features to estimate the intrinsic calibration parameters of the camera;
re-projecting feature points of images from a set of verification images to verify the estimated intrinsic calibration parameters of the camera;
computing a re-projecting error for the re-projected feature points for each image from the set of verification images;
responsive to the re-projection error of an image being less than an error threshold, denoting the image as an inlier, otherwise denoting the image as an outlier; and
responsive to the number of inlier images being greater than a first inlier threshold value, using the inliers to compute the intrinsic calibration parameters as final intrinsic calibration parameters for the camera.

2. The non-transitory computer-readable medium or media of claim 1 further comprising one or more sequences of instructions which, when executed by one or more processors, causes steps comprising:
responsive to the number of inlier images not being greater than a first inlier threshold value, returning to the step of selecting images from the set of captured images to form a set of selected images in order to supplement the set of selected images or to select a new set of selected images.

3. The non-transitory computer-readable medium or media of claim 1 wherein the step of responsive to the number of inlier images being greater than a first inlier threshold value, using the inliers to compute the intrinsic calibration parameters as final intrinsic calibration parameters for the camera comprises:
outputting the estimated intrinsic calibration parameters of the camera as the final intrinsic calibration parameters for the camera responsive to the number of inlier images being greater than a second inlier threshold value that is higher than the first inlier threshold value.

4. The non-transitory computer-readable medium or media of claim 1 further comprising one or more sequences of instructions which, when executed by one or more processors, causes steps comprising:
responsive to not obtaining final intrinsic calibration parameters for the camera after a number of attempts, deeming the set of captured images insufficient for calibration.

5. The non-transitory computer-readable medium or media of claim 1 further comprising one or more sequences of instructions which, when executed by one or more processors, causes steps comprising:
responsive to one or more of the regions not containing a number of detected features above a detected features threshold, providing a prompt that a particular region or regions requires additional detected features.

6. The non-transitory computer-readable medium or media of claim 5 wherein the prompt comprises a real-time view of what the camera is viewing plus an overlay that highlights the one or more regions that do not contain a number of detected features above the detected features threshold.

7. The non-transitory computer-readable medium or media of claim 5 wherein at least some of the regions are assigned a unique identifier and wherein the prompt identifies a region by its unique identifier.

8. The non-transitory computer-readable medium or media of claim 5 wherein the prompt indicates how many more detected features are required for each of the one or more regions that do not contain a number of detected features above the detected features threshold.

9. The non-transitory computer-readable medium or media of claim 1 wherein the regions are not uniformly sized, and wherein the image frame is divided with more regions in areas of the image frame that more accuracy for the camera calibration is wanted.

10. The non-transitory computer-readable medium or media of claim 1 further comprising one or more sequences of instructions which, when executed by one or more processors, causes steps comprising:
responsive to a gap between a boundary of the image frame and a detected feature closest to the boundary being below a maximum gap limit, a gap between two detected features that are closest to each other being below the maximum gap limit, or both, prompting to capture more feature points in a region or regions comprising the gap or gaps.

11. A processor-implemented method for performing intrinsic parameter calibration of a camera comprising:
capturing a set of captured images of a calibration target, the calibration target comprising a set of unique calibration markers and is positioned so that the camera captures a view of at least a portion of the calibration target in an image frame of the camera;
dividing the image frame of the camera into regions;
for each captured image of the set of captured images, detecting one or more features of one or more calibration markers of the calibration target that are viewable in the captured image;
counting a number of detected features for each region of the image frame; and
responsive to the regions containing a number of detected features above a detected features threshold:
selecting images from the set of captured images to form a set of selected images;
responsive to the set of selected images having detected features at or above a detected features threshold for each region of the image frame, selecting a subset of detected features to determine an estimate of intrinsic calibration parameters of the camera;
using the subset of detected features to estimate the intrinsic calibration parameters of the camera;
re-projecting feature points of images from a set of verification images to verify the estimated intrinsic calibration parameters of the camera;
computing a re-projecting error for the re-projected feature points for each image from the set of verification images;
responsive to the re-projection error of an image being less than an error threshold, denoting the image as an inlier, otherwise denoting the image as an outlier; and
responsive to the number of inlier images being greater than a first inlier threshold value, using the inliers to compute the intrinsic calibration parameters as final intrinsic calibration parameters for the camera.

12. The processor-implemented method of claim 11 further comprising the step of:
responsive to the number of inlier images not being greater than a first inlier threshold value, returning to the step of selecting images from the set of captured images to form a set of selected images in order to supplement the set of selecte

Assignees

• Baidu Usa Llc

Inventors

• Bao Yingze
• Chen Mingyu

Classifications

• G06T7/11

  Region-based segmentation · CPC title

• G06T7/85 Primary

  Stereo camera calibration · CPC title

• G06T7/80 Primary

  Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration · CPC title

• G06T2207/30208

  Marker matrix · CPC title

• G06T2207/30204

  Marker · CPC title