System and method for dynamic stereoscopic calibration

Patent metadata

Field	Value
Publication number	US-11315276-B2
Application number	US-202016975726-A
Country	US
Kind code	B2
Filing date	Mar 6, 2020
Priority date	Mar 9, 2019
Publication date	Apr 26, 2022
Grant date	Apr 26, 2022

Abstract

Methods for stereo calibration of a dual-camera that includes a first camera and a second camera and system for performing such methods. In some embodiments, a method comprises obtaining optimized extrinsic and intrinsic parameters using initial intrinsic parameters and, optionally, initial extrinsic parameters of the cameras, estimating an infinity offset e using the optimized extrinsic and extrinsic parameters, and estimating a scaling factor s using the optimized extrinsic and extrinsic parameters and infinity offset parameter e, wherein the optimized extrinsic and extrinsic parameters, infinity offset e and scaling factor s are used together to provide stereo calibration that leads to improved depth estimation.

First claim

What is claimed is:

1. A method for stereo calibration of a dual-camera that includes a first camera and a second camera, the method comprising:
a) obtaining optimized extrinsic and intrinsic parameters using initial intrinsic parameters and, optionally, initial extrinsic parameters, by obtaining with the dual-camera a set of left and right images, matching corresponding points in the left and right images to obtain matched feature points, calculating extrinsic parameters, and refining the initial intrinsic parameters based on the matched feature points;
b) estimating an infinity offset e using the optimized extrinsic and extrinsic parameters;
c) estimating a scaling factor s using the optimized extrinsic and extrinsic parameters and infinity offset e; and
d) using the optimized extrinsic and extrinsic parameters, infinity offset e and scaling factor s to provide stereo calibration that leads to improved depth estimation.

2. The method of claim 1 , wherein the stereo calibration includes dynamic stereo calibration.

3. The method of claim 2 , wherein the dynamic stereo calibration is performed in a moving vehicle that includes the dual-camera.

4. The method of claim 1 , wherein the initial intrinsic parameters include nominal values of intrinsic parameters of the first and second cameras.

5. The method of claim 1 , wherein the initial intrinsic parameters include factory calibrated initial intrinsic parameters of the first and second cameras.

6. The method of claim 1 , wherein the initial intrinsic parameters include initial intrinsic parameters of the first and second cameras estimated independently from bundle adjustment.

7. The method of claim 1 , wherein the initial intrinsic parameters include initial intrinsic parameters of the first and second cameras estimated independently from structure from motion.

8. The method of claim 1 , further comprising iterating between the calculating extrinsic parameters and the matching corresponding points in the left and right images until sufficient matched points are gathered.

9. The method of claim 1 , wherein the refining the initial intrinsic parameters includes calculating a difference in image location for each matched feature point along a non-disparity axis and wherein the obtaining optimized intrinsic parameters include fulfilling a stop condition.

10. The method of claim 8 , wherein the refining the initial intrinsic parameters includes calculating a difference in image location for each matched feature point along a non-disparity axis and wherein the obtaining optimized intrinsic parameters include fulfilling a stop condition.

11. A method for stereo calibration of a dual-camera that includes a first camera and a second camera, comprising:
obtaining optimized extrinsic and intrinsic parameters using initial intrinsic parameters and, optionally, initial extrinsic parameters by obtaining dynamically at least two sets of stereo images, wherein each stereo image set includes a left image and a right image, matching corresponding points in left and right images of each set, matching corresponding points in, respectively, left images and right images of at least two sets, generating a disparity map by calculating disparity values in the two time frames for all features matched in same sets and between sets, labeling pixels with constant disparity over different time steps as respective points at infinity and estimating infinity offset e, from a respective disparity of the points at infinity,
estimating an infinity offset e using the optimized extrinsic and extrinsic parameters;
estimating a scaling factor s using the optimized extrinsic and extrinsic parameters and infinity offset e; and
using the optimized extrinsic and extrinsic parameters, infinity offset e and scaling factor s to provide stereo calibration that leads to improved depth estimation.

12. A method for stereo calibration of a dual-camera that includes a first camera and a second camera, the method comprising:
obtaining optimized extrinsic and intrinsic parameters using initial intrinsic parameters and, optionally, initial extrinsic parameters,
estimating an infinity offset e using the optimized extrinsic and extrinsic parameters;
estimating a scaling factor s using the optimized extrinsic and extrinsic parameters and infinity offset e by obtaining dynamically at least one set of stereo images, detecting in the set at least one object of known dimensions (OKD) to obtain a detected OKD marked X OKD , matching corresponding points in X OKD , calculating a size of X OKD , calculating a disparity of X OKD , calculating a distance of X OKD from the dual-camera and estimating scaling factor s using the size, the disparity and the distance; and
using the optimized extrinsic and extrinsic parameters, infinity offset e and scaling factor s to provide stereo calibration that leads to improved depth estimation.

13. The method of claim 2 , wherein the estimating scaling factor s using the optimized extrinsic and extrinsic parameters and infinity offset e includes obtaining dynamically at least one set of stereo images, detecting in the set at least one object of known dimensions (OKD) to obtain a detected OKD marked X OKD , matching corresponding points in X OKD , calculating a size of X OKD , calculating a disparity of X OKD , calculating a distance of X OKD from the dual-camera and estimating scaling factor s using the size, the disparity and the distance.

14. The method of claim 3 , wherein the estimating scaling factor s using the optimized extrinsic and extrinsic parameters and infinity offset e includes obtaining dynamically at least one set of stereo images, detecting in the set at least one object of known dimensions (OKD) to obtain a detected OKD marked X OKD , matching corresponding points in X OKD , calculating a size of X OKD , calculating a disparity of X OKD , calculating a distance of X OKD from the dual-camera and estimating scaling factor s using the size, the disparity and the distance.

15. The method of claim 11 , wherein the estimating scaling factor s using the optimized extrinsic and extrinsic parameters and infinity offset e includes obtaining dynamically at least one set of stereo images, detecting in the set at least one object of known dimensions (OKD) to obtain a detected OKD marked X OKD , matching corresponding points in X OKD , calculating a size of X OKD , calculating a disparity of X OKD , calculating a distance of X OKD from the dual-camera and estimating scaling factor s using the size, the disparity and the distance.

16. A method for stereo calibration of a dual-camera that includes a first camera and a second camera, the method comprising:
obtaining optimized extrinsic and intrinsic parameters using initial intrinsic parameters and, optionally, initial extrinsic parameters,
estimating an infinity offset e using the optimized extrinsic and extrinsic parameters;
estimating a scaling factor s using the optimized extrinsic and extrinsic parameters and infinity offset e by estimating scaling factor s using the optimized extrinsic and extrinsic parameters and infinity offset e includes obtaining dynamically at least two sets of stereo images, detecting in the sets at least one stationary object X OS , matching corresponding points in X OS to obtain a disparity, obtaining a distance driven between the obtaining of the at least two sets, and estimating scaling factor s using the disparity and the distance; and
using the optimized extrinsic and extrinsic parameters, infinity offset e and scaling factor s to provide stereo calibration that leads to improved depth esti

Assignees

• Corephotonics Ltd

Inventors

• Geva Nadav
• Ilan Paz
• Gigushinski Oded
• Goldenberg Ephraim
• Shabtay Gal

Classifications

• H04N13/246 Primary

  Calibration of cameras · CPC title

• H04N13/239 Primary

  using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance (H04N13/243 takes precedence) · CPC title

• G06T7/85 Primary

  Stereo camera calibration · CPC title

• G06T7/593 Primary

  from stereo images · CPC title

• G06T2207/10012

  Stereo images · CPC title