Stereo auto-calibration from structure-from-motion

Patent metadata

Field	Value
Publication number	US-2015145965-A1
Application number	US-201414554544-A
Country	US
Kind code	A1
Filing date	Nov 26, 2014
Priority date	Nov 26, 2013
Publication date	May 28, 2015
Grant date	—

Abstract

Auto-calibration of stereo cameras installable behind the windshield of a host vehicle and oriented to view the environment through the windshield. Multiple first image points are located of one of the first images captured from the first camera at a first time and matched with first image points of at least one other of the first images captured from the first camera at a second time to produce pairs of corresponding first image points respectively in the first images captured at the different times. World coordinates are computed from the corresponding first image points. Second image points in the second images captured from the second camera are matched to at least a portion of the first image points. The world coordinates as determined from the first camera are used, to solve for camera parameters of the second camera from the matching second image points of the second camera.

First claim

What is claimed is:

1 . A method for auto-calibration of stereo cameras including a first camera and second camera installable behind the windshield of a host vehicle and orientable to view the environment through the windshield, the method comprising:
during motion of the host vehicle, capturing by a processor multiple series of images respectively from the stereo cameras including a first time series of first images from the first camera and a second time series of second images from the second camera; locating a plurality of first image points of at least one of the first images captured from the first camera at a first time; matching at least a portion of the first image points with first image points of at least one other of the first images captured from the first camera at a second time to produce pairs of corresponding first image points respectively in the first images captured at the different times; responsive to the motion of the host vehicle, computing world coordinates from the pairs of corresponding first image points of at least two of the first images captured at the different times from the first camera; matching a plurality of second image points in at least one of the second images captured from the second camera so that the matching second image points correspond to at least a portion of the first image points, using the world coordinates determined from the first camera solving for a plurality of camera parameters of the second camera.

2 . The method of claim 1 , wherein the solution of the camera parameters of the second camera predicts a depth map based on stereo disparity consistent with the world coordinates computed from the first images responsive to the motion of the host vehicle.

3 . The method of claim 2 , further comprising:
re-computing the camera parameters of the second camera with an initial estimate of the center of distortion of the second camera to compute further camera parameters including radial distortion parameters of the second camera.

4 . The method of claim 2 , wherein said solving the camera parameters includes solving distortion parameters intrinsic to the second camera.

5 . The method of claim 2 , further comprising:
performing the auto-calibration while correcting for further distortion caused by the first camera and the second camera having rolling shutters.

6 . The method of claim 5 , further comprising:
using the world coordinates as the constraints which correspond to the times when the picture elements of the first and second image points are actually captured.

7 . A system for auto-calibration of stereo cameras including a first camera and second camera installed behind the windshield of a host vehicle and oriented to view the environment through the windshield, and a processor connectible to the stereo cameras and operable to:
during motion of the host vehicle, capture multiple series of images respectively from the stereo cameras including a first time series of first images from the first camera and a second time series of second images from the second camera; locate a plurality of first image points of at least one of the first images captured from the first camera at a first time; match at least a portion of the first image points with first image points of at least one other of the first images captured from the first camera at a second time to produce pairs of corresponding first image points respectively in the first images captured at the different times; responsive to the motion of the host vehicle, compute world coordinates from the pairs of corresponding first image points of at least two of the first images captured at the different times from the first camera; match a plurality of second image points in at least one of the second images captured from the second camera so that the matching second image points correspond to at least a portion of the first image points using the world coordinates determined from the first camera, to solve for a plurality of camera parameters of the second camera.

8 . The system of claim 7 , wherein the solution of the camera parameters of the second camera predicts a depth map based on stereo disparity consistent with the world coordinates computed from the first images responsive to the motion of the host vehicle.

9 . The system of claim 8 , wherein the processor is further operable to re-compute the camera parameters of the second camera with an initial estimate of the center of distortion of the second camera to compute further camera parameters including radial distortion parameters of the second camera.

10 . The system of claim 8 , wherein the camera parameters of the second camera include intrinsic distortion parameters.

11 . The system of claim 8 , wherein the processor is further operable to perform the auto-calibration while correcting for further distortion caused by the first camera and the second camera having rolling shutters.

12 . The system of claim 11 , wherein the world coordinates are used as constraints which correspond to the times when the picture elements of the first and second image points are actually captured.

Assignees

• Mobileye Vision Technologies Ltd

Inventors

• Livyatan Harel
• Berberian Oded

Classifications

• H04N25/531

  by controlling rolling shutters in CMOS SSIS · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• H04N13/0246 Primary

  Electricity · mapped topic

• G06T7/20

  Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) · CPC title

• H04N13/0271

  Electricity · mapped topic