Automatic orientation estimation of camera system relative to vehicle

Patent metadata

Field	Value
Publication number	US-2016284087-A1
Application number	US-201514667031-A
Country	US
Kind code	A1
Filing date	Mar 24, 2015
Priority date	Mar 24, 2015
Publication date	Sep 29, 2016
Grant date	—

Abstract

Methods, apparatuses, and systems may provide for using the motion of a vehicle to estimate the orientation of a camera system of a vehicle relative to the vehicle. Image data may be received from a plurality of cameras positioned on the vehicle, and a first constraint set may be determined for the plurality of cameras based on a plurality of feature points in a ground plane proximate to the vehicle. A second constraint set may be determined based on one or more borders of the vehicle. One or more of the cameras may be automatically calibrated based on the first constraint set and the second constraint set.

First claim

We claim:

1 . A camera-based driver-assistance system comprising:
a plurality of cameras each comprising an electro-optical component to capture images, wherein the cameras are located in different areas on a vehicle hosting the camera-based driver-assistance system; a feature point analyzer to determine, for the plurality of cameras, a first constraint set based on a plurality of feature points in a ground plane proximate to the vehicle; a border analyzer to determine, for the plurality of cameras, a second constraint set based on one or more borders of the vehicle; a calibrator to conduct an automatic calibration of one or more of the plurality of cameras based on the first constraint set and the second constraint set; and a display to visually output a result of the automatic calibration.

2 . The system of claim 1 , wherein the border analyzer includes:
an image data interface to receive image data associated with the plurality of cameras; a frame analyzer to determine an optical flow based on a plurality of frames in the image data; a border detector to identify the one or more borders based on the optical flow.

3 . The system of claim 1 , further including a sensor interface to receive additional sensor data from one or more sensors on the vehicle, wherein the border analyzer is to determine the second constraint set based on the additional sensor data and the additional sensor data is to include one or more of velocity data, steering angle data, wheel information, and inertia data.

4 . The system of claim 1 , wherein the feature point analyzer is to include:
an orientation estimator to estimate an orientation of each feature point relative to a coordinate system of the plurality of cameras; and a location estimator to estimate a location of each feature point relative to the coordinate system of the plurality of cameras.

5 . The system of claim 1 , further including a feature point extractor to receive image data associated with the plurality of cameras, and detect the plurality of feature points based on the image data.

6 . The system of claim 1 , wherein the calibrator is to estimate an orientation of one or more of the plurality of cameras to conduct the automatic calibration.

7 . An apparatus comprising:
a feature point analyzer to determine, for the plurality of cameras, a first constraint set based on a plurality of feature points in a ground plane proximate to the vehicle; a border analyzer to determine, for the plurality of cameras, a second constraint set based on one or more borders of the vehicle; and a calibrator to conduct an automatic calibration one or more of the plurality of cameras based on the first constraint set and the second constraint set.

8 . The apparatus of claim 7 , wherein the border analyzer includes:
an image data interface to receive image data associated with the plurality of cameras; a frame analyzer to determine an optical flow based on a plurality of frames in the image data; and a border detector to identify the one or more borders based on the optical flow.

9 . The apparatus of claim 7 , further including a sensor interface to receive additional sensor data from one or more sensors on the vehicle, wherein the border analyzer is to determine the second constraint set based on the additional sensor data and the additional sensor data is to include one or more of velocity data, steering angle data, wheel information, and inertia data.

10 . The apparatus of claim 7 , wherein the frame analyzer includes:
an orientation estimator to estimate an orientation of each feature point relative to a coordinate system of the plurality of cameras; and a location estimator to estimate a location of each feature point relative to the coordinate system of the plurality of cameras.

11 . The apparatus of claim 7 , further including a feature point extractor to receive image data associated with the plurality of cameras, and detect the plurality of feature points based on the received image data.

12 . The apparatus of claim 7 , wherein the calibrator is to estimate an orientation of one or more of the plurality of cameras to conduct the automatic calibration.

13 . A method comprising:
determining, for a plurality of cameras, a first constraint set based on a plurality of feature points in a ground plane proximate to a vehicle; determining, for the plurality of cameras, a second constraint set based on one or more borders of the vehicle; and conducting an automatic calibration of one or more of the plurality of cameras based on the first constraint set and the second constraint set.

14 . The method of claim 13 , wherein determining the second constraint set includes:
receiving image data associated with the plurality of cameras; determining an optical flow based on a plurality of frames in the image data; and identifying the one or more borders based on the optical flow.

15 . The method of claim 13 , further including receiving additional sensor data from one or more sensors on the vehicle, wherein the second constraint set is determined based on the additional sensor data and the additional sensor data includes one or more of velocity data, steering angle data, wheel information, and inertia data.

16 . The method of claim 13 , wherein determining the first constraint set includes:
estimating an orientation of each feature point relative to a coordinate system of the plurality of cameras; and estimating a location of each feature point relative to the coordinate system of the plurality of cameras.

17 . The method of claim 13 , further including:
receiving image data associated with the plurality of cameras; and detecting the plurality of feature points based on the image data.

18 . The method of claim 13 , wherein conducting the automatic calibration includes estimating an orientation of one or more of the plurality of cameras.

19 . At least one non-transitory computer readable storage medium, comprising a set of instructions which, when executed by a computing device causes the computing device to:
determine, for a plurality of cameras, a first constraint set based on a plurality of feature points in a ground plane proximate to a vehicle; determine, for the plurality of cameras, a second constraint set based on one or more borders of the vehicle; and conduct an automatic calibration of one or more of the plurality of cameras based on the first constraint set and the second constraint set.

20 . The at least one computer readable storage medium of claim 19 , wherein the instructions, when executed, cause the computing device to:
receive image data associated with the plurality of cameras; determine an optical flow based on a plurality of frames in the image data; identify the one or more borders based on the optical flow to determine the second constraint set.

21 . The at least one computer readable storage medium of claim 19 , wherein the instructions, when executed, cause the computing device to receive additional sensor data from one or more sensors on the vehicle, wherein the second constraint set is determined based on the additional sensor data and the additional sensor data includes one or more of velocity data, steering angle data, wheel information, inertia data.

22 . The at least one computer readable storage medium of claim 19 , wherein the instructions, when executed, cause the computing device to:
estimate an orientation of each feature point relativ

Assignees

• Intel Corp

Inventors

• Natroshvili Koba
• Scholl Kay-Ulrich

Classifications

• B60R2300/301

  combining image information with other obstacle sensor information, e.g. using RADAR/LIDAR/SONAR sensors for estimating risk of collision · CPC title

• G06T2207/30252

  Vehicle exterior; Vicinity of vehicle · CPC title

• G06T7/2066

  Physics · mapped topic

• B60R1/00

  Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles · CPC title

• B60R2300/105

  using multiple cameras · CPC title