Online calibration of a motor vehicle camera system

Patent metadata

Field	Value
Publication number	US-10163226-B2
Application number	US-201515521909-A
Country	US
Kind code	B2
Filing date	Nov 26, 2015
Priority date	Dec 4, 2014
Publication date	Dec 25, 2018
Grant date	Dec 25, 2018

Abstract

The invention relates to a method for calibrating a camera system (7) of a motor vehicle (1). At least one camera (8-11) respectively sequentially generates camera images (21-28) from an environment (2) of the motor vehicle (1). A computing device (12) generates a virtual view (14) of the environment (2) from a virtual perspective from the camera images (21-28) by means of a projection (P). In traveling, the computing device (12) passes at least once a calibration cycle for each camera (8-11). Based on current projection parameters (30) of the projection (P), therein, camera images (23, 24) of the camera (9) are transformed. Motion vectors are determined from it. For at least one geometric characteristic of the motion vectors, a reference value is set. From a difference between the at least one geometric characteristic of the motion vectors and the respective corresponding reference value, an error value is determined. Depending on the error value, new projection parameters are determined (30).

First claim

The invention claimed is:

1. A method for calibrating a camera system of a motor vehicle, wherein at least one camera of the camera system sequentially generates camera images respectively from an environment of the motor vehicle and a processor of the camera system respectively generates a virtual plan view of the environment from a predetermined virtual perspective, from the camera images by means of a projection, the method comprising:
a) recognizing, by the processor, a straight vehicle motion of the motor vehicle along a predetermined direction of motion;
b) passing, at least once, with recognized travel along the predetermined direction of motion and by the processor, a calibration cycle for each camera, wherein for the calibration cycle, the processor performs the following steps:
transforming camera images of the camera based on current projection parameters of the projection;
determining motion vectors in the virtual plan view based on the transformed camera images;
for at least one geometric characteristic of the motion vectors selected from a group consisting of a length and a direction, setting a respective reference value depending on a direction of motion;
determining an error value from a difference between the at least one geometric characteristic of the motion vectors and the respective corresponding reference value; and
determining new projection parameters from the current projection parameters depending on the error value,

wherein in each calibration cycle, the error value is iteratively reduced by determining intermediate projection parameters depending on the error value and newly orienting only the motion vectors in the virtual plan view based on the intermediate projection parameters by newly transforming by means of the projection, until the error value satisfies a predetermined optimization criterion; and
wherein for a y component of each motion vector oriented parallel to a transverse axis of the vehicle, the y reference value zero is set, and for an x component oriented parallel to a longitudinal axis of the vehicle, an average value of all of the x components of the motion vectors is set as an x reference value, and a sum of squared differences is formed as the error value.

2. The method according to claim 1 , wherein at least one spatial orientation of the at least one camera selected from a group consisting of: a pitch angle, yaw angle, roll angle, and an installation height of the camera on the motor vehicle is described by the new projection parameters.

3. The method according to claim 1 , wherein the at least one camera captures a travel ground as the environment and the motion vectors are determined based on a texture of the travel ground.

4. The method according to claim 1 , wherein multiple motion vectors are determined by dividing each transformed camera image into a matrix of image blocks adjacent to each other or overlapping and determining one motion vector for each image block.

5. The method according to claim 1 , wherein a straight-ahead travel is recognized as travel along the direction of motion, and as the geometric characteristic, it is checked if the motion vectors are disposed parallel to each other and/or parallel to an image edge of the virtual plan view and/or identically long.

6. The method according to claim 1 , wherein for selecting two camera images from an image sequence of the respective camera, at least one camera image is skipped between a first one of the two camera images and a second one of the two camera images.

7. The method according to claim 1 , wherein at least one of the motion vectors, in which a contribution to the error value is greater than a predetermined maximum value and/or which violates a similarity criterion with respect to the remaining motion vectors, is deleted from the motion vectors as an outlier.

8. The method according to claim 1 , further comprising:
performing multiple calibration cycles;
for each calibration cycle, determining new projection parameters, respectively;
combining corresponding new projection parameters from different calibration cycles to a respective averaged projection parameter; and
using the averaged projection parameters as a basis for current projection parameters of the projection in a respectively next calibration cycle.

9. The method according to claim 1 , further comprising:
performing multiple calibration cycles; and
from the error value of one of the calibration cycles and a respective error value of at least one prior calibration cycle, determining an average error value, wherein the new projection parameters are only determined if the average error value is greater than a predetermined threshold value.

10. The method according to claim 1 , further comprising:
generating, by multiple cameras, camera images; and
matching the motion vectors formed to each of the cameras to each other by adjusting an installation height of at least one of the cameras as a projection parameter.

11. The method according to claim 1 , further comprising:
generating, by a first and a second camera, camera images; determining an overlap error value in the transformed camera images of the two cameras; and determining the new projection parameters for the first and/or the second camera depending on the overlap error value, wherein:
a) at least one inter-camera motion vector is determined depending on one of the transformed camera images of the first camera image and one of the transformed camera images of the second camera and the overlap error value is formed depending on at least one geometric characteristic of the at least one inter-camera motion vector, or
b) in an overlapping region an image block is determined in one of the transformed camera images of the first camera and a corresponding image block is determined in one of the transformed camera images of the second camera and the overlap error value is determined depending on a distance of the two image blocks, or
c) in an overlapping region a correlation value is generated from one of the transformed camera images of the first camera and one of the transformed camera images of the second camera in the overlapping region and a the comparative value is used as the overlap error value.

12. A camera system for a motor vehicle, comprising:
at least one camera for respectively generating an image sequence from camera images of an environment of the motor vehicle; and
a processor configured to respectively generate a transformed camera image with a virtual plan view of the environment from a predetermined virtual perspective, from the camera images by means of a projection; and
a display screen that displays the transformed camera images as a composed all-around view around the motor vehicle,
wherein the processor performs a method according to claim 1 .

13. A motor vehicle with a camera system according to claim 12 , wherein:
the camera system has multiple cameras;
a capturing range of a first one of the multiple cameras is oriented to a first lateral region beside the motor vehicle;
a capturing range of a second one of the multiple cameras is oriented to a second lateral region opposing the first lateral region beside the motor vehicle;
a capturing range of a third one of the multiple cameras is oriented to a front region in front of the motor vehicle; and
a capturing range of a fourth one of the multiple cameras is oriented to a rear region behind the motor vehicle,
wherein the camera system is configured to generate a respective transformed camera image from each one of the camera images of each camera and an all-around view from each one of the camera images, and to display th

Assignees

• Connaught Electronics Ltd

Inventors

• Ermilios Pantelis

Classifications

• G06T2207/30252

  Vehicle exterior; Vicinity of vehicle · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T7/80 Primary

  Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration · CPC title