Method and system for dynamically calibrating vehicular cameras

Patent metadata

Field	Value
Publication number	US-9834153-B2
Application number	US-201214113412-A
Country	US
Kind code	B2
Filing date	Apr 25, 2012
Priority date	Apr 25, 2011
Publication date	Dec 5, 2017
Grant date	Dec 5, 2017

Abstract

A method of dynamically calibrating a given camera relative to a reference camera of a vehicle includes identifying an overlapping region in an image frame provided by the given camera and an image frame provided by the reference camera and selecting at least a portion of an object in the overlapped region of the reference image frame. Expected pixel positions of the selected object portion in the given image frame is determined based on the location of the selected object portion in the reference image frame, and pixel positions of the selected object portion are located as detected in the given image frame. An alignment of the given camera is determined based on a comparison of the pixel positions of the selected object portion in the given image frame to the expected pixel positions of the selected object portion in the given image frame.

First claim

The invention claimed is:

1. A method of dynamically calibrating a given vehicular camera relative to a reference camera attached to the vehicle, the given camera and the reference camera comprising pixelated imaging arrays, said method comprising:
providing a plurality of cameras at the vehicle so that the cameras have respective exterior fields of view, the cameras capturing image data;
selecting from the plurality of cameras a reference camera and a given camera, wherein the selected reference camera and the selected given camera have overlapping fields of view;
capturing an actual given image frame of image data by the given camera;
capturing an actual reference image frame of image data by the reference camera;
identifying an overlapping region in the actual given image frame of image data captured by the given camera and the actual reference image frame of image data captured by the reference camera;
selecting at least one object or at least a portion of an object present in the overlapping region of the actual reference image frame captured by the reference camera;
determining expected pixel positions of the selected object or object portion for a predicted given image frame as would be captured by the given camera based on the location of the selected object or object portion in the actual reference image frame captured by the reference camera;
locating, via processing of the actual given image frame captured by the given camera, actual pixel positions of the selected object or object portion as detected in the actual given image frame captured by the given camera; and
determining alignment of the given camera based at least in part on a comparison of the actual pixel positions of the selected object or object portion in the actual given image frame captured by the given camera to the expected pixel positions of the selected object or object portion for the predicted given image frame.

2. The method of claim 1 , wherein the reference camera comprises a forward viewing camera of the vehicle and the given camera comprises a camera disposed at a side of the vehicle.

3. The method of claim 2 , wherein said method first determines whether the forward viewing camera is calibrated.

4. The method of claim 3 , wherein said method determines the calibration of the forward viewing camera by processing the image frame captured by the forward viewing camera to determine a location of a vanishing point in the image frame and comparing the determined vanishing point to a predetermined location in the image frame.

5. The method of claim 1 , including removing distortion in the location of the pixel positions in the actual given image frame for the selected object or object portion caused by use of a wide angle camera lens in the given camera.

6. The method of claim 5 , including removing distortion in locations of pixel positions in the actual reference image frame for the selected object or object portion caused by use of a wide angle camera lens in the reference camera.

7. The method of claim 5 , wherein the reference camera comprises a forward viewing camera and the given camera comprises a side camera of the vehicle.

8. The method of claim 1 , wherein the reference camera comprises a forward viewing camera and the given camera comprises a side camera of the vehicle, and wherein the given camera is disposed at a side mirror of the vehicle.

9. The method of claim 1 , wherein the reference camera comprises a rearward viewing camera and the given camera comprises a side camera of the vehicle, and wherein the given camera is disposed at a side mirror of the vehicle.

10. The method of claim 1 , wherein the given camera comprises a side camera of the vehicle.

11. The method of claim 1 , wherein the reference camera and the given camera are part of a surround view vision system of the equipped vehicle.

12. The method of claim 11 , wherein the reference camera comprises one of a forward viewing camera and a rearward viewing camera, and wherein the given camera comprises one of a driver side camera and a passenger side camera.

13. The method of claim 12 , wherein the system comprises multiple given cameras including a driver side camera and a passenger side camera.

14. A method of dynamically calibrating a given vehicular camera relative to a reference camera attached to the vehicle, the given camera and the reference camera comprising pixelated imaging arrays, said method comprising:
providing a plurality of cameras at the vehicle so that the cameras have respective exterior fields of view, the cameras capturing image data;
selecting from the plurality of cameras a reference camera and a given camera, wherein the selected reference camera and the selected given camera have overlapping fields of view;
capturing an actual given image frame of image data by the given camera;
capturing an actual reference image frame of image data by the reference camera;
identifying an overlapping region in the actual given image frame of image data captured by the given camera and the reference image frame of image data captured by the reference camera;
wherein the reference camera and the given camera are part of a surround view vision system of the vehicle;
wherein the given camera comprises one of a driver side camera at a driver side mirror of the vehicle and a passenger side camera at a passenger side mirror of the vehicle;
wherein the reference camera comprises one of a forward viewing camera at a front portion of the vehicle and a rearward viewing camera at a rear portion of the vehicle;
selecting at least one object or at least a portion of an object present in the overlapping region of the actual reference image frame captured by the reference camera;
determining expected pixel positions of the selected object or object portion for a predicted given image frame as would be captured by the given camera based on the location of the selected object or object portion in the actual reference image frame captured by the reference camera;
locating, via processing of the actual given image frame captured by the given camera, actual pixel positions of the selected object or object portion as detected in the actual given image frame captured by the given camera; and
determining alignment of the given camera based at least in part on a comparison of the actual pixel positions of the selected object or object portion in the actual given image frame captured by the given camera to the expected pixel positions of the selected object or object portion for the predicted given image frame.

15. The method of claim 14 , including removing distortion in the location of the pixel positions in the actual given image frame for the selected object or object portion caused by use of a wide angle camera lens in the given camera.

16. The method of claim 15 , including removing distortion in locations of pixel positions in the actual reference image frame for the selected object or object portion caused by use of a wide angle camera lens in the reference camera.

17. A method of dynamically calibrating a given vehicular camera relative to a reference camera attached to the vehicle, the given camera and the reference camera comprising pixelated imaging arrays, said method comprising:
providing a plurality of cameras at the vehicle so that the cameras have respective exterior fields of view, the cameras capturing image data;
selecting from the plurality of cameras a reference camera and a given camera, wherein the selected reference camera and the selected given camera have overlapping fields of view;
capturing

Assignees

• Gupta Nikhil
• Faraji Hilda
• He Daan
• Rathi Ghanshyam
• Magna Electronics Inc

Inventors

• Gupta Nikhil
• Faraji Hilda
• He Daan
• Rathi Ghanshyam

Classifications

• G06T2207/30252

  Vehicle exterior; Vicinity of vehicle · CPC title

• G06T3/4038

  Image mosaicing, e.g. composing plane images from plane sub-images · CPC title

• G03B43/00 Primary

  Testing correct operation of photographic apparatus or parts thereof · CPC title

• G06T7/80

  Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration · CPC title

• G06T7/85

  Stereo camera calibration · CPC title