Automatic calibration of extrinsic and intrinsic camera parameters for surround-view camera system

Patent metadata

Field	Value
Publication number	US-9386302-B2
Application number	US-201414283518-A
Country	US
Kind code	B2
Filing date	May 21, 2014
Priority date	May 21, 2014
Publication date	Jul 5, 2016
Grant date	Jul 5, 2016

Abstract

A method of calibrating multiple vehicle-based image capture devices of a vehicle. An image is captured by at least one image capture device. A reference object is identified in the captured image. The reference object has known world coordinates. Known features of the vehicle are extracted in the captured image. A relative location and orientation of the vehicle in world coordinates is determined relative to the reference object. Each of the multiple image capture devices is calibrated utilizing intrinsic and extrinsic parameters of the at least one image capture device as a function of the relative location and orientation of the vehicle in world coordinates.

First claim

What is claimed is:

1. A method of calibrating multiple vehicle-based image capture devices of a vehicle, the method comprising the steps of:
capturing an image by at least one image capture device;
identifying, by a processor, a reference object in the captured image, the reference object having known world coordinates, the identification of the reference object is used to correlate coordinates of the image capture device to world coordinates;
determining a location and orientation of the vehicle, by the processor, the location and orientation of the vehicle used to correlate coordinates of the vehicle to world coordinates, wherein determining the location and orientation of the vehicle includes:
capturing a top-down view image of the vehicle on an assembly line by an overhead non-vehicle-based image capture device;
applying feature extraction on the vehicle for detecting a vehicle boundary;
comparing an orientation of the assembly line with the vehicle boundary; and
identifying the relative location and orientation of the vehicle relative to the assembly line;

correlating, by the processor, the vehicle coordinates to the image capture device coordinates as a function of the relationship between the world coordinates and the vehicle coordinates and the relationship between the world coordinates and the image capture device coordinates;
calibrating the at least one image capture device utilizing intrinsic parameters and extrinsic parameters of the at least one image capture device as a function of the relationship of the vehicle coordinates to the image capture device coordinates.

2. The method of claim 1 wherein the identifying a relative location and orientation between the assembly line and the vehicle boundary comprises the following steps:
identifying a longitudinal axis of the assembly line;
identifying a longitudinal axis of the vehicle;
comparing the longitudinal axis of the assembly line with the longitudinal axis of the vehicle for determining an orientation between the assembly line and the vehicle.

3. The method of claim 2 where template matching is applied for determining an orientation of the assembly line.

4. The method of claim 3 wherein template matching is applied for determining the vehicle boundary and vehicle orientation.

5. The method of claim 1 wherein feature extraction of an exterior panel of the vehicle is applied for determining the vehicle boundary.

6. The method of claim 1 wherein applying feature extraction on the vehicle for detecting the vehicle boundary includes a feature comparison with a ground-truth database.

7. The method of claim 1 wherein adjusting the extrinsic parameters includes adjusting a rotation and translation of a respective image capture device based on the determined position and pose relative to the patterned display.

8. The method of claim 1 wherein refining intrinsic parameters of a respective image capture device includes refining a focal length and image center of the image capture device.

9. The method of claim 1 wherein refining intrinsic parameters of a respective image capture device includes refining a distortion from a lens of the image capture device.

10. A method of calibrating multiple vehicle-based image capture devices of a vehicle, the method comprising the steps of:
capturing images exterior of the vehicle by a plurality of vehicle-based image capture devices, wherein each vehicle-based image capture device captures a respective reference object, wherein each respective reference object includes a patterned display, and wherein known features of the vehicle include a respective vehicle boundary;
identifying, by a processor, a reference object in the captured image, the reference object having known world coordinates, the identification of the reference object is used to correlate coordinates of the image capture device to world coordinates;
determining a location and orientation of the vehicle, by the processor, the location and orientation of the vehicle used to correlate coordinates of the vehicle to world coordinates;
correlating, by the processor, the vehicle coordinates to the image capture device coordinates as a function of the relationship between the world coordinates and the vehicle coordinates and the relationship between the world coordinates and the image capture device coordinates;
calibrating the at least one image capture device utilizing intrinsic parameters and extrinsic parameters of the at least one image capture device as a function of the relationship of the vehicle coordinates to the image capture device coordinates.

11. The method of claim 10 wherein each image capture device captures a respective vehicle boundary.

12. The method of claim 11 further comprising the steps of:
traversing the vehicle past the plurality of patterned displays;
each image capture device capturing an image of at least one patterned display exterior of the vehicle when the vehicle reaches a respective checkpoint, the respective checkpoint being at a position that is a known distance from each of the plurality of patterned displays:
identifying, by a processor, common landmarks of the patterned display in each respective captured image;
identifying, by a processor, vehicle boundaries of the vehicle in each respective captured image;
determining a relative location and orientation of the vehicle relative to the common landmarks.

13. The method of claim 12 wherein traversing the vehicle across the patterned display includes traversing the vehicle across a plurality of checker board patterns, wherein each respective vehicle-based image capture device captures at least of a portion of a at least one patterned display.

14. The method of claim 12 wherein identifying common landmarks within the patterned display includes identifying corners of an outer boundary of the patterned display.

15. The method of claim 12 wherein identifying common landmarks within the patterned display includes identifying common edges that distinguish patterns within the patterned display among each of the captured images.

16. The method of claim 12 wherein identifying common landmarks within the patterned display includes identifying common points generated by common intersecting edges within the patterned display.

17. The method of claim 11 wherein a plurality of checkpoints are provided, wherein images of at least one patterned display is captured at each respective checkpoint.

18. The method of claim 17 wherein images of at least one of the patterned displays are captured at each respective checkpoint, wherein calibration is performed utilizing each of the images captured at each respective checkpoint by refining intrinsic parameters and extrinsic parameters of the image capture devices a function of the relationship of the vehicle coordinates to the image capture device coordinates.

Assignees

• Gm Global Tech Operations Inc

Inventors

• Zeng Shuqing
• Zhang Wende
• Wang Jinsong

Classifications

• H04N17/002 Primary

  for television cameras · CPC title

• G06K9/00791

  Physics · mapped topic

• G06T7/0018

  Physics · mapped topic

• G06T2207/30264

  Parking · CPC title

• G06K9/52

  Physics · mapped topic