Machine vision image sensor calibration

Patent metadata

Field	Value
Publication number	US-9582881-B2
Application number	US-201514662026-A
Country	US
Kind code	B2
Filing date	Mar 18, 2015
Priority date	Mar 18, 2015
Publication date	Feb 28, 2017
Grant date	Feb 28, 2017

Abstract

A system, apparatus, method, and computer readable media for calibration of one or more extrinsic image sensor parameters. A system may calibrate a multi-camera surround view system, for example as may be employed in a vehicle, based on image data comprising one or more ground plane landmarks. The system may determine a ground plane projection of the image data, for example through a Radon transformation. A signal associated with at least one of the one or more ground plane landmarks in the ground plane projection(s) may be determined, for example through application of the projection-slice theorem. The landmark signal may be utilized as a response in an automated calibration loop to optimize one or more extrinsic parameter values associated with the sensors.

First claim

What is claimed is:

1. A machine vision calibration system, comprising:
a plurality of image sensors to collect image data associated with multiple fields of view (FOV) encompassing adjacent portions of a ground plane having one or more ground plane calibration landmarks that comprise a pair of parallel lines, wherein:
a first of the image sensors is to collect first image data associated with a first FOV including at least a portion of a first of the parallel lines, but excluding a second of the parallel lines; and
a second of the image sensors is to collect second image data associated with a second FOV including at least a portion of a second of the parallel lines, but excluding the first of the parallel lines; and

an image sensor calibration module coupled to the image sensors and including logic to:
compute a two-dimensional ground plane projection of the collected image data based on a Radon transform of the image data for an angle corresponding to one or more extrinsic parameter values associated with the sensor that collected the image data, wherein to compute the projection, the calibration module includes logic to:
project the first image data in a first direction based on one or more first extrinsic parameter values associated with the first image sensor; and
project the second image data in a second direction based on one or more second extrinsic parameter values associated with the second image sensor;

compute, from the ground plane projection, a first landmark signal associated with the pair of lines;
evaluate the first landmark signal based on a parameterization of a pair of peaks in the signal associated with the pair of parallel lines; and
select one or more first extrinsic parameter values and one or more second extrinsic parameter values based on the evaluation of the first landmark signal.

2. The system of claim 1 , wherein the calibration module includes logic to:
compute the first landmark signal by processing the projected image data into a signal associated with a line perpendicular to a projection direction and passing through the pair of lines.

3. The system of claim 1 , wherein the calibration module includes logic to:
modify one or more of the first or second extrinsic parameter values from a prior value;
compute a modified projection of the image data into the ground plane based on the one or more modified extrinsic parameter values;
compute, from the modified projection, a second landmark signal associated with the parallel lines; and
select between the prior and modified value of the first or second extrinsic parameter values based on a comparison of the first landmark signal relative to the second landmark signal.

4. The system of claim 1 , wherein:
a third of the image sensors is to collect third image data associated with a third FOV including at least a portion of both of the parallel lines;
a fourth of the image sensors is to collect fourth image data associated with a fourth FOV including at least a portion of both of the parallel lines; and
the calibration module includes logic to:
project third image data, received from the third of the image sensors, in a third direction based on one or more extrinsic parameter values associated with the third image sensor; and
project fourth image data, received from the fourth of the image sensors, in a fourth direction based on one or more extrinsic parameter values associated with the fourth image sensor.

5. The system of claim 1 , wherein the image sensor calibration module comprises a central slice module including logic to:
generate two-dimensional spectral data by transforming the projected image data from spatial domain to frequency domain;
slice the two-dimensional spectral data perpendicular to at least one of the projection directions; and
generate the first landmark signal by inverse transforming the spectral data slice back to spatial domain.

6. The system of claim 5 , wherein the central slice module includes logic to slice the two-dimensional spectral data into a one-dimensional spectral signal including frequencies associated with the one or more ground plane calibration landmarks.

7. The system of claim 5 , wherein the central slice module includes logic to generate the spectral data by performing a Fourier transform, discrete sine transform, or discrete cosine transform.

8. A method of calibrating a machine vision system, the method comprising:
receiving image data associated with multiple fields of view (FOV) encompassing adjacent portions of a ground plane having one or more ground plane calibration landmarks that comprise a pair of parallel lines, wherein:
a first of the image sensors is to collect first image data associated with a first FOV including at least a portion of a first of the parallel lines, but excluding a second of the parallel lines; and
a second of the image sensors is to collect second image data associated with a second FOV including at least a portion of a second of the parallel lines, but excluding the first of the parallel lines;

computing a two-dimensional ground plane projection of the collected image data based on a Radon transform of the image data for an angle corresponding to one or more extrinsic parameter values associated with the sensor that collected the image data, wherein computing the projection further comprises:
projecting the first image data in a first direction based on one or more first extrinsic parameter values associated with the first image sensor; and
projecting the second image data in a second direction based on one or more second extrinsic parameter values associated with the second image sensor;

computing, from the ground plane projection, a first landmark signal associated with the pair of lines;
evaluating the first landmark signal based on a parameterization of a pair of peaks in the signal associated with the pair of parallel lines; and
selecting one or more first extrinsic parameter values and one or more second extrinsic parameter values based on the evaluation of the first landmark signal.

9. The method of claim 8 , wherein:
computing the first landmark signal comprises processing the projected image data into a signal associated with a line perpendicular to the projection direction and passing through the pair of lines.

10. The method of claim 8 , further comprising:
modifying one or more of the first or second extrinsic parameter values from a prior value;
computing a modified projection of the image data into the ground plane based on the one or more modified extrinsic parameter values;
computing, from the modified projection, a second landmark signal associated with the parallel lines; and
selecting between the prior and modified value of the first or second extrinsic parameter values based on a comparison of the first landmark signal relative to the second landmark signal.

11. The method of claim 8 , wherein computing the first landmark signal further comprises:
generating two-dimensional spectral data by transforming the projected image data from spatial domain to frequency domain;
slicing the spectral data perpendicular to at least one of the projection directions; and
generate the first landmark signal by inverse transforming the spectral data slice back to spatial domain.

12. The method of claim 11 , wherein:
computing the first landmark signal further comprises slicing the two-dimensional spectral data into a one-dimensional spectral signal including frequencies associated with the pair of lines.

13. A non-transitory computer readable media, including instructions stored thereon, which when executed by a processing system, cause the

Assignees

• Intel Corp

Inventors

• Natroshvili Koba

Classifications

• G06F17/142

  Fast Fourier transforms, e.g. using a Cooley-Tukey type algorithm · CPC title

• H04N17/002

  for television cameras · CPC title

• G06T7/0018 Primary

  Physics · mapped topic

• G06T3/0018

  Physics · mapped topic

• G06T2207/20056

  Discrete and fast Fourier transform, [DFT, FFT] · CPC title