Image processing device, stereo camera device, vehicle, and image processing method

Patent metadata

Field	Value
Publication number	US-10360462-B2
Application number	US-201615738593-A
Country	US
Kind code	B2
Filing date	Jun 24, 2016
Priority date	Jun 24, 2015
Publication date	Jul 23, 2019
Grant date	Jul 23, 2019

Abstract

Provided is an image processing device including: an input interface, which is configured to acquire a first captured image and a second captured image that are captured by a plurality of imaging units and; and a controller, which is configured to calculate parallax by performing a one-dimensional matching based on pixel values of the first captured image and pixel values of the second captured image, extracts one or more first feature points from a region in the first captured image that includes continuous pixels having a difference in parallax which is within a predetermined range, extract one or more second feature points corresponding respectively to the first feature points by performing a two-dimensional matching with the first feature points, and calibrate the imaging unit based on positions of the first feature points and positions of the second feature points.

First claim

The invention claimed is:

1. An image processing device, comprising:
an input interface configured to receive input of a first captured image and a second captured image captured by a plurality of imaging units of a stereo camera; and
a controller configured to calculate parallax by performing a one-dimensional matching based on pixel values of the first captured image and pixel values of the second captured image, extract one or more first feature points from a region in the first captured image that includes continuous pixels having a difference in parallax between adjacent pixels which is within a predetermined range, extract one or more second feature points corresponding respectively to the one or more first feature points by performing a two-dimensional matching between the one or more first feature points and pixels in the second captured image, and calibrate at least one of the plurality of imaging units based on positions of the one or more first feature points and positions of the one or more second feature points.

2. The image processing device of claim 1 , wherein
the controller is configured to perform the two-dimensional matching based on at least one first feature point in the one or more first feature points that is different from the first feature points corresponding to a part of at least one of a linear edge and a repetitive pattern.

3. The image processing device of claim 1 , wherein
the controller is configured to determine a starting point for the two-dimensional matching based on positions of the one or more first feature points and parallax at the positions.

4. The image processing device of claim 1 , wherein
the controller is configured to perform the two-dimensional matching with sub-pixel precision.

5. The image processing device of claim 1 , wherein
the one or more first feature points extracted from the first captured image by the controller comprise a plurality of first feature points, and the one or more second feature points extracted from the second captured image by the controller comprise a plurality of second feature points corresponding respectively to the plurality of first feature points, and the controller is configured to calibrate at least one of the plurality of imaging units based on positions of the plurality of first feature points and positions of the plurality of second feature points corresponding to the plurality of first feature points.

6. The image processing device of claim 5 , wherein
the controller is configured to calibrate a posture of the stereo camera based on the positions of the plurality of first feature points and the positions of the plurality of second feature points.

7. A stereo camera device, comprising:
a plurality of imaging units; and
an image processing device including an input interface configured to receive input of a first captured image and a second captured image captured by the plurality of imaging units, and a controller configured to calculate parallax by performing a one-dimensional matching based on pixel values of the first captured image and pixel values of the second captured image, extract one or more first feature points from a region in the first captured image that includes continuous pixels having a difference in parallax which is within a predetermined range, extract one or more second feature points corresponding respectively to the one or more first feature points by performing a two-dimensional matching between the one or more first feature points and pixels in the second captured image, and calibrate at least one of the plurality of imaging units based on positions of the one or more first feature points and positions of the one or more second feature points.

8. A vehicle, comprising
a stereo camera device, including:
a plurality of imaging units; and
a stereo camera device including an input interface configured to receive input of a first captured image and a second captured image captured by the plurality of imaging units, and a controller configured to calculate parallax by performing a one-dimensional matching based on pixel values of the first captured image and pixel values of the second captured image, extract one or more first feature points from a region in the first captured image that includes continuous pixels having a difference in parallax which is within a predetermined range, extract one or more second feature points corresponding respectively to the one or more first feature points by performing a two-dimensional matching between the one or more first feature points and pixels in the second captured image, and calibrate at least one of the plurality of imaging units based on positions of the one or more first feature points and positions of the one or more second feature points.

9. An image processing method, in which a controller in an image processing device performs steps comprising
calculating parallax by performing a one-dimensional matching based on pixel values in a first captured image captured by a first imaging unit in a plurality of imaging units and pixel values in a second captured image captured by a second imaging unit in the plurality of imaging units that is different from the first imaging unit;
extracting one or more first feature points from a region in the first captured image that includes continuous pixels having a difference in parallax which is within a predetermined range;
extract one or more second feature points corresponding respectively to the one or more first feature points by performing a two-dimensional matching between the one or more first feature points and pixels in the second captured image; and
calibrating at least one of the plurality of imaging units based on positions of the one or more first feature points and positions of the one or more second feature points.

Assignees

• Kyocera Corp

Inventors

• Koishi Tomofumi
• Inoue Shushin

Classifications

• H04N13/246

  Calibration of cameras · CPC title

• G06T7/85

  Stereo camera calibration · CPC title

• G03B35/08

  by simultaneous recording · CPC title

• G01C25/00

  Manufacturing, calibrating, cleaning, or repairing instruments or devices referred to in the other groups of this subclass (testing, calibrating or compensating compasses G01C17/38) · CPC title

• G01C3/14 Primary

  with binocular observation at a single point, e.g. stereoscopic type (G01C3/20 takes precedence) · CPC title