Disparity value deriving device, equipment control system, movable apparatus, robot, and disparity value deriving method

Patent metadata

Field	Value
Publication number	US-9813694-B2
Application number	US-201514680245-A
Country	US
Kind code	B2
Filing date	Apr 7, 2015
Priority date	Apr 11, 2014
Publication date	Nov 7, 2017
Grant date	Nov 7, 2017

Abstract

An image reducer executes a reduction process on a comparison image to generate a reduced comparison image, and executes a reduction process on a reference image to generate a reduced reference image. A disparity value deriver (a deriving unit) derives a disparity value by setting the minimum value of synthesized costs of pixels in the reduced comparison image in connection with a reference pixel in the reduced reference image, which have been calculated by a cost synthesizer. A disparity image generator generates a reduced disparity image, which is an image obtained by replacing the luminance values of the respective pixels in the reduced reference image with the disparity values corresponding to the pixels. The disparity image enlarger executes an enlargement process for enlarging the image size of the reduced disparity image generated by the disparity image generator to generate a disparity image.

First claim

What is claimed is:

1. A disparity value deriving device for deriving a disparity value representing disparity for an object from a first reference image obtained by a first imaging unit for imaging the object and a first comparison image obtained by a second imaging unit for imaging the object, the second imaging unit being located at a different position from the first imaging unit, the disparity value deriving device comprising:
a reducer configured to generate a second reference image and a second comparison image by reducing the first reference image and the first comparison image, respectively, by a first ratio;
a calculator configured to calculate degrees of matching of a plurality of regions in a predetermined range in the second comparison image with a first reference region in the second reference image, the predetermined range including a corresponding region that corresponds to the first reference region;
a synthesizer configured to obtain a synthesis degree of matching for each of the regions by aggregating the degrees of matching in the second comparison image for a second reference region located in neighborhood of the first reference region, with the degree of matching of the regions;
a deriving unit configured to derive the disparity value between the first reference region and the corresponding region, based on the synthesis degrees of matching;
a generator configured to generate a first disparity image based on disparity values derived by the deriving unit; and
an enlarging unit configured to enlarge the first disparity image by a second ratio to generate a second disparity image, by placing at least one first correction pixel between referential pixels that are consecutive in column and row directions and by placing a second correction pixel in a space generated between consecutive ones of the first correction pixels, the referential pixels being pixels included in the first disparity image.

2. The disparity value deriving device according to claim 1 , wherein, in the second disparity image, the enlarging unit obtains a pixel value of the first correction pixel placed between the consecutive referential pixels, based on pixel values of the referential pixels, and obtains a pixel value of the second correction pixel placed between the consecutive first correction pixels that have already-determined pixel values, based on the pixel values of the first correction pixels.

3. The disparity value deriving device according to claim 1 , wherein
the calculator calculates the degree of matching of each of a plurality of candidate regions based on a luminance value of the first reference region and luminance values of the candidate regions, the candidate regions being specified by shifting positions by a shift amount in the predetermined range from a region corresponding to the position of the first reference region in the second comparison image and being set as candidates for the corresponding region,
the deriving unit derives the disparity value based on the shift amount that corresponds to an extreme of the degrees of matching of the respective candidate regions in the second comparison image, and
the generator generates the first disparity image by replacing luminance values of the second reference image with the disparity values derived by the deriving unit.

4. The disparity value deriving device according to claim 1 , wherein, in the second disparity image, the enlarging unit obtains a pixel value of the first correction pixel placed between the consecutive referential pixels, based on a difference between pixel values of the referential pixels, and obtains a pixel value of the second correction pixel placed between the consecutive first correction pixels that have already-determined pixel values, based on a difference between pixel values of the first correction pixels.

5. The disparity value deriving device according to claim 4 , wherein, when the corresponding difference between the pixel values is larger than a predetermined threshold, the enlarging unit sets either the larger value or the smaller value of these two pixel values as each of the pixel values of the first correction pixel and the second correction pixel in the second disparity image, and, when the corresponding difference between the pixel values is smaller than the predetermined threshold, the enlarging unit sets an average of these two pixel values as each of the pixel values of the first correction pixel and the second correction pixel.

6. The disparity value deriving device according to claim 5 , wherein, when the corresponding difference between the pixel values is larger than the predetermined threshold, the enlarging unit is enabled to switch between these two pixel values as each of the pixel values of the first correction pixel and the second correction pixel.

7. The disparity value deriving device according to claim 1 , wherein the second ratio is an inverse of the first ratio.

8. The disparity value deriving device according to claim 1 , wherein the first ratio and the second ratio are changeable to any desirable ratios.

9. An equipment control system comprising:
the disparity value deriving device according to claim 1 ; and
a control device configured to control a control target according to information on a distance from the disparity value deriving device and the object, the information having been obtained from the disparity value derived by the disparity value deriving device.

10. A movable apparatus comprising the equipment control system according to claim 9 .

11. A robot comprising the equipment control system according to claim 9 .

12. A disparity value deriving method for deriving a disparity value representing disparity for an object from a first reference image obtained by a first imaging unit for imaging the object and a first comparison image obtained by a second imaging unit for imaging the object, the second imaging unit being located at a different position from the first imaging unit, the disparity value deriving device comprising:
generating a second reference image and a second comparison image by reducing the first reference image and the first comparison image, respectively, by a first ratio;
calculating degrees of matching of a plurality of regions in a predetermined range in the second comparison image with a first reference region in the second reference image, the predetermined range including a corresponding region that corresponds to the first reference region;
obtaining a synthesis degree of matching for each of the regions by aggregating the degrees of matching in the second comparison image for a second reference region located in neighborhood of the first reference region, with the degree of matching of the regions;
deriving the disparity value between the first reference region and the corresponding region, based on the synthesis degrees of matching;
generating a first disparity image based on disparity values derived by the deriving unit; and
enlarging the first disparity image by a second ratio to generate a second disparity image, by placing at least one first correction pixel between referential pixels that are consecutive in column and row directions and by placing a second correction pixel in a space generated between consecutive ones of the first correction pixels, the referential pixels being pixels included in the first disparity image.

Assignees

• Zhong Wei
• Saito Kiichiro
• Ricoh Co Ltd

Inventors

• Zhong Wei
• Saito Kiichiro

Classifications

• G06V10/751

  Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching · CPC title

• B60R2300/107

  using stereoscopic cameras · CPC title

• H04N13/239

  using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance (H04N13/243 takes precedence) · CPC title

• G06T7/593

  from stereo images · CPC title

• H04N13/296 Primary

  Synchronisation thereof; Control thereof · CPC title