Range image generating method, range image generating apparatus, and on-board device having range image generating function

Patent metadata

Field	Value
Publication number	US-10891751-B2
Application number	US-201916253839-A
Country	US
Kind code	B2
Filing date	Jan 22, 2019
Priority date	Jan 23, 2018
Publication date	Jan 12, 2021
Grant date	Jan 12, 2021

Abstract

An image processing method of generating a range image is provided. The method includes a step of detecting a distance to the object as a measured distance, by a distance measurement unit; a step of performing, by an image processing unit, integration of the distance with a matching evaluation value of a pixel in a stereogram corresponding to a location on the object of which the distance is detected by the distance measurement unit, among a plurality of pixels in the stereogram each including a corresponding matching evaluation value; and a step of generating a range image based on a result of the integration. The generating step is performed after the integration.

First claim

What is claimed is:

1. An image processing method of generating a range image, the method comprising:
detecting, by a distance measurement unit including a light outputting device, a distance to an object as a measured distance by emitting light from the light outputting device to the object;
performing, by circuitry, matching of a stereogram by shifting a comparison image with respect to a reference image, to calculate matching cost values of pixels in the stereogram in a parallax space, each of the matching cost values in the parallax space corresponding to a corresponding shift amount of a pixel between the reference image and the comparison image;
converting, by the circuitry, the matching cost values in the parallax space into a set of matching cost values in a distance space, each of the matching cost values in the distance space corresponding to a corresponding distance;
performing, by the circuitry, integration of a distance evaluation value related to the measured distance, with a matching cost value corresponding to the measured distance among the matching cost values of a pixel in the stereogram corresponding to the location on the object of which the distance is detected by the distance measurement unit; and
generating by the circuitry, after the integration, a range image for measuring a distance to an object, based on a result of the integration.

2. The method according to claim 1 , wherein
the distance evaluation value is expressed as a first function of a length from the pixel corresponding to the location on the object of which the distance is detected by the distance measurement unit, the first function taking a minimum when the length is zero and becoming larger as the length becomes larger.

3. The method according to claim 2 , wherein, in integrating the distance evaluation value,
a value of the first function, with respect to the length between the pixel corresponding to the location on the object of which the distance is detected by the distance measurement unit and a surrounding pixel of the pixel corresponding to the location on the object of which the distance is detected by the distance measurement unit, is added to a matching cost value corresponding to the measured distance among matching cost values of the surrounding pixel.

4. The method according to claim 2 , wherein
the distance evaluation value is a product of a value of the first function and a value of a second function of distance, the second function taking a minimum when the distance is equal to the measured distance and becoming larger as the distance deviates from the measured distance.

5. The method according to claim 4 , wherein
the integration includes
integrating, with a matching cost value corresponding to the measured distance among matching cost values of the pixel corresponding to the location on the object of which the distance is detected by the distance measurement unit, a product of a minimum value of the first function and a minimum value of the second function as the distance evaluation value; and
integrating, with a matching cost value corresponding to a distance deviating from the measured distance by a given amount, a product of the minimum value of the first function and a value of the second function with respect to the given amount as the distance evaluation value.

6. The method according to claim 4 , wherein
the integration includes
integrating, with a matching cost value corresponding to the measured distance among matching cost values of a surrounding pixel of the pixel corresponding to the location on the object of which the distance is detected by the distance measurement unit, a product of a value of the first function, with respect to a length between the surrounding pixel and the pixel corresponding to the location on the object of which the distance is detected by the distance measurement unit, and a minimum value of the second function, as the distance evaluation value; and
integrating, with a matching cost value corresponding to a distance deviating from the measured distance by a given amount among matching cost values of a surrounding pixel of the pixel corresponding to the location on the object of which the distance is detected by the distance measurement unit, a product of a value of the first function with respect to a length between the surrounding pixel and the pixel corresponding to the location on the object of which the distance is detected by the distance measurement unit, and a value of the second function obtained based on the given amount, as the distance evaluation value.

7. The method according to claim 1 , further comprising
calculating, for each pixel, an evaluation value by applying a dense matching algorithm to a value obtained by integration of the distance evaluation value with the matching cost value;
calculating an energy cost of a given pixel, by adding the evaluation value of the given pixel and the evaluation values of a surrounding pixel of the given pixel; and
determining a distance that minimizes the energy cost as distance information of the given pixel.

8. The method according to claim 1 , further comprising
displaying information indicating that the distance measurement unit or a capturing unit has failed, in response to a detection of failure of the distance measurement unit or the capturing unit, wherein the capturing unit is configured to capture an image contained in the stereogram.

9. The method according to claim 8 , wherein
the generating of the range image is performed by using the matching cost values of the stereogram, in a case in which failure of the distance measurement unit is detected; and
the method further comprises outputting the measured distance detected by the distance measurement unit in a case in which failure of the capturing unit is detected.

10. The method according to claim 1 , further comprising detecting a light reception level curve, each value on the light reception level curve being associated with a distance to the object; wherein
the integration of the distance evaluation value with the matching cost value of the pixel corresponding to the location on the object of which the distance is detected by the distance measurement unit is performed in a case in which the matching cost values in the distance space is flat and a number of a minimum value of the light reception level curve is one, or in which the light reception level curve includes a plurality of peaks and the number of the minimum value of the matching cost values in the distance space is one.

11. The method according to claim 1 , wherein the pixel corresponding to the location on the object of which the distance is detected by the distance measurement unit is identified based on a direction of measuring a distance of the object by the distance measurement unit.

12. The method according to claim 1 , wherein, in the generating of the range image, distance information determined based on the result of the integration is associated with the pixel.

13. An image processing apparatus configured to generate a range image, the image processing apparatus comprising:
a distance measurement unit including a light outputting device, the distance measurement unit being configured to detect a distance to an object by emitting light from the light outputting device to the object; and
circuitry configured
to cause the distance measurement unit to detect the distance to the object as a measured distance;
to perform matching of the stereogram by shifting a comparison image with respect to a reference image, to calculate matching cost values of pixels in the stereogram in a parallax space, each of the matc

Assignees

• Sekiguchi Hiroyoshi
• Yokota Soichiro
• Saisho Kenichiroh
• Ricoh Co Ltd

Inventors

• Sekiguchi Hiroyoshi
• Yokota Soichiro
• Saisho Kenichiroh

Classifications

• H04N23/80

  Camera processing pipelines; Components thereof · CPC title

• H04N23/951

  by using two or more images to influence resolution, frame rate or aspect ratio · CPC title

• G01S17/86

  Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G01S17/931

  of land vehicles · CPC title