Disparity deriving apparatus, movable apparatus, robot, method of deriving disparity, method of producing disparity, and storage medium

Patent metadata

Field	Value
Publication number	US-9898823-B2
Application number	US-201514662519-A
Country	US
Kind code	B2
Filing date	Mar 19, 2015
Priority date	Mar 27, 2014
Publication date	Feb 20, 2018
Grant date	Feb 20, 2018

Abstract

A disparity deriving apparatus for deriving disparity of an object includes a calculator to calculate costs between a first reference area in a reference image and each one of corresponding areas corresponding to the first reference area in the comparison image, and costs between a second reference area, surrounding the first reference area in the reference image, and each one of corresponding areas corresponding to the second reference area in a comparison image; a synthesizer to synthesize the costs of the first reference area and the second reference area as synthesis costs; a deriving unit to derive a disparity value of the object captured in the first reference area based on the synthesized synthesis costs; and a correction unit to apply a spatial filter to a local region, composed of a plurality of areas, in the first reference image to correct a disparity value of the local region.

First claim

What is claimed is:

1. A disparity deriving apparatus for deriving disparity of an object based on a reference image of the object captured at a first image capturing position and a comparison image of the object captured at a second image capturing position, the disparity deriving apparatus comprising:
processing circuitry configured to
calculate costs between a first reference area in the reference image and each one of corresponding areas corresponding to the first reference area in a given region in the comparison image, and costs between a second reference area, surrounding the first reference area in the reference image, and each one of corresponding areas corresponding to the second reference area in a given region in the comparison image;
synthesize the costs of the first reference area, and the costs of the second reference area as synthesis costs;
derive a disparity value of the object captured in the first reference area based on the synthesized synthesis costs; and
apply a spatial filter to a local region, composed of a plurality of areas, each area of the plurality of areas having a corresponding disparity value, in the first reference area to correct a specific disparity value in the local region by changing the specific disparity value in the local region to a new value using the spatial filter, wherein, in applying the spatial filter to the disparity values of the local region, excluding a disparity value of one area of the local region having low reliability from the application of the spatial filter.

2. The disparity deriving apparatus of claim 1 , wherein when a difference between two local minimums of the synthesis costs is smaller than a given value at the one area of the areas of the local region in the first reference image, the processing circuitry corrects the specific disparity value of the local region while excluding the one area from the application of the spatial filter.

3. The disparity deriving apparatus of claim 1 , wherein the processing circuitry is further configured to:
determine whether a difference between two local minimums of the synthesis costs is smaller than a given value at the one area of the areas of the local region in the first reference area, wherein the processing circuitry corrects the specific disparity value of the local region while excluding the one area from an application of the spatial filter when the processing circuitry determines that the difference is smaller than the given value.

4. The disparity deriving apparatus of claim 3 , wherein the processing circuitry conducts the determination for the local region on a specific directional path of directional paths in the first reference image.

5. The disparity deriving apparatus of claim 1 , wherein the given region is changeable.

6. A movable apparatus comprising:
the disparity deriving apparatus of claim 1 .

7. The movable apparatus of claim 6 , wherein the movable apparatus is one of a vehicle and a robot.

8. A robot comprising the disparity deriving apparatus of claim 1 .

9. The robot of claim 8 , wherein the robot is a fixed industrial robot.

10. The disparity deriving apparatus of claim 1 , wherein the processing circuitry is further configured to identify the one area to be excluded from application of the spatial filter when a difference between two local minimums of the synthesis costs for the one area is smaller than a given value.

11. The disparity deriving apparatus of claim 1 , wherein the processing circuitry is further configured to exclude the disparity of the one area of the local region in applying the spatial filter to correct the specific disparity value, the specific disparity value corresponding to an area of the local region other than the one area of the local region.

12. A method of deriving disparity of an object based on a reference image of the object captured at a first image capturing position and a comparison image of the object captured at a second image capturing position, the method comprising:
calculating costs between a first reference area in the reference image and each one of corresponding areas corresponding to the first reference area in a given region in the comparison image, and costs between a second reference area, surrounding the first reference area in the reference image, and each one of corresponding areas corresponding to the second reference area in a given region in the comparison image;
synthesizing the costs of the first reference area, and the costs of the second reference area as synthesis costs;
deriving a disparity value of the object captured in the first reference area based on the synthesis costs synthesized by the synthesizing step; and
applying a spatial filter to a local region, composed of a plurality of areas, each area of the plurality of areas having a corresponding disparity value, in the first reference area to correct a specific disparity value in the local region by changing the specific disparity value in the local region to a new value using the spatial filter, wherein the applying step includes, in applying the spatial filter to the disparity values of the local region, excluding a disparity value of one area of the local region having low reliability from the application of the spatial filter.

13. A method of producing disparity of an object based on the reference image of the object captured at the first image capturing position and the comparison image of the object captured at the second image capturing position, the method comprising the steps of claim 12 .

14. A non-transitory computer-readable carrier medium storing a program that, when executed by a computer, causes the computer to execute a method of deriving disparity of an object based on a reference image of the object captured at a first image capturing position and a comparison image of the object captured at a second image capturing position, the method comprising the steps of claim 12 .

15. The method of claim 12 , wherein when a difference between two local minimums of the synthesis costs is smaller than a given value at the one area of the areas of the local region in the first reference image, the applying step includes correcting the specific disparity value of the local region while excluding the one area from the application of the spatial filter.

16. The method of claim 12 , further comprising determining whether a difference between two local minimums of the synthesis costs is smaller than a given value at the one area of the areas of the local region in the first reference area, wherein the applying step includes correcting the specific disparity value of the local region while excluding the one area from an application of the spatial filter when it is determined that the difference is smaller than the given value.

17. The method of claim 16 , wherein the determining step further comprises determining whether the difference is smaller than the given value for the local region on a specific directional path of directional paths in the first reference image.

18. The method of claim 17 , wherein the given region is changeable.

19. A disparity deriving apparatus for deriving disparity of an object based on a reference image of the object captured at a first image capturing position and a comparison image of the object captured at a second image capturing position, the disparity deriving apparatus comprising:
processing circuitry configured to
calculate costs between a first reference area in the reference image and each one of corresponding areas corresponding to the first reference ar

Assignees

• Zhong Wei
• Saitoh Kiichiroh
• Ricoh Co Ltd

Inventors

• Zhong Wei
• Saitoh Kiichiroh

Classifications

• G06T2207/30261

  Obstacle · CPC title

• G06T7/97

  Determining parameters from multiple pictures (depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85) · CPC title

• G06T7/593 Primary

  from stereo images · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• H04N13/239

  using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance (H04N13/243 takes precedence) · CPC title