Stereo matching method and apparatus

Patent metadata

Field	Value
Publication number	US-10818025-B2
Application number	US-201715707048-A
Country	US
Kind code	B2
Filing date	Sep 18, 2017
Priority date	Jan 26, 2017
Publication date	Oct 27, 2020
Grant date	Oct 27, 2020

Abstract

A stereo matching method includes extracting feature points of a first image and feature points of a second image, the first image and the second image together constituting a stereo image, determining reference points by matching the feature points of the second image to the feature points of the first image, classifying the reference points, and performing stereo matching on pixels of which disparities are not determined in the first image and the second image based on disparities of the reference points in the pixels determined based on a result of the classifying.

First claim

What is claimed is:

1. A stereo matching method, comprising:
extracting feature points of a first image and feature points of a second image, the first image and the second image together constituting a stereo image;
determining reference points by matching the feature points of the second image to the feature points of the first image;
classifying the reference points comprises any one or any combination of:
classifying a reference point for which a consistency is not maintained by the matching among the reference points into a first class,
classifying a reference point for which the consistency is maintained and a ratio between the first optimal cost and the second optimal cost among correlation costs of the reference points calculated based on a cost analysis for measuring a window-based correlation is less than a threshold among the reference points into a second class, and
classifying a reference point for which the consistency is maintained and the ratio between the first optimal cost and the second optimal cost reaches the threshold among the reference points into a third class;

resetting a window based on at least one of a shifted window having an adjustable angle and multiple windows having identical sizes with respect to a reference point of which a depth is discontinued from adjacent reference points among reference points classified into the first class and the second class;
resetting the window based on an extension window obtained by extending a size of the window with respect to a reference point of which a depth is continued from the adjacent reference points among the reference points classified into the first class and the second class; and
performing stereo matching on pixels based on the reset window.

2. The stereo matching method of claim 1 , wherein the determining of the reference points comprises:
matching the feature points of the second image to the feature points of the first image using a window-based correlation; and
determining, as the reference points, feature points having an optimal cost among the matched feature points using a cost analysis for measuring the window-based correlation.

3. The stereo matching method of claim 1 , wherein the classifying of the reference points comprises classifying the reference points into a class based on at least one of:
whether the reference points are present in a region in which a depth discontinuity occurs,
whether the reference points are present in a region in which an occlusion occurs,
whether the reference points are present in a region having a texture value less than or equal to a preset reference, and
whether the reference points have disparity values greater than or equal to a preset reliability.

4. The stereo matching method of claim 1 , wherein the classifying of the reference points further comprises labeling the reference points based on the respective classes of the reference points.

5. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the method of claim 1 .

6. A stereo matching apparatus, comprising:
a processor; and
a memory configured to store a computer-readable instruction,
wherein, in response to execution of the instruction at the processor, the processor is configured to:
extract feature points of a first image and feature points of a second image, the first image and the second image together constituting a stereo image,
determine reference points by matching the feature points of the second image to the feature points of the first image,
classify the reference points comprising any one or any combination of:
classify a reference point for which the consistency is not maintained by the matching among the reference points into a first class,
classify a reference point for which the consistency is maintained and the ratio between the first optimal cost and the second optimal cost is less than a preset threshold among the reference points into a second class, and
classify a reference point for which the consistency is maintained and the ratio between the first optimal cost and the second optimal cost reaches the preset threshold among the reference points into a third class,

reset a window based on at least one of a shifted window having an adjustable angle and multiple windows having identical sizes with respect to a reference point of which a depth is discontinued from adjacent reference points among reference points classified into the first class and the second class;
reset the window based on an extension window obtained by extending a size of the window with respect to a reference point of which a depth is continued from the adjacent reference points among the reference points classified into the first class and the second class; and
perform stereo matching on pixels based on the reset window.

7. The stereo matching apparatus of claim 6 , wherein the processor is further configured to:
match the feature points of the second image to the feature points of the first image using a window-based correlation, and
determine, as the reference points, feature points having an optimal cost among the matched feature points using a cost analysis for measuring the window-based correlation, and the memory is configured to store information including disparity values corresponding to the reference points.

8. The stereo matching apparatus of claim 6 , wherein the processor is further configured to classify the reference points into a class based on at least one of:
whether the reference points are present in a region in which a depth discontinuity occurs,
whether the reference points are present in a region in which an occlusion occurs,
whether the reference points are present in a region having a texture value less than or equal to a preset reference, and
whether the reference points have disparity values greater than or equal to a preset reliability.

9. The stereo matching apparatus of claim 6 , wherein the processor is further configured to:
classify the reference points into a class based on at least one of:
whether a consistency is maintained by the matching,
a ratio between a first optimal cost and a second optimal cost among correlation costs of the reference points calculated based on a cost analysis for measuring a window-based correlation, and
whether optimal costs are detected among the correlation costs of the reference points.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Park Seungin
• Ahn Minsu
• Kim Jiyeon

Classifications

• G06V30/413

  Classification of content, e.g. text, photographs or tables · CPC title

• H04N13/122

  Improving the three-dimensional [3D] impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues (H04N13/128 takes precedence) · CPC title

• H04N2013/0081

  Depth or disparity estimation from stereoscopic image signals · CPC title

• H04N13/128

  Adjusting depth or disparity · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title