Hardware disparity evaluation for stereo matching

Patent metadata

Field	Value
Publication number	US-10529085-B2
Application number	US-201815941721-A
Country	US
Kind code	B2
Filing date	Mar 30, 2018
Priority date	Mar 30, 2018
Publication date	Jan 7, 2020
Grant date	Jan 7, 2020

Abstract

A method for calculating disparity in a pair of images includes receiving a first image of a scene and designating the first image as a master image. A second image of the scene is received, and the second image is designated as a slave image. The master image is binarized to produce a binarized master image. The slave image is binarized to produce a binarized slave image. A matching cost associated with matching each pixel within the binarized master image with a corresponding set of candidate pixels within the binarized slave image is calculated. A probability density function is created based on the calculated matching costs associated with each pixel within the binarized master image. The created probability density function is used to produce a disparity for the master image and the slave image and to produce a confidence for the produced disparity.

First claim

What is claimed is:

1. A method for calculating disparity in a pair of images, comprising:
receiving a first image of a scene and designating the first image as a master image;
receiving a second image of the scene and designating the second image as a slave image;
binarizing the master image to produce a binarized master image;
binarizing the slave image to produce a binarized slave image;
calculating a matching cost associated with matching each pixel within the binarized master image with a corresponding set of candidate pixels within the binarized slave image;
creating a probability density function based on the calculated matching costs associated with each pixel within the binarized master image; and
using the created probability density function to produce a disparity for the master image and the slave image and to produce a confidence for the produced disparity.

2. The method of claim 1 , wherein the steps of binarizing the master and slave images, calculating the matching costs, creating the probability density function, and producing the disparity and confidence are performed within a special-purpose digital signal processor.

3. The method of claim 2 , wherein the special-purpose digital signal processor operates in real-time.

4. The method of claim 1 , wherein binarizing each of the master image and slave image comprises:
defining a list of supports of increasing size;
selecting a pixel;
selecting a support for the selected pixel from the defined list of supports;
calculating a mean and standard deviation for the selected support about the selected pixel;
assigning a binarized pixel value or selecting a next support according to a difference between the calculated mean and the selected pixel relative to a predetermined fraction of the calculated standard deviation; and
repeating the steps of selecting the pixel, selecting the support, calculating the mean and standard deviation, and assigning a binarized pixel value for each pixel of the master image.

5. The method of claim 4 , wherein assigning a binarized pixel value or selecting a next support according to a difference between the calculated mean and the selected pixel relative to a predetermined fraction of the calculated standard deviation, comprises:
assigning the binarized pixel value as “1” when it is determined that the difference between the calculated mean and the selected pixel exceeds the predetermined fraction of the calculated standard deviation;
assigning the binarized pixel value as “0” when it is determined that the difference between the calculated mean and the selected pixel is below the predetermined fraction of the calculated standard deviation; and
selecting a next support when it is determined that the difference between the calculated mean and the selected pixel is within the predetermined fraction of the calculated standard deviation.

6. The method of claim 1 , wherein calculating the matching cost, comprises:
selecting a pixel in the binarized master image;
selecting a match candidate in the binarized slave image;
calculating a matching cost for the selected pixel in the binarized master image and the selected match candidate in the binarized slave image; and
repeating the steps of selecting the pixel in the binarized master image, selecting the match candidate in the binarized slave image, and calculating the matching cost, for each pixel of the binarized master image and each corresponding match candidate in the binarized slave image.

7. The method of claim 6 , wherein the probability density function is determined based on the calculated matching costs for each pixel of the binarized master image and each corresponding match candidate in the binarized slave image.

8. The method of claim 1 , wherein the first image and the second image are acquired from two cameras, a single camera with two image sensors, or a single camera moved between two locations and/or two angles.

9. The method of claim 1 , wherein the produced disparity is instantiated as an image comprising a plurality of pixels, the value of each being set according to a degree of disparity between the first image and the second image at a corresponding pixel location.

10. The method of claim 9 , wherein the value of each of the plurality of pixels of the disparity image are greater for greater disparities and lesser for lesser disparities.

11. The method of claim 9 , wherein the produced confidence is instantiated as an image comprising a plurality of pixels, the value of each being set according to a degree of confidence for a corresponding pixel of the disparity image.

12. The method of claim 11 , wherein the value of each of the plurality of pixels of the confidence image are greater for greater degrees of confidence and lesser for lesser degrees of confidence.

13. A system for generating a 3D model, comprising:
one or more cameras for acquiring a first image of a scene and a second image of the scene;
a special-purpose digital signal processor configured to:
binarize the acquired first image to produce a binarized master image;
binarize the acquired second image to produce a binarized slave image;
calculate a matching cost associated with matching each pixel within the binarized master image with a corresponding set of candidate pixels within the binarized slave image;
create a probability density function based on the calculated matching costs associated with each pixel within the binarized master image; and
use the created probability density function to produce a disparity image for the master image and the slave image and to produce a confidence image for the produced disparity; and
a display device configured to display a three-dimensional rendering based on the produced disparity image and/or the produced confidence image.

14. The system of claim 13 , wherein the special-purpose digital signal processor is configured to operate in real-time.

15. The system of claim 13 , wherein the special-purpose digital signal processor is configured to binarize each of the master image and slave image by:
defining a list of supports of increasing size;
selecting a pixel;
selecting a support for the selected pixel from the defined list of supports;
calculating a mean and standard deviation for the selected support about the selected pixel;
assigning a binarized pixel value or selecting a next support according to a difference between the calculated mean and the selected pixel relative to a predetermined fraction of the calculated standard deviation; and
repeating the steps of selecting the pixel, selecting the support, calculating the mean and standard deviation, and assigning a binarized pixel value for each pixel of the master image.

16. The system of claim 13 , wherein the special-purpose digital signal processor is configured to calculate the matching cost, by:
selecting a pixel in the binarized master image;
selecting a match candidate in the binarized slave image;
calculating a matching cost for the selected pixel in the binarized master image and the selected match candidate in the binarized slave image; and
repeating the steps of selecting the pixel in the binarized master image, selecting the match candidate in the binarized slave image, and calculating the matching cost, for each pixel of the binarized master image and each corresponding match candidate in the binarized slave image.

17. The system of claim 13 , wherein the special-purpose digital signal processor is configured:
to produce the disparity image such that a value of each pixel of the disparity image is set

Assignees

• Samsung Electronics Co Ltd

Inventors

• Bitan Gal
• Azaria Haim

Classifications

• G06T2207/20228

  Disparity calculation for image-based rendering · CPC title

• G06T2207/20076

  Probabilistic image processing · CPC title

• G06T7/593 Primary

  from stereo images · CPC title

• G06T2207/10012

  Stereo images · CPC title

• H04N13/271

  wherein the generated image signals comprise depth maps or disparity maps · CPC title