Method for generating three dimensional images

Patent metadata

Field	Value
Publication number	US-10504235-B2
Application number	US-201615391267-A
Country	US
Kind code	B2
Filing date	Dec 27, 2016
Priority date	Dec 27, 2016
Publication date	Dec 10, 2019
Grant date	Dec 10, 2019

Abstract

A method is provided for reconstructing a three dimensional image, comprising: providing a pair of stereoscopic images captured by two image capturing devices; retrieving disparity data from the pair of stereoscopic images; using retrieved disparity data to determine contours of an object present in the pair of the stereoscopic images captured; selecting a pixel belonging to that object; establishing a search area at the vicinity of the selected pixel; determining values for various disparity candidates, wherein the determined values correspond to one or more chosen hypothesis associated with the search area; testing the one or more chosen hypothesis and selecting an hypothesis which best fits the selected pixel and its neighboring pixels; executing a stereo matching algorithm using data retrieved from the captured images while using the values of disparity candidates that correspond to the chosen hypothesis; and reconstructing the three dimensional image based on the results obtained.

First claim

The invention claimed is:

1. A method for reconstructing a three dimensional image, said method comprises the steps of:
receiving a pair of stereoscopic images captured essentially simultaneously by two image capturing devices;
retrieving disparity data from said pair of stereoscopic images and calculating disparity edges therefrom based on intensity gradients that exist between pixels comprised in said stereoscopic images;
using the calculated disparity edges to determine contours of an object present in the pair of the stereoscopic images captured;
selecting at least one pixel belonging to said object;
establishing a search area at the vicinity of the at least one selected pixel to enable identifying disparity candidates associated with objects located at the vicinity of said object whose contours were determined, wherein a size of the search area is adapted to a disparity level of the selected at least one pixel;
determining values for the identified disparity candidates, wherein the determined values correspond to one or more chosen hypothesis associated with said search area;
testing the one or more chosen hypothesis and selecting a hypothesis which best fits the at least one selected pixel and its neighboring pixels;
executing a stereo matching algorithm using data retrieved from the pair of corresponding captured images while using the values of one or more identified disparity candidates that correspond to the chosen hypothesis; and
reconstructing the three dimensional image based on the results of the stereo matching algorithm.

2. The method of claim 1 , wherein said method comprises an iterative process which comprises receiving data that relates to disparity candidates obtained from a low-noise iteration and executing a new high-resolution iteration.

3. The method of claim 1 , wherein said method comprises retrieving data that relates to disparity candidates over a group of pixels present in the area located at the vicinity of the at least one selected pixel.

4. The method of claim 3 , further comprising a step of selecting a subset of disparity candidates from among the retrieved data that relates to said disparity, for re-evaluating said at least one selected pixel.

5. The method of claim 1 , wherein said step of testing the one or more chosen hypothesis comprises using a high-resolution adaptive block matching of the area located around the at least one selected pixel.

6. The method of claim 1 , wherein said objects located at the vicinity of the object whose contour was determined are present at different depths of the image.

7. The method of claim 1 , wherein said disparity edges are calculated by using an edge detector or by taking into account changes that occurred in said disparity data that exceed a pre-determined threshold, or any combination thereof.

8. The method of claim 1 , wherein said disparity edges are calculated by using a discrete differentiation operator for computing an approximation of intensities' gradients in the stereoscopic images.

9. A system for reconstructing a three dimensional image, the system comprising:
at least two image capturing devices for capturing essentially simultaneously at least two stereoscopic images;
at least one processor configured to:
retrieve disparity data from said at least two stereoscopic images;
calculating disparity edges therefrom based on intensity gradients that exist between pixels comprised in said stereoscopic images;
apply retrieved the calculated disparity edges to determine contours of an object present in the at least two stereoscopic images captured;
select at least one pixel belonging to the object;
establish a search area at the vicinity of the at least one selected pixel to enable identifying disparity candidates associated with objects located at the vicinity of said object whose contours were determined, wherein a size of the search area is adapted to a disparity level of the selected at least one pixel;
determine values for the identified disparity candidates, wherein the determined values correspond to one or more chosen hypothesis associated with said search area;
test the one or more chosen hypothesis and select a hypothesis which best describes the at least one selected pixel and its neighboring pixels;
execute a stereo matching algorithm using data retrieved from the at least two captured images while using the values of one or more identified disparity candidates that correspond to the chosen hypothesis; and
reconstruct the three dimensional image based on the results of the stereo matching algorithm;

a memory means configured for storing information retrieved from the at least two image capturing devices along with disparity information that relates to said search area; and
a delay buffer configured to enable synchronization between data received from the at least two image capturing devices.

10. The system of claim 9 , wherein the at least one processor is further configured to determine that disparity candidates available around the contours belong to different objects, while disparity candidates associated with pixels located relatively far from the determined contours belong to said object.

11. The system of claim 10 , wherein the at least one processor is further configured to establish a smaller search area for the pixels located relatively far from the determined contours.

12. The system of claim 9 , wherein the size of the search area is different than a size of an area used for computing low-noise disparity.

13. The system of claim 9 , wherein the at least one processor is further configured to determine depth from the at least two captured images.

14. The system of claim 9 , wherein the at least one processor is further configured to determine a depth of the three dimensional image by which low-noise disparity candidates are selected over a certain area for a secondary high-resolution disparity search carried out over selected candidates.

15. The system of claim 9 , wherein the at least one processor is further configured to determine a depth of the three dimensional image by selecting a disparity calculating mechanism which is adapted to enable balancing between noise level and resolution level according to user preferences.

Assignees

• Inuitive Ltd

Inventors

• Goldentouch Lev
• Tsoref Ziv
• Lifsches Samuel

Classifications

• G06T7/593 Primary

  from stereo images · CPC title

• G06T2207/10024

  Color image · CPC title

• G06T2207/20076

  Probabilistic image processing · CPC title

• H04N2013/0081

  Depth or disparity estimation from stereoscopic image signals · CPC title

• G06T2207/10012

  Stereo images · CPC title