Detection of objects located closer than a minimal distance of hardware stereo systems

Patent metadata

Field	Value
Publication number	US-12488484-B2
Application number	US-202318164754-A
Country	US
Kind code	B2
Filing date	Feb 6, 2023
Priority date	Feb 6, 2023
Publication date	Dec 2, 2025
Grant date	Dec 2, 2025

Abstract

A method for use in a stereoscopic image generating system, wherein the image generating system comprises a hardware module associated with at least one pair of image capturing devices, at least one memory means and at least one processor, wherein information retrieved from the capturing devices is processed by the at least one processor, which is configured to implement a hardware module stereo algorithm for identifying objects included in the captured scene at a distance that is equal to or greater than a minimal distance defined by the geometry and disparity range of said hardware module, and wherein the at least one processor is further configured to implement a software stereo algorithm adapted to identify objects included in the captured scene at a distance that is less than the minimal distance required for detecting objects by the hardware module stereo algorithm.

First claim

The invention claimed is:

1 . A method for use in a stereoscopic image generating system, wherein the image generating system comprises a hardware module that is associated with at least one pair of image capturing devices, at least one memory means and at least one processor,
wherein information retrieved from the capturing devices is processed by the at least one processor, which is configured to implement a hardware module stereo algorithm for identifying objects included in the captured scene at a distance that is equal to or greater than a minimal distance defined by geometry and disparity range of said hardware module, wherein the at least one processor is further configured to implement a software stereo algorithm adapted to detect objects included in the captured scene at a distance that is less than the minimal distance required for identifying objects by the hardware module stereo algorithm, wherein said method comprises an initialization phase that comprises the steps of: (i) receiving at least one pair of images from said two image capturing devices, storing information derived from the received images by columns and rows and retrieving information contained in pixels comprised in said rows/columns; (ii) assigning a small set of initial random disparities to each of said columns, wherein the same set of initial random disparities is assigned to all pixels that belong to a single column; (iii) selecting a small set of random disparities from among random permutations, in a way that creates a generally uniform coverage and determining a permutation index i for column c by using a relationship i=P[c mod S], where S is the permutation size and P is a random permutation of the numbers 0, . . . , S−1, and disparity samples for said column c are: i, i+S, i+2S, . . . until the edge of the image is reached; (iv) computing matching costs for each pixel located at the first row of the captured image for its initial set of random disparities, wherein the disparity that corresponds the lowest score is stored at said at least one memory means; and (v) completing the computation for all image rows included in the captured image.

2 . The method of claim 1 , wherein said method is further configured to generate a combined depth map based on combining the objects located at a distance that is equal to or greater than said minimal distance that are identifiable by the hardware module stereo algorithm, and objects located at a distance that is less than said minimal distance that are identifiable by the software stereo algorithm.

3 . The method of claim 1 , wherein said method is further configured to generate an indication upon identifying objects included within the captured scene at a distance that is less than said minimal distance.

4 . The method of claim 1 , wherein said method is further configured to eliminate from information received from the image capturing devices, values of pixels associated with objects that are located within the captured scene at a distance that is less than said minimal distance.

5 . The method of claim 1 , wherein said hardware module stereo algorithm is configured to scan a full range of disparities, ensuring that all objects located within the scene at a distance that is equal to or greater than said minimal distance, are detected.

6 . The method of claim 5 , wherein said software stereo algorithm is configured to identify objects which are located at a distance less than the minimal distance, which are not detectable by the hardware module scan.

7 . The method of claim 1 , wherein the software stereo algorithm is configured to detect distance at which objects are located by using the hardware module and/or the pair of image capturing devices, wherein in case that said distance is equal to or greater than the minimal distance, depth results will be determined by using the results obtained by the hardware module stereo algorithm, and in case that the detected distance is less than the minimal distance, the depth results will be determined by using the results obtained by the software stereo algorithm.

8 . The method of claim 1 , comprises a phase of calculating lowest-found cost and best-found disparity, wherein said phase comprises the steps of:
1) Making a number of passes over the image in order to update for each pixel its best-found disparity with a respective lowest-found cost in order to update for each pixel its best-found disparity with a respective lowest found cost; 2) For each pass, comparing the cost of each given pixel's current best disparity with two proposed disparities, one of which is a disparity of said given pixel row's neighbor whereas the other is that of its column's neighbor; and 3) Evaluating the cost of disparities d+1 and d−1, where d is current disparity having lowest cost, and updating the current best disparity and current lowest cost if the cost is smaller than the current best cost of a given pixel.

9 . The method of claim 8 , comprising a phase of combining disparity maps retrieved while processing information received from the image capturing devices to enable detection of objects included in the captured scene at the distance that is equal to or greater than the pre-defined minimal distance and information received from applying the software stereo algorithm to enable detection of closer objects, wherein said phase comprises the steps of:
(a) initializing a combined depth map of the disparity obtained from processing information received from the image capturing devices; (b) identifying pixels in said combined depth map derived from the output of said software stereo algorithm, wherein the identified pixels have a disparity higher than a maximal disparity of the hardware software module algorithm; (c) implementing morphological operations onto said combined depth map, so that each pixel in the image is adjusted based on values of other pixels located at its neighborhood, to eliminate small holes and remove small blobs from the captured image being processed; (d) after eliminating the small holes and the removal of the small blobs, setting the disparity of the image pixels to the disparity obtained from the software stereo algorithm; (e) if the matching cost of the pixels set in step (d) is higher than a pre-defined threshold, their disparity is set to an invalid value; (f) setting the disparity of pixels that were eliminated/removed by applying said morphological operations, to an invalid disparity; and (g) generating a combined depth map.

10 . The method of claim 9 , wherein the combined depth map is applied as a full combined disparity map, or applied as a truncated map that matches a disparity range of the hardware module's algorithm, or applied for generating an indication if a large number of close pixels is detected by the software stereo algorithm.

11 . The method of claim 1 , adapted for use in robotics field of technology.

Assignees

• Inuitive Ltd

Inventors

• Ecker Ady

Classifications

• G06T5/77

  Retouching; Inpainting; Scratch removal · CPC title

• G06V10/761 Primary

  Proximity, similarity or dissimilarity measures · CPC title

• G06V2201/07

  Target detection · CPC title

• H04N13/106

  Processing image signals (for multi-view video sequence encoding H04N19/597) · CPC title

• G06T2207/20036

  Morphological image processing · CPC title