Method and device for determining a parallax problem in sensor data of two sensors

Patent metadata

Field	Value
Publication number	US-11719825-B2
Application number	US-202016985256-A
Country	US
Kind code	B2
Filing date	Aug 5, 2020
Priority date	Aug 9, 2019
Publication date	Aug 8, 2023
Grant date	Aug 8, 2023

Abstract

A method for detecting a parallax problem in sensor data from two sensors, wherein the sensors are spaced apart at different positions and at least partly capture the same environment, and one of the sensors provides distance information items. The method includes receiving the acquired sensor data from the sensors; receiving or estimating sensor visual range information items from the other of the sensors; assigning measured values in the acquired sensor data of the one sensor to measured values corresponding to each of them from the other sensor, the assignment taking into account respective imaging conditions; comparing the distance information items and the received or estimated sensor visual range information items on each of the measured values of the sensor data, wherein a parallax problem is detected in response to a distance exceeding a threshold criterion; and outputting a comparison result.

First claim

The invention claimed is:

1. An apparatus for detecting a parallax problem, the apparatus comprising:
a first sensor and a second sensor arranged in positions that are spaced apart from each other and at least partially capturing the same environment, the first sensor configured to provide distance information items;
an input device;
a computing device; and
an output device,
wherein the input device receives acquired sensor data from the first and second sensors, the acquired sensor data from the first sensor includes at least one first distance value for at least one first point in the environment based on the first sensor, the at least one first distance value measuring a distance from the first sensor to the at least one first point,
wherein the computing device:
one of (i) receives sensor visual range information items for the at least one first point from the second sensor or (ii) estimates the sensor visual range information items for the at least one first point of the second sensor based on the acquired sensor data from the second sensor such that the computing device is configured to determine at least one second distance value for the at least one first point based on the second sensor, the at least one second distance value measuring a distance from the second sensor to the at least one first point or to at least one object located in the environment between the second sensor and the at least one first point;
assigns measured values, which include the at least one first distance value, in the acquired sensor data of the first sensor to corresponding measured values, which include the at least one second distance value, in the acquired sensor data from the second sensor such that the at least one first point in the environment is represented by at least the at least one first distance value as measured by the first sensor and the at least one second distance value as measured by the second sensor, the assignment taking into account respective imaging conditions of the two sensors;
compares the at least one first distance value to the at least one second distance value in order to determine a comparison distance measure indicative of a distance between a point on the at least one object at which the second sensor measures against and the at least one first point; and
determines whether a parallax problem exists in response to the comparison distance measure exceeding a threshold criterion value, and

wherein the output device outputs a comparison result of the comparison between the at least one first distance value and the at least one second distance value.

2. The apparatus of claim 1 , wherein the computing device further carries out the assignment of the measured values by back projection of the measured values of the sensor data of the first sensor into an image space formed from the measured values of the sensor data of the second sensor.

3. The apparatus of claim 1 , wherein the computing device is further designed that the sensor visual range information items are estimated based on estimated free zones and performs a pattern recognition on the sensor data of the second sensor to estimate the free zones.

4. A transportation vehicle comprising the apparatus of claim 1 .

5. The transportation vehicle of claim 4 , wherein the first sensor is a LIDAR sensor or a 3D radar sensor and the second sensor is a camera.

6. A method for detecting a parallax problem in sensor data from a first sensor and a second sensor, wherein the first and second sensors are spaced apart at different positions and at least partly capture the same environment, the first sensor configured to provide distance information items, the method comprising:
receiving the acquired sensor data from the first and second sensors, the acquired sensor data from the first sensor includes at least one first distance value for at least one first point in the environment based on the first sensor, the at least one first distance value measuring a distance from the first sensor to the at least one first point;
receiving or estimating visual range information items for the at least one first point from the second sensor based on sensor data acquired from the second sensor such that the computing device is configured to determine at least one second distance value for the at least one first point based on the second sensor, the at least one second distance value measuring a distance from the second sensor to the at least one first point or to at least one object located in the environment between the second sensor and the at least one first point;
assigning measured values, which include the at least one first distance value, in the acquired sensor data of the first sensor to corresponding measured values, which include the at least one second distance value in the acquired sensor data of the second sensor such that the at least one first point in the environment is represented by at least the at least one first distance value as measured by the first sensor and the at least one second distance value as measured by the second sensor, the assignment taking into account the respective imaging conditions of the two sensors;
comparing the at least one first distance value to the at least one second distance value in order to determine a comparison distance measure indicative of a distance between a point on the at least one object at which the second sensor measures against and the at least one first point;
determining whether a parallax problem exists in response to the comparison distance measure exceeding a threshold criterion value; and
outputting a comparison result of the comparison between the at least one first distance value and the at least one second distance value.

7. The method of claim 6 , wherein the assignment of the measured values is carried out by back projection of the measured values of the sensor data of the first sensor into an image space formed from the measured values of the sensor data of the second sensor.

8. The method of claim 6 , wherein the sensor visual range information items are estimated at least partly based on estimated free zones, wherein a pattern recognition is carried out on the sensor data of the second sensor to estimate the free zones.

9. The method of claim 8 , wherein an object recognition is carried out within the context of the pattern recognition and the free zones are estimated based on the at least one object recognized in the environment.

10. The method of claim 6 , wherein the sensor visual range information items of the second sensor are estimated based on sensor data of at least one other sensor.

Assignees

• Volkswagen Ag

Inventors

• Steinmeyer Simon
• Graf Thorsten
• Schlichte Michael

Classifications

• G01S17/931 Primary

  of land vehicles · CPC title

• G01C3/085

  with electronic parallax measurement · CPC title

• G01S17/894

  Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06T7/521

  from laser ranging, e.g. using interferometry; from the projection of structured light · CPC title