Multipath interference error correction in a time of flight sensor

Patent metadata

Field	Value
Publication number	US-10914817-B2
Application number	US-201916370016-A
Country	US
Kind code	B2
Filing date	Mar 29, 2019
Priority date	Mar 29, 2019
Publication date	Feb 9, 2021
Grant date	Feb 9, 2021

Abstract

A three-dimensional (3D) sensor device achieves accurate distance measurements in the presence of multipath illumination interference (MPII) by estimating the effect of indirect illumination on a given pixel's optical signal measurement and correcting the signal measurement to remove the estimated contribution from indirect illumination. After generating an initial point cloud image of a scene, the sensor device identifies target objects and background objects within the scene, where the background objects are potential sources of indirect illumination. The sensor device estimates the total amount of indirect illumination directed to a point on the target object by the background objects and received at a pixel corresponding to the point, and corrects the measured signal at the pixel to remove the contribution of this estimated indirect illumination. The sensor device performs this MPII estimation without a priori knowledge of the background objects' characteristics, and is therefore response to changes in the background.

First claim

What is claimed is:

1. A sensor device, comprising:
an emitter component configured to emit light pulses into a monitored scene;
a photo-sensor component comprising an array of pixels, wherein respective pixels of the array of pixels are configured to convert a subset of the light pulses received from the monitored scene to electrical energy proportional to the subset of the light pulses received at the pixels;
a distance determination component configured to determine distance values associated with the respective pixels based on analysis of the electrical energy and to generate a point cloud image comprising the distance values; and
a multipath correction component configured to
identify a first portion of the point cloud image representing a target object,
identify a second portion of the point cloud image representing a background object,
model the background object as an array of elementary point sources based on the distance values of the point cloud image,
estimate amounts of indirect irradiance directed to a point on the target object by the elementary point sources and received at a pixel, of the array of pixels, corresponding to the point on the target object,
aggregate the amounts of indirect irradiance to yield a total estimated indirect irradiance directed to the point on the target object and received at the pixel, and
adjust a distance value, of the distance values, associated with the pixel based on the total estimated indirect irradiance to yield an updated distance value for the point on the target object.

2. The sensor device of claim 1 , wherein the multipath correction component is configured to define the array of elementary point sources as projections of fields of view of the array of pixels onto a surface of the background object.

3. The sensor device of claim 1 , wherein the multipath correction component is configured to,
for each elementary point source of the array of elementary point sources, determine a set of point source characteristics based on analysis of the point cloud image and a measured strength of an optical signal received at another pixel, of the array of pixels, corresponding to the elementary point source, and
estimate, as one of the amounts of indirect irradiance, an amount of indirect irradiance directed to the point on the target object by the elementary point source based on the set of point source characteristics.

4. The sensor device of claim 3 , wherein the set of point source characteristics comprise at least one of a distance of the elementary point source from the sensor device, a three-dimensional coordinate of the elementary point source within the point cloud image, a scattering area of the elementary point source, a normal vector to the elementary point source, or a product of a reflectivity index of the elementary point source and an incoming irradiance received at the elementary point source.

5. The sensor device of claim 3 , wherein the multipath correction component is configured to
determine a rotational angle of the target object based on analysis of the point cloud image, and
estimate the amount of indirect irradiance further based on the rotational angle.

6. The sensor device of claim 5 , wherein the multipath correction component is configured to determine the rotational angle of the target object based on analysis of a subset of the distance values corresponding to an area of the target object within a defined proximity of the point on the target object.

7. The sensor device of claim 1 , wherein the multipath correction component is configured to
translate the total estimated indirect irradiance to an electrical measure representing a contribution of the total estimated indirect irradiance to a total electrical energy generated by the pixel corresponding to the point on the target object,
subtract the electrical measure from the total electrical energy generated by the pixel to yield a corrected electrical measure, and
adjust the distance value associated with the pixel based on the corrected electrical measure to yield the updated distance value.

8. The sensor device of claim 1 , wherein the multipath correction component is further configured to
re-estimate the amounts of indirect irradiance based on the updated distance value to yield updated amounts of indirect irradiance,
aggregate the updated amounts of indirect irradiance to yield an updated total estimated indirect irradiance, and
adjust the updated distance value based on the updated total estimated indirect irradiance to yield a refined updated distance value.

9. The sensor device of claim 1 , further comprising a control output component configured to generate a safety output signal in response to determining that the updated distance value satisfies a defined criterion.

10. A method for correcting errors in measured distance values caused by multipath illumination interference, comprising:
emitting, by a sensor device comprising a processor, light pulses into a monitored area;
for respective pixels of a pixel array of the sensor device, generating, by the sensor device, electrical energy proportional to a subset of the light pulses received at the respective pixels from surfaces within the monitored area;
determining, by the sensor device based on measurements of the electrical energy for the respective pixels, distance values associated with the respective pixels;
generating, by the sensor device, a point cloud image comprising the distance values;
identifying, by the sensor device, a target object represented by a first portion of the point cloud image;
identifying, by the sensor device, a background object represented by a second portion of the point cloud image;
modeling, by the sensor device, an array of elementary point sources across a surface of the background object based on analysis of the second portion of the point cloud image;
estimating, by the sensor device, respective levels of indirect irradiance directed to a point on the target object by the elementary point sources and received at a pixel, of the array of pixels, corresponding to the point on the target object;
integrating, by the sensor device, the levels of indirect irradiance to yield a total estimated indirect irradiance directed to the point on the target object and received at the pixel; and
modifying, by the sensor device, a distance value, of the distance values, associated with the pixel based on the total estimated indirect irradiance to yield an updated distance value for the point on the target object.

11. The method of claim 10 , wherein the modeling comprises
modeling projections of respective fields of view of the array of pixels onto the surface of the background object, and
defining the array of elementary point sources as intersections of the respective fields of view with the surface of the background object.

12. The method of claim 10 , wherein the estimating the respective levels of indirect irradiance comprises:
for each elementary point source of the array of elementary point sources, determining characteristics of the elementary point source based on analysis of the point cloud image and a measured strength of an optical signal received at another pixel, of the array of pixels, corresponding to the elementary point source, and
estimating, as one of the levels of indirect radiance, a level of indirect irradiance directed to the point on the target object by the elementary point source based on the characteristics of the elementary point source.

13. The method of claim 12 , wherein the determining the characteristics comprises determining at least one of a distance of the elementary point sou

Assignees

• Rockwell Automation Tech Inc

Inventors

• Boutaud Frederic

Classifications

• G01S17/894 Primary

  Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar · CPC title

• G06T2207/20076

  Probabilistic image processing · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06T7/521

  from laser ranging, e.g. using interferometry; from the projection of structured light · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title