Multipath mitigation for time of flight system

Patent metadata

Field	Value
Publication number	US-11029149-B2
Application number	US-201916262440-A
Country	US
Kind code	B2
Filing date	Jan 30, 2019
Priority date	Jan 30, 2019
Publication date	Jun 8, 2021
Grant date	Jun 8, 2021

Abstract

A time-of-flight (ToF) system disclosed herein provides a method of a method of separating a direct component of light collected by a time of flight (ToF) detector from a global component of light collected by the ToF detector, the method comprising acquiring three or more images represented by three or more matrices in response to illuminating a target with a light source using a first spatial pattern at three or more different modulation frequencies, acquiring an additional image represented by an additional matrix in response to illuminating the target with the light source using a second spatial pattern, the second spatial pattern being different than the first spatial pattern, and determining one or more parameters of the direct component of light and the global component of light based on analysis of the three or more matrices and the additional matrix.

First claim

What is claimed is:

1. A physical hardware system to provide multipath mitigation in a time-of-flight (ToF) system, comprising:
a camera configured to acquire k images, k being two or more, represented by k matrices in response to illuminating a target with a light source using a first spatial pattern at k different light frequencies and to acquire an additional k+1 th image represented by an additional matrix in response to illuminating the target with the light source using a second spatial pattern, the second spatial pattern being different than the first spatial pattern;
a denoising module configured to generate a denoising coefficient matrix based on a kernel matrix applied to each of the k+1 images; and
an image analyzer to apply the denoising coefficient to the two or more matrices and the additional k+1 th matrix and to determine one or more parameters of a direct component of light and a global component of light received at the camera from the target.

2. The physical hardware system of claim 1 , wherein the first spatial pattern is a uniform spatial pattern.

3. The physical hardware system of claim 1 , wherein the second spatial pattern is a non-uniform spatial pattern.

4. The physical hardware system of claim 1 , wherein each of the first spatial pattern and the second spatial pattern is a non-uniform spatial pattern.

5. The physical hardware system of claim 1 , wherein the additional image is acquired in response to illuminating the target with the light source using the second spatial pattern at a light frequency similar to one of the two or more different light frequencies used when illuminating the target with the first spatial pattern.

6. The physical hardware system of claim 1 , wherein the camera is further configured to acquire three or more images represented by three or more matrices in response to illuminating the target with the light source using the first spatial pattern at three or more different light frequencies.

7. The physical hardware system of claim 1 , wherein the non-uniform spatial pattern is at least one of a dot-pattern, a vertical-line pattern, and a horizontal line pattern.

8. The physical hardware system of claim 1 , wherein the kernel matrix is an m×n matrix with m being different than n.

9. A method of separating a direct component of light collected by a time of flight (ToF) detector from a global component of light collected by the ToF detector, the method comprising:
acquiring k images, k being two or more, represented by k matrices in response to illuminating a target with a light source using a first spatial pattern at k different modulation frequencies;
acquiring an additional k+1 th image represented by an additional matrix in response to illuminating the target with the light source using a second spatial pattern, the second spatial pattern being different than the first spatial pattern;
generate a denoising coefficient matrix based on a kernel matrix applied to each of the k+1 images; and
applying the denoising coefficient to the two or more matrices and the additional k+1 th matrix and to determine one or more parameters of the direct component of light and the global component of light based on analysis of the three or more matrices and the additional matrix.

10. The method of claim 9 , wherein the first spatial pattern is a uniform spatial pattern.

11. The method of claim 9 , wherein the second spatial pattern is a non-uniform spatial pattern.

12. The method of claim 9 , wherein each of the first spatial pattern and the second spatial pattern is a non-uniform spatial pattern.

13. The method of claim 9 , wherein the additional image is acquired in response to illuminating the target with the light source using the second spatial pattern at a light frequency similar to a one of the three or more different light frequencies used when illuminating the target with the first spatial pattern.

14. The method of claim 9 , wherein the non-uniform spatial pattern is at least one of a dot-pattern, a vertical-line pattern, and a horizontal line pattern.

15. The method of claim 9 , wherein the kernel matrix is an m×n matrix with m being different than n.

16. A physical article of manufacture including one or more non-transitory computer-readable storage media, encoding computer-executable instructions for executing on a computer system a computer process, the computer process comprising:
acquiring k images, k being two or more, represented by k matrices in response to illuminating a target with a light source using a first spatial pattern at k different modulation frequencies;
acquiring an additional k+1 th image represented by an additional matrix in response to illuminating the target with the light source using a second spatial pattern, the second spatial pattern being different than the first spatial pattern;
generating a denoising coefficient matrix based on a kernel matrix applied to each of the k+1 images; and
applying the denoising coefficient to the two or more matrices and the additional k+1 th matrix and to determine one or more parameters of the direct component of light and the global component of light based on analysis of the three or more matrices and the additional matrix.

17. The physical article of manufacture of claim 16 , wherein the first spatial pattern is a uniform spatial pattern and the second spatial pattern is a non-uniform spatial pattern.

18. The physical article of manufacture of claim 17 , wherein the process further comprising performing a frequency conversion operation to transform a phase into a frequency K.

19. The physical article of manufacture of claim 17 , wherein each of the first spatial pattern and the second spatial pattern is a non-uniform spatial pattern.

20. The physical article of manufacture of claim 19 , wherein the light source illuminates a target using N temporal modulation frequencies with the uniform spatial pattern and N temporal modulation frequencies with the non-uniform spatial pattern.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Ortiz Egea Sergio
• Lin Hung-Ming

Classifications

• H04N23/56

  provided with illuminating means · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G01S7/493

  Extracting wanted echo signals · CPC title

• G06T2207/10152

  Varying illumination · CPC title

• G06T5/50

  using two or more images, e.g. averaging or subtraction · CPC title