Adaptive processing in time of flight imaging

Patent metadata

Field	Value
Publication number	US-11520052-B2
Application number	US-202016913676-A
Country	US
Kind code	B2
Filing date	Jun 26, 2020
Priority date	Jun 26, 2020
Publication date	Dec 6, 2022
Grant date	Dec 6, 2022

Abstract

Examples are disclosed herein relating to signal processing in a time-of-flight (ToF) system. One example provides, a method comprising emitting, via a light source, amplitude-modulated light toward an object, acquiring, via an image sensor comprising a plurality of pixels, a plurality of image frames capturing light emitted from the light source that is reflected by the object, wherein the plurality of image frames are acquired at two or more different frequencies of the amplitude-modulated light and collectively form a multifrequency frame, and for each pixel of the multifrequency frame, determining a brightness level, applying an adaptive denoising process by setting a kernel size based on the brightness level, and performing a phase unwrapping process to determine a depth value for the pixel.

First claim

The invention claimed is:

1. A method for signal processing in a time of flight system, the method comprising:
emitting, via a light source, amplitude-modulated light toward an object;
acquiring, via an image sensor comprising a plurality of pixels, a plurality of image frames capturing light emitted from the light source that is reflected by the object, wherein the plurality of image frames are acquired at two or more different frequencies of the amplitude-modulated light and collectively form a multifrequency frame; and
for each pixel of the multifrequency frame,
determining a brightness level,
applying an adaptive denoising process by setting a kernel size based on the brightness level, and
performing a phase unwrapping process to determine a depth value for the pixel.

2. The method of claim 1 , wherein setting the kernel size comprises setting a larger kernel for a lower brightness level, and setting a smaller kernel for a higher brightness level.

3. The method of claim 2 , wherein setting a smaller kernel comprises adding zeroes to a kernel.

4. The method of claim 1 , wherein applying the adaptive denoising process comprises applying:

S
⁡

(

m
,
n
,
k

)

=

∑

i
=

-
I

I

⁢

⁢

∑

j
=

-
J

J

⁢

⁢

λ
⁡

(

i
,
j
,
k

)

⁢

⁢

S
~

⁡

(

m
,
n
,
k

)

⁢

⁢
with
⁢

⁢

∑

i
=

-
I

I

⁢

⁢

∑

j
=

-
J

J

⁢

⁢

λ
⁡

(

i
,
j
,
k

)

=
1

wherein
{tilde over (S)}(m, n, k) is a signal corresponding to pixel (m,n) for a frequency k, of an array of size (M, N) with {m∈1, 2, . . . , M}, and {n∈1, 2, . . . , N}, for a frequency k∈1, 2, . . . , K,
S(m, n, k) is a signal with reduced noise, represented as a weighted sum of neighboring pixels, {i∈−I, −I+1, . . . , I} and {j∈−J, −J+1, . . . , J}, and
λ(i, j, k) are edge-preserving normalized weighting coefficients.

5. The method of claim 4 , wherein

λ
⁡

(

i
,
j
,
k

)

=

1

Norm
⁡

(

m
,
n
,
k

)

⁢

ξ
⁡

(

i
,
j
,
k

)

,

wherein

ξ
⁡

(

i
,
j
,
k

)

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Ortiz Egea Sergio

Classifications

• G01S17/894 Primary

  Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar · CPC title

• G01S7/484

  Transmitters · CPC title

• G01S17/26

  wherein the transmitted pulses use a frequency-modulated or phase-modulated carrier wave, e.g. for pulse compression of received signals · CPC title

• G01S7/4863

  Detector arrays, e.g. charge-transfer gates · CPC title

• G01S17/36

  with phase comparison between the received signal and the contemporaneously transmitted signal · CPC title