Edge preserving depth filtering

Patent metadata

Field	Value
Publication number	US-9584790-B2
Application number	US-201313908923-A
Country	US
Kind code	B2
Filing date	Jun 3, 2013
Priority date	Jun 3, 2013
Publication date	Feb 28, 2017
Grant date	Feb 28, 2017

Abstract

A scene is illuminated with modulated illumination light that reflects from surfaces in the scene as modulated reflection light. Each of a plurality of pixels of a depth camera receive the modulated reflection light and observe a phase difference between the modulated illumination light and the modulated reflection light. For each of the plurality of pixels, an edginess of that pixel is recognized, and the phase difference of that pixel is smoothed as a function of the edginess of that pixel.

First claim

The invention claimed is:

1. A method of filtering depth information, the method comprising:
illuminating a scene with a modulated illumination light having a plurality of different modulation frequencies, the modulated illumination light reflecting from surfaces in the scene as modulated reflection light;
receiving at each of a plurality of pixels the modulated reflection light;
observing a phase difference between the modulated illumination light and the modulated reflection light at each of the plurality of pixels for each of the modulation frequencies;
for each of the plurality of pixels and modulation frequencies, recognizing an edginess of that pixel corresponding to a relative difference between the phase difference of that pixel and phase differences of one or more neighboring pixels; and
for each of the plurality of pixels and modulation frequencies, smoothing the phase difference of that pixel relative to phase differences of one or more neighboring pixels as a function of the edginess of that pixel such that greater edginess corresponds to less smoothing and lesser edginess corresponds to more smoothing.

2. The method of claim 1 , wherein recognizing an edginess of a pixel includes calculating an Edge i of the pixel relative to a neighboring pixel, such that

Edge
i

=

(

1
-

P
i

·

P
j



P
i



⁢



P
j



)

2

,

where P i is a vector with an angle equal to a phase difference of the pixel, and P j is a vector with an angle equal to a phase difference of the neighboring pixel.

3. The method of claim 2 , wherein recognizing an edginess of the pixel includes calculating an Edge ij of the pixel relative to each pixel in a set N of neighboring pixels, such that

Edge
ij

=

∑

j
∈
N

⁢

⁢

(

1
-

P
i

·

P
j



P
i



⁢



P
j



)

2

.

4. The method of claim 3 , wherein smoothing the phase difference of the pixel relative to phase differences of one or more neighboring pixels includes smoothing with an edge-preserving factor of e −k×Edge ij , where k is a tunable constant.

5. The method of claim 4 , wherein smoothing the phase difference of the pixel relative to phase differences of one or more neighboring pixels includes Gaussian smoothing the phase difference.

6. The method of claim 1 , wherein smoothing the phase difference of the pixel relative to phase differences of one or more neighboring pixels includes Gaussian smoothing the phase difference.

7. The method of claim 6 , wherein smoothing the phase difference of the pixel relative to phase differences of one or more neighboring pixels includes Gaussian smoothing with an edge-preserving factor.

8. The method of claim 1 , further comprising noise filtering.

9. The method of claim 8 , further comprising noise filtering smoothed phase differences.

10. The method of claim 8 , wherein noise filtering includes ignoring pixels having noisy phase differences.

11. The method of claim 8 , wherein noise filtering includes filtering pixels having an edge intensity greater than a tunable threshold.

12. A method of filtering depth information, the method comprising:
illuminating a scene with a modulated illumination light, the modulated illumination light reflecting from surfaces in the scene as modulated reflection light;
receiving at each of a plurality of pixels the modulated reflection light;
observing a phase difference between the modulated illumination light and the modulated reflection light at each of the plurality of pixels;
for each of the plurality of pixels, calculating an Edge ij of the pixel relative to each pixel in a set of N neighboring pixels, such that

Edge
ij

=

∑

j
∈
N

⁢

⁢

(

1
-

P
i

·

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Barsoum Emad
• Peeper Craig

Classifications

• G01S7/4912 Primary

  Receivers · CPC title

• H04N13/122 Primary

  Improving the three-dimensional [3D] impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues (H04N13/128 takes precedence) · CPC title

• G01S17/894

  Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar · CPC title

• H04N13/0018 Primary

  Electricity · mapped topic

• G01S17/89

  for mapping or imaging · CPC title