Active stereo matching for depth applications

Patent metadata

Field	Value
Publication number	US-2023069179-A1
Application number	US-202217981781-A
Country	US
Kind code	A1
Filing date	Nov 7, 2022
Priority date	Mar 22, 2018
Publication date	Mar 2, 2023
Grant date	—

Abstract

A head-mounted device (HMD) is configured to perform depth detection with a stereo camera pair comprising a first camera and a second camera, both of which are configured to detect/capture visible light and IR light. The fields of view for both of the cameras overlap to form an overlapping field of view. The HMD also includes an IR dot-pattern illuminator that is mounted on the HMD with the cameras and that is configured to emit an IR dot-pattern illumination. The IR dot-pattern illuminator emits a dot-pattern illumination that spans at least a part of the overlapping field of view. The IR dot-pattern illumination adds texture to objects in the environment and enables the HMD to determine depth for those objects, even if they have textureless/smooth surfaces.

First claim

What is claimed is:

1 . A system that adjusts a depth map in an attempt to eliminate a potential jitter effect when neighboring pixels in the depth map have depth values and when a difference between those depth values exceeds a difference threshold, the system comprising:
at least one processor; and at least one hardware storage device that stores instructions that are executable by the at least one processor to cause the system to:
access a depth map that includes a plurality of pixels comprising depth values;
identify, from within the depth map, a pair of neighboring pixels comprising a first pixel having a first depth value and a second pixel having a second depth value;
determine that a difference between the first depth value and the second depth value exceeds a predefined difference threshold; and
based on said determination that the difference between the first depth value and the second depth value exceeds the predefined difference threshold, adjust the depth map to eliminate a potential for a jitter effect that potentially occurs in depth maps when neighboring depth pixels have depth values that exceed the predefined difference threshold.

2 . The system of claim 1 , wherein adjusting the depth map includes replacing the second depth value with a new depth value.

3 . The system of claim 2 , wherein the new depth value is selected such that a difference between the new depth value and the first depth value does not exceed the predefined difference threshold.

4 . The system of claim 1 , wherein adjusting the depth map includes imposing a filter on the depth map.

5 . The system of claim 1 , wherein the system includes a pattern illuminator that illuminates an observable pattern.

6 . The system of claim 5 , wherein an image of the observable pattern is stored as a reference.

7 . The system of claim 1 , wherein at least one of the first pixel or the second pixel is partially representative of an object that includes a textureless surface.

8 . The system of claim 1 , wherein the depth map is generated using stereo depth imaging.

9 . A method for adjusting a depth map in an attempt to eliminate a potential jitter effect when neighboring pixels in the depth map have depth values and when a difference between those depth values exceeds a difference threshold, the method comprising:
accessing a depth map that includes a plurality of pixels comprising depth values; identifying, from within the depth map, a pair of neighboring pixels comprising a first pixel having a first depth value and a second pixel having a second depth value; determining that a difference between the first depth value and the second depth value exceeds a predefined difference threshold; and based on said determination that the difference between the first depth value and the second depth value exceeds the predefined difference threshold, adjusting the depth map to eliminate a potential for a jitter effect that potentially occurs in depth maps when neighboring depth pixels have depth values that exceed the predefined difference threshold.

10 . The method of claim 9 , wherein a geometric surface is constructed for an environment using the depth map.

11 . The method of claim 9 , wherein the method is performed by a mixed-reality (MR) system.

12 . The method of claim 9 , wherein adjusting the depth map includes replacing the first depth value with a new depth value.

13 . The method of claim 12 , wherein the new depth value is selected such that a difference between the new depth value and the second depth value does not exceed the predefined difference threshold.

14 . The method of claim 9 , wherein a filter is used to adjust the depth map.

15 . The method of claim 9 , wherein the depth map is generated based on a plurality of images that are generated by head tracking cameras.

16 . The method of claim 9 , wherein the depth map is generated based on a plurality of images that are generated by hand tracking cameras.

17 . A method for adjusting a depth map in an attempt to eliminate a potential jitter effect when neighboring pixels in the depth map have depth values and when a difference between those depth values exceeds a difference threshold, the method comprising:
accessing a depth map that includes a plurality of pixels comprising depth values; identifying, from within the depth map, a set of pixels comprising a first pixel having a first depth value and a second pixel having a second depth value; determining that a difference between the first depth value and the second depth value exceeds a predefined difference threshold; and based on said determination that the difference between the first depth value and the second depth value exceeds the predefined difference threshold, adjusting the depth map to eliminate a potential for a jitter effect that potentially occurs in depth maps when certain selected depth pixels have depth values that exceed the predefined difference threshold.

18 . The method of claim 17 , wherein adjusting the depth map includes replacing the second depth value with a new depth value, and wherein the new depth value is selected such that a difference between the new depth value and the first depth value does not exceed the predefined difference threshold.

19 . The method of claim 17 , wherein the first pixel is a neighbor to the second pixel in the depth map.

20 . The method of claim 17 , wherein at least one of the first pixel or the second pixel is partially representative of an object that includes a textureless surface.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Bleyer Michael
• Pekelny Yuri
• Price Raymond Kirk

Classifications

• H04N23/11

  for generating image signals from visible and infrared light wavelengths · CPC title

• G06F3/0346

  with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors · CPC title

• G06T15/04

  Texture mapping · CPC title

• G06T7/593

  from stereo images · CPC title

• H04N13/271 Primary

  wherein the generated image signals comprise depth maps or disparity maps · CPC title