Error reduction of depth maps

Patent metadata

Field	Value
Publication number	US-11074704-B2
Application number	US-201916240596-A
Country	US
Kind code	B2
Filing date	Jan 4, 2019
Priority date	Jan 4, 2019
Publication date	Jul 27, 2021
Grant date	Jul 27, 2021

Abstract

The described implementations relate to reducing distance errors in depth maps. One example can detect blooming in a depth map. This example can identify a depth range affected by the blooming and take an action relating to the identified depth range to address the blooming.

First claim

The invention claimed is:

1. A method, comprising:
receiving a depth map of a scene and a corresponding active brightness image of the scene;
detecting blooming in the depth map at least in part from saturated pixels in the active brightness image;
identifying a depth range of the saturated pixels;
invalidating the depth range of the saturated pixels from the depth map; and,
generating another depth map that includes invalidated depths for the saturated pixels.

2. The method of claim 1 , wherein the receiving comprises receiving the depth map from a time-of-flight camera.

3. The method of claim 1 , wherein the invalidating the depth range of the saturated pixels comprises invalidating only saturated pixels.

4. The method of claim 1 , wherein the invalidating the depth range of the saturated pixels comprises invalidating saturated pixels and unsaturated pixels.

5. The method of claim 1 , wherein the detecting blooming in the depth map comprises making a binary determination whether blooming occurs in the depth map.

6. The method of claim 5 , wherein the making a binary determination is based at least in part upon a number of saturated pixels in the active brightness image.

7. The method of claim 6 , wherein the making a binary determination is based at least in part upon a number of saturated pixels in the active brightness image that are adjacent to one another.

8. The method of claim 1 , wherein the detecting comprises computing a set of all valid depth pixels that have a spatial neighbor that is saturated.

9. The method of claim 8 , wherein the identifying comprises generating a histogram of depth values for the set.

10. The method of claim 9 , further comprising smoothing the histogram.

11. The method of claim 10 , wherein the smoothing comprises replacing a value of individual bins of the histogram with an average value.

12. The method of claim 11 , wherein the replacing comprises calculating the average value as the value of an individual bin averaged with at least two adjacent neighboring bins.

13. A system, comprising:
a depth camera configured to capture a depth map and a corresponding active brightness image of a scene; and,
a processor configured to:
detect blooming in the depth map at least in part from saturated pixels in the active brightness image,
identify a depth range of the saturated pixels, invalidate the depth range of the saturated pixels from the depth map, and generate another depth map that includes invalidated depths for the depth range.

14. The system of claim 13 , wherein the depth camera and the processor are implemented on a single device.

15. The system of claim 13 , wherein the depth camera is manifest on a first device and the processor is implemented on a second device.

16. The system of claim 13 , wherein the processor is configured to identify the depth range by identifying saturated pixels and adjacent pixels of the depth map.

17. The system of claim 13 , wherein the processor is configured to perform the invalidating only on regions of the depth range that are aligned with saturated pixels in the active brightness image that corresponds to the depth map.

18. The system of claim 13 , wherein the depth camera and the processor are implemented on separate devices.

19. A device comprising:
a depth camera configured to generate a depth map and a corresponding active brightness image of a scene; and,
a bloom reduction computation component configured to detect blooming in the depth map based at least in part on saturated pixels of the active brightness image, identify a depth range of the saturated pixels, invalidate the depth range of the saturated pixels from the depth map, and generate another depth map that includes invalidated depths for the saturated pixels.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Bleyer Michael
• Edmonds Christopher D
• Pekelny Yuri

Classifications

• G06T7/0002 Primary

  Inspection of images, e.g. flaw detection · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06T2207/30168

  Image quality inspection · CPC title

• G06T5/40

  using histogram techniques · CPC title

• G06T2207/10048

  Infrared image · CPC title