Color noise reduction in 3D depth map

Patent metadata

Field	Value
Publication number	US-10536684-B2
Application number	US-201615371433-A
Country	US
Kind code	B2
Filing date	Dec 7, 2016
Priority date	Dec 7, 2016
Publication date	Jan 14, 2020
Grant date	Jan 14, 2020

Abstract

A camera based depth mapping system. Depth information is coded with colors to make a laser-generated 3D depth map easier to interpret. In the event that the laser illumination is not sufficient, the depth information can have a low signal to noise ratio, i.e., the depth map can be noisy. Color noise reduction techniques are used to alleviate this problem.

First claim

What is claimed is:

1. A device comprising:
at least one computer medium that is not a transitory signal and that comprises instructions executable by at least one processor to:
generate, using at least one laser, a three dimensional (3D) depth map;
apply color noise reduction to the depth map to increase a signal-to-noise ratio (SNR) of the depth map compared to a SNR of the depth map prior to application of noise reduction;
access a transmission schedule; and
transmit laser range-finding beams using the at least one laser at transmission times according to the transmission schedule to generate the depth map.

2. The device of claim 1 , comprising the at least one processor.

3. The device of claim 2 , comprising at least one camera providing signals to the at least one processor representative of reflections of the laser range-finding beams.

4. The device of claim 1 , wherein the instructions are executable to synchronize at least one computer clock of the device with a heartbeat signal of a master device in an optical micromesh that includes the device and at least one other device also synchronized to the heartbeat signal, transmission of laser range-finding beams according to the transmission schedule being executed at least in part by accessing the computer clock.

5. The device of claim 1 , wherein the instructions are executable to assign slots in the transmission schedule to other devices in an optical micromesh.

6. The device of claim 1 , wherein the depth map is of at least one object, and the instructions are executable to:
access a color image of the object; and
based at least in part on the color image, augment the depth map with color.

7. The device of claim 1 , wherein the depth map is of at least one object, and the instructions are executable to:
access default colors; and
based at least in part on the defeat colors, augment the depth map with color.

8. An assembly comprising:
plural computerized devices;
each computerized device comprising at least one laser emitter configured to output signals useful for generating at least one three dimensional (3D) depth map; and
at least one processor programmed with instructions to augment at least one of the 3D depth maps with color and to apply noise reduction to 3D depth maps augmented with color to reduce noise in the depth map augmented with color.

9. The assembly of claim 8 , wherein the devices are programmed with instructions to emit laser beams to establish the at least one 3D depth map using time division emission rules.

10. The assembly of claim 8 , comprising at least one camera providing signals to at least one processor representative of reflections of the laser range-finding beams.

11. The assembly of claim 8 , wherein at least one of the depth maps is of at least one object, and the instructions are executable to:
access a color image of the object; and
based at least in part on the color image, augment the depth map with color.

12. The assembly of claim 8 , wherein at least one of the depth maps is of at least one object, and the instructions are executable to:
access default colors; and
based at least in part on the defeat colors, augment the depth map with color.

13. A method comprising:
emitting laser emissions;
using reflections of the emissions, generating a three dimensional (3D) depth map; and
applying color to the 3D depth map to render an augmented depth map by associating a first color with a first region of the 3D depth map representing a “closest” surface and associating a second color with at least a second region of the 3D depth map on the basis that the first color is warmer than the second color.

14. The method of claim 13 , comprising:
accessing a transmission schedule; and
transmitting the laser emissions at transmission times according to the transmission schedule.

15. The method of claim 13 , comprising receiving signals from at least one camera representative of reflections of the laser emissions.

16. The method of claim 14 , comprising synchronizing at least one computer clock with a heartbeat signal of a master device in an optical micromesh, transmission of laser range-finding beams according to the transmission schedule being executed at least in part by accessing the computer clock.

17. The method of claim 14 , comprising assigning slots in the transmission schedule to other devices in an optical micromesh.

18. The method of claim 13 , wherein the depth map is of at least one object, and the method comprises:
accessing a color image of the object; and
based at least in part on the color image, augmenting the depth map with color.

19. The method of claim 13 , wherein the depth map is of at least one object, and the method comprises:
accessing default colors; and
based at least in part on the defeat colors, augmenting the depth map with color.

Assignees

• Sony Corp
• Sony Interactive Entertainment Inc

Inventors

• Shintani Peter
• Gonzalez Hernandez Jose Omar
• Clay William
• Espinosa Pablo Antonio
• Ansfield Fred
• Mohapatra Bibhudendu
• Resch Keith
• Usami Morio

Classifications

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• H04N13/128 Primary

  Adjusting depth or disparity · CPC title

• H04N13/15

  for colour aspects of image signals · CPC title

• G06T2207/10024

  Color image · CPC title

• G06T7/521

  from laser ranging, e.g. using interferometry; from the projection of structured light · CPC title