Depth imaging system based on stereo vision and infrared radiation

Patent metadata

Field	Value
Publication number	US-9928420-B2
Application number	US-201313925762-A
Country	US
Kind code	B2
Filing date	Jun 24, 2013
Priority date	Apr 15, 2013
Publication date	Mar 27, 2018
Grant date	Mar 27, 2018

Abstract

The subject disclosure is directed towards a high resolution, high frame rate, robust stereo depth system. The system provides depth data in varying conditions based upon stereo matching of images, including actively illuminated IR images in some implementations. A clean IR or RGB image may be captured and used with any other captured images in some implementations. Clean IR images may be obtained by using a notch filter to filter out the active illumination pattern. IR stereo cameras, a projector, broad spectrum IR LEDs and one or more other cameras may be incorporated into a single device, which may also include image processing components to internally compute depth data in the device for subsequent output.

First claim

What is claimed is:

1. A system comprising:
a projector configured to illuminate a scene with an unknown light pattern, the unknown light pattern emitted at a predetermined infrared (IR) wavelength;
a first IR camera configured to capture a first image of the scene illuminated with the unknown light pattern;
a second IR camera configured to capture a second image of the scene illuminated with the unknown light pattern;
an optical notch filter configured to block the predetermined IR wavelength and pass other wavelengths of light;
a third IR camera coupled to the optical notch filter, the third IR camera configured to capture a third image of the scene illuminated with the unknown light pattern, the third image comprising an IR image without the unknown light pattern; and
an image processing system configured to compute depth data based upon the first, second, and third images captured by the first, second, and third IR cameras, and output the depth data from the device.

2. The system of claim 1 further comprising a visible light spectrum camera configured to capture a color image of the scene.

3. The system of claim 2 , wherein the third IR camera and the visible light spectrum camera are combined into a single camera.

4. The system of claim 1 further comprising at least one IR light source configured to illuminate the scene with at least some IR light at a second IR wavelength that is not filtered out by the optical notch filter.

5. The system of claim 1 , wherein at least one of the first, second, and third IR cameras is coupled to a narrow bandpass filter.

6. The system of claim 1 , wherein at least two of the first, second, and third IR cameras are arranged as stereo IR cameras.

7. The system of claim 6 , wherein the projector is positioned between the stereo IR cameras.

8. The system of claim 6 further comprising an additional camera configured to capture a fourth image of the scene, wherein the additional camera is positioned adjacent one of the first and second IR cameras.

9. The system of claim 6 further comprising an IR light source configured to illuminate the scene with at least some IR light at a second IR wavelength that is not filtered out by the optical notch filter, wherein the IR light source is positioned adjacent one of the first and second IR cameras, and the third IR camera is positioned adjacent the IR light source.

10. The system of claim 6 further comprising first and second IR light sources configured to illuminate the scene with at least some IR light at a second wavelength that is not filtered out by the optical notch filter, wherein the first IR light source is positioned adjacent one of the first and second IR cameras, and the third IR camera is positioned between the first IR light source and the second IR light source.

11. The system of claim 1 wherein first, second, and third IR cameras and projector are incorporated into a single device.

12. The system of claim 11 wherein the device includes an image processing system configured to compute the depth data within the device.

13. The system of claim 1 wherein the first, second, and third IR cameras and the projector are arranged linearly in a single device, and wherein the device comprises at least one IR light source configured to illuminate the scene with at least some IR light at a second wavelength that is not filtered out by the optical notch filter.

14. A system comprising:
a plurality of infrared (IR) cameras configured to sense stereo images of a scene illuminated with an unknown IR light pattern, the unknown IR light pattern emitted at a predetermined IR wavelength;
a plurality of narrow bandpass filters corresponding to the plurality of IR cameras, wherein a respective narrow bandpass filter of the plurality of narrow bandpass filters is coupled to a respective IR camera of the plurality of IR cameras, and wherein the plurality of narrow bandpass filters pass the predetermined IR wavelength and block other IR wavelengths of light;
a notch-filtered camera configured with a notch filter to capture a clean image of the scene illuminated with the unknown IR light pattern, the clean image comprising an IR image without the unknown IR light pattern; and
an image processor configured to compute depth data based upon the stereo images captured by the plurality of IR cameras and the clean image captured by the notch-filtered camera, and output the depth data from the device.

15. The system of claim 14 further comprising a projector configured to illuminate the scene with the unknown IR light pattern.

16. The system of claim 15 , wherein the plurality of IR cameras, the notch-filtered camera, and the projector are arranged linearly in a device, and wherein the projector is positioned between two of the plurality of IR cameras.

17. The system of claim 14 , wherein the notch-filtered camera configured with the notch filter is further configured to capture a visible light spectrum color image.

18. The system of claim 14 further comprising at least one IR light source that emits at least some IR light at a second wavelength that is not filtered out by the notch filter.

19. The system of claim 14 , wherein the notch filter is configured to block the predetermined IR wavelength and pass visible and other IR wavelengths of light.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Kirk Adam G
• Whyte Oliver A
• Kang Sing Bing
• Zitnick Iii Charles Lawrence
• Szeliski Richard S
• Izadi Shahram
• Rhemann Christoph
• Georgiou Andreas
• Bhattacharjee Avronil

Classifications

• G06F2218/12

  Classification; Matching · CPC title

• H04N23/56

  provided with illuminating means · CPC title

• H04N23/11

  for generating image signals from visible and infrared light wavelengths · CPC title

• H04N25/611

  Correction of chromatic aberration · CPC title

• H04N2013/0081

  Depth or disparity estimation from stereoscopic image signals · CPC title