Multispectral binary coded projection using multiple projectors

Patent metadata

Field	Value
Publication number	US-10012496-B2
Application number	US-201514927248-A
Country	US
Kind code	B2
Filing date	Oct 29, 2015
Priority date	Oct 29, 2015
Publication date	Jul 3, 2018
Grant date	Jul 3, 2018

Abstract

A method of measuring a depth map for an object or a material property of an object involves positioning multiple projectors at respectively multiple angular positions relative to an object. Each of the projectors comprises color channels whose colors differ from that of other projectors. An image capture device is positioned relative to the multiple projectors. A combined color pattern is projected onto the object using the multiple projectors. Each projector projects a color striped pattern whose resolution differs from that of others of the projectors and the resolution of the color striped pattern is related to the angular position of the projector. An image of the object is captured with the combined color pattern using the image capture device and a depth map or a material property of the object is recovered by calculations using the captured image.

First claim

The invention claimed is:

1. A method of measuring a depth map for an object, comprising:
positioning multiple projectors at respectively multiple angular positions relative to the object, wherein each of the projectors comprises color channels whose colors differ from that of others of the multiple projectors;
positioning an image capture device relative to the multiple projectors, wherein the image capture device comprises a multispectral capture device having a capture sensitivity spectral range that includes the spectral range of the totality of colors of the multiple projectors;
projecting a first combined color pattern onto the object using the multiple projectors, wherein each projector projects a color striped pattern whose resolution differs from that of others of the projectors, and wherein the resolution of the color striped pattern is related to the angular position of the projector and is characterized by a minimum stripe width such that a small minimum stripe width corresponds to a high resolution pattern and a large minimum stripe width corresponds to a low resolution pattern;
capturing an image of the object with the first combined color pattern using the image capture device; and
recovering the depth map of the object by calculations using the captured image.

2. The method according to claim 1 , wherein the resolution of the color striped pattern a projector projects is inversely related to the angle between the projector and the camera.

3. A method of measuring a depth map for an object, comprising:
positioning multiple projectors at respectively multiple angular positions relative to the object, wherein each of the projectors comprises color channels whose colors differ from that of others of the multiple projectors;
positioning an image capture device relative to the multiple projectors, wherein the image capture device comprises a multispectral capture device having a capture sensitivity spectral range that includes the spectral range of the totality of colors of the multiple projectors;
projecting a first combined color pattern onto the object using the multiple projectors, wherein each projector projects a color striped pattern whose resolution differs from that of others of the projectors and wherein the resolution of the color striped pattern is related to the angular position of the projector;
capturing an image of the object with the first combined color pattern using the image capture device;
projecting a second combined color pattern onto the object using the multiple projectors, the second combined color pattern differing from the first combined color pattern, wherein each projector projects a color striped pattern whose resolution differs from that of others of the projectors and wherein the resolution of the color striped pattern is related to the angular position of the projector;
capturing a second image of the object illuminated by the second combined color pattern by using the image capture device; and
recovering the depth map of the object by analyzing the captured first image and the captured second image.

4. The method according to claim 3 , wherein the second combined color pattern is inverted in color values relative to the first combined color pattern.

5. The method according to claim 1 , wherein each projector projects a periodic color striped pattern that partitions depth values along a line emanating from the image capture device into periods of ranges of depth values, and wherein a range of depth values for a projector projecting a periodic color striped pattern is contained in a full period of ranges of depth values for one of the other projectors projecting a periodic color striped pattern at a higher resolution.

6. The method according to claim 5 , wherein the depth map of the object is recovered by determining a range of depth values corresponding to a stripe position in the projector projecting the highest resolution color striped pattern.

7. The method according to claim 6 , wherein determining a range of depth values corresponding to a stripe position in the projector projecting the highest resolution color striped pattern includes recursively determining a range of depth values corresponding to a stripe position in a projector projecting a lower resolution color striped pattern.

8. The method according to claim 1 , wherein the multiple projectors include at least first, second and third projectors,
wherein the first projector comprises spectrally relatively narrow color channels with at least three characteristic peak wavelengths λ 1 , λ 2 , λ 3 , the second projector comprises spectrally relatively narrow color channels with at least three characteristic peak wavelengths λ 4 , λ 5 , λ 6 , the third projector comprises spectrally relatively narrow color channels with at least three characteristic peak wavelengths λ 7 , λ 8 , λ 9 , and all of the λ i 's are different, and
wherein the image capture device comprises at least nine color channels each with spectrally relatively broad capture sensitivity such that the combined capture sensitivity spectral range for all the color channels includes all of the λ i 's.

9. The method according to claim 1 , further comprising estimating a material property of the object from the recovered depth map by determining surface normal vectors from the depth map, calculating an empirical reflectance map from the determined surface normal vectors and the captured image, and fitting a reflectance model to the calculated empirical reflectance map so as to obtain an estimate of a material property parameter.

10. The method according to claim 9 , wherein the reflectance model being fitted is a bidirectional reflectance distribution function (BRDF) model, and wherein a fitted model parameter of the BRDF model provides an estimate of a material property of the object.

11. A system for measuring a depth map for an object comprising:
a projector positioning unit constructed to fix multiple projectors at respectively multiple angular positions relative to the object, wherein each of the projectors comprises color channels whose colors differ from that of others of the multiple projectors;
a camera positioning unit constructed to fix a position of an image capture device relative to the multiple projectors, wherein the image capture device comprises a multispectral capture device having a capture sensitivity spectral range that includes the spectral range of the totality of colors of the multiple projectors;
a projection unit constructed to project a first combined color pattern onto the object using the multiple projectors, wherein each projector projects a color striped pattern whose resolution differs from that of others of the projectors, and wherein the resolution of the color striped pattern is related to the angular position of the projector and is characterized by a minimum stripe width such that a small minimum stripe width corresponds to a high resolution pattern and a large minimum stripe width corresponds to a low resolution pattern;
an image capturing unit constructed to capture an image of the object with the first combined color pattern using the image capture device; and
a depth recovery unit constructed to recover the depth map of the object by calculations using the captured image.

12. A non-transitory computer-readable storage medium storing a program for causing a computer to implement the method according to claim 1 .

13. A non-transitory computer-readable storage medium storing a program for causing a computer to implement the method according to claim 3 .

14. A system for measuring a depth map for an object, comprising:
a projector

Assignees

• Canon Kk

Inventors

• Tin Siu-Kei

Classifications

• H04N9/045

  Electricity · mapped topic

• G06T7/514 Primary

  from specularities · CPC title

• G06T7/55

  from multiple images · CPC title

• H04N2209/047

  using multispectral pick-up elements · CPC title

• G06T7/521

  from laser ranging, e.g. using interferometry; from the projection of structured light · CPC title