3D scanner using structured lighting

Patent metadata

Field	Value
Publication number	US-9982995-B2
Application number	US-201214117446-A
Country	US
Kind code	B2
Filing date	May 11, 2012
Priority date	May 24, 2011
Publication date	May 29, 2018
Grant date	May 29, 2018

Abstract

A series of structured lighting patterns are projected on an object. Each successive structured lighting pattern has a first and second subset of intensity features such as edges between light and dark areas. The intensity features of the first set coincide spatially with intensity features from either the first or second subset from a preceding structured lighting pattern in the series. Image positions are detected where the intensity features of the first and second subset of the structured lighting patterns are visible in the images. Image positions where the intensity features of the first subset are visible are associated with the intensity features of the first subset, based on the associated intensity features of closest detected image positions with associated intensity features in the image obtained with a preceding structured lighting pattern in said series. Image positions where the intensity features of the second subset are visible, between pairs of the image positions associated with intensity features of the first subset, with intensity features of the second subset between the intensity features associated with the pair of positions. The associated intensity features in a final structured lighting pattern of the series are used to identify the intensity features of the final structured lighting pattern for the determination of 3D surface position information.

First claim

The invention claimed is:

1. A method of determining 3D surface position information from an object using structured lighting, the method comprises:
providing a series of successive structured lighting patterns, each successive structured lighting pattern in the series comprising a first subset and a second subset of intensity features, each intensity feature of the first subset coinciding spatially with a respective one of the intensity features from either the first subset or the second subset from a preceding one of the successive structured lighting pattern in the series;
illuminating the object with different ones of the successive structured lighting patterns of said series each at least at a different time point;
capturing 2D images of the object, each when the object is illuminated with a respective one of the successive structured lighting patterns;
detecting image positions within the 2D images where the intensity features of the first and second subset of the successive structured lighting patterns are visible in the 2D images;
for each particular one of the successive structured lighting patterns in the series:
storing labels of the intensity features of the first subset of the particular one of the successive structured lighting patterns in combination with image positions within the 2D image captured using the particular one of the successive structured lighting patterns where the intensity features of the first subset are visible based on previously stored intensity feature label and image position combinations previously determined using a preceding one of the successive structured lighting patterns in said series and the 2D image captured using the preceding one of the successive structured lighting patterns in said series, and
associating intensity features of the second subset with image positions where the intensity features of the second subset are visible, in respective position ranges between pairs of the image positions associated with mutually adjacent intensity features of the first subset, the association being based on the associations of the mutually adjacent intensity features;
using the associations between intensity features and image positions in a final one of the successive structured lighting patterns of the series to identify the intensity features of the final one of the successive structured lighting patterns for the determination of 3D surface position information.

2. A method according to claim 1 , wherein the object is illuminated with the successive structured lighting patterns in a temporal sequence corresponding to their sequence in the series.

3. A method according to claim 1 , wherein the intensity features are edges between areas of mutually different light intensity in the successive structured lighting patterns.

4. A method according to claim 3 , wherein the intensity features of the first and second subset are edges in mutually opposite directions of intensity change along an image line.

5. A method according to claim 1 , wherein the method comprising selecting one image position where an intensity features of the second subset is visible between the image positions of one of the pairs, if a plurality of intensity features of the second subset is detected between the image positions of said one of the pairs, and associating the selected image position with the intensity features of the second subset between the intensity features associated with said one of the pairs of positions.

6. A method according to claim 1 , comprising:
assigning feature detection score values for detection of intensity features of the second subset to respective image positions,
selecting one intermediate image position for each of said pair of image positions, based on the assigned detection score values of a range of image positions between the image positions of the pair, and
associating the selected image position with the intensity features of the second subset between the intensity features associated with the pair of positions.

7. A method according to claim 1 , wherein for each successive structured lighting pattern the associating of the intensity features of the first subset with detected image positions is based on the intensity feature and image position association information determined using spatially closest detected image positions from 2D images captured using either an immediately preceding successive structured lighting pattern of the series or a further preceding successive structured lighting in the series.

8. A method according to claim 1 , comprising for a current successive structured lighting pattern in said series:
first associating image positions where the intensity features of the second subset are visible, between pairs of the image positions associated with intensity features of the first subset in the image obtained with a preceding successive structured lighting pattern in said series, with intensity features of the second subset between the intensity features associated with the pair of positions in the image obtained with the current structured lighting pattern in said series;
defining a search range between the image positions associated with successive intensity features of the second subset in the image obtained with the current successive structured lighting pattern in said series;
subsequently searching the search range for the image position where the intensity feature of the first subset is visible and associating that image position with the associated intensity features in the image obtained with the current successive structured lighting pattern in said series.

9. A method according to claim 1 , comprising
additionally repeating illumination of the object with one or more successive structured lighting patterns from only a part of the series when the association between image positions and the intensity features of at least one of the first and second subset has been lost,
capturing one or more further 2D images of the scene, each when the scene is illuminated with a respective one of said part of the successive structured lighting patterns; and
re-associating intensity features with image positions where the intensity features of the first subset are visible in the one or more further images.

10. A method according to claim 1 , comprising using the detected positions of the identified intensity features in the final structured lighting pattern of the series to determine 3D surface position information of object positions where the intensity features of the final successive structured lighting pattern intersect the object.

11. A method according to claim 1 , comprising
illuminating the object with a further structured lighting pattern that has a predetermined spatial relation to said final successive structured lighting pattern;
capturing a further image while the object is illuminated with the further structured lighting pattern at a time closer to a time of illumination of the object with the final successive structured lighting pattern than a time of illumination of the object with an initial successive structured lighting pattern in the series;
using the associated intensity features of positions obtained with the intensity features of the final successive structured lighting pattern and the predetermined spatial relation to identify detected intensity features in the further image;
using the detected positions of the identified intensity features obtained with the further structured lighting pattern to determine 3D surface position information of object positions where the intensity features of the further structured lighting pattern intersect the object.

12. A method according to claim

Assignees

• Znamenskiy Dmitry Nikolayevich
• Vlutters Ruud
• Van Bree Karl Catharina
• Koninklijke Philips Nv

Inventors

• Znamenskiy Dmitry Nikolayevich
• Vlutters Ruud
• Van Bree Karl Catharina

Classifications

• G06T2207/10012

  Stereo images · CPC title

• G01B11/2536

  using several gratings with variable grating pitch, projected on the object with the same angle of incidence · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06T7/586

  from multiple light sources, e.g. photometric stereo · CPC title

• G06T2207/10152

  Varying illumination · CPC title