3D shape reconstruction using reflection onto electronic light diffusing layers

Patent metadata

Field	Value
Publication number	US-9818234-B2
Application number	US-201615072116-A
Country	US
Kind code	B2
Filing date	Mar 16, 2016
Priority date	Mar 16, 2016
Publication date	Nov 14, 2017
Grant date	Nov 14, 2017

Abstract

3D shape reconstruction of objects including specular objects in which structured light patterns are projected toward an object which reflects the patterns onto multiple layers of EGlass arranged in spaced-apart relation with each other while each layer is in turn controlled to be in the diffuse mode with all others being in the transparent mode. Images are captured of the structured light pattern as reflected onto the EGlass layers by the mirror-like surface of the specular object. By projecting multiple different patterns, such as multiple different Gray code patterns, and by sequencing through each EGlass layer for each pattern, the 3D shape of the entirety of the visible surface of the object can be reconstructed by analysis of captured images of the distorted reflections of the patterns by the surface of the object. Embodiments are also described which can be used without reconfiguration for surface reconstruction of the shape of diffuse objects.

First claim

What is claimed is:

1. An apparatus for depth estimation of the surface of an object positioned at an inspection station, the apparatus comprising:
a projector positioned and configured to project patterned light toward the inspection station;
first and second transparency-controllable layers positioned to receive a reflection from a direction of the inspection station, the first and second layers being positioned in spaced-apart relation relative to each other, wherein both of the first and second layers are controllably switchable between a transparent mode in which the layer is transparent, and a diffuse mode in which the layer diffuses light;
an image capture device positioned and configured to capture images of the first and second layers; and
a controller configured to control transparency of the first and second layers and to control projection of patterned light by the projector, and configured to estimate depth of the surface of an object positioned at the inspection station by calculations using images of the first and second layers captured by the image capture device.

2. The apparatus according to claim 1 , further comprising a second image capture device positioned and configured to capture images of an object at the inspection station,
wherein the controller controls the projector to project patterned light onto the object, and
wherein based on diffuse components of the captured images of the second image capture device, the controller estimates depth of the surface of an object having at least some diffuse characteristics.

3. The apparatus according to claim 2 , wherein the patterned light comprises a sequence of patterns, different from each other, such that each pixel at the layer is uniquely identifiable.

4. The apparatus according to claim 3 , wherein the controller determines depth at multiple points on the surface of the object by decoding patterns in the captured images.

5. The apparatus according to claim 4 , wherein the patterned light comprises a sequence of binary patterns of horizontal and vertical stripe patterns, each pattern in the sequence having a spatial frequency that differs from others of the patterns in the sequence.

6. The apparatus according to claim 5 , wherein the patterns are formed using binary Gray codes.

7. The apparatus according to claim 2 , further comprising three or more transparency-controllable layers, wherein the layers are arbitrarily controlled to either of the transparent mode or the diffuse mode.

8. The apparatus according to claim 1 , wherein the controller controls both of the first and second layers to the transparent mode, and controls the projector to project patterned light onto the object,
wherein the image capture device captures images through the first and second layers of the object illuminated by the patterned light, and
wherein based on diffuse components of the captured images, the controller estimates depth of the surface of an object having at least some diffuse characteristics.

9. The apparatus according to claim 8 , wherein the controller determines depth at multiple points on the surface of the object by decoding the binary patterns in the captured images.

10. The apparatus according to claim 9 , wherein the patterned light comprises a sequence of patterns, different from each other, such that each pixel at the layer is uniquely identifiable.

11. The apparatus according to claim 9 , wherein the patterned light comprises a sequence of binary patterns of horizontal and vertical stripe patterns, each pattern in the sequence having a spatial frequency that differs from others of the patterns in the sequence.

12. The apparatus according to claim 11 , wherein the patterns are formed using binary Gray codes.

13. The apparatus according to claim 8 , further comprising three or more transparency-controllable layers, wherein all of the layers are controlled to the transparent mode.

14. The apparatus according to claim 1 , wherein in a first phase the controller controls the first layer to the diffuse mode and the second layer to the transparent mode, and controls the projector to project a first sequence of patterned light onto an object at the inspection station, thereby to cause a distorted reflection of the patterned light to be reflected by the object onto the first layer,
wherein in a second phase the controller controls the first layer to the transparent mode and the second layer to the diffuse mode, and controls the projector to project a second sequence of patterned light onto an object at the inspection station, thereby to cause a distorted reflection of the patterned light to be reflected by the object onto the second layer, and
wherein based on specular components of the captured images, the controller estimates depth of the surface of an object having at least some specular characteristics.

15. The apparatus according to claim 14 , wherein the patterned light comprises a sequence of patterns, different from each other, such that each pixel at the layer is uniquely identifiable.

16. The apparatus according to claim 15 , wherein the controller determines depth at multiple points on the surface of the object by decoding the binary patterns in the captured images of deformed patterns reflected by the surface of the object from the first and second layers.

17. The apparatus according to claim 16 , wherein each of the first and second sequences of patterned light comprises a sequence of binary patterns of horizontal and vertical stripe patterns, each pattern in the sequence having a spatial frequency that differs from others of the patterns in the sequence.

18. The apparatus according to claim 17 , wherein the first and second sequence of patterns are formed using binary Gray codes.

19. The apparatus according to claim 16 , wherein the spaced-apart relation of the first and second layers is determined by calibration, and
wherein the controller determines depth at multiple points on the surface of the object by tracing paths of light rays from the first layer and the second layer using the calibration.

20. The apparatus according to claim 14 , further comprising three or more transparency-controllable layers, wherein each of the layers in turn is controlled to the diffuse mode while all other layers are controlled to the transparent mode.

21. The apparatus according to claim 1 , wherein each of the first and second layers is comprised of a polymer dispersed liquid crystal (PDLC) or polymer stabilized cholesteric texture (PSCT) film sandwiched between two layers of glass with two layers of conductive interlayers, so as to allow control of the layer between the transparent mode and the diffuse mode.

22. The apparatus according to claim 8 , wherein a mirror is arranged behind both of the first and second layers, and wherein the projector projects directly onto the object having at least some diffuse characteristics.

23. A method for estimating depth of the surface of an object having at least some specular characteristics, the method comprising:
controlling first and second transparency-controllable layers between a transparent mode in which the layer is transparent and a diffuse mode in which the layer diffuses light, wherein the first and second layers are positioned in spaced-apart relation relative to each other;
projecting a first sequence of light patterns onto an object while the first layer is in the diffuse mode and the second layer is in the transparent mode, wherein the first

Assignees

• Canon Kk

Inventors

• Nezamabadi Mahdi
• Martinello Manuel

Classifications

• G06T7/521 Primary

  from laser ranging, e.g. using interferometry; from the projection of structured light · CPC title

• G02F1/1334

  based on polymer dispersed liquid crystals, e.g. microencapsulated liquid crystals · CPC title

• G01B11/2513

  with several lines being projected in more than one direction, e.g. grids, patterns · CPC title

• G06T7/0057

  Physics · mapped topic

• G02F1/137

  characterised by the electro-optical or magneto-optical effect, e.g. field-induced phase transition, orientation effect, guest-host interaction or dynamic scattering · CPC title