Method and apparatus for generating a 3D reconstruction of an object

Patent metadata

Field	Value
Publication number	US-12033280-B2
Application number	US-202016860006-A
Country	US
Kind code	B2
Filing date	Apr 27, 2020
Priority date	Dec 20, 2018
Publication date	Jul 9, 2024
Grant date	Jul 9, 2024

Abstract

The generation of a 3D reconstruction of an object is disclosed, which includes illuminating the object, capturing image data in relation to the object, and calculating the 3D reconstruction of the object from the image data. The image data contains first image data and second image data, wherein the first image data are captured when the object is illuminated with illumination light, at least some of which, in relation to an object imaging beam path, is reflected light which illuminates the object, wherein the second image data are captured from different recording directions when the object is illuminated with illumination light, at least some of which is guided in the object imaging beam path, and wherein the 3D reconstruction of the object is calculated from the first image data and the second image data.

First claim

The invention claimed is:

1. A method for generating a 3D reconstruction of an object, the method comprising:
illuminating an object with illumination light emanating from a light source configuration containing at least one light source;
capturing image data in relation to the object, wherein the image data contains first image data and second image data;
capturing the first image data from different recording directions, the object being illuminated with the illumination light emanating from the light source configuration incident on an optically effective surface of the object, at least some of the illumination light incident on the optically effective surface being reflected into an object imaging beam path;
capturing the second image data from the different recording directions, the object being illuminated with the same illumination light emanating from the light source configuration, at least some of which, in relation to the object imaging beam path, is background light which back-illuminates the object;
analyzing an intensity profile along an object point trajectory to estimate a reflection property of a 3D reconstruction of the object; and
calculating the 3D reconstruction of the object from the first image data and the second image data.

2. The method according to claim 1 , wherein a tomographic reconstruction of the object, in which the first and second image data are back-projected into a 3D voxel grid according to spatially dependent weighting, is ascertained to calculate the 3D reconstruction of the object.

3. The method according to claim 1 , wherein the calculation of the 3D reconstruction of the object comprises a calculation of object feature images for at least some of the first image data.

4. The method according to claim 3 , wherein at least one of:
the object feature images have a single object feature or a plurality of object features from a group of edges, corners, Gabor features as a feature;
the object feature images are calculated by machine learning or by a neural network; or the calculation of the 3D reconstruction of the object includes a determination of segmented image masks and the cutting of an object part from at least some of at least one of the first image data or the second image data with segmented image masks.

5. The method according to claim 3 , wherein the calculation of the 3D reconstruction of the object comprises a calculation of edge information images from the at least one of the first image data or the second image data.

6. The method according to claim 5 , wherein the calculation of the 3D reconstruction of the object comprises a calculation of epipolar plane images from at least one of the object feature images or the edge information images.

7. The method according to claim 6 , wherein the calculation of the 3D reconstruction of the object comprises the calculation of object point trajectories from the epipolar plane images and the measurement of the gradient of the calculated object point trajectories and the estimation of depth information by means of triangulation to form an epipolar geometric 3D reconstruction of the object.

8. The method according to claim 7 , wherein a tomographic reconstruction of the object, in which the first image data and the second image data are back-projected into a 3D voxel grid following spatially dependent weighting, is ascertained for the purposes of calculating the 3D reconstruction of the object, and wherein the epipolar geometric 3D reconstruction of the object is combined by calculation with the 3D voxel grid to form the 3D reconstruction of the object.

9. The method according to claim 7 , wherein at least one of:
a mesh for describing the surface of the object is calculated from the 3D voxel grid, or mesh normals are calculated from the 3D voxel grid with a derivative filter.

10. The method according to claim 1 , further comprising performing at least one of:
calculating a texture transparency from the second image data, wherein the 3D reconstruction of the object contains information of the calculated texture transparency; calculating a specular texture image, wherein the 3D reconstruction of the object contains information of the calculated texture image;
capturing color images as images with image data;
determining the 3D reconstruction of the object by combining by calculation a plurality of the 3D color channel reconstructions of the object, wherein each of the 3D color channel reconstructions is calculated for at least one of a color channel or a texture transparency channel from the first image data and the second image data;
determining a 3D reconstruction having texture information; and
reflecting a view of the object into the image capturing device.

11. The method according to claim 10 , further comprising:
averaging corresponding data within the at least one of the color channel or the texture transparency channel.

12. The method according to claim 1 , wherein the calculation of the 3D reconstruction of the object comprises a calculation of a visual shell from silhouettes of the object calculated from the second image data.

13. The method according to claim 1 , further comprising: capturing the first image data and the second image data at a same time.

14. The method according to claim 1 , wherein the at least one light source contains an areal illuminate configured to provide diffuse illumination light, and illuminating the object with the diffuse illumination light.

15. An apparatus for generating a 3D reconstruction of an object, the apparatus being configured to carry out the method according to claim 1 and comprising:
a device configured to illuminate with the object with the illumination light;
a device configured to capture the first image data and the second image data of the object; and
a device configured to calculate the 3D reconstruction of the object from the first image data and the second image data.

16. A non-transitory storage medium having a computer program stored thereon and comprising commands that, when executed on a computer, cause an apparatus for generating a 3D reconstruction of an object to carry out the method of claim 1 ,
wherein the apparatus contains:
a device configured to illuminate the object with the illumination light;
a device configured to capture the first image data and the second image data of the object; and
a device configured to calculate the 3D reconstruction of the object from the first image data and the second image data.

17. An apparatus for generating a 3D reconstruction of an object, the apparatus comprising:
a device configured to illuminate an object with illumination light emanating from a light source configuration containing at least one light source;
a device configured to capture image data in relation to the object; and a device configured to calculate the 3D reconstruction of the object from the captured image data,
wherein the captured image data include first image data and second image data,
wherein the first image data are captured from different recording directions, the object being illuminated with the illumination light emanating from the light source configuration incident on an optically effective surface of the object, at least some of the illumination light incident on the optically effective surface being reflected into an object imaging beam path,
wherein the second image data are captured from the different recording directions, the object being illuminated with the same illumination light, at least some of which, in relation to the object imaging beam path, is background

Assignees

• Zeiss Carl Vision Int Gmbh

Inventors

• Schwarz Oliver
• Omlor Lars

Classifications

• G06T12/20 Primary

  Inverse problem, i.e. transformations from projection space into object space · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06T2207/10024

  Color image · CPC title