Method for reconstructing three-dimensional object combining structured light and photometry and terminal device

Patent metadata

Field	Value
Publication number	US-12586223-B2
Application number	US-202017927692-A
Country	US
Kind code	B2
Filing date	Nov 17, 2020
Priority date	Jul 28, 2020
Publication date	Mar 24, 2026
Grant date	Mar 24, 2026

Abstract

The present application is applicable to the technical field of computer vision, and provides a method for reconstructing a three-dimensional object combining structured light and photometry and a terminal device, wherein the method comprises: acquiring N first images, wherein each first image is obtained by shooting after a coded pattern having a coding stripe sequence is projected to a three-dimensional object, and N is a positive integer; determining structured light depth information of the three-dimensional object based on the N first images; acquiring M second images, wherein the M second images are obtained by shooting after P light sources are respectively projected to the three-dimensional object from different directions, and M and P are positive integers; determining photometric information of the three-dimensional object based on the M second images; and reconstructing the three-dimensional object based on the structured light depth information and the photometric information. Therefore, the structured light system and the photometric system are combined to reconstruct the three-dimensional object, and the precision of a three-dimensional reconstruction result of the three-dimensional object with complex surfaces is improved.

First claim

What is claimed is:

1 . A method for reconstructing a three-dimensional object combining structured light and photometry, comprising:
acquiring N first images, wherein each first image is obtained by shooting after a coded pattern having a coding stripe sequence is projected to a three-dimensional object, and N is a positive integer; determining structured light depth information of the three-dimensional object based on the N first images; acquiring M second images, wherein the M second images are obtained by shooting after P light sources are respectively projected to the three-dimensional object from different directions, and M and P are positive integers; determining photometric information of the three-dimensional object based on the M second images; and reconstructing the three-dimensional object based on the structured light depth information and the photometric information, wherein reconstructing the three-dimensional object based on the structured light depth information and the photometric information comprises: iteratively calibrating the structured light depth information based on the photometric information to obtain calibration deviation information relative to the structured light depth information; determining whether the calibration deviation information meets a preset iteration termination condition or not; and if yes, reconstructing the three-dimensional object based on calibrated structured light depth information correspondingly meeting the iteration termination condition.

2 . The method for reconstructing a three-dimensional object combining structured light and photometry according to claim 1 , wherein
prior to acquiring M second images, the method further comprises: determining positions and projection directions of the P light sources according to the structured light depth information.

3 . The method for reconstructing a three-dimensional object combining structured light and photometry according to claim 1 , wherein
determining photometric information of the three-dimensional object based on the M second images specifically comprises: inputting the M second images into a reflection model matched with a target surface type of the three-dimensional object to output the photometric information of the three-dimensional object by the reflection model.

4 . The method for reconstructing a three-dimensional object combining structured light and photometry according to claim 1 , wherein
the photometric information comprises a first photometric information and a second photometric information, wherein iteratively calibrating the structured light depth information based on the photometric information to obtain first calibration deviation information relative to the structured light depth information comprises: iteratively calibrating the structured light depth information based on the first photometric information to obtain first calibration deviation information relative to the structured light depth information; correspondingly, determining whether the calibration deviation information meets a preset iteration termination condition or not comprises: calibrating the second photometric information with the calibrated structured light depth information to obtain second calibration deviation information relative to the second photometric information; and determining whether the first calibration deviation information and the second calibration deviation information meet a preset deviation condition or not so as to correspondingly determine whether the calibration deviation information meets the preset iteration termination condition or not.

5 . The method for reconstructing a three-dimensional object combining structured light and photometry according to claim 4 , wherein
the first photometric information is a reflectivity and the second photometric information is normal information.

6 . The method for reconstructing a three-dimensional object combining structured light and photometry according to claim 4 , wherein
determining whether the first calibration deviation information and the second calibration deviation information meet a preset deviation condition or not comprises: determining target deviation information of a preset deviation weight allocation corresponding to the first calibration deviation information and the second calibration deviation information; and correspondingly determining whether the first calibration deviation information and the second calibration deviation information meet the preset deviation condition or not according to a comparison result of whether the target deviation information is smaller than preset deviation threshold information or not.

7 . The method for reconstructing a three-dimensional object combining structured light and photometry according to claim 1 , wherein
iteratively calibrating the structured light depth information based on the photometric information comprises: inputting the photometric information and the structured light depth information into a depth information calibration model to correspondingly output the calibrated structured light depth information, wherein each training sample of a training sample set in the depth information calibration model comprises structured light depth information corresponding to Q pieces of photometric information, and Q is a positive integer.

Assignees

• Shenzhen Inst Of Adv Tech Cas

Inventors

• Song Zhan
• Song Zhao

Classifications

• G01B11/254

  Projection of a pattern, viewing through a pattern, e.g. moiré · CPC title

• G06V10/60

  relating to illumination properties, e.g. using a reflectance or lighting model · CPC title

• G06T7/80

  Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration · CPC title

• G01B11/24

  for measuring contours or curvatures · CPC title

• G01B11/2536

  using several gratings with variable grating pitch, projected on the object with the same angle of incidence · CPC title