High-precision dynamic real-time 360-degree omnidirectional point cloud acquisition method based on fringe projection

Patent metadata

Field	Value
Publication number	US-11961244-B2
Application number	US-202017769230-A
Country	US
Kind code	B2
Filing date	Aug 27, 2020
Priority date	Nov 8, 2019
Publication date	Apr 16, 2024
Grant date	Apr 16, 2024

Abstract

Disclosed is a high-precision dynamic real-time 360-degree omnidirectional point cloud acquisition method based on fringe projection. The method comprises: firstly, by means of the fringe projection technology based on a stereoscopic phase unwrapping method, and with the assistance of an adaptive dynamic depth constraint mechanism, acquiring high-precision three-dimensional (3D) data of an object in real time without any additional auxiliary fringe pattern; and then, after a two-dimensional (2D) matching points optimized by the means of corresponding 3D information is rapidly acquired, by means of a two-thread parallel mechanism, carrying out coarse registration based on Simultaneous Localization and Mapping (SLAM) technology and fine registration based on Iterative Closest Point (ICP) technology. By means of the invention, low-cost, high-speed, high-precision, unconstrained and rapid-feedback omnidirectional 3D real-time molding becomes possible, and a new gate is opened into the fields of 360-degree workpiece 3D surface defect detection, rapid reverse forming, etc.

First claim

The invention claimed is:

1. A high-precision dynamic real-time 360-degree omnidirectional point cloud acquisition method based on fringe projection, comprising the steps of:
step_1: building a quad-camera fringe projection system based on a stereo phase unwrapping (SPU) method, and completing a system calibration;
step_2: obtaining an absolute phase of an object, so as to realize an acquisition of real-time high-precision 3D data under a single viewing angle of the object;
step_3: in a coarse registration thread, using a scale-invariant feature transformation (SIFT) method to obtain 2D matching points of adjacent images, and realizing the coarse registration based on a simultaneous localization and mapping (SLAM) technology by solving a “perspective n-point” (PnP) problem, and at the same time quantifying an amount of motion of the object during each coarse registration; and
step_4: in a fine registration thread, in response to an accumulated motion amount of the coarse registration reaching a threshold, performing a fine registration based on a closest point iterative (ICP) method,
wherein the step 2 further comprises the sub-steps of:
projecting, by a projector, three-step phase-shifting fringe patterns to the object and simultaneously capturing three-step phase-shifting fringe images by using a plurality of cameras;
obtaining a wrapped phase of the object from three-step phase-shifting fringe images captured by a first camera of the plurality of cameras; and
obtaining the absolute phase of the object only through three-step phase-shifting fringe images captured by the plurality of cameras,

wherein the step 3 further comprises the sub-steps of:
finding the 2D matching points of the adjacent images by assuming two adjacent 3D frames;
optimizing the 2D matching points by using 3D information;
obtaining rotation and translation matrices between adjacent 3D frames; and
quantifying an amount of motion in each coarse registration, and

wherein the step 4 further comprises the sub-steps of:
comparing a cumulative motion of coarse registration and a size of a motion threshold to determine whether the fine registration can be performed; and
performing the fine registration.

2. The method according to claim 1 , wherein the quad-camera fringe projection system includes a computer, four cameras, and a projector; the projector and the four cameras are connected by four trigger lines, and the four cameras and the computer are connected by four data lines,
wherein the first camera and the projector are kept at a long distance, there is a gap between the first camera and the projector where another camera is placed; the second camera is placed between the first camera and the projector and is placed close to the first camera; the fourth camera and the first camera are placed symmetrically for the projector; the third camera and the first camera are placed symmetrically for the projector, and
wherein the step 1 further comprises calibrating the entire system to unified world coordinates, obtaining internal and external parameters of the four cameras and projectors, and converting the internal and external parameters into 2D to 3D, 3D to 2D, and 2D to 2D mapping parameters.

3. The method according to claim 2 , wherein the step 2 comprise the substeps of:
(1) projecting three three-step phase-shifting fringe patterns to the object by the projector, and simultaneously triggering the four cameras to capture the images wherein three-step phase-shifting fringes captured by the first camera is expressed as:
I 1 C ( u C ,v C )= A C ( u C ,v C )+ B C ( u C ,v C )cos(Φ C ( u C ,v C )),

I 2 C ( u C ,v C )= A C ( u C ,v C )+ B C ( u C ,v C )cos(Φ C ( u C ,v C )+2π/3),

I 3 C ( u C ,v C )= A C ( u C ,v C )+ B C ( u C ,v C )cos(Φ C ( u C ,v C )+4π/3),

where (u C , v C ) denotes coordinates of a pixel point on the first camera, I 1 C , I 2 C , I 3 C denote the three fringe images captured by the first camera, A C is an average light intensity, B C represents a modulation degree light intensity, and Φ C denotes an absolute phase of the fringe images;
(2) obtaining the wrapped phase of the object
from the three fringe images collected by the first camera according to:

ϕ
C

(

u
C

,

v
C

)

=

arc
⁢

tan
⁢

(

3

⁢

(

I
2
C

(

u
C

,

v
C

)

-

I
3
C

(

u
C

,

v
C

)

)

2
⁢

I
1
C

(

u
C

,

v
C

)

-

I
2
C

(

Assignees

• Univ Nanjing Sci & Tech

Inventors

• Zuo Chao
• Qian Jiaming
• Chen Qian
• Feng Shijie
• Tao tianyang
• Hu Yan
• Yin Wei
• Zhang Liang
• Liu Kai
• Wu Shuaijie
• Xu Mingzhu
• Wang Jiaye

Classifications

• G06T12/00

  Tomographic reconstruction from projections · CPC title

• G06T7/337 Primary

  involving reference images or patches · CPC title

• G06T3/14

  Transformations for image registration, e.g. adjusting or mapping for alignment of images · CPC title

• G06T7/85

  Stereo camera calibration · CPC title

• H04N23/90 Primary

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title