System and method for reconstructing 3-D shapes of objects from reflection images

Patent metadata

Field	Value
Publication number	US-11671580-B2
Application number	US-202016874632-A
Country	US
Kind code	B2
Filing date	May 14, 2020
Priority date	May 14, 2019
Publication date	Jun 6, 2023
Grant date	Jun 6, 2023

Abstract

A system and method are provided for reconstructing 3-D point cloud. A light source generates light that is received by a polarization field generator, which generates a polarization field that illuminates the target object being imaged such that each outgoing ray has a unique polarization state. A camera captures images of the illuminated target object and the captured images are received by a processor that: (1) performs a polarization field decoding algorithm that decodes the polarization field to obtain a set of incident rays; (2) performs a camera ray decoding algorithm to obtain a set of camera rays; (3) performs a ray-ray intersection algorithm that determines intersection points where the incident rays and the camera rays intersect; and (4) performs a 3-D reconstruction algorithm that uses the set of incident rays, the set of camera rays and the intersection points to reconstruct a 3-D point cloud of the target object.

First claim

What is claimed is:

1. A system for reconstructing a three-dimensional (3-D) point cloud of a target object, the system comprising:
a light source that generates light;
a polarization field generator that receives the light generated by the light source and generates a polarization field that illuminates the target object;
a camera that captures images of the illuminated target object; and
a processor that controls operations of the light source and of the polarization field generator to generate the polarization field such that the target object is illuminated with a preselected sequence of Gray Code images, wherein the processor receives the captured images and processes the captured images by:
performing a polarization field decoding algorithm that decodes the polarization field to obtain a set of incident rays, wherein the polarization field decoding algorithm decodes Gray Code images that are captured by the camera to determine which pixel of the polarization field generator originated each incident ray of the set of incident rays to thereby determine a direction of each incident ray of the set of incident rays, wherein when the processor performs the polarization field decoding algorithm that decodes the polarization field to obtain the set of incident rays, the processor performs an optimization algorithm that uses the captured images to create a reflection image formation model that is used to optimize the directions of the incident rays of said set of incident rays;
performing a camera ray decoding algorithm to obtain a set of camera rays;
performing a ray-ray intersection algorithm that determines intersection points where the incident rays and the camera rays intersect; and
performing a 3-D reconstruction algorithm that uses the set of incident rays, the set of camera rays and the intersection points to reconstruct the 3-D point cloud of the target object.

2. The system of claim 1 , wherein the 3-D reconstruction algorithm includes a Poisson integration algorithm.

3. The system of claim 1 , wherein the polarization field generator comprises a polarizer panel of a liquid crystal display (LCD) device.

4. The system of claim 1 , wherein the target object is an object that exhibits reflection of the environment.

5. The system of claim 1 , wherein the polarization field generator is configured to alter a polarization state of the polarization field in response to an applied voltage.

6. The system of claim 5 , wherein the polarization state varies between linear polarization to elliptical polarization.

7. A method for reconstructing a three-dimensional (3-D) point cloud of a target object, the method comprising:
with a light source, emitting light;
with a polarization field generator, receiving at least a portion of the light and generating a polarization field that illuminates the target object;
with a camera, capturing images of the illuminated target object; and
with a processor, controlling operations of the light source and of the polarization field generator to generate the polarization field such that the target object is illuminated with a preselected sequence of Gray Code images, receiving the captured images, and processing the captured images by:
performing a polarization field decoding algorithm that decodes the polarization field to obtain a set of incident rays, wherein the polarization field decoding algorithm decodes Gray Code images that are captured by the camera to determine which pixel of the polarization field generator originated each incident ray of the set of incident rays to thereby determine a direction of each incident ray of the set of incident rays, wherein when the processor performs the polarization field decoding algorithm that decodes the polarization field to obtain the set of incident rays, the processor performs an optimization algorithm that uses the captured images to create a reflection image formation model that the processor uses to optimize the directions of the incident rays of said set of incident rays;
performing a camera ray decoding algorithm to obtain a set of camera rays;
performing a ray-ray intersection algorithm that determines intersection points where the incident rays and the camera rays intersect; and
performing a 3-D reconstruction algorithm that uses the set of incident rays, the set of camera rays and the intersection points to reconstruct the 3-D point cloud of the target object.

8. The method of claim 7 , wherein the 3-D reconstruction algorithm includes a Poisson integration algorithm.

9. The method of claim 7 , wherein the polarization field generator comprises a polarizer panel of a liquid crystal display (LCD) device.

10. The method of claim 7 , wherein the target object is an object that exhibits reflection of the environment.

11. The method of claim 7 , wherein the polarization field generator is configured to alter a polarization state of the polarization field in response to an applied voltage.

12. The method of claim 11 , wherein the polarization state varies between linear polarization to elliptical polarization.

13. A computer program comprising computer instructions embodied on a non-transitory computer readable medium, the computer program being designed to reconstruct a three-dimensional (3-D) point cloud of a target object, wherein a light source emits light that is received by a polarization field generator that generates a polarization field that illuminates the target object, and wherein a camera captures images of the illuminated target object, the computer program being executed by a processor that receives the captured images from the camera, the computer instructions comprising:
a first instruction set that performs a polarization field decoding algorithm that decodes the polarization field to obtain a set of incident rays, wherein the polarization field decoding algorithm includes instructions that decode Gray Code images that are captured by the camera to determine which pixel of the polarization field generator originated each incident ray of the set of incident rays to thereby determine a direction of each incident ray of the set of incident rays, wherein the first instruction set includes computer instructions that perform an optimization algorithm that uses the captured images to create a reflection image formation model that the first instruction set uses to optimize the directions of the incident rays of said set of incident rays;
a second instruction set that performs a camera ray decoding algorithm to obtain a set of camera rays;
a third instruction set that performs a ray-ray intersection algorithm that determines intersection points where the incident rays and the camera rays intersect;
a fourth instruction set that performs a 3-D reconstruction algorithm that uses the set of incident rays, the set of camera rays and the intersection points to reconstruct the 3-D point cloud of the target object; and
a fifth instruction set that controls operations of the light source and of the polarization field generator to generate the polarization field such that the target object is illuminated with a preselected sequence of Gray Code images.

14. The computer program of claim 13 , wherein the 3-D reconstruction algorithm includes computer instructions for performing a Poisson integration algorithm.

15. The computer program of claim 13 , wherein the polarization field generator comprises a polarizer panel of a liquid crystal display (LCD) device.

16. The computer program of claim 13 , wherein the target object is an object that exhibits reflection of the environment.

17. The

Assignees

• Univ Louisiana State

Inventors

• Ye Jinwei
• Lu Jie

Classifications

• G01B11/25

  by projecting a pattern, e.g. {one or more lines,} moiré fringes on the object (G01B11/255 takes precedence {; image analysis for depth or shape recovery G06T7/50}) · CPC title

• G06T17/00

  Three-dimensional [3D] modelling for computer graphics · CPC title

• G02F1/133536

  Reflective polarizers (G02F1/13362 takes precedence) · CPC title

• H04N13/207

  using a single two-dimensional [2D] image sensor · CPC title

• H04N13/254 Primary

  in combination with electromagnetic radiation sources for illuminating objects · CPC title