System and method for three-dimensional image reconstruction using an absolute orientation sensor

Patent metadata

Field	Value
Publication number	US-10460462-B2
Application number	US-201816172067-A
Country	US
Kind code	B2
Filing date	Oct 26, 2018
Priority date	Jul 18, 2016
Publication date	Oct 29, 2019
Grant date	Oct 29, 2019

Abstract

A three-dimensional image reconstruction system includes an image capture device, an inertial measurement unit (IMU), and an image processor. The image capture device captures image data. The inertial measurement unit (IMU) is affixed to the image capture device and records IMU data associated with the image data. The image processor includes one or more processing units and memory for storing instructions that are executed by the one or more processing units, wherein the image processor receives the image data and the IMU data as inputs and utilizes the IMU data to pre-align the first image and the second image, and wherein the image processor utilizes a registration algorithm to register the pre-aligned first and second images.

First claim

The invention claimed is:

1. A three-dimensional image reconstruction system comprising:
an image capture device that captures image;
an inertial measurement unit (IMU) affixed to the image capture device that records IMU data associated with image data of the captured image; and
an image processor that includes one or more processing units and memory for storing instructions that are executed by the one or more processing units, wherein the image processor receives the image data and the IMU data as inputs, and calculates rigid body motion (R,t) that includes a rotational component R and a translational component t between the image data and a current reconstructed point cloud, wherein the image processor constrains the rotational component R of the rigid body motion (R,t) between the image data and the current reconstructed point cloud based on the IMU data and a transformation value X, wherein the image processor calculates only the translational component t of the rigid body motion (R,t) between the image data and the current reconstructed point cloud, and wherein the image processor utilizes a registration algorithm to register the image data with the current reconstructed point cloud based on the calculated rigid body motion (R, t) between the image data and the current reconstructed point cloud to generate a three-dimensional image.

2. The three-dimensional image reconstruction system of claim 1 , wherein the transformation value X is generated by a calibration module based on two or more pairs of image data and the IMU data.

3. The three-dimensional image reconstruction system of claim 1 , wherein the image processor generates a partial point cloud based on the image data.

4. The three-dimensional image reconstruction system of claim 3 , wherein the IMU data and the transformation value X is utilized by the image processor to pre-align the partial point cloud with the current reconstructed point cloud.

5. The three-dimensional image reconstruction system of claim 4 , wherein the image processor calculates distances between corresponding points in the partial point cloud and the reconstructed point cloud and compares the calculated distances to a threshold.

6. The three-dimensional image reconstruction system of claim 5 , wherein the image processor discards those points in which the calculated distance exceeds the threshold.

7. The three-dimensional image reconstruction system of claim 6 , wherein the threshold is calculated based on an average distance d between the corresponding points and a tolerance t.

Assignees

• Univ King Abdullah Sci & Tech

Inventors

• Giancola Silvio
• Ghanem Bernard Semaan
• Schneider Jens
• Wonka Peter

Classifications

• G06T7/55 Primary

  from multiple images · CPC title

• G06T17/00 Primary

  Three-dimensional [3D] modelling for computer graphics · CPC title

• G06T2210/56

  Particle system, point based geometry or rendering · CPC title

• G06T2200/08

  involving all processing steps from image acquisition to 3D model generation · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title