System and method for reconstructing 3D model

Patent metadata

Field	Value
Publication number	US-9574874-B2
Application number	US-201414219224-A
Country	US
Kind code	B2
Filing date	Mar 19, 2014
Priority date	Oct 17, 2013
Publication date	Feb 21, 2017
Grant date	Feb 21, 2017

Abstract

A system and method for reconstructing a three-dimensional (3D) model are described. The 3D model reconstruction system includes: a low resolution reconstruction unit that converts a first depth map, acquired by scanning a scene with a depth camera, into a second depth map having a low resolution, processes the second depth map to extract pose change information about a pose change of the depth camera, and reconstructs a low resolution 3D model in real-time; and a high resolution reconstruction unit that processes the first depth map by using the pose change information of the depth camera that is extracted from the second depth map and reconstructs a high resolution 3D model.

First claim

What is claimed is:

1. A system for reconstructing a three-dimensional (3D) model, the system comprising:
a low resolution reconstruction unit that comprises:
a scale adjuster that converts a first depth map, acquired by scanning a scene with a depth camera, into a second depth map having a low resolution by downsizing the first depth map,
a first map converter that converts depth data in the second depth map into point cloud data in a coordinate system in 3D space,
a first sensor tracker that matches point cloud data in the second depth map with point cloud data in a previous second depth map by using an iterative closest point (ICP) algorithm and extracts pose change information about a pose change of the depth camera based on the result of matching, and
a first volume constructor that integrates together point cloud data in the second depth map and the previous second depth map and generates, a low resolution 3D model in real-time, based on the pose change information of the depth camera; and a high resolution reconstruction unit that processes the first depth map by using the pose change information of the depth camera that is extracted from the second depth map and reconstructs a high resolution 3D model.

2. The system of claim 1 , wherein the low resolution reconstruction unit further comprises a first map estimator for estimating an estimated depth map from the low resolution 3D volume.

3. The system of claim 2 , wherein the estimated depth map is visualized based on a user's viewpoint, displayed, and stored as the previous second depth map.

4. The system of claim 1 , wherein the high resolution reconstruction unit comprises:
a second map converter that converts the depth data in the first depth map into point cloud data in a coordinate system in 3D space;
a second sensor tracker that matches point cloud data in the first depth map with point cloud data in a previous first depth map by using an iterative closest point (ICP) algorithm and extracts the pose change information of the depth camera based on the result of matching; and
a second volume constructor that integrates together point cloud data in the first depth map and the previous first depth map and generates a high resolution 3D volume, based on the pose change information of the depth camera, and
wherein the second sensor tracker performs the ICP algorithm by using the pose change information of the depth camera that is extracted from the second depth map.

5. The system of claim 4 , wherein the high resolution reconstruction unit further comprises a second map estimator for estimating an estimated depth map from the high resolution 3D volume.

6. The system of claim 4 , wherein the second sensor tracker excludes overlapping depth maps by using the pose change information of the depth camera that is extracted from the second depth map and performs the ICP algorithm on each of the selected depth maps except for the overlapping depth maps.

7. The system of claim 5 , wherein the estimated depth map is stored as the previous first depth map.

8. The system of claim 1 , wherein the high resolution reconstruction unit receives a depth map sequence and the pose change information of the depth camera from the low resolution reconstruction unit and reconstructs a high resolution 3D model, either after the depth camera completes a scan or while the depth camera performs the scan.

9. The system of claim 1 , wherein the low resolution reconstruction unit is an embedded system,
wherein the high resolution reconstruction unit is a cloud server, and
wherein the embedded system uploads a depth map sequence and the pose change information of the depth camera to the cloud server and downloads the high resolution 3D model from the cloud server.

10. A method of reconstructing a three-dimensional (3D) model, the method comprising:
converting a first depth map acquired by scanning a scene through a depth camera into a second depth map having a low resolution, processing the second depth map to extract pose change information about a pose change of the depth camera, and reconstructing a low resolution 3 D model in real-time; and
processing the first depth map by using the pose change information of the depth camera that is extracted from the second depth map and reconstructing a high resolution 3D model;
wherein reconstructing the low resolution 3D model in real-time comprises:
generating the second depth map by downsizing the first depth map,
converting depth data in the second depth map into point cloud data in a coordinate system in 3D space,
matching point cloud data in the second depth map with point cloud data in a previous second depth map by using an iterative closest point (ICP) algorithm and extracting the pose change information of the depth camera based on the result of matching, and
integrating together point cloud data in the second depth map and the previous second depth map and generating a low resolution 3D volume in real-time, based on the pose change information of the depth camera.

11. The method of claim 10 , wherein the reconstructing of the low resolution 3D model further comprises estimating an estimated depth map from the low resolution 3D volume.

12. The method of claim 11 , wherein the estimated depth map is visualized based on a user's viewpoint, displayed, and stored as the previous second depth map.

13. The method of claim 10 , wherein the reconstructing of the high resolution 3D model comprises:
converting the depth data in the first depth map into point cloud data in a coordinate system in 3D space;
matching point cloud data in the first depth map with point cloud data in a previous depth map by using an ICP algorithm and extracting the pose change information of the depth camera based on the result of matching; and
integrating together point cloud data in the first depth map and the previous first depth map and generating a high resolution 3D volume, based on the pose change information of the depth camera, and
wherein in the extracting of the pose change information of the depth camera, the ICP algorithm is performed by using the pose change information of the depth camera that is extracted from the second depth map.

14. The method of claim 13 , wherein the reconstructing of the high resolution 3D model further comprises estimating an estimated depth map from the high resolution 3D volume.

15. The method of claim 13 , wherein the extracting of the pose change information of the depth camera comprises excluding overlapping depth maps by using the pose change information of the depth camera that is extracted from the second depth map and performing the ICP algorithm on each of the selected depth maps except for the overlapping depth maps.

16. The method of claim 14 , wherein the estimated depth map is stored as the previous first depth map.

17. The method of claim 10 , wherein the reconstructing of the high resolution 3D model comprises reconstructing the high resolution 3D model by using a depth map sequence and the pose change information of the depth camera, either after the depth camera completes a scan or while the depth camera performs the scan.

18. The method of claim 10 , wherein the reconstructing of the low resolution 3D model comprises reconstructing the low resolution 3D model in each of a plurality of embedded systems, and
wherein the reconstructing of the high resolution 3D model comprises reconstructing the high resolution 3D model for the scene by using the pose change information of the depth camera that is received from the embedded system by a c

Assignees

• Samsung Electronics Co Ltd

Inventors

• Son Woo-Ram
• Oh Han

Classifications

• H04N23/64

  Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image · CPC title

• H04N5/23222

  Electricity · mapped topic

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G01B11/22 Primary

  for measuring depth · CPC title

• G06T2207/20016

  Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform · CPC title