Three-dimensional reconstruction method, electronic device and storage medium

Patent metadata

Field	Value
Publication number	US-11436709-B2
Application number	US-202017082761-A
Country	US
Kind code	B2
Filing date	Oct 28, 2020
Priority date	Mar 27, 2020
Publication date	Sep 6, 2022
Grant date	Sep 6, 2022

Abstract

The disclosure provides a 3-dimensional reconstruction method, an electronic device and a storage medium. The method includes: obtaining an image sequence pre-collected by a camera; obtaining nominal intrinsic parameters of the camera, and preset initial values of distortion coefficients of the camera, the camera being a camera for collecting the image sequence; calculating fluctuation ranges of the distortion coefficients; performing the 3D reconstruction on the image sequence based on the nominal intrinsic parameters, the initial values of the distortion coefficients, and the fluctuation ranges of the distortion coefficients; and during the 3D reconstruction, optimizing an obtained visual point cloud and a camera pose.

First claim

What is claimed is:

1. A 3-dimensional reconstruction method, comprising:
obtaining an image sequence pre-collected by a camera;
obtaining nominal intrinsic parameters of the camera, and preset initial values of distortion coefficients of the camera, the camera being a camera for collecting the image sequence;
calculating fluctuation ranges of the distortion coefficients, comprising: extracting distorted line features of a part of images in the image sequence; and calculating the fluctuation ranges of the distortion coefficients according to the distorted line features;
performing the 3-dimensional reconstruction on the image sequence based on the nominal intrinsic parameters, the initial values of the distortion coefficients, and the fluctuation ranges of the distortion coefficients; and
during the 3-dimensional reconstruction, optimizing an obtained visual point cloud and a camera pose.

2. The method according to claim 1 , further comprising:
during the 3-dimensional reconstruction, optimizing a semantic element in the obtained visual point cloud, so that a point cloud of the semantic element meets geometric features corresponding to the semantic element.

3. The method according to claim 1 , wherein the performing the 3-dimensional reconstruction on the image sequence comprises:
selecting a first image and a second image from the image sequence;
performing the 3-dimensional reconstruction on the first image and the second image to obtain an initial visual point cloud; and
registering each of new images in the image sequence to obtain a point cloud of each of the new images; and
combining the point cloud of each of the new images into the initial visual point cloud.

4. The method according to claim 3 , further comprising:
during registering each of the new images, determining whether to optimize the distortion coefficients according to a change curve of distortion coefficient.

5. The method according to claim 1 , wherein when the 3-dimensional reconstruction is performed on a target area of each image of the image sequence, the initial values of the distortion coefficients are 0, and the target area does not comprise an edge area of the image.

6. The method according to claim 1 , wherein the nominal intrinsic parameters of the camera comprise a nominal focal length and a nominal optical center value of the camera.

7. The method according to claim 1 , wherein the image sequence is an image sequence collected by crowdsourcing.

8. An electronic device, comprising:
at least one processor; and
a memory coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor are caused to implement the following actions:
obtaining an image sequence pre-collected by a camera;
obtaining nominal intrinsic parameters of the camera, and preset initial values of distortion coefficients of the camera, the camera being a camera for collecting the image sequence;
calculating fluctuation ranges of the distortion coefficients, comprising: extracting distorted line features of a part of images in the image sequence; and calculating the fluctuation ranges of the distortion coefficients according to the distorted line features;
performing the 3-dimensional reconstruction on the image sequence based on the nominal intrinsic parameters, the initial values of the distortion coefficients, and the fluctuation ranges of the distortion coefficients; and
during the 3-dimensional reconstruction, optimizing an obtained visual point cloud and a camera pose.

9. The electronic device according to claim 8 , wherein when the instructions are executed by the at least one processor, the at least one processor are caused to further implement the following actions:
during the 3-dimensional reconstruction, optimizing a semantic element in the obtained visual point cloud, so that a point cloud of the semantic element meets geometric features corresponding to the semantic element.

10. The electronic device according to claim 8 , wherein the performing the 3-dimensional reconstruction on the image sequence comprises:
selecting a first image and a second image from the image sequence;
performing the 3-dimensional reconstruction on the first image and the second image to obtain an initial visual point cloud; and
registering each of new images in the image sequence to obtain a point cloud of each of the new images; and
combining the point cloud of each of the new images into the initial visual point cloud.

11. The electronic device according to claim 10 , wherein when the instructions are executed by the at least one processor, the at least one processor are caused to further implement the following actions:
during registering each of the new images, determining whether to optimize the distortion coefficients according to a change curve of distortion coefficient.

12. The electronic device according to claim 8 , wherein when the 3-dimensional reconstruction is performed on a target area of each image of the image sequence, the initial values of the distortion coefficients are 0, and the target area does not comprise an edge area of the image.

13. The electronic device according to claim 8 , wherein the nominal intrinsic parameters of the camera comprise a nominal focal length and a nominal optical center value of the camera.

14. The electronic device according to claim 8 , wherein the image sequence is an image sequence collected by crowdsourcing.

15. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to make the computer implement a 3-dimensional reconstruction method, the method comprising:
obtaining an image sequence pre-collected by a camera;
obtaining nominal intrinsic parameters of the camera, and preset initial values of distortion coefficients of the camera, the camera being a camera for collecting the image sequence;
calculating fluctuation ranges of the distortion coefficients, comprising: extracting distorted line features of a part of images in the image sequence; and calculating the fluctuation ranges of the distortion coefficients according to the distorted line features;
performing the 3-dimensional reconstruction on the image sequence based on the nominal intrinsic parameters, the initial values of the distortion coefficients, and the fluctuation ranges of the distortion coefficients; and
during the 3-dimensional reconstruction, optimizing an obtained visual point cloud and a camera pose.

16. The non-transitory computer-readable storage medium according to claim 15 , wherein the method further comprises:
during the 3-dimensional reconstruction, optimizing a semantic element in the obtained visual point cloud, so that a point cloud of the semantic element meets geometric features corresponding to the semantic element.

Assignees

• Beijing Baidu Netcom Sci & Tech Co Ltd
• Apollo Intelligent Driving Tech Beijing Co Ltd

Inventors

• Cai Yuzhan
• Zheng Chao
• Yan Chao
• Zhang Hantian

Classifications

• G06T7/75

  involving models · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06T17/00 Primary

  Three-dimensional [3D] modelling for computer graphics · CPC title

• G06T7/80

  Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration · CPC title

• G06T7/579 Primary

  from motion · CPC title