Method, device, computer system, and mobile apparatus for generating three-dimensional point cloud

Patent metadata

Field	Value
Publication number	US-11004261-B2
Application number	US-201916413313-A
Country	US
Kind code	B2
Filing date	May 15, 2019
Priority date	Nov 16, 2016
Publication date	May 11, 2021
Grant date	May 11, 2021

Abstract

A method of generating a three-dimensional point cloud includes obtaining a plurality of image data sources through a plurality of sensors, and performing fusion processing according to the plurality of image data sources to obtain the three-dimensional point cloud.

First claim

What is claimed is:

1. A method of generating a three-dimensional point cloud, comprising:
obtaining a plurality of image data sources through a plurality of sensors, and
performing fusion processing according to the plurality of image data sources to obtain the three-dimensional point cloud,
wherein performing the fusion processing according to the plurality of image data sources includes:
determining, according to a first image data source in the plurality of image data sources, an initial parameter for processing a second image data source in the plurality of image data sources; and
processing the second image data source according to the initial parameter.

2. The method according to claim 1 , wherein obtaining the plurality of image data sources through the plurality of sensors includes obtaining the plurality of image data sources through at least two of a depth camera, a binocular camera, a monocular camera, or a position angle sensor.

3. The method according to claim 1 , wherein performing the fusion processing further includes performing fusion processing on a plurality of depth data sources in the plurality of image data sources to obtain the three-dimensional point cloud.

4. The method according to claim 3 , wherein performing the fusion processing on the plurality of depth data sources in the plurality of image data sources includes performing point matching and screening operations according to the plurality of depth data sources.

5. The method according to claim 1 , wherein:
the first image data source includes an image data source obtained by a depth camera, and the second image data source includes an image data source obtained by a binocular camera, and
determining the initial parameter for processing the second image data source according to the first image data source includes determining an initial feature point correspondence for processing the image data source obtained by the binocular camera according to the image data source obtained by the depth camera.

6. The method according to claim 5 , wherein determining the initial feature point correspondence for processing the image data source obtained by the binocular camera according to the image data source obtained by the depth camera includes:
determining a depth of a feature point according to the image data source obtained by the depth camera;
determining a parallax of the feature point with respect to the binocular camera according to the depth of the feature point; and
determining the initial feature point correspondence according to the parallax of the feature point with respect to the binocular camera.

7. The method according to claim 5 , wherein processing the second image data source according to the initial parameter includes:
determining a correspondence of feature points of two images obtained by the binocular camera according to the initial feature point correspondence;
determining parallax information according to the correspondence of the feature points of the two images;
determining depth information according to the parallax information; and
generating the three-dimensional point cloud according to the depth information.

8. The method according to claim 1 , wherein:
the first image data source includes a camera position and attitude data source, and the second image data source includes a depth data source, and
determining the initial parameter for processing the second image data source according to the first image data source includes determining initial camera position and attitude for performing fusion processing on the depth data source according to the camera position and attitude data source.

9. The method according to claim 8 , wherein processing the second image data source according to the initial parameter includes:
determining camera position and attitude corresponding to depth data in the depth data source according to the initial camera position and attitude; and
performing fusion processing on the depth data in the depth data source according to the camera position and attitude to obtain the three-dimensional point cloud.

10. The method according to claim 1 , wherein:
the first image data source includes a camera position and attitude data source, and the second image data source includes an image data source obtained by a monocular camera; and
determining the initial parameter for processing the second image data source according to the first image data source includes determining initial camera position and attitude for performing point cloud modeling on the image data source obtained by the monocular camera according to the camera position and attitude data source.

11. The method according to claim 10 , wherein processing the second image data source according to the initial parameter includes constructing a point cloud model using the image data source obtained by the monocular camera according to the initial camera position and attitude.

12. A mobile apparatus comprising:
a plurality of sensors configured to obtain a plurality of image data sources; and
a processor configured to perform fusion processing according to the plurality of image data sources to obtain a three-dimensional point cloud,
wherein the processor is configured to:
determine, according to a first image data source in the plurality of image data sources, an initial parameter for processing a second image data source in the plurality of image data sources; and
processing the second image data source according to the initial parameter.

13. The mobile apparatus according to claim 12 , wherein the plurality of sensors include at least two of a depth camera, a binocular camera, a monocular camera, or a position angle sensor.

14. The mobile apparatus according to claim 12 , wherein the processor is further configured to perform fusion processing on a plurality of depth data sources in the plurality of image data sources to obtain the three-dimensional point cloud.

15. The mobile apparatus according to claim 12 , wherein:
the first image data source includes an image data source obtained by a depth camera, and the second image data source includes an image data source obtained by a binocular camera, and
the processor is further configured to determine an initial feature point correspondence for processing the image data source obtained by the binocular camera, according to the image data source obtained by the depth camera.

16. The mobile apparatus according to claim 12 , wherein:
the first image data source includes a camera position and attitude data source, and the second image data source includes a depth data source; and
the processor is further configured to determine initial camera position and attitude for performing fusion processing on the depth data source, according to the camera position and attitude data source.

17. The mobile apparatus according to claim 12 , wherein:
the first image data source includes a camera position and attitude data source, and the second image data source includes an image data source obtained by a monocular camera, and
the processor is further configured to determine initial camera position and attitude for performing point cloud modeling on the image data source obtained by the monocular camera, according to the camera position and attitude data source.

18. The mobile apparatus according to claim 12 , wherein the mobile apparatus is one of a drone, an unmanned vessel, or a robot.

Assignees

• Sz Dji Technology Co Ltd

Inventors

• Yao Yao
• Zhao Kaiyong
• Zheng Shizhen
• Pan Cihui

Classifications

• G06T7/55 Primary

  from multiple images · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06T7/593

  from stereo images · CPC title

• G06T17/00 Primary

  Three-dimensional [3D] modelling for computer graphics · CPC title