Method and apparatus for fusing point cloud data

Patent metadata

Field	Value
Publication number	US-10789771-B2
Application number	US-201816232721-A
Country	US
Kind code	B2
Filing date	Dec 26, 2018
Priority date	Dec 29, 2017
Publication date	Sep 29, 2020
Grant date	Sep 29, 2020

Abstract

A method and apparatus for fusing point cloud data, and a computer readable storage medium are provided. Some embodiments of the method can include: acquiring a first image and a second image, the first image and the second image being respectively associated with a first frame of point cloud data and a second frame of point cloud data acquired for a given scene; determining a point cloud movement matrix between the first frame of point cloud data and the second frame of point cloud data on the basis of the first image and the second image; and fusing the first frame of point cloud data with the second frame of point cloud data on the basis of the point cloud movement matrix.

First claim

What is claimed is:

1. A method for fusing point cloud data, comprising:
acquiring a first image and a second image, the first image and the second image being respectively associated with a first frame of point cloud data and a second frame of point cloud data acquired for a given scene;
determining a point cloud movement matrix between the first frame of point cloud data and the second frame of point cloud data on the basis of the first image and the second image; and
fusing the first frame of point cloud data with the second frame of point cloud data on the basis of the point cloud movement matrix,
wherein the determining a point cloud movement matrix between the first frame of point cloud data and the second frame of point cloud data comprises:
determining an image movement matrix between the first image and the second image;
acquiring a position and orientation movement matrix between the images and the point cloud data; and
determining the point cloud movement matrix on the basis of the image movement matrix and the position and orientation movement matrix,
wherein the method is performed by at least one processor.

2. The method according to claim 1 , wherein the acquiring a first image and a second image comprises:
acquiring information associated with the first frame of point cloud data and the second frame of point cloud data, wherein the information includes at least one of a positioning signal, an inertial navigation signal, or a matching quality of historical point cloud data; and
acquiring, in response to the information not satisfying a predetermined condition, the first image and the second image.

3. The method according to claim 2 , wherein the acquiring, in response to the information not satisfying a predetermined condition, the first image and the second image comprises:
acquiring the first image and the second image, in response to a quality of the positioning signal being lower than a first threshold quality or the quality of the inertial navigation signal being lower than a second threshold quality, and the matching quality of the historical point cloud data being lower than a third threshold quality.

4. The method according to claim 2 , further comprising:
determining, in response to the information satisfying the predetermined condition, the point cloud movement matrix between the first frame of point cloud data and the second frame of point cloud data on the basis of at least one of:
the inertial navigation signal, or
matching between the first frame of point cloud data and the second frame of point cloud data; and
fusing the first frame of point cloud data with the second frame of point cloud data on the basis of the point cloud movement matrix.

5. The method according to claim 1 , wherein the determining an image movement matrix between the first image and the second image comprises:
extracting matching characteristics in the first image and the second image; and
determining the image movement matrix on the basis of the matching characteristics.

6. The method according to claim 1 , wherein the acquiring a position and orientation movement matrix comprises:
determining a position of a ladar for acquiring the point cloud data disposed on an acquisition entity and the position of a camera for acquiring the images; and
determining the position and orientation movement matrix on the basis of the position of the ladar and the position of the camera.

7. The method according to claim 1 , wherein the fusing the first frame of point cloud data with the second frame of point cloud data on the basis of the point cloud movement matrix comprises:
determining point cloud frame positions and orientations of the first frame of point cloud data and the second frame of point cloud data in a world coordinate system;
determining, on the basis of positions of points in the first frame of point cloud data and the second frame of point cloud data in a local coordinate system of the point cloud data and the point cloud frame positions and orientations, point positions and orientations of the points in the world coordinate system; and
fusing on the basis of the point positions and orientations.

8. An apparatus for fusing point cloud data, comprising:
at least one processor; and
a memory storing instructions, the instructions when executed by the at least one processor, cause the at least one processor to perform operations, the operations comprising:
acquiring a first image and a second image, the first image and the second image being respectively associated with a first frame of point cloud data and a second frame of point cloud data acquired for a given scene;
determining a point cloud movement matrix between the first frame of point cloud data and the second frame of point cloud data on the basis of the first image and the second image; and
fusing the first frame of point cloud data with the second frame of point cloud data on the basis of the point cloud movement matrix,
wherein the determining a point cloud movement matrix between the first frame of point cloud data and the second frame of point cloud data comprises:
determining an image movement matrix between the first image and the second image;
acquiring a position and orientation movement matrix between the images and the point cloud data; and
determining the point cloud movement matrix on the basis of the image movement matrix and the position and orientation movement matrix.

9. The apparatus according to claim 8 , wherein the acquiring a first image and a second image comprises:
acquiring information associated with the first frame of point cloud data and the second frame of point cloud data, wherein the information includes at least one of a positioning signal, an inertial navigation signal, or a matching quality of historical point cloud data; and
acquiring, in response to the information not satisfying a predetermined condition, the first image and the second image.

10. The apparatus according to claim 9 , wherein the acquiring, in response to the information not satisfying a predetermined condition, the first image and the second image comprises:
acquiring the first image and the second image, in response to a quality of the positioning signal being lower than a first threshold quality or the quality of the inertial navigation signal being lower than a second threshold quality, and the matching quality of the historical point cloud data being lower than a third threshold quality.

11. The apparatus according to claim 8 , wherein the operations further comprise:
determining, in response to the information satisfying the predetermined condition, the point cloud movement matrix between the first frame of point cloud data and the second frame of point cloud data on the basis of at least one of:
the inertial navigation signal, or
matching between the first frame of point cloud data and the second frame of point cloud data; and
fusing the first frame of point cloud data with the second frame of point cloud data on the basis of the point cloud movement matrix.

12. The apparatus according to claim 8 , wherein the determining an image movement matrix between the first image and the second image comprises:
extracting matching characteristics in the first image and the second image; and
determining the image movement matrix on the basis of the matching characteristics.

13. The apparatus according to claim 8 , wherein the acquiring a position and orientation movement matrix comprises:
determining a position of a ladar for acquiring the point cloud data disposed on an acquisition entity and the position of a camera for acquiring the images; an

Assignees

• Baidu online network technology beijing co ltd

Inventors

• Zhou Wang
• Yan Miao
• Zhan Yifei
• Duan Xiong
• Ma Changjie
• Lang Xianpeng
• Jin Yonggang

Classifications

• G06V10/751

  Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching · CPC title

• G06T2200/04

  involving 3D image data · CPC title

• G06T7/30 Primary

  Determination of transform parameters for the alignment of images, i.e. image registration · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06T7/74

  involving reference images or patches · CPC title