Providing a point cloud using a surveying instrument and a camera device

Patent metadata

Field	Value
Publication number	US-10041793-B2
Application number	US-201615132837-A
Country	US
Kind code	B2
Filing date	Apr 19, 2016
Priority date	Apr 21, 2015
Publication date	Aug 7, 2018
Grant date	Aug 7, 2018

Abstract

Some embodiments of the invention include a method for providing a 3D-point cloud using a geodetic surveying instrument. Some embodiments also include an image capturing unit. A method, according to some embodiments may include scanning a surrounding with the surveying instrument according to a defined scanning region with at least partly covering the object and generating a scanning point cloud corresponding to the scanning region with reference to a surveying coordinate system which is defined by the surveying instrument and generating a first image on side of the surveying instrument covering a region basically corresponding to the scanning region, the first image representing a reference image the pose of which is known with reference to the surveying coordinate system due to the position and orientation of the surveying instrument for acquiring data the first image is base on.

First claim

What is claimed is:

1. A method for providing a three dimensional (3D-) point cloud using a geodetic surveying instrument, the method comprising:
scanning a surrounding with the geodetic surveying instrument according to a defined scanning region;
generating a scanning point cloud corresponding to the scanning region with reference to a surveying coordinate system which is defined by the surveying instrument;
generating a first image on a side of the surveying instrument covering a region at least basically corresponding to the scanning region, the first image representing a reference image whereby a pose of the reference image is known with reference to the surveying coordinate system due to the position and orientation of the surveying instrument for acquiring data the first image is based on;
capturing a first series of at least two images with an image capturing unit, the images having respective poses and partly cover the scanning region and an imaging region; and
applying a Structure-from-Motion- (SfM) or a Simultaneous Localization and Mapping- (SLAM-) algorithm at least comprising:
identifying a first set of image points based on the reference image and on the first series of at least two images, the image points representing a first set of reference points of a reference point field, wherein each point of the first set of reference points appears in the reference image and in at least one of the images of the first series of images and position information regarding the points of the first set of reference points is provided by the scanning point cloud, and
determining the poses for the images of the first series of images with reference to the surveying coordinate system, using the first set of reference points and the position information, the poses representing respective positions and orientations of the image capturing unit relative to the surveying instrument at capturing the images of the first series of images.

2. The method according to claim 1 , further comprising:
determining a second set of reference points in an imaging region based on the poses for the images of the first series of images, wherein
the imaging region is different from the scanning region, and
a position information regarding a 3D-position of at least one point of the second set of reference points is derived, the position information being referenced to the surveying coordinate system,
in particular wherein the scanning point cloud is actualised by adding the derived 3D-position information to the scanning point cloud.

3. The method according to claim 2 , further comprising:
capturing a second series of at least one image with the image capturing unit, wherein the at least one image of the second series at least partly covers the second set of reference points and/or at least one point of the first set of reference points,
determining the pose for the at least one image of the second series with reference to the surveying coordinate system, using at least three reference points of the first and/or the second set of reference points covered with the at least one image and using respective position information.

4. The method according to claim 3 , further comprising:
determining the pose for at least one image of the second series with reference to the surveying coordinate system solely using reference points of the second set of reference points.

5. The method according to claim 3 , wherein:
at least one of the images of the second series of at least one image partly covers the scanning region, wherein a further reference point (T) is determined based on the reference image and the at least one of the images of the second series of at least one image.

6. The method according to claim 3 , further comprising:
applying a bundle adjustment procedure based on the poses of the images of the first series of at least two images and second series of at least one image and the first and the second set of reference points.

7. The method according to claim 1 , wherein:
computing an image point cloud for at least a part of the imaging region based on the poses of at least two of the images captured with the image capturing unit the image point cloud at least comprising 3D-position information according to a respective commonly covered region of the at least two captured images and being referenced with respect to the surveying coordinate system.

8. The method according to claim 7 , wherein:
quality information concerning the position accuracy of points of the scanning point cloud and of points of the image point cloud is generated.

9. The method according to claim 8 , wherein:
the quality information is displayed in an image,
wherein a first region representing the scanning point cloud is overlaid by a first marking and a second region representing the image point cloud is overlaid by a second marking, the markings being represented by regions of different colours, wherein
a region covered by both the scanning and the image point cloud is overlaid by a third marking or
the points of the region covered by both the scanning and the image point cloud are filtered so that only points of the scanning point cloud or only points of the image point cloud are provided, wherein the filtering is induced by an input of an operator.

10. The method according to claim 7 , wherein the computing an image point cloud includes applying a dense-matching algorithm.

11. The method according to claim 1 , further comprising:
compensating position errors occurring with calculating the 3D-position information or the poses of images on basis of scanning point cloud data, wherein the compensation is performed for each successive image data provided by successively captured images with the image capturing unit.

12. The method according to claim 1 , wherein:
the surveying instrument and the image capturing unit being structurally separated from each other, wherein the images capturing unit is embodied as a mobile digital camera or provided by a mobile phone.

13. The method according to claim 12 , wherein the geodetic surveying instrument comprises a laser scanner or total station.

14. The method according to claim 1 , wherein:
the method being conducted directly and instantaneously during a measuring process, wherein data processing, the processing unit receiving data with respect to the:
scanning of the surrounding,
generating of the first image on the side of the surveying instrument, and
capturing of at least two further images with the image capturing unit.

15. The method according to claim 1 , wherein:
a live-image is displayed on side of the image capturing unit imaging an actual field of view of the image capturing unit, wherein a region which both is actually imaged by the image capturing unit and covered or uncovered by the scanning point cloud is marked in the live-image, in particular wherein a part of the object being covered by one of the images captured with the image capturing unit is marked in the live-image.

16. One or more non-transitory computer-readable media storing one or more programs that are configured, when executed, to cause one or more processors to execute the method as recited in claim 1 .

17. The method according to claim 1 , wherein the data processing regarding the Structure-from-Motion- (SfM) or the SLAM-algorithm is performed by a processing unit during the measuring process.

18. A system for providing a three dimensional (3D-) point cloud comprising:
a geodetic surveying instrument;
a support structure being mounted on a base of th

Assignees

• Hexagon Technology Ct Gmbh

Inventors

• Metzler Bernhard
• Velizhev Alexander
• Fidler Thomas

Classifications

• G06V10/806

  of extracted features · CPC title

• G06F18/253

  of extracted features · CPC title

• H04N23/698

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• G01C15/002 Primary

  Active optical surveying means (optical plumbing G01C15/105) · CPC title

• G06T7/521

  from laser ranging, e.g. using interferometry; from the projection of structured light · CPC title