Creation of a 3D city model from oblique imaging and lidar data

Patent metadata

Field	Value
Publication number	US-11222204-B2
Application number	US-201715468057-A
Country	US
Kind code	B2
Filing date	Mar 23, 2017
Priority date	Mar 23, 2016
Publication date	Jan 11, 2022
Grant date	Jan 11, 2022

Abstract

A method and a hybrid 3D-imaging device for surveying of a city scape for creation of a 3D city model. According to the invention, lidar data is acquired simultaneously with the acquisition of imaging data for stereoscopic imaging, i.e. acquisition of imaging and lidar data in one go during the same measuring process. The lidar data is combined with the imaging data for generating a 3D point cloud for extraction of a 3D city model, wherein the lidar data is used for compensating and addressing particular problem areas of generic stereoscopic image processing, in particular areas with unfavourable lighting conditions and areas where the accuracy and efficiency of stereoscopic point matching and point extraction is strongly reduced.

First claim

The invention claimed is:

1. A hybrid 3D-imaging device for surveying of a city scape to create a 3D city model of the surveyed city scape, the hybrid 3D-imaging device comprising:
a stereo imaging device for generating stereoscopic imaging data for an area of the city scape;
a lidar device for generating lidar data for the area of the city scape, wherein the hybrid 3D-imaging device is configured to acquire the stereoscopic-imaging data and the lidar data during the same measuring process in one go when surveying the city scape; and
a control and processing unit being adapted for:
controlling the stereo imaging device and the lidar device,
generating a 3D point cloud for the area of the city scape based on the stereoscopic imaging and the lidar data by stereoscopic image processing, and
generating a 3D city model with automated building model extraction, based on the 3D point cloud,

wherein:
the control and processing unit is adapted to:
processing of the stereoscopic imaging data within a stereoscopic method,
assessing in real time with the generating of the stereoscopic imaging data a quality of the stereoscopic imaging data for stereoscopic imaging, wherein at least part of the stereoscopic imaging data being assigned to a first class, the first class indicating a critical area of the stereoscopic method where accuracy and efficiency of the processing of the stereoscopic imaging data within the stereoscopic method are below average, the first class being defined by at least one of:
a defined first classification criterion for a 2D classification of the stereoscopic imaging data, and
a generation of a first auxiliary 3D point cloud, by using the stereoscopic imaging data while disregarding the lidar data, and a defined second classification criterion within the first auxiliary 3D point cloud, and

combining the stereoscopic imaging data of the first class with lidar data corresponding to the critical area determined based on the real time quality assessment of the stereoscopic imaging data and comprising a fraction of the area of the city scape defined by the stereoscopic imaging data of the first class.

2. The hybrid 3D-imaging device according to claim 1 , wherein the control and processing unit is adapted for generating the 3D point cloud with a quality assessment of the lidar data, wherein at least part of the lidar data being assigned to a second class being defined by at least one of:
a defined third classification criterion within a second auxiliary 3D point cloud solely based on the lidar data, and
a defined first comparison criterion for comparing the first auxiliary 3D point cloud solely based on the stereoscopic imaging data with a third auxiliary point cloud based on a combination of the stereoscopic imaging and the lidar data,
wherein the stereoscopic imaging data of the first class is only combined with lidar data of the critical area where the critical area is overlapping the fraction of the area of the city scape defined by the lidar data of the second class.

3. The hybrid 3D-imaging device according to claim 1 , wherein the control and processing unit being adapted for generating the 3D point cloud with a quality assessment of the lidar data based on data classification, wherein at least part of the lidar data is assigned to a third class being defined by at least one of:
a defined fourth classification criterion within the second auxiliary 3D point cloud solely based on the lidar data, and
a defined second comparison criterion for comparing the first auxiliary 3D point cloud solely based on the stereoscopic imaging data with the third auxiliary point cloud based on a combination of the stereoscopic imaging and the lidar data,
wherein the stereoscopic imaging data corresponds to the fraction of the area of the city scape defined by the lidar data of the third class being combined with the lidar data of the third class.

4. The hybrid 3D-imaging device according to claim 3 , wherein at least one of the first to fourth classification criteria being based on a semantic classification, in particular wherein the semantic classification comprises semantic classifiers defining at least one of:
shadowing,
a region with an occlusion,
a region with vegetation, and
a region with a homogeneous surface.

5. The hybrid 3D-imaging device according to claim 1 , wherein at least one of the first to second comparison criteria being based on at least one of:
a signal to noise threshold,
a resolution threshold, and
a systematic error threshold.

6. The hybrid 3D-imaging device according to claim 1 , wherein:
the hybrid 3D-imaging device is built as one single hybrid 3D-imaging device, the single hybrid 3D-imaging device being adapted for acquiring lidar data for a selected region of the area within the surveyed city scape, based on at least one of:
an a-priori model of the surveyed city scape, and
an analysis of the stereoscopic imaging data, and

the control and processing unit being adapted for generating the 3D point cloud with a photogrammetric method, and being adapted for processing at least one of:
nadir and/or oblique images,
multispectral images,
normalized difference vegetation index images,
building footprints, and
a reference model comprising at least one of a digital terrain model, a digital elevation model, and a digital surface model.

7. A hybrid 3D-imaging device for surveying of a city scape to create a 3D city model of the surveyed city scape, the hybrid 3D-imaging device comprising:
a stereo imaging device for generating stereoscopic imaging data for an area of the city scape;
a lidar device for generating lidar data for the area of the city scape, wherein the hybrid 3D-imaging device is configured to acquire the stereoscopic imaging data and the lidar data during the same measuring process in one go when surveying the city scape; and
a control and processing unit being adapted for:
controlling the stereo imaging device and the lidar device,
generating a 3D point cloud for the area of the city scape based on the stereoscopic imaging and the lidar data by stereoscopic image processing, and
generating a 3D city model with automated building model extraction, based on the 3D point cloud,

wherein:
the control and processing unit is adapted to:
processing of the stereoscopic imaging data within a stereoscopic method,
assessing in real time with the generating of the stereoscopic imaging data a quality of the stereoscopic imaging data for stereoscopic imaging, wherein at least part of the stereoscopic imaging data being assigned to a first class, the first class indicating a critical area of the stereoscopic method where accuracy and efficiency of the processing of the stereoscopic imaging data within the stereoscopic method are below average, the first class being defined by at least:
a generation of a first auxiliary 3D point cloud by the stereoscopic method, by using the stereoscopic imaging data while disregarding the lidar data, and a defined first classification criterion within the first auxiliary 3D point cloud, and

combining the stereoscopic imaging data of the first class with lidar data corresponding to the critical area determined based on the real time quality assessment of the stereoscopic imaging data and comprising a fraction of the area of the city scape defined by the stereoscopic imaging data of the first class.

8. The hybrid 3D-imaging device according to claim 7 , wherein the control and processing unit is adapted for generating the 3D point cloud with a quality assessment of the lidar data, wherein at least part of the lidar data being assigned to a second class being defined by at least one of:
a defined second classification criterion within a second a

Assignees

• Leica Geosystems Ag
• Airborne Hydrography Ab

Inventors

• Wagner Rüdiger
• Ekelund Anders
• Axelsson Andreas
• Steinmann Patrick

Classifications

• G01S17/89 Primary

  for mapping or imaging · CPC title

• G06V20/17

  taken from planes or by drones · CPC title

• G06V20/10

  Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) · CPC title

• G06V20/176

  Urban or other man-made structures · CPC title

• G06F18/2431

  Multiple classes · CPC title