1.

INTRODUCTION

We are currently living in an unprecedented era of urbanization, with around 3.5 billion people living in urban areas and it is expected that more than two-thirds of the population will live in cities by 2050¹. At the same time, global temperatures are steadily increasing over time, primarily caused by anthropogenic activities, known as climate change. Climate change has significant impacts on the earth’s climate and weather conditions, including increased likelihood of excessive rainfall, prolonged periods of heat waves and droughts and increased risks of flooding².

Cities are becoming increasingly vulnerable to the negative impacts of climate change and, consequently, the urgent need for incorporating climate adaptation strategies into urban planning is frequently highlighted. In this context, Urban Digital Twins (UDTs) have emerged as a promising technology that can potentially aid urban planning processes and facilitate the implementation of climate adaptive initiatives3.

Digital twins (DTs) have been found useful in manufacturing, construction, and maintenance. Adapting DTs to serve cities, the question arises of what an urban digital twin should contain and how it should be orchestrated to serve a city’s dynamical ecosystem, along with how to enhance the efficiency of the city⁴. Talking about “digital twins”, we refer to management of the “smart cities” and creating better living conditions for their residents and a better business environment through the use of digital technologies.

There are many definitions for “smart cities”. A digital city is one whose procedures, communication and information have all been digitalized. An intelligent city is a digital city that has a layer of intelligence that can make high-level decisions based on a level of artificial intelligence. A smart city is an intelligent city where application is focused on practical use and user experience⁵.

Remote sensing is one of the most effective tools for quickly and accurately collecting and updating spatial data and information. Combining various UAV surveys such as photogrammetry and airborne laser scanning with ground-based laser scanners allows the creation of precise digital twins of cities. One of the main challenges in combining data collected by different sensors is their georeferencing in a chosen coordinate system with the required accuracy, especially in densely urbanized environments where the massive use of GNSS does not always provide sufficiently reliable results. In the present study, a methodology for combining the application of SLAM lidar imagery in urban environments georeferenced by control points from orthophotos collected by UAVs is investigated.

Creating digital twins of urban areas requires combining data and measurements from different sensors and instruments:

The densely built urban environment poses additional difficulties in the collection and processing of spatial data. Aerial photography should only be carried out with unmanned aerial vehicles certified to fly over populated areas. Furthermore, aerial photography is not sufficient to create an accurate digital twin. Ground scanning in street view mode is needed to help collect data on infrastructure networks, building facades and spaces between buildings, the street network, sealed spaces, vegetation and other characteristic objects in the urban environment.

Apart from the actual data collection with different sensors and instruments, an important element is their georeferencing and integration into a single model. In order to be able to georeference the data acquired with a SLAM scanner, it is necessary to mark and define control points of sufficient density. Their coordinates have to be measured with GNSS. In urban environments, however, the use of GNSS is limited by multiple factors - insufficient open horizon and poor satellite geometry, reception of reflected signal from high objects, inability to achieve a fixed solution in RTK mode, etc. This often necessitates the use of classical geodetic methods to determine the coordinates of the control points via a total station, which significantly increases the measurement time.

In the present study, a fast method is developed to combine aerial photogrametry and ground-based laser scanning, in which control points for georeferencing the point cloud are determined directly from the orthopho images without the need for prior GNSS measurements. The accuracy of the acquired data is evaluated, demonstrating the effectiveness of the method and its applicability for acquiring high-precision spatial data in urban environments.

In order to demonstrate the potential of the methodological approach described above, it was applied to the Lyulin residential complex - one of the largest in the Bulgarian capital Sofia. This complex is a typical example of the residential complexes of Eastern European cities with many high-rise residential buildings combined with sealed internal spaces and unstructured green areas. The results obtained through the proposed methodology would support the rapid collection of high fidelity spatial data for the creation of digital doubles for sustainable urban planning and management, especially in the context of the growing need for effective adaptation to climate change.

2.

METHODS

2.2

Observation Equipment

The observation equipment includes three type of instruments.

Photogrammetry was performed with fixed-wing UAV, model eBeeX (Figure 1a), manufactured by the company AgEagle Aerial Systems. This type of UAV is the only certified Class A2 platform that can fly over people as well as in beyond constant visual line-of-sight (BVLOS) situations. With the on-demand RTK / PPK capabilities of survey drone eBee X, you can gather data without ground control points (GCPs) while maintaining a GSD down to 1.5 cm. The eBee X professional drone is compatible with the industry’s leading base stations. (https://ageagle.com/drones/ebee-x/).

Figure 1.

Fixed-wing UAV, model eBeeX (a) and photogrammetric sensor S.O.D.A 3D (b)

To gather the necessary information, the platform is equipped with a dedicated integrated photogrammetric sensor S.O.D.A 3D (Figure 1b). The S.O.D.A. 3D mapping camera is a professional drone photogrammetry camera that changes orientation during flight to capture three images (2 oblique, 1 nadir) every time, instead of just one, for a much wider field of view.

For flight planning and data capture eMotion flight planning software was used. The software is easy to use and reliable allowing multi-drone flights.

The inner space was scanned with geoSLAM ZEB Horizon laser scanners (Figure 2). GeoSLAM technology gives opportunities to collect geospatial data from some of the most difficult environments, whether they are indoor, outdoors, underground – everywhere. (https://geoslam.com/)., SLAM (Simultaneous Localization and Mapping) is the process of mapping an area while keeping track of the location of the device within that area. This is what makes mobile mapping possible, allowing the digitization of large areas in much shorter spaces of time. SLAM systems simplify data collection, providing an avenue for scanning outdoor or indoor environments.

Figure 2.

GeoSLAM ZEB Horizon laser scanners

The ZEB Horizon provides fast work scanning both simple and complex environments with 300,000 scanner points per second and 3D scanning with a survey grade relative accuracy of up to 6mm dependent on the environment. This gives an accurate scans and fine details. ZEB Horizon can be mounted to a drone, a backpack or a vehicle and deliver a range of 100m. It is one of the best decision for scanning urban territories – facades of the buildings with all details, inner spaces, infrastructural network, transport network, green infrastructure etc.

Used Handheld SLAM system ZEB Horizon does not have internal GNSS receivers to provide accurate coordinates. There is therefore a requirement for external methods to georeference data. This is done using pre-determined control points that are identified during a scan. Adding more control points will increase the accuracy of the georeferencing process – a minimum of 3 is required, but more will yield better results. When using a GNSS receiver to measure coordinates of control points, they have to be sufficiently spread out to ensure better coverage and easier identification in post processing. There are the 3 methods for accurately georeferencing data with GEOSLAM LIDAR.

The first method requires at least 3 reference points to be marked and measured in the scan area, using a GNSS receiver. As a handheld LiDAR system performs a scan, like GeoSLAM’s ZEB Horizon, it is placed on the points for a minimum of 10 seconds via the reference base. The scanner notes that the system has stopped for a prolonged amount of time and registers the point locally. This method requires more time and it is applicable for small objects.

The second method like the method above, georeferencing using a reflective target requires a minimum of 3 control points using GNSS. Placing the targets on the known control points gives users the freedom to move through their data without stopping. The targets give off higher intensity values due to the reflectiveness, and can therefore be highlighted within the processed scan and the software will georeference the data automatically. This method provide the opportunity for georeferencing data captured with accessories like backpack or vehicle mounts because there isn’t a requirement to stop.

In both described methods, a GNSS receiver is used for pre-determination of the control points. This requires the control points to be placed on an open area where the GNSS can provide fixed decisions. Into inner spaces surrounded with high buildings or trees, GNSS measurements are not very efficient because of bad satellite geometry and multipath of the signal.

The third way to georeference mobile LiDAR data is via a backpack and GNSS receiver. By capturing real-time coordinates during a scan and matching the scanning and GNSS trajectories, you can simply georeference the resulting data. This method is useful for those wanting to capture and georeference large datasets but it requires good GNNS signal during the whole period of scanning. The method is applicable in open areas such along the main streets but not very effective into inner small streets and between high buildings and trees.

A Geodetic GNSS Receiver i50 (Figure 3) has been used for control measurements. The i50 GNSS receiver incorporates a GNSS engine, GNSS antenna, internal radio, 4G cellular modem, Bluetooth, Wi-Fi, and dual-battery in a ruggedized and miniature unit that is easy for you to set up an all-in-one RTK rover or mobile base station. Bluetooth and Wi-Fi technology provide cable-free communication between the receiver and controller. The receiver is used as the part of a RTK GNSS system with CHC LandStar 7 software.

Figure 3.

Geodetic GNSS Receiver i50

2.3

Description of the method

The present investigation involved the integrated use of three technological tools, the use of which in a systematic algorithm to achieve a high level of geometric and semantic digital replication of the urban environment for the purpose of sustainable urban planning aimed at effective climate adaptation. The applied algorithm involves the use of the three technological tools in the following sequence:

• 1) Selection and marking of ground control pointsu using highly reflective markers that are visible both as a consequence of the laser scanning used and in the photogrammetric data processing. Control points were planned with sufficient density and even distribution in the study area. All points were marked with reflective marks with size 100x100 mm (Figure 4).

• 2) Terrestrial laser scanning using the ZEB Horizon handheld SLAM system along a pre-mapped route. In this stage, the selected area was crawled and scanned with a backpack-mounted ZEB Horizon geoSLAM. A method for determining the control points without GNSS is applied and analyzed. The coordinates of the marked control points are read directly from the created orthophoto map by aerial photogrammetry. For this purpose, all control points are marked with highly reflective marks. In this way, the control points have higher intensity values due to the reflectivity and they are easily identified both on the orthophoto map (Figure 5a and 5b) and in the laser scan point cloud.

• 3) For accuracy control purposes, some of the ground points were measured by Geodetic GNSS Receiver i50 in RTK mode in the WGS84 UTM34N coordinate system. These data were subsequently used to evaluate and analyse the final results. For some of the points located in the inner spaces between the buildings, GNSS coordinates could not be determined to obtain a fixed RTK solution. This once again confirms the theory that in densely urbanized areas it is not possible to determine all control points with GNSS and necessary precision.

• 4) Territory capture by UAS EbeeX, by S.O.D.A 3D sensor and in RTK mode

• 5) Photogrammetric processing of the photogrammetric capture data in the Pix4D mapper platform and generation of an orthophotomosaic, DSM and high spatial resolution point cloud. Aerial photography was performed from a flight altitude of 120 m with a photogrammetric sensor S.O.D.A 3D and spatial resolution of 2.5 cm of the images. This ensured identification of control points at the ortophoto images with sufficient accuracy. During the flight, 155 aerial images were collected and total 0.365 sq. km were covered. The figure below (Figure 6) shows a view of the orthophotomosaic that was obtained as a result of the processing of the individual images. The data were processed in the Pix4Mapper platform environment and the resulting data have the following geolocation details (Table 1):

• 6) Taking the coordinates of the ground control points from the prepared models and using them to georeference the acquired laser scan data through SLAM ZEB Horizon (Figure 7), using point cloud coloring by intensity, which allows the identification of ground control points. The coordinates from the photogrammetric survey are assigned to these.

  The data were subsequently processed in the GEOSLAM Connect specialized platform environment and the resulting data were georeferenced based on the recognized control points in the intensity visualization mode of the resulting point cloud. Its georeferencing was performed in another platform environment- GeoSLAM Draw (Figure 8), using the coordinates of the ground points from the photogrammetric models (orthophotomap and DSM)

• 7) Integration of the two datasets - photogrammetry and terrestrial laser scanning - to obtain a complete geometric digital model of the urban area under study (Figure 9).

• 8) Analysis of the results and their accuracy;

Figure 4.

Control points marked with reflective signs on the area. The red points on the map are controlled via GNSS measurements

Figure 5.

Reflective mark on a sidewalk (a) and on the water shaft (b) into the ortophoto image

Figure 6.

Ortophoto map of the study area with detail

Figure 7.

Results from ground laser scanning with a backpack-mounted ZEB Horizon geoSLAM

Figure 8.

Georeferencing in GeoSLAM Draw, using the coordinates of the ground points from the photogrammetric models

Figure 9.

Integrated digital model of the urban area under study

Table 1.

Geolocation details of resulting photogrammetric data

Mn Enror [m]	Max Error [m]	Geolocation Error X[%]	Geolocation Error Y[%]	Geolocation Error Z[%]
-	-0.98	0.00	0.00	0.00
-0.98	-0.79	0.00	0.00	0.00
-0.79	-0.59	0.00	0.00	0.00
-0.59	-0.39	0.00	0.00	0.00
-0.39	-0.20	0.00	0.00	0.00
-0.20	0.00	45.16	49.03	48.39
0.00	0.20	54.84	50.97	51.61
0.20	0.39	0.00	0.00	0.00
0.39	0.59	0.00	0.00	0.00
0.59	0.79	0.00	0.00	0.00
0.79	0.98	0.00	0.00	0.00
0.98	-	0.00	0.00	0.00
Mean[m]	0.000071	-0.000191	-0.000104
Sigma [m]	0.013158	0.010208	0.019838
RMS Error [m]	0.013158	0.010210	0.019838

2.4

Analysis of the results accuracy

Direct Geo-referencing using RTK correction protocol does not need any GCPs. So, all GCPs are changed to Independent Check Points “ICP” which used for to checking the accuracy of this method. RMSE are calculated by using SFM approach to measure GCPs coordinates “from models generated by linear EO parameter from GNSS” and compare them to original coordinates from GNSS assumed as true value⁶ (Figure 10)

Figure 10.

Coordinates differences of check points

In the present research, 8 Independent Check Points are used and compared (Table 2).

Table 2

Check points measurement comparison

Marker	Y_ortho	X_ortho	H_ortho	Y_gnss	X_gnss	H_gnss	ΔY (m)	ΔX (m)	ΔH (m)
2	683892,975	4731069,925	637,133	683893,005	4731069,963	637,09	-0,030	-0,038	0,043
3	683909,246	4731080,105	637,370	683909,241	4731080,117	637,362	0,005	-0,012	0,008
4	683922,250	4731107,580	637,349	683922,233	4731107,620	637,394	0,017	-0,040	-0,045
5	683961,756	4731159,281	637,786	683961,774	4731159,320	637,781	-0,018	-0,039	0,005
6	683980,219	4731149,885	637,068	683980,222	4731149,925	637,057	-0,003	-0,040	0,011
7	683926,157	4731079,104	636,628	683926,123	4731079,110	636,671	0,034	-0,006	-0,043
8	683899,096	4731019,647	636,026	683899,087	4731019,676	635,982	0,009	-0,029	0,044
9	683975,502	4731059,412	634,950	683975,516	4731059,403	634,915	-0,014	0,009	0,035

The root mean squared errors of the coordinates are calculated using the formulas:

where ΔX, ΔY and ΔH represent the differences between the measured GNSS coordinates and the orthophoto map-reported coordinates of the validation points, and n is the total number of validation points. Xrmse and Yrmse are used to calculate the mean squared error in horizontal position XYrmse of a survey points:

A value of 5 cm was taken as the tolerance in the difference in position and height of the check points. This is the defined acceptable mean square error for geodetic reference points in the National regulation of the content, creation and maintenance of the cadastral map. The results show that differences between coordinates of the check points from areal images compared with coordinates from GNSS measurements do not exceed 4 cm in spite of difficult urban area.

3.

RESULTS

The results obtained from the scanning and photogrammetric imaging allow achieving high geometric accuracy and possible semantic classification of the parameters and structure of the urban environment. In this case, the photogrammetrically created point cloud provides an adequate mapping of the roof structures of buildings and provides data on the different land cover types, while the point cloud obtained from the ground laser scanning allows a geometrically accurate reflection of the parameters of the facades, vegetation and urban infrastructure elements. In this way, an effective representation of the multidimensional and complex structure of the urban environment is achieved, which is essential for its effective planning and management, including for the purposes of processes related to its adaptation to climate change.

4.

DISCUSSION

Within the past years, the development of high-quality Inertial Measurement Units (IMU) and GNSS technology and dedicated RTK (Real Time Kinematic) and PPK (Post-Processing Kinematic) solutions for UAVs promise accurate measurements of the exterior orientation (EO) parameters which allow direct georeferencing of the images⁷. In direct image georeferencing we have precise calculation of the position and orientation of the camera at the time of shooting, so that each pixel can be georeferenced to the Earth’s coordinate system without being ground measurements are required. The GNSS receiver of the UAV may communicate with the reference station in real time (using a radio link), adjustments can be applied simultaneously during the flight for real-time kinematic (RTK) correction. The use of this method can lead to significant reductions in field georeferencing activities and costs, saving time and resources for stabilization and GCP surveying.

The experimental results illustrate using of RTK systems on the UAV is fast and accurate than using of GCPs to register point cloud. It can be concluded the RTK system for direct geo-referencing and eliminating the GCPs from point cloud is sufficient and useful for rapid mapping systems. Moreover, the high resolution imagery collected from platforms have the capabilities to provide a quick overview of the disaster area with decimeter accuracy level as mentioned⁸.

RTK is advantageous for many surveying professionals because it increases safety. The technique eliminates the need for teams to maneuver through dangerous terrain to set GCPs while also efficiently saving time and productivity. RTK provides corrections to the drone onsite and is ideal for geo-tagging in absolute accuracy throughout flights in real-time. No GNSS post-processing is necessary with real-time correction. As an operational best practice, it’s ideal to use RTK on flights in open terrain and within two or three kilometers of the ground station to maintain the communications link. These flights can deliver highly accurate results without the need for using GCPs. This is an extremely helpful advantage for land surveyors working in dense vegetation, crops and other hard to distinguish terrain.

Another innovative technology that can complement and integrate successfully with drone-based photogrammetry is SLAM-based ground-based laser scanning systems for localization. They allow for accurate and detailed inentarization of complex multidimensional spaces, such as urban areas. Simultaneous Localization and Mapping (SLAM) accomplishes the goal of concurrent localization and map creation based on self-recognition. LiDAR-based SLAM technology has advanced exponentially as a result of the widespread use of LiDAR sensors in a variety of technological sectors⁹.

By integrating these technologies, results are achieved that are not only impressive in terms of visualization, but also in terms of their ability to maximize the sub-representation of the components of the urban environment and the geosystems that form it. This is particularly important in the context of the need to adequately inform the planning and management of these areas, including their adaptation to climate change. These technological solutions and their integration underpin both the concept of digital geo-twinning and that of smart cities. In this context, the results of this research show one possible way of integrating new technological solutions in the field of geospatial technologies to achieve quality information provision.

5.

CONCLUSIONS

Sensors and intelligent data acquisition are helping to improve the life cycle of any asset and process, including the urban planning procedures. The complex and multi-component nature of urban areas makes them extremely demanding in terms of information provision for their planning and management. This is especially true for the processes related to the adaptation of these territories to the changing climatic conditions and the associated changes in the geographical environment and the different geosystems that form it. The capabilities of modern geoinformation technologies and their integration to achieve a more complete and relational digital representation of these territories are of key importance in these processes. The present investigation is an example of how these technologies can be integrated and complemented to achieve this digital replication, and in a rapid and efficient manner.

Direct georeferencing is a new technique in photogrammetry, especially in the aerial photogrammetry. This method does not require Ground Control Points “GCPs”, to process aerial photographs into desired ground coordinates systems. Compared with the old method, this method has four main advantages: faster field work, faster data processing, simple workflow and less cost⁶.

The ability of intelligent unmanned platforms to achieve autonomous navigation and positioning in a large-scale environment has become increasingly demanding, in which LIDAR-based Simultaneous Localization and Mapping (SLAM) is the mainstream of research schemes¹⁰.

Used Handheld SLAM system ZEB Horizon does not have internal GNSS receivers to provide accurate coordinates. There is therefore a requirement for external methods to georeference data. This is done using pre-determined control points that are identified during a scan. Georeferencing the point cloud is performed using a reflective target on the control points. Their coordinates can be extracted with high accuracy from the areal images without using GNSS. This method gives users the freedom to move through their data without stopping. The targets give off higher intensity values due to the reflectiveness, and can therefore be highlighted within the processed scan and the software will georeference the data automatically.