Extracting building footprints from UAV imagery for urban digital transformation in Ho Chi Minh City
Article information
Abstract
Background and objective
Rapid urbanization in Vietnam's municipalities and in particular in Ho Chi Minh City has placed complex issues in the practices of urban management and governance. The underlying challenge is the availability of spatial data on urban infrastructure and settings. This has instigated a national program on digitalizing urban infrastructure for better urban governance and efficient management. With respect to urban infrastructure data, this study introduces a deep learning workflow to enable building footprints extraction from very high-resolution UAV imagery and converts them into a structured geospatial layer to support the spatial management practices of urban environmental settings.
Methods
The workflow relies on the DeepLabV3+ architecture with a ResNet-101 backbone, chosen for its strong semantic segmentation capability. Due to a small set of available labeled UAV samples, a combination of deep learning and transfer learning techniques was deployed by first pretraining the model on a large collection of open images. Then, it was fine-tuned with manually annotated photographs acquired over the Phuoc Thien resettlement area in Long Binh Ward, the eastern side of Ho Chi Minh City. To facilitate this, survey flights were conducted at an altitude of 100 m to produce orthomosaics with a Ground Sampling Distance (GSD) of 2.82 cm/pixel, which were processed through contrast enhancement, geometric correction, orthomosaic generation, and tiling before segmentation; subsequent refinement converts raster masks to vector polygons and assigns spatial references.
Results
The resulting building footprint layer attains an Intersection over Union (IoU) of 0.91 and a Precision of 0.90, confirming reliable delineation under diverse building roof geometries and illumination. Delivered as a georeferenced vector layer, these footprints can be ingested directly into municipal spatial databases, providing an up-to-date dataset to support cadastral maintenance, zoning evaluation, and infrastructure monitoring.
Conclusion
The proposed method shows that deep learning combined with transfer learning can effectively extract urban building boundaries from UAV imagery in the context of labeled data is limited. The workflow ensures the accuracy in both geometric and semantic aspects, which creates a spatial data layer that can be easily integrated into urban databases. The findings offer an insight into enabling the national strategy for urban digital transformation.
Introduction
Background and Motivation
Recent years, the rapid urbanization in major Vietnam's municipalities has promoted a multiple demand for constructing enormous spatial databases to support the management practices of planning, fluctuation tracking, and detailed urban planning. However, conventional fact resources, which include paper cadastral maps or commercial satellite images, are still restricted in terms of resolution, frequency of updates, and integration into Geographic Information Systems (GIS). Given this context, Unmanned Aerial Vehicles (UAVs) have emerged as a powerful device for obtaining orthophotos which are high-resolution, temporal flexibility, and a wide surveyed area. This requirement is crucially appropriate to urban regions with excessive population density and various architectural morphologies and settings.
However, converting UAV imagery into structured layers of spatial data, such as building footprints, remains challenging, especially in the context of the requirement for automated and accurate processing in complex urban settings. In the field of remote sensing image analysis, Deep Learning (DL) models, especially Convolutional Neural Networks (CNNs), have proven outstanding effectiveness in semantic segmentation problems. Owing to its ability to automatically learn spatial features and data semantics, DL has been widely employed to overcome many limitations of traditional methods that rely on manual feature design.
Literature Review of Deep Learning-based Techniques in Urban Applications
Since the turning point of the AlexNet model proposed by Krizhevsky et a l. (2012), a series of effective DL architectures, such as FCN, U-Net, ResNet, DeepLab, and SegNet have been widely used in image segmentation problems, opening up great advances in spatial data analysis (Badrinarayanan et al., 2017; Chen et al., 2017; Kumaraswamy and Patil, 2022; Long et al., 2015; Ronneberger et al., 2015). Maggiori et al. (2017) deployed CNN for subject segmentation from aerial photographs. Meanwhile, Duarte et al. (2018) combined satellite and UAV images to analyze post-disaster house damage. Additionally, Ramadhani and Alya (2024) have applied Mask R-CNN for detecting building boundaries from urban UAV imagery. Recently, Farajzadeh et al. (2023) proposed combining U-Net with ResNet on UAV orthomosaic and height modeling to increase segmentation accuracy.
Among the methods of extracting building footprints, Ren et al. (2024) proposed an automated framework from a reconstructed 3D model using TIN meshing and geometric analysis, achieving an Intersection of Union (IoU) over 99%. However, recent studies have increasingly focused on the application of DL to improve both accuracy and automation. Ayala et al. (2021) used the U-Net architecture that improves training on integrated data Sentinel-1, Sentinel-2, and OpenStreetMap to detect buildings and road networks at a resolution of 2.5 m. Şenol and Çöltekin (2022) demonstrated that DL models are not only superior in speed but also more accurate than manual and traditional methods, particularly for monitoring unplanned urban areas. These studies reaffirm the growing dominance of DL in the exploitation of the building footprint, especially in complex urban environmental settings.
In addition, Transfer Learning (TL) techniques are being considered as an optimal solution in the context of limited labeling data. Recently, Khdir and Sadeq (2023) applied the Mask R-CNN model to automatically extract building footprints from UAV imagery and showed that the application of TL techniques significantly improves identification accuracy, especially in the context of limited UAV data. Subsequently, Neupane et al. (2024) used fine-tuning on oblique images to improve the recognition of low-rise buildings, which is a typical study on TL. TL is a technique in DL that enables a model to leverage the knowledge learned from a source task and apply it to a relevant target task. Instead of training from scratch, the model will use parameters that have been pre-trained and refined on the new dataset. This approach not only reduces training computational costs but also improves the efficiency of identification in the context of limited availability of labeling data (Hosna et al., 2022; Weiss et al., 2016; Zhuang et al., 2020).
In Vietnam, the majority of spatial data layers are still built manually by various measurement methods, the most common of which is digitization from remote sensing images, including satellite and aerial images. Although the application of DL models in this domain is not yet widespread, several studies have been conducted to explore their potentialities, which primarily focused on information extraction and land cover classification from remotely sensed imagery (Nguyen et al., 2025a; Phan et al., 2021; Phan et a l., 2018). In parallel, there are a few studies on DL applications in this field. Le et al. (2022) applied DL model development for detecting the changes in land use of coffee trees based on Sentinel-2 imagery. Nguyen et al. (2022) conducted DL techniques to predict dengue fever outbreaks using climate-related data. Notably, Nguyen et al. (2025b) carried out DL application to automatically extract the roofs of buildings and constructions directly from UAV imagery. However, most of the current studies have merely implemented model testing and have not established an overall integrated process from image collection, model training, to building complete geometric data for urban management practices.
International studies are mostly conducted on large-scale standard datasets, with pre-labels or accompanied by ancillary data such as oblique images and height models, which are not suitable for the actual context in Vietnam, where the construction of manual labeling datasets faces many difficulties in terms of human resources, funding, and technical infrastructure. In addition, current models have not been fully verified under the condition of using only 2D UAV orthophotos, with the characteristics of roof materials and specific architectural morphology in Vietnamese cities (Le et al., 2024; Tarekegn et al., 2024; Zhang et al., 2018).
A major challenge in DL adoption is the need for large-scale labeling datasets, which are expensive and difficult to implement in field surveys. To overcome this, many studies have applied TL techniques, which allow models to be initialised from pre-trained weights on large datasets, then fine-tuned with smaller amounts of data in the study area. This is considered an effective strategy to reduce training costs and increase the ability of the model to adapt to specific data domains. However, the effectiveness of TL in the context of using high-resolution urban UAV imagery in Vietnam has not been systematically evaluated.
From the above-mentioned analysis, it can be seen that the existing gap lies in the lack of experiments to verify the effectiveness of the DL model coupled with TL in extracting building boundaries from orthophotos. Overcoming this gap is practically essential for Vietnamese urban areas with complex architectural features and diverse lighting conditions. While current studies have primarily focused on satellite imagery or UAV imagery with ancillary data, the effectiveness of using DL only on pure RGB UAV imagery has not been fully investigated.
Therefore, this study proposes and validates a process for extracting residential boundaries from high-resolution UAV imagery in urban areas, using the DeepLabV3+ architecture combined with TL. DeepLabV3+ is an advanced semantic segmentation architecture, with feature extraction capabilities at multiple scales through the Atrous Spatial Pyramid Pooling (ASPP) mechanism and accurate edge reconstruction thanks to the encoder-decoder structure (Jia et al., 2023; Li and Zhao, 2022; Mahara et al., 2025; Sunandini et al., 2023). Despite the existence of various semantic segmentation architectures, DeepLabV3+ was deliberately chosen for its ability to balance contextual awareness and boundary accuracy in dense urban environments. In comparison with U-Net, a widely used baseline in UAV-based remote sensing, DeepLabV3+ offers superior performance in preserving small subject boundaries and capturing wider spatial contexts. These characteristics are important for extracting residential traces from UAV imagery. Though the deployment of DeepLabV3+ is a relatively higher computational demand, this model remains suitable for its segmentation quality and robustness in complex scenes.
Combined with the ResNet-101 backbone, the model is expected to be highly effective in the problem of UAV imagery segmentation. The output is a vector data layer with spatial positioning, ready to be integrated into GIS systems to support the management practices of urban construction and planning.
Research Methods
This study was carried out according to a general process consisting of three main stages: (i) collecting UAV data; (ii) processing of UAV data; and (iii) constructing DL training models to extract building footprints from UAV imagery by using TL techniques. The overall diagram of the workflow is shown in Fig. 1.
Overall workflow of the proposed research.
Study area
The study area is the Phuoc Thien resettlement area, in Long Binh ward, located in the fast-growing area on the eastern side of Ho Chi Minh City (Fig. 2). This is a new urban area that has synchronous planning with an area of about 36 hectares. The area is characterized by a set of modern technical infrastructure, high construction density, reasonably planned internal roads, and completely underground electrical and telecommunications systems, creating a neat and stable urban space.
Geographical location of the study area within the Phuoc Thien resettlement, Long Binh Ward, Ho Chi Minh City.
Importantly, the study has a diverse urban landscape with an alternation of urban buildings, urban greenery, public parks, and transportation systems. However, building roofs have a very rich morphology, including flat roofs, tile roofs, and corrugated iron roofs, with many different designs, sizes, and lighting directions. This diverse morphology is a favourable condition to test the generalization and stability of the DL model in the extraction of housing boundaries from high-resolution UAV imagery.
UAV data acquisition and processing
In urban environments with high construction density, the use of rotary-wing UAVs is the optimal choice for collecting aerial images thanks to their maneuverability, easy take-off and landing in narrow spaces, and precise control in areas with many structures. This study uses the DJI Phantom 4 Pro PPK device, which integrates a 1-inch CMOS sensor and a GNSS positioning system that supports PPK (Post-Processed Kinematic) processing, to improve positioning accuracy when combined with ground measurement.
Two flights configured at an altitude of 100 m were implemented to ensure the image quality is suitable for the extraction of geometric information on buildings and other construction objects in urban environments. At this altitude, the spatial resolution is approximately 2.82 cm, and consistent with both configurations.
Nadir flight (90°): Conducted over two flight shifts, 458 images were captured with an overlapping of 80% for both the along-track and the across-track, suitable for creating orthophotos and digital elevation models (overlap shown in Fig. 3). Nadir images provide a direct top-down view, which helps to clearly show the boundaries of the grounds of the subjects, thereby effectively supporting data labeling and training the image segmentation model (Fig. 4a).
Illustration diagram of flight layout in along-track and across-track image overlap.
UAV flight trajectories and image acquisition positions.
Oblique flight (80°): Conducted over four flight shifts, 981 images were captured with an overlapping of 70% for both the along-track and the across-track, in order to supplement geometric information for the vertical and inclined façades of structures. The combination of nadir and oblique imagery helps to diversify the input sample, providing data from various perspectives of the same subject (Fig. 4b). This is especially important during DL model training, which aims to increase generalization and improve accuracy when the model applies to diverse structured real-world data.
A total of 1,439 high-quality UAV images were obtained, ensuring good contrast, uniform sharpness, and comprehensive coverage of the survey area, creating a solid foundation for the steps of processing and extracting spatial information using DL techniques.
To ensure the geometric accuracy of the UAV image data, ten (10) Ground Control Points (GCPs) are evenly arranged to cover the entire survey area. The horizontal coordinates of the points are measured using the RTK GNSS receiver, while the vertical coordinates are determined by high-precision geometric leveling using Leica Sprinter 50. These GCPs help correct spatial positions during image processing and support bundle adjustment, ensuring highly accurate 3D model reconstruction. The use of GCPs minimizes geometric errors, improves the reliability of orthophotos, and ensures that the output geospatial products meet the technical requirements for large-scale maps.
The UAV image processing is conducted using Agisoft Metashape software, using the Structure from Motion (SfM) technique combined with Multi-View Stereo (MVS) to reconstruct 3D geometry, build surface models, and create an orthomosaic. The output is an orthomosaic with high positional accuracy and serves as a basis for the analysis and extraction of urban objects in the next stages.
Application of Deep Learning models and Transfer Learning techniques
DeepLabV3+ architecture with the ResNet-101 backbone
The DeepLabV3+ architecture in combination with the ResNet-101 backbone network is a supervised DL architecture designed to optimize feature extraction at multiple spatial and semantic levels, thereby improving the accuracy of object segmentation and classification (Chen et al., 2018; Yuan et al., 2022).
Fig. 5 illustrates an overview of the DeepLabV3+ architecture with key components including the feature extractor (ResNet-101), dilated convolution, Atrous Spatial Pyramid Pooling (ASPP) module, and decoder. In particular, ResNet-101 serves as the main characteristic extractor. The early layers learn low-level local features such as edges and surface patterns, which are necessary to determine the exact boundaries of the building at the pixel level. Meanwhile, deep layers learn high-level semantic features such as the overall shape of the building, repetitive structures, and spatial relationships with surrounding objects (Li et al., 2023). The combination of shallow and deep features allows the model to simultaneously maintain sharpness at the boundary and understand the overall context within complex urban environments.
Diagram illustrating the DeepLabV3+ architecture incorporating the ResNet-101 backbone (Chen et al., 2018).
In addition, DeepLabV3+ integrates a dilated convolution mechanism along with an ASPP module to expand the information reception area without compromising the characteristic resolution. This makes the DL model capable of acquiring spatial information at various scales, from local details to global context. As a result, the model can distinguish building footprints even in complex visual conditions such as shadows, obscured tree canopies, or areas that overlap with other infrastructure.
Model Selection and Evaluation of Training Strategies
Choosing an appropriate DL architecture plays a key role in ensuring high accuracy in object extraction from high-resolution UAV imagery. Notably, the goal of this study is to demonstrate the feasibility of the proposed processing chain for building extraction from UAV imagery with TL, and it does not make a comparative evaluation among different segmentation architectures. Consequently, we chose the DeepLabV3+ with ResNet-101 backbone as the state-of-the-art of representative model to compare the training methods on the small set of local training data. Accordingly, the DeepLabV3+ architecture combined with the ResNet-101 backbone was selected to solve the problem of extracting building footprints in urban areas. The reason for this selection comes from the geometric characteristics of the subject, including: building footprints are often complex in shape, easily obscured, or contiguous in dense urban environments. The DeepLabV3+ model with multi-scale feature extraction and the use of atrous convolution enhances the retention of detailed boundaries while capturing spatial context at multiple levels.
The selection of this architecture is further substantiated by numerous recent studies that have demonstrated the superior efficacy of DeepLabV3+ in segmenting urban features from high-resolution remote sensing images. The model has been effectively employed for problems such as residential demarcation, surface cover determination, or urban morphology analysis, especially in areas with complex and fragmented landscape structures (Atik et al., 2022; Li and Dong, 2022). Recent studies reported that DeepLabV3+ has shown improved boundary preservation and segmentation accuracy compared to earlier architectures such as FCN, U–Net, or SegNet. In addition, DeepLabV3+ demonstrates notable superiority in segmentation precision and the ability to keep the geometric boundaries of the object intact. Apart from the above-mentioned advancements, the DeepLabV3+ model is built into the ArcGIS Pro software, which simplifies the process of experimental deployment and enhances integration in processing UAV imagery on the GIS platform.
On that basis, this study proposes and verifies the approach with three different training strategies to evaluate the effectiveness of applying the DeepLabV3+ architecture with ResNet-101 under the conditions of actual data in the research area:
Strategy 1 – Training from scratch: all weights of the model are randomly initialized and trained entirely using local sample data. In the context of this study, this strategy was only applied to datasets generated from UAV images of the study area, to test the ability to learn from local data.
Strategy 2 – Direct application of pre-trained feature extraction: use the pre-trained DeepLabV3+ model on large datasets and apply it directly to segment the images of the study area, without performing fine-tuning. This method is time-saving and computationally efficient but can reduce the effectiveness of identification by not adapting to specific UAV image characteristics.
Strategy 3 – Fine-tuning: Selectively retrain a weighted portion of the pre-trained model with local data. In the DeepLabV3+ architecture using the ResNet-101 backbone, the total number of parameters that need to be trained from scratch is about 48 million. According to the empirical formula to avoid overfitting, the number of training samples required to train the entire model is estimated to be greater than 270,000 samples (Chollet, 2021; Hestness et al., 2017; LeCun et al., 2015). To overcome the limitation of dataset size, the study applied TL techniques from pre-trained models on large datasets. The refinement factor is set to r = 0.1, which only updates about 10% of the parameters of the original model, equivalent to 4.8 million parameters. The reduction in the number of parameters that need to be updated significantly lowers the requirement for the number of training samples, estimated to be around 31,200 samples (Kornblith et al., 2019). In addition, the data augmentation technique is deployed with a factor of 16 times, through geometric and optical transformations such as rotation, flipping, brightness adjustment, and random cropping. As a result, the number of original samples that need to be labeled is reduced to only 1, 960 samples, in line with the size of the survey data, while still ensuring the diversity required for the training process.
Data Sampling
From the processed UAV image dataset, a data subset was selected to support the construction of a training dataset for the DL model. The selection process ensures that it fully reflects the morphological characteristics of the surveyed area, including the diversity of scale, architecture, and spatial layout of urban constructions.
On that basis, manual annotations were conducted to accurately determine the ground boundaries of construction works, acting as a label layer for model training. Data were digitized directly from high-resolution UAV imagery in a dedicated software environment, ensuring the geometric fidelity and spatial accuracy of the objects. The annotation process was consistently carried out according to a single editorial convention, focusing on accurately showing continuous and non-overlapping boundaries between neighbouring buildings.
The selection of buildings for annotation was intentionally applied to the representation of the entire study area. Specifically, the digitized objects include many types of housing with distinct differences in shape (rectangular, L-shaped, or irregular polygons) and roof structures (flat roofs, single-sided sloped roofs, double-sided sloped roofs, or combined roofs). Boundary determination was done manually based on their geometric features and spatial relationships to surrounding structures. This approach helps the model to learn generalized patterns, identify the diverse morphological characteristics of urban buildings, and improve segmentation robustness in different contexts.
A total of 1,960 building footprints were labeled and divided into three volumes according to the standard ratio: 70% training data, 20% validation data, and 10% independent test data. This division not only ensures objectivity in model evaluation but also facilitates the process of refining parameters and testing the level of generalization in the training process.
Extract and transform the resulting data
The DeepLabV3+ model, after completing training data, was deployed to segment the RGB orthomosaic of the study area through pixel classification into two semantic classes: the building footprint layer (label 1) and the non-building footprint layer (label 0). The output is a binary raster in which each pixel is assigned a label based on the predicted probability of the building footprint inferred by the DL model. This process creates a two-layer segment map, which serves as an input for subsequent processing and analysis steps in the GIS-based environment.
Pixel-level segmentation offers a high accuracy of classification, but is not refined because of usual occasions in a mixture of a large number of small, fragmented regions and irregular object boundaries, mainly due to surface texture, shadows, and occlusions in high-resolution UAV imagery. To ensure the consistency and reliability of the extracted building layer for GIS use, a post-processing step is applied to reduce spatial noise and enhance the continuity of the objects.
Noise removal is carried out on the basis of geometric and spatial criteria, including minimum-area thresholds, region-connectivity analysis, and gap-filling operations. Segmented regions with a small area, which do not meet the geometric building conditions, are considered noise interference and are removed to avoid the inclusion of objects that do not reflect the entity in the field. At the same time, the analysis of the connection and the filling of the gaps inside the segment area helps to maintain the geometric integrity of the building objects. This processing step plays an important role in limiting the formation of small, discrete polygons when converting raster data to vector forms, thereby minimizing structural aberrations and pseudo-increments in the number of objects performed in the vector data layer.
The cleaned binary raster is then converted into vector-polygon features by applying a region-based polygonization method. Each connected component is transformed into a single polygon, followed by contour simplification to reduce geometric noise, such as spikes or jagged edges, while preserving the essential shape and boundary fidelity of the original segmented region. The result is a clean vector layer that can be used in any GIS system for spatial queries, measurements, and overlays.
To assess the generalization quality, final field-tested vector is compared against an existing cadastral boundary layer for spatial fit, completeness, and boundary recognition in visually complex urban scenes. The evaluation is conducted on the vectorized building features rather than raw pixel outputs, ensuring that the reported accuracy metrics reflect the true utility of the data in practical urban analysis and spatial decision-making contexts.
Evaluate the effectiveness of the model
Upon completion of the training process, the DeepLabV3+ model was applied to the semantic segmentation of the building footprint across the orthomosaic of the study area. Among various evaluation metrics, this study selected Precision and IoU as the most relevant indicators to evaluate both classification correctness and geometric boundary accuracy, which are critical for GIS-based building footprint analysis. These metrics were selected as representative indicators to ensure focused and meaningful evaluation aligned with the objective of spatial data construction for urban management.
The Precision metric reflects the ratio of the building footprints correctly identified by the model to the total number of building footprints that have been labeled by the model. This value is calculated according to Formula (1) as follows:
Whereas TP (True Positive) is the number of building footprints identified by the model to match the actual data, and FP (False Positive) is the number of areas that are mistakenly recognized as constructions even though they are not. The higher the precision, the better the level of accuracy in identifying building footprints, and at the same time, it reflects the model's ability to minimize errors.
In addition, the IoU metric is used to measure the degree of geometric coincidence between the predicted region and the ground-truth region. The IoU is determined by the ratio between the intersection area and the union area of each corresponding pair of objects, helping to assess the degree of boundary matching between the model and reality, according to Formula (2). This index plays an important role in evaluating not only the detectability, but also the quality of the construction segment profile.
Results and Discussion
DeepLabV3+ model training results
To evaluate the effectiveness of the DeepLabV3+ model in the problem of building footprint segmentation from UAV imagery, three different strategies were implemented. In particular, two strategies (Strategy 1 and Strategy 3) require model training, while Strategy 2 only applies the pre-trained model directly without going through the retraining phase on local data.
Two training and validation loss charts corresponding to Strategy 1, training from scratch, and Strategy 3, fine-tuned TL, are presented in Fig. 6a and Fig. 6b to clarify the differences in training between the two methods.
Training and validation loss charts.
The results from Strategy 1 indicated that the initial loss value is very high, hovering around 2.9, due to the entire weighting of the pattern being randomly initialized. Although the training losses gradually decreased throughout the process, the curve showing the loss still fluctuated significantly, especially in the validation set. The large gap between training and validation losses from the middle of the training process shows obvious signs of overfitting. This reflects that a model with more than 48 million parameters cannot learn effectively from a limited training dataset.
In contrast, the chart of Strategy 3 – fine-tuned TL – showed a significantly lower initial loss value, starting at approximately 1.7. Both the training and validation loss curves decreased rapidly and converged early, reaching a stable level after about 1,000 batches. The gap between the two curves remains minimal and almost parallel throughout the training process, indicating that the learning model is stable, well-adapted to the local dataset, and does not experience overfitting.
These results suggested that, in the case of limited labeled data, a fine-tuned TL strategy is the appropriate and effective option. This method not only saves computational time and resources but also allows the model to inherit spatial and semantic features from large datasets and adapt them to the specific urban visual context of the study area.
Building footprints extraction
Based on the trained DeepLabV3+ model, the RGB orthomosaic of the study area was segmented into two semantic classes: the building footprint class (label 1) and the non-building class (label 0). The segmentation result is a binary raster in which each pixel is assigned a label based on the predicted probability of belonging to the "building footprint" object inferred by the model. This raster is then post-processed and converted into a vector layer in a GIS environment for quantitative evaluation as well as integration into the mapping system.
The results of pre-segmented UAV imagery processing showed that the orthomosaic achieved a GSD of 2.82 cm/pixel, ensuring the ability to represent the geometric details of the construction works. The Root Mean Square Error (RMSE) when tested by five independent checkpoints is 4.6 cm, in line with the geometric accuracy requirements for the problem of extracting objects from high-resolution images.
Fig. 7 visually illustrates the results of building footprint segmentation from UAV imagery under the three strategies described in the methodology section:
Results of building footprints segmentation.
Fig. 7a shows an RGB orthomosaic with great detail, clear construction boundaries, residential layouts, roads, trees, and urban green areas.
Fig. 7b (Strategy 1 – training from scratch): Although a large number of objects could be detected by the model, the noise in the resulting segmentation was quite severe, and the boundaries of the results were not well defined. This performance constraint may be attributed to the fact that the model had not been trained on pre-learned features, and the local dataset is small, which prevents it from reducing the loss and converging. Consequently, the training is hard to converge to a stable and generalised result, especially for cases with sloped roofs, strong shadows, and partial occlusions (e.g., tree canopies). These results highlight the challenges associated with training deep models from scratch on small, locally specific datasets, particularly in complex urban settings.
Fig. 7c (Strategy 2 – no refinement): The model produces unrefined outputs, with incomplete or misidentified buildings. This suggests that the pre-trained weights do not fully adapt to the morphological and spectral characteristics of local UAV imagery. It reflects the risk of directly applying generic models without local adaptation.
Fig. 7d (Strategy 3 – fine-tuned TL): The segmentation results showed a clear improvement in both object completeness and boundary accuracy compared to other strategies. The model can detect buildings with different shapes and sizes, such as small, irregular, and densely arranged houses. It also works well in unfavourable regions that are affected by shadows, changes in roof slope, or partially obscured by trees – cases where previous strategies have struggled. This improvement reflects the benefits of fine-tuning, in which a pre-trained model is tuned using a limited amount of local data. Through this process, the model is able to update the high-level characteristic representations learned from a common dataset and adapt them more closely to the specific morphological and visual features of the UAV imagery of the study area. As a result, converged training is more efficient and provides a better generalization. As a result, fine-tuned TL achieves a balance between model robustness and local adaptability, making it more suitable for actual urban mapping applications, especially when training data is limited.
Besides the enhancements in object segmentation, the results from strategy 3 also offer practical advantages in real-world applications. The segmented outputs in Fig. 7d, with their clear building outlines and high visual coherence, enable efficient vectorization to create building footprint layers without requiring extensive manual correction. This is especially useful in a complex bundle of urban data systems, because the availability of real-time, accurate, and clean geospatial data is an essential ingredient for planning, monitoring, and enforcement. According to these outputs, Fig. 8 shows that the result of the segmentation was successfully converted into a vector layer with each building footprint.
Results of building footprints extraction.
Results of quantitative evaluation of deep learning models
The results presented in Table 1 show a marked improvement in accuracy when applying fine-tuned TL techniques. Specifically, when comparing Strategy 1 and Strategy 3, the Precision index increased from 0.68 to 0.90, corresponding to an improvement of 32.4%. More notably, the IoU metric improved significantly from 0.57 to 0.91, corresponding to an increase of 59.7%, indicating that the possibility of geometric overlap between the predicted area and reality has been improved considerably. This improvement can be attributed to the model's good adaptability when refined with a local dataset. The typical geometric features and spatial layout of the study area – such as common roof shapes, clearly defined building boundaries, and dense, organized arrangements in a grid-like urban pattern – helped the DL model to more accurately identify building footprints from UAV imagery.
The accuracy evaluation of extracting the building footprint of each strategy
Strategy 3 achieved the highest results on both metrics, confirming the effectiveness of fine-tuning techniques in combining the generalized learning capabilities of the pre-training model with the ability to adapt to the actual data context. In contrast, Strategy 1 gave the lowest results because the entire model was trained from scratch with a limited dataset, which was insufficient for training a deep model with over 48 million parameters. Meanwhile, Strategy 2 – which uses a pre-trained model without fine-tuning – outperforms Strategy 1 but is still significantly lower than Strategy 3, suggesting that the failure to update the weights according to local features has limited model performance.
Overall, the quantitative evaluation results confirm that fine-tuned TL is an effective and practical strategy for accurately extracting building footprints from high-resolution UAV imagery, especially when training data resources are limited.
One of the highlights of the study was the use of a combination of nadir (90°) and oblique (80°) images in the acquisition of UAV data. This flight configuration not only enhances the overlap of the survey area but also provides additional information about the inclined geometry of the objects, especially in areas with complex or partially obscured building facades. From the perspective of DL, this is a factor that helps to increase the diversity of input data, thereby improving the model's ability to learn in identifying object boundaries in different visual conditions.
Another notable advantage is that the study implemented the entire processing process – from UAV image collection, manual labeling, model training, to post-processing and vector conversion – using high-resolution RGB data, without the need for ancillary data sources such as Digital Surface Model (DSM) or multispectral imagery. This shows the potential application of the method in practice, especially in localities with limited resources that still want to effectively exploit data from UAV imagery to build spatial data.
However, the analysis also reveals some technical shortcomings that need to be considered. In areas with strong shadows, similar material colours or obscuring tree canopy, the model's ability to identify building boundaries is still not entirely stable. This is a common limitation when using only RGB images, due to the lack of information about height or surface structure to distinguish between roofs and ground or other objects. In addition, some cases where the building has a small area or irregular shape also increase the risk of incorrect segmentation or omissions.
Regarding spatial representativeness, the current study was carried out in a newly planned resettlement area with a very regular and standardized housing layout. Buildings in this area are characterized by rectangular footprints, well-defined boundaries, and consistent spacing, creating favourable geometric conditions for object detection. As a result, the DL model works well in this context, benefiting from low levels of background noise and uniform spatial morphology.
However, these ideal conditions do not fully reflect the diverse characteristics of the urban environments. The model has yet to be evaluated in unplanned settlements, historic city centers, or low-density residential zones, where buildings are often irregularly shaped, fragmented, or spatially disordered. Such areas often have a lot of visual clutter, obscurity, and heterogeneity in object scale and texture, which can adversely affect segmentation accuracy. Therefore, while the results obtained in the study area are promising, they may not lead to the model's generalization. Future studies should validate the model in more complex urban environments to fully assess its robustness and applicability under different spatial conditions.
Conclusions
This study has developed and validated a process of using DL techniques combined with TL to extract building footprints from high-resolution UAV imagery applied to a newly developed urban area in Long Binh Ward, Ho Chi Minh City. The workflow is implemented in three main stages: UAV data acquisition, image processing, and semantic segmentation using a deep neural network. The flexible flight configuration that allows for the collection of nadir and oblique images, combined with evenly distributed GCPs, has ensured the geometric accuracy required for the next processing steps.
The DeepLabV3+ model with the ResNet-101 backbone was chosen due to its ability to handle geometric and semantic features well. The evaluation process was carried out with three different training strategies, in which the fine-tuning TL strategy showed superior efficiency, achieving Precision = 0.90 and IoU = 0.91 – significantly higher than the other strategies. This result is not only confirmed by quantitative metrics but also by the visual quality of the post-segmented vector layer, which meets the requirements for urban data integration and management.
Despite these promising outcomes, several limitations remain. The model still has difficulty identifying building footprints in areas with complex visual conditions, such as shadows, obscured roofs, or colours similar to roadbeds. Moreover, the current implementation of the workflow is defined in urban areas with synchronous planning; the generality of the model has not been verified in areas with irregular spatial morphology, such as spontaneous residential areas or old urban districts.
However, the experimental results demonstrated that the proposed method has proven its effectiveness and practicality in extracting construction boundaries from UAV imagery, even with limited sources of labeling data. This is a useful platform for developing solutions to build a digital dataset for urban management in the existing emergent need for digital transformation and the rapid urbanization in Vietnam.