Spatial Data Acquisition and Data Quality
Open in the Padhaiko app4.1 Data capture methods: primary and secondary
Spatial data acquisition is the most resource-intensive stage of building a GIS. Depending on whether observations are measured directly from the physical world or extracted from existing media, acquisition methods fall into primary and secondary categories.
| Aspect | Primary Data Capture | Secondary Data Capture |
|---|---|---|
| Source | Direct measurement of real-world phenomena in situ or remotely | Extraction from existing paper maps, photographs, tabular records, or digital archives |
| Techniques | Total station / theodolite field survey, GNSS surveying (RTK/DGPS), satellite/aerial remote sensing, LiDAR | Manual/heads-up digitizing, flatbed/drum scanning + vectorization, table geocoding, dataset import |
| Cost & Speed | Higher per-area cost and field logistics, but produces current, application-specific accuracy | Lower cost and rapid desktop turnaround, but inherits age and scale errors of the source map |
Primary Data Capture Methods
- Ground Field Survey: Uses total stations, digital levels, and laser rangefinders to measure horizontal angles, slope distances, and elevation differences with millimeter-to-centimeter precision for engineering and cadastral boundaries.
- GNSS Surveying: Uses Differential GNSS (DGNSS) or Real-Time Kinematic (RTK) base–rover pairs to achieve centimeter-level coordinates in the field, or handheld recreational receivers for meter-level asset inventories.
- Remote Sensing: Measures reflected or emitted electromagnetic energy from satellite, aircraft, or UAV sensors to map large inaccessible regions rapidly and repeatedly.
Secondary Data Capture Methods
- Digitizing: Tracing vector features from paper maps on a digitizing tablet (puck) or via heads-up (on-screen) digitizing over a scanned and georeferenced base image.
- Scanning and Raster-to-Vector Conversion: Converting analog maps into high-resolution TIFF rasters using drum or large-format flatbed scanners, followed by automated or semi-automated line tracing.
- Existing Digital Datasets & Geocoding: Importing census tables, CAD drawings, or address lists (geocoding street addresses into point coordinates using a reference street centerline network).
4.2 Georeferencing, digitizing, vectorization, editing and topological cleaning
Raw scanned maps, aerial images, and freshly digitized linework must pass through a strict geometric and topological conditioning pipeline before entering a production geodatabase.
- Georeferencing: Assigns real-world coordinates in a target CRS to a raster image or unreferenced CAD drawing using Ground Control Points (GCPs) (clearly identifiable intersections, benchmarks, or map grid corners).
- Fit quality is quantified by the Root Mean Square Error (RMSE) over GCPs:
- Resampling methods after geometric warping: Nearest Neighbor (preserves exact discrete class codes), Bilinear Interpolation ( nearest cells, smooths continuous data), and Cubic Convolution ( nearest cells, sharpest continuous imagery).
- Digitizing Modes:
- Point Mode: Operator clicks each vertex deliberately (best for straight road segments and building corners).
- Stream Mode: Vertices are recorded automatically at fixed time or distance intervals along sinuous curves (rivers, contours), followed by Douglas–Peucker line simplification to remove redundant vertices.
- Vectorization: Converting raster lines/zones into vector polylines/polygons via skeletonization (thinning) of binary line masks, centerline extraction, and boundary tracing—often requiring manual verification where text labels cross contour lines.
- Editing and Topological Cleaning: Manual tracing introduces geometric defects that break overlay and network routing:
- Undershoots and Overshoots (Dangles): A line stops short of or extends past its intended intersection; fixed by setting a snap tolerance (snapping vertices within radius ) and trimming dangles shorter than a threshold.
- Sliver Polygons: Thin gap or overlap polygons formed when a shared boundary between adjacent polygons is digitized twice; fixed by edge snapping / shared-boundary editing and eliminating sliver areas below a minimum tolerance.
- Duplicate Vertices, Self-Intersections, and Unclosed Polygons: Fixed via geometry validation (
ST_MakeValid), node splitting at crossings (planarization), and polygon building.
1
While georeferencing a scanned municipal map using GCPs, the residual errors in meters are , , , and .
- Compute , , and total .
- Which resampling method should be chosen if the raster is a discrete soil-class map rather than an aerial photograph, and why?
Compute and :
Compute total :
Select the resampling method: For a discrete soil-class map, Nearest Neighbor resampling must be used because it copies the original integer class value of the closest input pixel without averaging neighboring classes (unlike Bilinear or Cubic, which would invent invalid fractional codes like between Forest and Urban ).
, , and total . Nearest Neighbor resampling must be used for discrete soil-class maps so original category codes are preserved without interpolation averaging.
4.3 Remote sensing fundamentals
Remote sensing acquires information about Earth's surface without physical contact by recording reflected solar radiation, emitted thermal infrared energy, or backscattered radar pulses.
Electromagnetic Spectrum, Platforms, and Sensors
- Electromagnetic Spectrum Bands:
- Visible (VIS, –): Blue (–), Green (–), Red (–); chlorophyll absorbs blue and red, reflecting green.
- Near-Infrared (NIR, –): Strongly reflected by healthy leaf cellular structure—fundamental for vegetation indices such as .
- Shortwave Infrared (SWIR, –): Sensitive to soil/vegetation moisture and mineral composition.
- Thermal Infrared (TIR, –): Measures emitted heat for land-surface temperature.
- Microwave / Radar (–): Penetrates clouds, smoke, and darkness (Synthetic Aperture Radar, SAR).
- Platforms & Sensors:
- Platforms: Ground-based tripods, UAVs/drones, aircraft, and satellites (Sun-synchronous polar-orbiting for high-resolution mapping vs. Geostationary for continuous weather monitoring).
- Passive Sensors: Measure natural reflected sunlight or emitted thermal energy (cannot see through thick clouds in optical bands).
- Active Sensors: Emit their own energy pulse and measure the return (LiDAR uses laser pulses; Radar/SAR uses microwaves).
Four Resolutions of Remote Sensing
| Resolution Type | Definition | Concrete Example |
|---|---|---|
| Spatial Resolution | Ground size of one pixel (Instantaneous Field of View, IFOV) | Sentinel-2 visible/NIR bands: ; Landsat OLI: |
| Spectral Resolution | Number and narrowness of wavelength bands recorded | Multispectral (– broad bands) vs. Hyperspectral (– narrow contiguous bands) |
| Radiometric Resolution | Bit depth ( quantization levels) of brightness sensitivity | -bit ( levels, –) vs. -bit ( levels) vs. -bit ( levels) |
| Temporal Resolution | Revisit time between successive observations of the same spot | Landsat-8/9 individually: ( combined); Sentinel-2A/2B: |
Overview of Image Classification
Image classification groups pixels into thematic land-cover classes based on their multi-band spectral signatures:
- Supervised Classification: Analyst supplies labeled training samples (ROIs) for each target class (Water, Forest, Built-up, Agriculture). Algorithms (Maximum Likelihood, Support Vector Machine, Random Forest) learn statistical decision boundaries and classify the full scene.
- Unsupervised Classification: Requires no prior labels; clustering algorithms (-means, ISODATA) automatically group pixels into spectrally similar clusters, which the analyst labels afterward.
- Object-Based Image Analysis (OBIA): Segments very-high-resolution imagery into homogeneous image objects (polygons) first, then classifies using spectral, shape, texture, and contextual features.
4.4 Modern data sources
Modern geospatial engineering combines traditional surveys with dense, open, and crowdsourced sensors:
- UAV / Drone Photogrammetry: Low-altitude drones capture overlapping nadir and oblique images processed via Structure from Motion (SfM) and Multi-View Stereo to generate centimeter-level orthomosaics, Digital Surface Models (DSMs), and 3D textured meshes over local corridors or landslide sites.
- LiDAR (Light Detection and Ranging): Active laser scanning (airborne, terrestrial, or mobile) emitting hundreds of thousands of pulses per second to measure range . Produces dense 3D point clouds capable of penetrating forest canopies (multiple returns: first return = canopy top, last return = bare ground DEM).
- Open Satellite Data:
- Landsat (USGS/NASA): Longest continuous medium-resolution Earth archive since 1972 ( optical/SWIR, thermal, free global catalog).
- Sentinel (ESA Copernicus): Sentinel-2 (– multispectral, -day revisit) and Sentinel-1 (C-band SAR all-weather radar).
- OpenStreetMap (OSM) and Volunteered Geographic Information (VGI):
- VGI harnesses citizen contributors using GPS traces, aerial tracing, and mobile apps to map local infrastructure.
- OpenStreetMap is the flagship global open vector database built on three elements—
nodes(points),ways(polylines/closed polygons), andrelations(route/multipolygon groupings)—with openkey=valuetags, widely used in humanitarian disaster mapping (e.g., Kathmandu Living Labs / HOT).
4.5 Spatial data quality
No spatial dataset is error-free; spatial data quality determines whether a dataset is "fit for use" in a given engineering decision. ISO 19157 defines six core quality elements:
Six Dimensions of Spatial Data Quality
- Positional Accuracy: Closeness of stored feature coordinates to their true ground positions (horizontal and vertical; often reported as or NSSDA confidence).
- Attribute Accuracy: Correctness of descriptive labels and measurements (e.g., confusion matrix / Overall Accuracy and Kappa coefficient for classified land cover).
- Temporal Accuracy: Currency (how up-to-date the data is), validity period, and agreement of recorded timestamps with real-world events.
- Lineage: The documented history of a dataset—source materials, acquisition dates, sensor parameters, coordinate transformations, and processing steps applied.
- Completeness: Absence of omission (missing real-world features) and commission (excess/duplicate/phantom features).
- Logical Consistency: Adherence to structural, format, domain, and topological rules (valid polygons, connected road graphs, valid attribute ranges).
2
According to the National Standard for Spatial Data Accuracy (NSSDA), circular horizontal positional accuracy is , where . If independent check points on an orthomosaic yield and , compute and the horizontal accuracy.
Compute radial :
Compute NSSDA horizontal accuracy:
This means of well-defined points on the orthomosaic lie within of their true surveyed ground positions.
, giving an NSSDA horizontal positional accuracy of ().
4.6 Metadata, ISO standards, data conversion and integration
Metadata ("data about data") describes the content, quality, coordinate system, lineage, spatial extent, resolution, licensing, and contact point of a geospatial dataset so external users can discover and evaluate fitness for use without opening huge raw files.
ISO Geospatial Metadata Standards
- ISO 19115: Core international standard defining the schema for geographic metadata (identification, spatial reference, distribution, constraints).
- ISO 19139: XML encoding implementation schema for ISO 19115 metadata records, used by OGC Catalog Service for the Web (CSW) geoportals.
- ISO 19157: Standard framework for quantifying and reporting spatial data quality measures.
Data Conversion and Integration
Integrating heterogeneous layers from multiple agencies requires handling both syntactic (format) and semantic (meaning) differences:
- Format & Geometry Conversion: Translating between vector formats (Shapefile, GeoJSON, GeoPackage, KML) and raster formats (GeoTIFF, NetCDF) using GDAL/OGR, as well as rasterization (vector raster) and vectorization (raster vector).
- CRS Reprojection & Datum Harmonization: Transforming every layer into a common target CRS (e.g., MUTM or UTM WGS 84) before overlay.
- Edge Matching & Conflation: Aligning geometries across adjacent map sheet boundaries (edge matching) and merging a geometrically accurate layer with an attribute-rich layer (conflation).
- Schema & Semantic Mapping: Reconciling differing attribute names, units, and classification taxonomies (e.g., harmonizing two ministries' differing forest definitions).
added
Distinguish between primary and secondary spatial data capture methods with two examples each, and describe three common topological digitizing errors along with their fixes.
Primary vs. Secondary capture:
- Primary capture directly measures the physical environment (e.g., RTK GNSS survey, airborne LiDAR, multispectral satellite imaging).
- Secondary capture converts existing analog or pre-existing records (e.g., scanning and heads-up digitizing paper topographic sheets, geocoding tabular address lists).
Three digitizing errors and fixes:
- Undershoots / Overshoots (Dangles): Lines fail to meet or extend past a target node; fixed by configuring snap tolerances and trimming dangles.
- Sliver Polygons: Narrow spurious gaps/overlaps along independently digitized shared boundaries; fixed by shared-edge snapping and eliminating polygons below an area threshold.
- Self-Intersections / Unclosed Rings: Bow-tie loops or unclosed polygon rings; fixed by geometry validation/repair (
ST_MakeValid) and polygon planarization.
Primary capture directly observes the Earth (GNSS, remote sensing), whereas secondary capture extracts data from existing maps or documents (scanning, digitizing). Common digitizing defects—dangles (undershoots/overshoots), sliver polygons, and self-intersections—are corrected via snapping, dangling-arc trimming, sliver elimination, and geometry validation.
Define the four resolutions of remote sensing. Using , compute the values for two Sentinel-2 pixels: Pixel (, ) and Pixel (, ), and identify which pixel represents healthy vegetation.
Four remote sensing resolutions:
- Spatial: Ground dimension of one pixel (IFOV).
- Spectral: Number and bandwidth of electromagnetic bands captured.
- Radiometric: Bit depth ( brightness levels) per pixel.
- Temporal: Revisit interval between repeated passes over the same location.
Compute NDVI for both pixels:
Interpret values: Healthy vegetation absorbs red light for photosynthesis and strongly reflects NIR light, yielding positive NDVI values ( to ). Water or wet bare surfaces reflect less NIR than red, producing negative NDVI. Thus Pixel represents healthy vegetation.
The four resolutions are spatial, spectral, radiometric, and temporal. (healthy vegetation) and (water/non-vegetated surface).
Explain how airborne LiDAR generates bare-earth Digital Elevation Models (DEMs) under forest canopy, and contrast UAV photogrammetry with OpenStreetMap (OSM) VGI as geospatial data sources. An airborne LiDAR pulse has a round-trip travel time of () at ; compute the slant range from the sensor to the target.
Compute LiDAR slant range :
Bare-earth DEM extraction and modern sources:
- LiDAR under canopy: Each laser footprint can record multiple return echoes—the first return reflects from treetops (producing a DSM), while the last return penetrates canopy gaps to hit the ground surface; filtering ground returns creates an accurate bare-earth DEM.
- UAV Photogrammetry vs. OSM/VGI: UAV photogrammetry produces ultra-high-resolution local orthomosaics and DSMs via Structure from Motion (ideal for engineering sites, though blocked under dense canopy), whereas OpenStreetMap (VGI) provides free, collaboratively updated global vector infrastructure (
nodes,ways,relations) with variable volunteer completeness.
The LiDAR slant range is . LiDAR penetrates canopy gaps via multiple returns so last-return ground points yield a bare-earth DEM; UAV photogrammetry provides local cm-level orthomosaics/DSMs, while OSM provides crowdsourced global vector features.
List the six ISO elements of spatial data quality. For a supervised land-cover classification assessed against ground-truth check points, the confusion matrix gives diagonal counts (correctly classified pixels) of (Forest), (Agriculture), and (Urban). Compute the Overall Accuracy percentage.
List the six ISO spatial data quality elements:
- Positional accuracy
- Attribute accuracy
- Temporal accuracy
- Lineage
- Completeness (omission and commission)
- Logical consistency (topological, domain, and format compliance)
Compute Overall Accuracy:
The six ISO quality elements are positional accuracy, attribute accuracy, temporal accuracy, lineage, completeness, and logical consistency. The classification achieves an Overall Accuracy of .
What is geospatial metadata, and what roles do ISO 19115 and ISO 19139 play? Briefly explain two challenges encountered when integrating datasets from different agencies.
Metadata and ISO standards: Metadata is structured documentation ("data about data") describing a dataset's CRS, extent, resolution, lineage, accuracy, and usage rights. ISO 19115 defines the conceptual metadata schema (mandatory and optional elements), while ISO 19139 specifies its standard XML encoding so catalogs (CSW) can exchange and search metadata uniformly.
Two data-integration challenges:
- CRS & Datum Mismatch: Combining layers in different datums (e.g., Everest 1830 vs. WGS 84) without proper datum transformation introduces systematic spatial shifts of tens to hundreds of meters.
- Semantic & Edge-Matching Conflicts: Different agencies use incompatible attribute taxonomies (e.g., varying thresholds for "forest") or produce mismatched geometry boundaries across adjacent map sheets requiring conflation and edge matching.
Geospatial metadata documents a dataset's origin, CRS, quality, and terms of use; ISO 19115 defines the content schema and ISO 19139 provides its XML exchange format. Integration challenges include CRS/datum misalignment and semantic/schema conflicts or edge-matching gaps across agency datasets.