RS Pattern Recognition and Classifications
Open in the Padhaiko app5.1 Traditional Statistical Classifiers
Image classification translates multi-band pixel vectors into discrete thematic land-use/land-cover (LULC) labels (e.g., forest, water, cropland, urban). Traditional statistical classifiers are grouped into unsupervised (data-driven clustering) and supervised (trained from labeled reference samples).
Unsupervised Classification: K-Means and ISODATA
Unsupervised classifiers partition pixels into natural spectral clusters without prior training labels; the analyst subsequently assigns semantic class names to each spectral cluster using reference imagery.
| Aspect | K-Means Clustering | ISODATA (Iterative Self-Organizing Data Analysis) |
|---|---|---|
| Number of Clusters () | Fixed upfront by the user | Dynamic: merges closely spaced clusters and splits elongated/high-variance clusters |
| Algorithmic Loop | 1. Initialize centroids; 2. Assign each pixel to nearest centroid (Euclidean); 3. Recompute cluster means until convergence | Adds user thresholds for min/max cluster size, max standard deviation (split criterion), and min centroid distance (merge criterion) |
| Sensitivity | Sensitive to initial seeds and outlier pixels; assumes spherical equal-size clusters | More adaptive to real natural RS distributions, at the cost of tuning more hyperparameters |
Supervised Classification: Minimum Distance, Parallelepiped, and Gaussian Maximum Likelihood
Supervised classification uses labeled training polygons (Regions of Interest, ROIs) to learn spectral class decision boundaries before classifying every pixel in the scene.
Three Classical Supervised Classifiers
Let class () have training mean vector and covariance matrix :
- Parallelepiped (Box) Classifier: Defines an axis-aligned hyper-rectangle around each class using its minimum/maximum or bounds in each band.
- Pros: Extremely fast threshold logic.
- Cons: Ignores band correlation (diagonal spectral clouds produce huge corners), causing overlapping boxes (ambiguous) and unclassified gaps.
- Minimum Distance to Means Classifier: Assigns pixel to the class whose mean vector is closest in Euclidean distance:
- Pros: Every pixel is classified; robust when training samples are small.
- Cons: Ignores differences in class variance and inter-band covariance (e.g., misclassifies points near a compact class toward a wide-variance class).
- Gaussian Maximum Likelihood Classifier (GMLC): Assumes each class follows a multivariate normal distribution . Assigning to maximize posterior probability is equivalent to maximizing the log-discriminant function:
- Uses Mahalanobis distance , naturally modeling elliptical, correlated spectral classes.
- Requires at least (practically ) independent training pixels per class so is invertible and well-conditioned.
5.2 Machine Learning in Earth Observation
Modern Earth-observation workflows frequently combine optical bands, spectral indices (NDVI, NDWI), SAR backscatter, and topographic derivatives (elevation, slope). Because these multi-source features violate multivariate Gaussian assumptions, non-parametric machine learning algorithms dominate operational mapping.
Decision Trees, Random Forest (RF), and Support Vector Machines (SVM)
| Classifier | Core Mechanism | Strengths in RS | Key Limitations |
|---|---|---|---|
| Decision Tree (CART) | Recursively splits feature space with axis-parallel thresholds maximizing Gini impurity or entropy reduction | White-box rules; handles categorical + continuous layers (elevation, soil type) | Single trees easily overfit noisy training pixels and show high variance |
| Random Forest (RF) | Ensemble of decorrelated trees built on bootstrap samples (bagging) and random feature subsets at each split; predicts by majority vote | Resistant to overfitting; handles high-dimensional multi-sensor stacks; provides Out-of-Bag (OOB) error & feature importance | Less interpretable than a single tree; cannot extrapolate outside training range in regression |
| Support Vector Machine (SVM) | Finds the hyperplane maximizing the margin between classes, using kernel tricks (RBF, polynomial) for non-linear boundaries | Excels with small training datasets and high-dimensional hyperspectral data; strong generalization | Sensitive to hyperparameters (); slower to train on millions of samples; multi-class requires one-vs-one/rest |
Feature Selection, Training, and Validation Datasets
- Feature Selection: Removes redundant or noisy bands to prevent overfitting and reduce compute time using filter methods (Jeffries–Matusita separability distance, ReliefF), wrapper methods (Recursive Feature Elimination, RFE), or embedded importance (RF Gini/permutation importance).
- Training and Validation Datasets: Training polygons must be representative, spatially distributed across the scene, and spectrally pure. Crucially, validation samples must be independent of training samples; splitting pixels randomly from inside the same training polygons leaks spatial autocorrelation and produces overly optimistic accuracy.
Object-Based Image Analysis (OBIA) and Segmentation
Traditional pixel-based classifiers treat each pixel in isolation, producing salt-and-pepper speckle noise on very-high-resolution (VHR, ) imagery where individual tree crowns, roofs, and cars span many pixels.
Object-Based Image Analysis (OBIA) resolves this in two stages:
- Image Segmentation: Groups contiguous, spectrally homogeneous pixels into meaningful image objects (segments)—most commonly via multiresolution segmentation (region-growing controlled by user-defined scale, shape/compactness, and color weights) or SLIC superpixels.
- Object Classification: Classifies each segment using not only its mean/std spectral values, but also geometric shape (area, rectangularity, elongation), GLCM texture, and contextual hierarchy (e.g., classifying a dark segment adjacent to a tall building as shadow rather than water).
5.3 Overview of Deep Learning and GeoAI
GeoAI integrates geospatial data science with deep learning—multi-layer neural networks that automatically learn hierarchical spatial, textural, and spectral representations directly from raw image patches rather than relying on hand-engineered features.
Fundamentals of Deep Learning for Earth Observation
Unlike pixel-wise classifiers, deep neural networks take 2D multi-band patches and learn stacked non-linear filters via backpropagation and stochastic gradient descent (Adam/SGD). Remote sensing poses unique challenges compared to everyday RGB photography:
- Imagery is multi-spectral/hyperspectral or SAR (), often -bit radiometric depth.
- Objects lack canonical orientation (rotation invariance), span huge scale ranges, and require georeferenced patch tiling with overlap to avoid boundary seams.
Convolutional Neural Networks (CNNs) for Image and Scene Classification
A Convolutional Neural Network (CNN) slides learnable 2D/3D spatial filters (kernels) across an input patch to extract translation-equivariant features:
- Convolutional layers: Early layers learn low-level edges and textures; deeper layers combine them into semantic motifs (rooftops, road junctions, crop rows). For an input of size , kernel size , padding , and stride , the output spatial dimension is:
- Activation functions (ReLU): introduces non-linearity.
- Pooling / strided convolutions: Downsample spatial dimensions ( max pooling) to expand the receptive field and aggregate global context.
- Fully connected / Global Average Pooling head: Outputs a single scene-level class probability vector (e.g., classifying a tile as Airport, Dense Residential, or Forest in scene benchmarks like EuroSAT or BigEarthNet).
U-Net Architectures for Semantic Pixel-Level Segmentation
Standard classification CNNs discard fine spatial coordinates during downsampling. Semantic segmentation requires predicting a class label for every single pixel (e.g., building footprint extraction, road network mapping, flood inundation masks).
U-Net Encoder–Decoder Architecture
The U-Net architecture (Ronneberger et al., 2015) has a symmetric U-shaped topology:
- Contracting Path (Encoder): Repeated convolutions + ReLU + max pooling halve spatial dimensions while doubling feature channels, capturing what is in the scene (semantic context).
- Bottleneck: Deepest coarse-resolution, high-channel feature representation.
- Expanding Path (Decoder): Transposed convolutions (or bilinear upsampling + convolutions) restore full spatial resolution to recover where objects lie.
- Skip Connections: Concatenate high-resolution feature maps from each encoder stage directly to the matching decoder stage, preserving crisp building edges and narrow roads lost during pooling.
Introductory Applications of Deep Learning in Remote Sensing
- Building & road extraction: U-Net / DeepLabv3+ segmentation of sub-meter VHR imagery for cadastral mapping and disaster preparedness.
- Flood & landslide mapping: Multi-temporal SAR + optical change-detection networks (Siamese CNNs / Vision Transformers).
- Cloud & cloud-shadow masking: Pixel-level semantic segmentation replacing simple spectral thresholds.
- Crop-type & burned-area mapping: Spatio-temporal 3D CNNs and Temporal Attention Encoders over Sentinel-2 time series.
5.4 Accuracy Assessment and Error Metrics
No classification map is complete without a rigorous, statistically sound accuracy assessment comparing map predictions against independent ground-truth or high-resolution reference data.
Confusion (Error) Matrix and Derived Metrics
A confusion matrix (error matrix) is a table where entry counts reference units predicted as class (rows) that truly belong to class (columns), with total samples , row totals , and column totals .
Accuracy Assessment Metrics from the Confusion Matrix
- Overall Accuracy (): Fraction of all reference samples correctly classified:
- Producer's Accuracy ( / Recall): Probability that a true ground-truth sample of class is correctly mapped (omission error ):
- User's Accuracy ( / Precision): Probability that a pixel labeled as class on the map truly belongs to class on the ground (commission error ):
- -Score: Harmonic mean of Precision () and Recall () for class :
- Cohen's Kappa Coefficient (): Measures agreement beyond pure chance (, expected chance agreement ):
(Note: Modern RS best practice reports , , , and —and area-adjusted estimates—alongside , recognizing that can be sensitive to class prevalence.)
Sampling Strategies and Spatial Cross-Validation
- Sampling Strategies:
- Simple Random Sampling: Unbiased, easy to analyze, but small/rare classes (e.g., water or urban in a forested basin) may receive too few samples.
- Stratified Random Sampling: Allocates sample points within each mapped class stratum to ensure adequate precision for rare classes (requiring area-weighted correction when reporting population and ).
- Systematic / Cluster Sampling: Uniform grid coverage or field-efficient plots.
- Spatial Cross-Validation: Because nearby pixels share high spatial autocorrelation (similar soils, lighting, sensor artifacts), ordinary random -fold cross-validation mixes adjacent training and test pixels, inflating accuracy estimates. Spatial cross-validation partitions data into geographically disjoint blocks (spatial folds) so the model is always tested on unseen geographic regions, measuring true spatial generalization.
Worked Out Examples
5.1
In a single-band NIR classification with equal prior probabilities (), training statistics are:
- Water (): , ()
- Vegetation (): , ()
Classify a pixel with value using (1) the Minimum Distance to Means classifier and (2) the Gaussian Maximum Likelihood Classifier (GMLC), and explain why the results differ.
Minimum Distance to Means classification:
Since , Minimum Distance assigns to Water ().
Gaussian Maximum Likelihood classification:
Using :
- For Water ():
- For Vegetation ():
Since , GMLC assigns to Vegetation ().
- Minimum Distance assigns Water () ().
- GMLC assigns Vegetation () () because is standard deviations away from Water's narrow mean () but only standard deviations from Vegetation's broad distribution ().
5.2
The following confusion matrix (rows = mapped class, columns = reference truth) summarizes validation points:
| Mapped \ Reference | Forest | Crop | Water | Row Total () |
|---|---|---|---|---|
| Forest | ||||
| Crop | ||||
| Water | ||||
| Col Total () |
Compute:
- Overall Accuracy ().
- Producer's Accuracy (Recall), User's Accuracy (Precision), and -score for the Crop class.
- Cohen's Kappa coefficient ().
Computing Overall Accuracy:
Computing Producer's Accuracy, User's Accuracy, and -score for Crop:
- Producer's Accuracy (Recall):
- User's Accuracy (Precision):
- -score:
Computing Cohen's Kappa ():
- .
- Crop: , , .
- (substantial agreement).
5.3
- Perform one full K-Means assignment and centroid-update pass on four 2-band pixels starting from initial centroids and .
- A U-Net encoder takes a Sentinel-2 image patch through one unpadded convolution (stride , padding ) followed by a max-pooling layer (stride ). What is the spatial resolution of the resulting feature map?
K-Means assignment and centroid update: Compute squared Euclidean distances :
- : ; .
- : ; .
- : ; .
- : ; .
Updated centroids:
CNN / U-Net encoder output dimensions:
- After unpadded convolution ():
- After max pooling ():
- Cluster 1 with ; Cluster 2 with .
- Feature-map size after convolution and pooling is .
Exercises
Exercise — RS Pattern Recognition, GeoAI & Accuracy Assessment
Contrast unsupervised classification (K-Means and ISODATA) with supervised classification (Parallelepiped, Minimum Distance, and Gaussian Maximum Likelihood), highlighting when GMLC fails and how ISODATA improves on K-Means.
Unsupervised clustering (K-Means vs. ISODATA): K-Means partitions pixels into a fixed number of spherical clusters via iterative Euclidean assignment and centroid updates. ISODATA extends K-Means by dynamically splitting high-variance clusters and merging closely spaced or tiny clusters using user thresholds.
Supervised statistical classifiers: Parallelepiped uses fast axis-aligned boxes but suffers overlaps/gaps with correlated bands; Minimum Distance assigns to the nearest class mean but ignores class variance; GMLC models class covariance via Mahalanobis distance, providing optimal Bayesian boundaries for Gaussian data—unless training samples per class are fewer than (singular ) or distributions are strongly multimodal.
ISODATA adapts cluster count by splitting/merging unlike fixed- K-Means. Among supervised methods, GMLC models band covariance unlike Parallelepiped and Minimum Distance, but requires sufficient training samples () and Gaussian class distributions.
Compare Random Forest (RF) and Support Vector Machines (SVM) for Earth-observation classification, explain why independent validation data are essential during feature selection and training, and describe how Object-Based Image Analysis (OBIA) reduces salt-and-pepper noise on high-resolution imagery.
RF vs. SVM and dataset independence: Random Forest averages an ensemble of bootstrapped decision trees with random feature subsets, tolerating noisy multi-sensor stacks and yielding OOB error and feature importance. SVM finds maximum-margin hyperplanes in kernel space, excelling with small training sets in high dimensions. Validation polygons must be spatially and sample-wise independent from training polygons (and feature selection) to avoid optimistic bias from spatial autocorrelation.
OBIA segmentation workflow: On sub-meter/meter VHR imagery, individual pixels capture shadows or roof tiles rather than whole objects, causing salt-and-pepper noise. OBIA first segments contiguous homogeneous pixels into coherent image objects (e.g., via multiresolution segmentation), then classifies objects using spectral statistics, shape metrics, texture, and neighborhood context.
RF excels on large multi-source stacks via bagging and feature importance, while SVM performs strongly on small high-dimensional datasets. OBIA segments homogeneous regions first and classifies objects using spectral, shape, texture, and contextual features, eliminating pixel-level noise.
Explain how Convolutional Neural Networks (CNNs) perform scene classification and why U-Net adds an expanding decoder with skip connections for semantic pixel-level segmentation in GeoAI. If a satellite patch passes through a convolution with stride and padding ("same" padding), what is the output spatial size?
CNNs vs. U-Net in GeoAI: A scene-classification CNN applies stacked convolutions and pooling layers to compress spatial grids into high-level semantic features and a single image-level class label. For pixel-level semantic segmentation, U-Net pairs this contracting encoder with a symmetric expanding decoder that upsamples back to full resolution, while skip connections copy fine spatial edge features from early encoder layers directly into the decoder.
Output spatial dimension:
Thus the output feature map preserves the resolution.
CNNs downsample to predict a single scene label, whereas U-Net uses a symmetric encoder–decoder with skip connections to preserve crisp spatial boundaries at every pixel. With , output size remains .
A binary land–water classification is evaluated on reference pixels: true Water pixels are mapped as Water, true Water pixels are mapped as Land, true Land pixels are mapped as Water, and true Land pixels are mapped as Land. Construct the confusion matrix and calculate (a) Overall Accuracy, (b) Producer's and User's Accuracy for Water, (c) -score for Water, and (d) Cohen's Kappa ().
Constructing the confusion matrix (rows = Mapped, cols = Reference):
- Row 1 (Mapped Water): , row total .
- Row 2 (Mapped Land): , row total .
- Column totals: Reference Water , Reference Land , .
Computing OA, PA, UA, , and :
- (a) ().
- (b) (); ().
- (c) .
- (d) .
(a) ; (b) Water ; (c) Water ; (d) .
Compare simple random sampling and stratified random sampling for map accuracy assessment, and explain why spatial cross-validation is necessary when evaluating machine learning models on satellite imagery.
Simple random vs. stratified random sampling: Simple random sampling selects points uniformly across the scene, yielding unbiased population estimates directly, but rare classes (e.g., wetlands or urban roads) may receive too few points for reliable User's Accuracy estimates. Stratified random sampling draws a minimum sample count within each mapped class stratum to secure tight per-class confidence intervals, then applies area-weighted adjustments to compute unbiased population OA and PA.
Need for spatial cross-validation: Remote sensing observations exhibit strong spatial autocorrelation: neighboring pixels share illumination, soil moisture, and sensor artifacts. Standard random -fold splits place adjacent pixels in both training and validation folds, causing data leakage and inflated accuracy. Spatial cross-validation separates folds into geographically disjoint blocks, testing true model generalization across unseen landscapes.
Stratified random sampling guarantees adequate sample counts for rare classes (with area-weighted population correction) unlike simple random sampling; spatial cross-validation uses geographically disjoint blocks to prevent spatial autocorrelation leakage from inflating accuracy metrics.