Digital RS Image Processing and Enhancements
Open in the Padhaiko app4.1 Spatial Enhancement and Filtering
While radiometric calibration and point-wise contrast stretches modify each pixel independently, spatial enhancement modifies a pixel's value based on its surrounding neighborhood. Spatial filtering suppresses random sensor noise, sharpens blurred boundaries, and isolates roads, faults, and field edges.
Digital Image Representation and Convolution Kernels
A single-band digital RS image is represented as a 2D discrete matrix of rows and columns, where each pixel holds a quantized Digital Number (DN) or reflectance value. A spatial filter slides a small odd-sized weight matrix (or convolution kernel / mask) of size across every pixel :
(Strictly, correlation slides without flipping while convolution rotates by ; for symmetric smoothing kernels both are identical.) Border pixels are handled by zero-padding, mirror reflection, or edge replication.
Low-Pass Smoothing and High-Pass Sharpening
| Aspect | Low-Pass (Smoothing) Filters | High-Pass (Sharpening) Filters |
|---|---|---|
| Frequency Response | Passes low spatial frequencies (broad homogeneous regions); attenuates high frequencies (noise, sharp edges) | Passes high spatial frequencies (edges, lineaments, fine detail); suppresses slowly varying background |
| Kernel Weights | All weights positive, summing to (e.g., box mean or Gaussian weights) | Positive center surrounded by negative neighbors; weights sum to (pure edge/detail) or (high-boost sharpening) |
| Example Kernel | (or ) | (high-pass) or (high-boost) |
| Non-Linear Alternative | Median filter (replaces center with neighborhood median; removes salt-and-pepper noise without blurring edges) | Unsharp masking: |
Gradient Operators and Edge Detection (Prewitt, Sobel, Laplacian)
Edges correspond to rapid spatial changes in image intensity, detected using first-order derivatives (gradients) or second-order derivatives (Laplacian).
First- and Second-Order Edge Detection Operators
- Gradient Vector and Magnitude (First-Order): For continuous , the gradient is , approximated by horizontal () and vertical () convolution masks:
- Prewitt Operators: Equal weights along the orthogonal axis:
- Sobel Operators: Give double weight () to the center row/column to smooth noise slightly before differentiation:
- Laplacian Operator (Second-Order, Isotropic): Measures the sum of unmixed second derivatives . Common discrete masks (4-neighbor and 8-neighbor) are:
The Laplacian is rotation-invariant (detects edges in all orientations in a single pass) and locates edges at zero-crossings, though it is twice as sensitive to high-frequency noise as first-order gradients (often paired with Gaussian smoothing as Laplacian of Gaussian, LoG).
4.2 Spectral Transformations and Indices
Multispectral enhancement combines pixel values across different wavelength bands at the same location to isolate biophysical properties (chlorophyll, water, burn severity) and remove inter-band correlation.
Normalized Difference Spectral Indices
Normalized difference indices take the ratio , which reduces topographic illumination, solar angle, and multiplicative atmospheric effects:
| Index | Formula | Typical Range & Interpretation | Primary Applications |
|---|---|---|---|
| NDVI (Normalized Difference Vegetation Index) | Dense vegetation: ; bare soil: ; water: | Vegetation greenness, LAI, crop vigor, drought & phenology monitoring | |
| NDWI (Normalized Difference Water Index — McFeeters / Gao) | (McFeeters) or (Gao) | Open water (McFeeters); high canopy water (Gao) | Surface water delineation & flood extent (McFeeters); leaf/canopy water content (Gao) |
| NBR (Normalized Burn Ratio) | Healthy forest: high positive; freshly burned scar: negative () | Wildfire perimeter mapping and burn-severity grading | |
| NDRE (Normalized Difference Red-Edge Index) | Sensitive across high-biomass crops where NDVI saturates () | Mid-to-late-season nitrogen stress, chlorophyll content, precision agriculture |
Principal Component Analysis (PCA) for Band Decorrelation
Adjacent spectral bands (e.g., Blue, Green, Red) are often strongly correlated, creating redundant information. Principal Component Analysis (PCA) (also called the Karhunen–Loève transform) applies an orthogonal linear transformation derived from the band covariance matrix (or correlation matrix):
where columns of are the orthonormal eigenvectors of ordered by decreasing eigenvalues :
- PC1 aligns along the direction of maximum scene variance (typically overall albedo/topography).
- PC2, PC3 capture orthogonal spectral contrasts (vegetation vs. soil, moisture).
- Higher components ( for large ) contain mostly uncorrelated random sensor noise and can be discarded for dimensionality reduction and noise suppression.
Tasseled Cap Transformation (Kauth–Thomas)
Unlike data-dependent PCA, the Tasseled Cap Transformation uses fixed, sensor-specific linear combinations of optical bands (originally derived for Landsat MSS/TM/OLI and Sentinel-2) to project reflectance into three physically interpretable orthogonal axes:
- Brightness: Weighted sum of all bands, capturing soil albedo and bare-rock reflectance.
- Greenness: Contrast between NIR and visible bands, measuring green vegetation vigor and biomass.
- Wetness: Contrast between VNIR and SWIR bands, sensitive to soil and canopy moisture content.
Because its coefficients are fixed for a given sensor, Tasseled Cap components can be compared consistently across scenes and seasons.
4.3 Imaging Spectrometry
Multispectral vs. Hyperspectral Remote Sensing
Multispectral sensors sample a few discrete, broad bands separated by spectral gaps, capturing overall spectral slope. Imaging spectrometry (hyperspectral remote sensing) records dozens to hundreds of narrow, contiguous spectral bands across the solar-reflected () or thermal spectrum, reconstructing a continuous reflectance curve for every pixel that resolves diagnostic narrow mineral, pigment, and gas absorption features.
Hyperspectral Data Cube Representation
A hyperspectral dataset forms a 3D data cube with two spatial dimensions ( rows columns) and one spectral dimension ( bands). Each pixel corresponds to a continuous -element spectral vector , while each slice at fixed is a monochrome 2D image. Standard storage interleavings are BSQ (Band Sequential), BIL (Band Interleaved by Line), and BIP (Band Interleaved by Pixel—fastest for pixel-wise spectral matching).
Spectral Sampling, Spectral Resolution, and FWHM
Spectral Sampling Interval vs. Spectral Resolution (FWHM)
For a spectrometer channel centered at with spectral response function :
- Spectral Sampling Interval (): The wavelength spacing between adjacent band centers ().
- Full Width at Half Maximum (): The spectral bandwidth between the two wavelengths where drops to of its peak value—this defines the true spectral resolution of the instrument.
- Nyquist Spectral Sampling: Critical sampling requires , though many spaceborne sensors operate near (e.g., bands at sampling).
4.4 High-Dimensional Data Processing
Curse of Dimensionality and the Hughes Phenomenon
Moving from multispectral bands to hyperspectral bands vastly expands feature-space volume (). In -dimensional space:
- Almost all volume concentrates in the outer shell and corners of the hypercube, making Euclidean distances between points nearly uniform.
- Hughes Phenomenon: For a fixed, limited number of labeled training samples, classification accuracy initially rises as bands are added, peaks at an optimal dimensionality, and then degrades sharply as grows further. Because estimating a covariance matrix requires estimating parameters, small training sets produce singular or poorly conditioned covariance estimates (overfitting).
Noise Reduction and Minimum Noise Fraction (MNF) Transformation
Before classification or unmixing, noisy spectral bands (e.g., strong absorption bands at and ) are dropped and noise is suppressed via the Minimum Noise Fraction (MNF) transform (Green et al., 1988).
Minimum Noise Fraction (MNF) Transformation
Unlike standard PCA—which orders components purely by total variance (so a very noisy band with high variance can dominate PC1)—MNF orders components strictly by Signal-to-Noise Ratio (SNR). It consists of two cascaded PCA rotations:
- Noise-whitening rotation: Estimate the noise covariance matrix (e.g., from homogeneous areas or shift-difference residuals between adjacent pixels) and transform the data so noise becomes uncorrelated with unit variance in every channel ().
- Standard PCA on noise-whitened data: Compute principal components of the whitened covariance. Resulting MNF eigenvalues equal :
- Low-order MNF bands () hold coherent spatial/spectral signal.
- High-order MNF bands () hold pure random noise and are discarded or smoothed before applying the inverse MNF transform for denoising.
4.5 Spectral Matching and Sub-pixel Unmixing
Spectral Angle Mapper (SAM) and Spectral Information Divergence (SID)
Spectral Angle Mapper (SAM) and Spectral Information Divergence (SID)
- Spectral Angle Mapper (SAM): Treats an observed pixel spectrum and a reference library spectrum as vectors in -dimensional space and measures the angle between them:
Smaller angle indicates closer spectral match. Because SAM depends only on vector direction (not vector magnitude), it is invariant to uniform illumination/topographic shading differences.
- Spectral Information Divergence (SID): Models each spectrum as a probability distribution and , and measures symmetric Kullback–Leibler divergence:
SID captures subtle probabilistic spectral variability that geometric angle measures may miss.
Linear Spectral Mixture Models (LSMM)
At spatial resolution, a single pixel often contains a mixture of materials (e.g., vegetation, soil, impervious roof). Under the Linear Spectral Mixture Model, each photon interacts with only one surface type, so observed pixel reflectance is a convex linear combination of pure material spectra (endmembers) :
subject to two physical abundance constraints (Full Constrained Least Squares, FCLS):
- Abundance Non-negativity Constraint (ANC): for all .
- Abundance Sum-to-One Constraint (ASC): .
Endmember Extraction: Pixel Purity Index (PPI) and N-FINDR
Under linear mixing, all mixed pixels lie inside a convex simplex in whose vertices are the pure endmembers:
- Pixel Purity Index (PPI): Repeatedly projects the MNF-reduced data cloud onto thousands of random unit vectors (skewers). Pixels that fall at the extreme minimum or maximum ends of each projection accumulate a high purity tally; pixels with the highest PPI scores identify candidate pure endmembers.
- N-FINDR: Automatically finds the set of image pixels whose convex simplex has the maximum possible volume in -dimensional MNF subspace (). Starting from random initial pixels, N-FINDR iteratively tests replacing each vertex with every pixel in the image whenever doing so increases simplex volume, converging to the purest endmember set without manual inspection.
Worked Out Examples
4.1
Consider a image window centered at pixel with DN values:
Compute the filtered output at the center pixel using:
- A low-pass box mean filter.
- Sobel gradient masks and , and the exact gradient magnitude .
- The 4-neighbor Laplacian operator .
Applying the box mean filter:
Applying the Sobel gradient masks ( and ):
- Horizontal gradient with :
- Vertical gradient with :
- Gradient magnitude:
Applying the 4-neighbor Laplacian :
- Low-pass box mean: .
- Sobel gradients: , , .
- 4-neighbor Laplacian: .
4.2
A calibrated Sentinel-2 pixel over a healthy forest canopy records surface reflectances: , , , , and . Calculate NDVI, NDWI (McFeeters), NBR, and NDRE, and briefly interpret the values.
Computing NDVI and NDRE:
Computing NDWI (McFeeters) and NBR:
(dense healthy canopy), (high chlorophyll without red-band saturation), (non-water land pixel), and (unburned healthy vegetation).
4.3
- A 3-band pixel spectrum is compared against a reference library spectrum using the Spectral Angle Mapper (SAM). Compute the spectral angle in radians and degrees.
- In a 2-band (Red, NIR) linear spectral mixture model, pure vegetation endmember is and bare soil endmember is . Unmix an observed pixel under ANC and ASC ().
Computing the SAM spectral angle:
- Dot product: .
- Norms: , .
- Cosine and angle:
Solving the linear mixture model:
From the Red band ():
Thus . Checking the NIR band:
Both ANC () and ASC () hold with zero residual.
- SAM angle ().
- Sub-pixel fractions: vegetation () and bare soil ().
Exercises
Exercise — Digital Image Processing, Spectrometry & Unmixing
Compare low-pass, high-pass, and Laplacian spatial convolution filters, and explain why the Sobel operator is often preferred over the Prewitt operator on noisy satellite imagery. Given a vertical step edge window where columns 1, 2, 3 have constant values across all three rows, compute the Sobel gradient components , , and magnitude at the center pixel.
Filter comparison and Sobel vs. Prewitt: Low-pass filters (positive weights summing to 1) blur noise and smooth variations; high-pass filters (weights summing to 0) isolate edges; the Laplacian is an isotropic second-derivative operator detecting zero-crossings in all directions at once. Sobel assigns weight to the central row/column, introducing mild directional smoothing that makes it less sensitive to random pixel noise than Prewitt.
Sobel calculation on the vertical step window:
Sobel suppresses noise better than Prewitt via center weighting. For the vertical ramp , , , and .
Define NDVI, NDWI, NBR, and NDRE, and explain the roles of PCA and the Tasseled Cap transformation in spectral enhancement. A forest site has pre-fire reflectances and post-fire reflectances . Compute , , and .
Indices, PCA, and Tasseled Cap: NDVI (NIR, Red) measures greenness; NDWI (Green–NIR or NIR–SWIR) tracks open/canopy water; NBR (NIR, SWIR2) quantifies burn severity; NDRE (NIR, Red-Edge) avoids saturation in dense crops. PCA orthogonalizes correlated bands along data-driven maximum-variance axes, whereas Tasseled Cap uses fixed sensor coefficients to yield Brightness, Greenness, and Wetness.
Computing pre-fire, post-fire, and differenced NBR:
, , and (indicating high burn severity).
Distinguish between multispectral and hyperspectral remote sensing using the concept of a hyperspectral data cube, spectral sampling interval, and Full Width at Half Maximum (FWHM). Why can hyperspectral sensors distinguish clay minerals or tree species that multispectral sensors merge together?
Multispectral vs. hyperspectral and FWHM: Multispectral sensors measure broad, separated bands (), whereas hyperspectral imaging spectrometers collect contiguous narrow bands forming a 3D data cube . Spectral sampling interval is the spacing between adjacent band centers, while FWHM is the bandwidth at peak response (defining true spectral resolution).
Resolving narrow diagnostic features: Broad multispectral bands average out narrow absorption dips ( wide) caused by molecular vibrations in clay minerals (e.g., kaolinite vs. montmorillonite near ) or subtle pigment shifts. Contiguous narrow FWHM channels preserve the exact band center, depth, and asymmetry of each spectrum.
Hyperspectral cubes sample contiguous narrow bands characterized by sampling interval and FWHM, resolving narrow diagnostic mineral and biochemical absorption features that broad multispectral bands average out.
Explain the curse of dimensionality and the Hughes phenomenon in hyperspectral classification, and describe how the Minimum Noise Fraction (MNF) transformation improves upon standard PCA for high-dimensional noise reduction.
Curse of dimensionality and Hughes phenomenon: As band count grows, feature-space volume expands exponentially and training samples become sparse. Estimating class covariance matrices requires parameters; with fixed training sample size, classification accuracy rises initially with extra bands and then drops once parameter estimation variance outweighs added spectral information (Hughes phenomenon).
Why MNF outperforms PCA for denoising: Standard PCA orders components by total variance without distinguishing signal from noise, so a high-variance noisy band can appear in early PCs. MNF performs a two-stage cascade—first whitening the noise covariance , then applying PCA—which ranks output components strictly by Signal-to-Noise Ratio (SNR), cleanly separating coherent signal () from random noise ().
The Hughes phenomenon occurs when adding bands beyond training-sample support degrades classifier accuracy due to poor parameter estimation. MNF whitens noise before PCA so components rank strictly by SNR rather than raw variance.
Contrast Spectral Angle Mapper (SAM) and Spectral Information Divergence (SID) for spectral matching, and explain how Pixel Purity Index (PPI) and N-FINDR extract endmembers for linear spectral unmixing. In a linear mixture of built roof () and dry sand (), if an observed pixel has , find the sub-pixel fractions of the two endmembers.
SAM vs. SID and endmember extraction: SAM measures the vector angle and is invariant to illumination scale; SID treats normalized spectra as probability distributions and computes symmetric Kullback–Leibler divergence. In linear unmixing ( with ), pure endmembers form the vertices of the data simplex: PPI counts how often pixels lie at extremes of random unit projections, while N-FINDR selects the pixels maximizing simplex volume.
Unmixing two endmembers (, , observed ): Using ():
SAM uses illumination-invariant vector angles while SID uses symmetric KL divergence; PPI scores projection extremes and N-FINDR maximizes simplex volume to find endmembers. Sub-pixel fractions are () and ().