xDEM: cross-DEM analysis
repository·main·Indexed 42 days ago
https://github.com/glaciohack/xdemA Python package for the robust analysis of digital elevation models (DEMs) and elevation point clouds (EPCs). Built on Xarray, xDEM provides modular tools for terrain analysis, 3D rigid coregistration (ICP, LZD, CPD), bias correction, and uncertainty analysis. It includes specialized classes for DEM and EPC data abstractions and a spatialstats module for quantifying heteroscedasticity and spatial correlation.
What's inside xDEM
- xDEM is a Python package designed for the analysis of elevation data, specifically Digital Elevation Models (DEMs). The name stands for cross-DEM analysis, reflecting its core functionality of performing cross-analysis of elevation data over static surfaces (e.g., for coregistration and uncertainty analysis). The package is built upon Xarray.
License information for xDEM
mainxDEM is licensed under the Apache License, Version 2.0 (January 2004). This license allows for the use, reproduction, modification, and distribution of the software, subject to certain conditions regarding copyright notices and redistribution requirements.Complementary tools for elevation data analysis
mainxDEM is designed to work within a broader ecosystem of geospatial elevation tools. While xDEM focuses on robust DEM analysis, you can combine it with the following tools for data retrieval and specialized analysis:
Data Retrieval and Pre-processing (Python)
- SlideRule: Cloud-based pre-processing and retrieval of high-resolution elevation data (e.g., ICESat-2, GEDI).
- pDEMtools: Pre-processing and retrieval of ArcticDEM and REMA high-resolution DEMs for polar regions.
- icepyx: Specifically for retrieving ICESat-2 data.
Specialized Analysis (Python)
- PDAL: For processing dense elevation point clouds.
- demcompare: For comparing two DEMs.
- RichDEM: For in-depth terrain and hydrological analysis.
Other Languages
- Julia: Use the
Geomorphometry.jlpackage for terrain analysis. - R: Use the
MultiscaleDTMpackage for modular multi-scale terrain analysis.
Data Sources
- OpenTopography and OpenAltimetry are recommended community resources for retrieving open elevation datasets.
What is bias correction in xDEM
mainIn xDEM, bias-correction methods are transformations used to correct systematic elevation errors and spatially-structured random errors. Unlike affine coregistration, bias corrections are not limited to elevation data; they can incorporate external variables (e.g., land cover type, processing metrics) to identify and correct biases.
While many methods rely on coordinates (e.g., deramping) or terrain (e.g., curvature-dependent corrections) derived from the elevation data, the framework allows for any number of simultaneous dimensions to be used for correction.
Overview of xDEM core concepts
mainxDEM is a Python package for analyzing Digital Elevation Models (DEMs) and Elevation Point Clouds (EPCs). The library is built around two primary data abstractions:
DEM: The central class for manipulating digital elevation models (rasters).EPC: The central class for manipulating elevation point clouds.
Terrain analysis is implemented modularly, re-implementing tools found in
gdalDEM. For complex workflows like 3D rigid coregistration or bias corrections, xDEM uses modularCoregobjects that interface with one another.Additionally, xDEM inherits geospatial functionalities from
GeoUtils, including implicit loading, numerical interfacing, and object-based methods for reprojection, cropping, and vector masking. It relies onRasterio,GeoPandas,Pyproj,NumPy, andSciPyfor its underlying computations.Build coregistration pipelines with `CoregPipeline`
mainWhen a single coregistration method is insufficient, you can combine multiple methods sequentially using the
xdem.coreg.CoregPipelineobject. This object follows a pattern similar to SciKit-Learn'sPipeline. You can construct a pipeline by passing a list of methods to the constructor or by using the+operator to sum them. A pipeline exposes the same interface as a singleCoregobject (e.g.,.fit(),.apply(),.fit_and_apply()) and its results can be exported using.to_matrix().# Option 1: List sequential methods pipeline = xdem.coreg.CoregPipeline([xdem.coreg.ICP(), xdem.coreg.NuthKaab()]) # Option 2: Sum methods pipeline = xdem.coreg.ICP() + xdem.coreg.NuthKaab() # Apply the pipeline aligned_dem = pipeline.fit_and_apply(ref_dem, tba_dem_shifted_rotated)How curvatures work in xDEM
mainCurvatures describe the convexity or concavity of terrain using second derivatives of elevation. xDEM provides six basic curvatures: profile, tangential, planform, flowline, maximal/maximum, and minimal/minimum.
Calculation Systems
You can choose between two mathematical systems using the
curv_methodparameter:"geometric"(default): Suitable for most users; defines curvature via the radius of a circle."directional": Defines curvatures as directional derivatives of the elevation field.
Requirements
All curvature calculations require second-order derivatives ($z_{xx}$, $z_{xy}$, or $z_{yy}$). Therefore, you must use either the
"ZevenbergThorne"or"Florinsky"surface_fitmethods. The"Horn"method is incompatible with curvatures.Note: Curvature values are returned in units of $m^{-1}$ but are multiplied by 100 by convention for easier handling.
Understand xDEM elevation data objects
mainElevation data objects in xDEM are built upon the raster and vector object abstractions from the GeoUtils package. When working with DEMs (Digital Elevation Models) or EPCs (Elevation Profile Curves) in xDEM, you are interacting with these inherited characteristics. For specific implementations, refer to thedem_classandepc_classdocumentation.How uncertainty analysis works in xDEM
mainxDEM's uncertainty analysis focuses on quantifying and propagating random errors (including structured errors), assuming systematic errors have already been addressed via
coregistrationorbiascorr.The analysis integrates spatial statistics and uncertainty quantification to provide three main capabilities:
- Estimating heteroscedasticity: Modeling variable random errors (e.g., errors that change based on terrain slope or curvature).
- Modeling spatial correlation: Estimating how errors are correlated over specific distances (e.g., due to instrument noise or spatial resolution).
- Error propagation: Propagating these errors to elevation derivatives like spatial averages, slope, or aspect.
When using elevation differences between two datasets as an error proxy, you can specify how to treat the precision of the 'other' dataset:
- If the other dataset is much more precise, the differences approximate the target dataset's uncertainty.
- If the other dataset has similar precision, the uncertainties combine quadratically.
Assumptions for spatial statistical inference in xDEM
mainTo perform reliable spatial uncertainty analysis (geostatistics) on elevation data, the data must satisfy the assumption of second-order stationarity. This means the following three conditions must be met:
- Stationary Mean: The mean of the variable of interest is constant over sufficiently large areas (no persistent elevation biases like shifts or tilts that don't average out).
- Stationary Variance: The variance of the variable of interest is constant over sufficiently large areas (no random elevation errors that vary significantly across space).
- Stationary Covariance: The covariance between two observations depends only on the spatial distance (spatial lag) between them, not on other factors.
If your data violates these (e.g., larger errors on steep slopes), you must estimate heteroscedasticity and perform standardization to achieve second-order stationarity.
Analyze georeferenced rasters, vectors, and point clouds with GeoUtils
mainxDEM is specialized for Digital Elevation Model (DEM) analysis. If your workflow requires analyzing georeferenced rasters, vectors, or point cloud data, use xDEM's sister-package GeoUtils instead.Work with xdem.EPC objects
mainThe
xdem.EPCclass represents an Estimated Point Cloud. It inherits all point cloud methods and attributes fromgeoutils.PointCloud.Core Capabilities:
- Creation: Create from XYZ arrays, tuples, or standard arrays via
.from_xyz(),.from_tuples(), or.from_array(). - I/O: Open/save via
EPC()orEPC.to_file(). - Georeferencing: Reproject, crop, or subsample using
.reproject(),.crop(), and.subsample(). - Raster Interface: Convert to a grid via
.grid()or rasterize via.rasterize(). - Coregistration: Perform 3D coregistration using
.coregister_3d(). - Vertical Referencing: Manage VCRS using
.set_vcrs()and.to_vcrs().
import xdem # Example: Creating an EPC from XYZ data x, y, z = [0, 1], [0, 1], [10, 11] epc = xdem.EPC.from_xyz(x, y, z) print(epc.point_count)- Creation: Create from XYZ arrays, tuples, or standard arrays via