xDEM: cross-DEM analysis

repository·main·Indexed 42 days ago

https://github.com/glaciohack/xdem

A Python package for the robust analysis of digital elevation models (DEMs) and elevation point clouds (EPCs). Built on Xarray, xDEM provides modular tools for terrain analysis, 3D rigid coregistration (ICP, LZD, CPD), bias correction, and uncertainty analysis. It includes specialized classes for DEM and EPC data abstractions and a spatialstats module for quantifying heteroscedasticity and spatial correlation.

Tokens
33.1K
Snippets
100
Records
156
Agent score
52%

What's inside xDEM

  1. What is xDEM?

    main
    xDEM is a Python package designed for the analysis of elevation data, specifically Digital Elevation Models (DEMs). The name stands for cross-DEM analysis, reflecting its core functionality of performing cross-analysis of elevation data over static surfaces (e.g., for coregistration and uncertainty analysis). The package is built upon Xarray.
  2. License information for xDEM

    main
    xDEM is licensed under the Apache License, Version 2.0 (January 2004). This license allows for the use, reproduction, modification, and distribution of the software, subject to certain conditions regarding copyright notices and redistribution requirements.
  3. Complementary tools for elevation data analysis

    main

    xDEM is designed to work within a broader ecosystem of geospatial elevation tools. While xDEM focuses on robust DEM analysis, you can combine it with the following tools for data retrieval and specialized analysis:

    Data Retrieval and Pre-processing (Python)

    • SlideRule: Cloud-based pre-processing and retrieval of high-resolution elevation data (e.g., ICESat-2, GEDI).
    • pDEMtools: Pre-processing and retrieval of ArcticDEM and REMA high-resolution DEMs for polar regions.
    • icepyx: Specifically for retrieving ICESat-2 data.

    Specialized Analysis (Python)

    • PDAL: For processing dense elevation point clouds.
    • demcompare: For comparing two DEMs.
    • RichDEM: For in-depth terrain and hydrological analysis.

    Other Languages

    • Julia: Use the Geomorphometry.jl package for terrain analysis.
    • R: Use the MultiscaleDTM package for modular multi-scale terrain analysis.

    Data Sources

    • OpenTopography and OpenAltimetry are recommended community resources for retrieving open elevation datasets.
  4. What is bias correction in xDEM

    main

    In xDEM, bias-correction methods are transformations used to correct systematic elevation errors and spatially-structured random errors. Unlike affine coregistration, bias corrections are not limited to elevation data; they can incorporate external variables (e.g., land cover type, processing metrics) to identify and correct biases.

    While many methods rely on coordinates (e.g., deramping) or terrain (e.g., curvature-dependent corrections) derived from the elevation data, the framework allows for any number of simultaneous dimensions to be used for correction.

  5. Overview of xDEM core concepts

    main

    xDEM is a Python package for analyzing Digital Elevation Models (DEMs) and Elevation Point Clouds (EPCs). The library is built around two primary data abstractions:

    • DEM: The central class for manipulating digital elevation models (rasters).
    • EPC: The central class for manipulating elevation point clouds.

    Terrain analysis is implemented modularly, re-implementing tools found in gdalDEM. For complex workflows like 3D rigid coregistration or bias corrections, xDEM uses modular Coreg objects that interface with one another.

    Additionally, xDEM inherits geospatial functionalities from GeoUtils, including implicit loading, numerical interfacing, and object-based methods for reprojection, cropping, and vector masking. It relies on Rasterio, GeoPandas, Pyproj, NumPy, and SciPy for its underlying computations.

  6. Build coregistration pipelines with `CoregPipeline`

    main

    When a single coregistration method is insufficient, you can combine multiple methods sequentially using the xdem.coreg.CoregPipeline object. This object follows a pattern similar to SciKit-Learn's Pipeline. You can construct a pipeline by passing a list of methods to the constructor or by using the + operator to sum them. A pipeline exposes the same interface as a single Coreg object (e.g., .fit(), .apply(), .fit_and_apply()) and its results can be exported using .to_matrix().

    # Option 1: List sequential methods
    pipeline = xdem.coreg.CoregPipeline([xdem.coreg.ICP(), xdem.coreg.NuthKaab()])
    
    # Option 2: Sum methods
    pipeline = xdem.coreg.ICP() + xdem.coreg.NuthKaab()
    
    # Apply the pipeline
    aligned_dem = pipeline.fit_and_apply(ref_dem, tba_dem_shifted_rotated)
  7. How curvatures work in xDEM

    main

    Curvatures describe the convexity or concavity of terrain using second derivatives of elevation. xDEM provides six basic curvatures: profile, tangential, planform, flowline, maximal/maximum, and minimal/minimum.

    Calculation Systems

    You can choose between two mathematical systems using the curv_method parameter:

    • "geometric" (default): Suitable for most users; defines curvature via the radius of a circle.
    • "directional": Defines curvatures as directional derivatives of the elevation field.

    Requirements

    All curvature calculations require second-order derivatives ($z_{xx}$, $z_{xy}$, or $z_{yy}$). Therefore, you must use either the "ZevenbergThorne" or "Florinsky" surface_fit methods. The "Horn" method is incompatible with curvatures.

    Note: Curvature values are returned in units of $m^{-1}$ but are multiplied by 100 by convention for easier handling.

  8. Understand xDEM elevation data objects

    main
    Elevation data objects in xDEM are built upon the raster and vector object abstractions from the GeoUtils package. When working with DEMs (Digital Elevation Models) or EPCs (Elevation Profile Curves) in xDEM, you are interacting with these inherited characteristics. For specific implementations, refer to the dem_class and epc_class documentation.
  9. How uncertainty analysis works in xDEM

    main

    xDEM's uncertainty analysis focuses on quantifying and propagating random errors (including structured errors), assuming systematic errors have already been addressed via coregistration or biascorr.

    The analysis integrates spatial statistics and uncertainty quantification to provide three main capabilities:

    1. Estimating heteroscedasticity: Modeling variable random errors (e.g., errors that change based on terrain slope or curvature).
    2. Modeling spatial correlation: Estimating how errors are correlated over specific distances (e.g., due to instrument noise or spatial resolution).
    3. Error propagation: Propagating these errors to elevation derivatives like spatial averages, slope, or aspect.

    When using elevation differences between two datasets as an error proxy, you can specify how to treat the precision of the 'other' dataset:

    • If the other dataset is much more precise, the differences approximate the target dataset's uncertainty.
    • If the other dataset has similar precision, the uncertainties combine quadratically.
  10. Assumptions for spatial statistical inference in xDEM

    main

    To perform reliable spatial uncertainty analysis (geostatistics) on elevation data, the data must satisfy the assumption of second-order stationarity. This means the following three conditions must be met:

    1. Stationary Mean: The mean of the variable of interest is constant over sufficiently large areas (no persistent elevation biases like shifts or tilts that don't average out).
    2. Stationary Variance: The variance of the variable of interest is constant over sufficiently large areas (no random elevation errors that vary significantly across space).
    3. Stationary Covariance: The covariance between two observations depends only on the spatial distance (spatial lag) between them, not on other factors.

    If your data violates these (e.g., larger errors on steep slopes), you must estimate heteroscedasticity and perform standardization to achieve second-order stationarity.

  11. Work with xdem.EPC objects

    main

    The xdem.EPC class represents an Estimated Point Cloud. It inherits all point cloud methods and attributes from geoutils.PointCloud.

    Core Capabilities:

    • Creation: Create from XYZ arrays, tuples, or standard arrays via .from_xyz(), .from_tuples(), or .from_array().
    • I/O: Open/save via EPC() or EPC.to_file().
    • Georeferencing: Reproject, crop, or subsample using .reproject(), .crop(), and .subsample().
    • Raster Interface: Convert to a grid via .grid() or rasterize via .rasterize().
    • Coregistration: Perform 3D coregistration using .coregister_3d().
    • Vertical Referencing: Manage VCRS using .set_vcrs() and .to_vcrs().
    import xdem
    
    # Example: Creating an EPC from XYZ data
    x, y, z = [0, 1], [0, 1], [10, 11]
    epc = xdem.EPC.from_xyz(x, y, z)
    print(epc.point_count)