xESMF: Universal Regridder for Geospatial Data

xESMF is designed to address the limitations of traditional interpolation tools (like SciPy's interp2d or MATLAB's interp2) when applied to geospatial data.

Key advantages include:

1. Scientific Correctness: Unlike standard image processing interpolation, xESMF accounts for the spherical geometry of the Earth to prevent distortion. It also supports algorithms that prioritize physical properties like conservation (e.g., total mass conservation) and monotonicity (e.g., preventing negative air density), which piecewise polynomial splines often fail to maintain.
2. Support for Complex Grids: xESMF supports regridding between general curvilinear grids (quadrilateral or "logically rectilinear" grids), provided that geographic latitude and longitude can be represented as 2D or 1D arrays. Supported examples include:
   • Cubed-Sphere grids (e.g., in GFDL-FV3)
   • Latitude-Longitude-Cap grids (e.g., in MITgcm)
   • Lambert Conformal grids (e.g., in WRF) Note: xESMF does not currently support non-quadrilateral grids like the hexagonal grid in MPAS.
3. Usability and Integration: xESMF integrates directly with the existing Python scientific stack. It works with numpy.ndarray and tracks metadata using xarray.DataArray and xarray.Dataset, minimizing the learning curve for users already familiar with these libraries.

If xESMF does not suit your specific needs, several other geospatial regridding tools exist for data on the sphere. xESMF is designed specifically to provide a friendly interface for the xarray community, whereas other tools may be more focused on Earth System Model (ESM) infrastructure or require more complex API interactions.

Key Alternatives:

• ESMPy: A Python interface to the Fortran-based ESMF. It provides access to ESMF's powerful regridding engine (which handles general irregular 2D or 3D meshes) but has a complex, non-intuitive API that often requires significant boilerplate code.
• TempestRemap: A modern C++ package supporting arbitrary-order conservative remapping and on-the-fly cubed-sphere grid generation.
• xCDAT: A Python package that offers regridding algorithms from both xESMF and regrid2 (originally from the CDAT cdutil package), including support for vertical regridding.
• Iris: A Python package with built-in interpolation and regridding capabilities.
• NCO: A command-line tool providing regridding functionality.
• NCL (NCAR Command Language): Provides bilinear and conservative algorithms for rectilinear grids, with an ESMF wrapper for more complex grid types.
• ESMF: A massive Fortran package used as infrastructure for Earth System Models. It is intended for building models rather than direct data analysis.

When source and target grids are too large for local memory, use the ESMF_RegridWeightGen command-line tool on an HPC system with MPI. This tool generates weight files that can be reused in xESMF.

NetCDF Attribute Requirements

To avoid file type not recognized errors, ensure your NetCDF files have correct units attributes for geographical coordinates:

• Longitude: Must use one of degrees_east, degree_east, degrees_E, degree_E, degreesE, or degreeE.
• Latitude: Must use one of degrees_north, degree_north, degrees_N, degree_N, degreesN, or degreeN.
• Conservative Regridding: For cell corners, use grid_corner_lon and grid_corner_lat.

Command Usage

You can run the generator in parallel using mpirun.

Key Flags:

• --netcdf4: Use this if the resulting weight file is very large (NetCDF3 has size limitations).
• --src_regional / --dst_regional: Use if one of the grids is not periodic in longitude.

$ mpirun -np 252 ESMF_RegridWeightGen -s source.nc -d destination.nc -w weights.nc -m bilinear

If you are installing ESMF via conda on an HPC, you may encounter MPI conflicts where the conda-provided mpirun does not work with the system's MPI libraries. To resolve this, you must recompile mpi4py from source, linking it specifically to your HPC's MPI implementation using a custom mpi.cfg file.

1. Create/edit an mpi.cfg file with your system's MPI paths (e.g., mpi_dir, include_dirs, libraries, mpicc, etc.).
2. Download the mpi4py source tarball.
3. Build and install using python setup.py build --mpi=<your_config_name>.

# Example mpi.cfg content
[gaea-gnu]
  mpi_dir              = /opt/cray/pe/mpt/7.7.11/gni/mpich-gnu/8.2
  include_dirs         = %(mpi_dir)s/include
  libraries            = mpich
  library_dirs         = %(mpi_dir)s/lib
  runtime_library_dirs = %(mpi_dir)s/lib
  mpicc                = /opt/gcc/8.2.0/bin/gcc
  mpicxx               = /opt/gcc/8.2.0/bin/g++

# Recompilation steps
wget https://bitbucket.org/mpi4py/mpi4py/downloads/mpi4py-3.0.3.tar.gz
tar -zxf mpi4py-3.0.3.tar.gz
conda activate myenv
cat gaea_mpi.cfg >> mpi4py-3.0.3/mpi.cfg
pushd mpi4py-3.0.3
python setup.py build --mpi=gaea-gnu
python setup.py install