Py-ART: Python ARM Radar Toolkit

repository·main·Indexed 20 days ago

https://github.com/arm-doe/pyart

An open-source Python module providing a collection of weather radar algorithms and utilities for examining, processing, and analyzing data from various precipitation and cloud radars. Built on the Scientific Python stack, it includes tools for reading NEXRAD archive data, visualizing radar products via RadarMapDisplay, and filtering data using GateFilter.

Tokens
10.6K
Snippets
33
Records
47
Agent score
69%

What's inside Py-ART

  1. Py-ART Core Capabilities

    main

    Py-ART is a collection of weather radar algorithms and utilities designed to examine, process, and analyze radar data.

    Key Features:

    • Data Ingestion: Reads common weather radar formats including Sigmet/IRIS, MDV, CF/Radial, UF, and NEXRAD Level II archive files.
    • Data Export: Writes radar data to NetCDF files conforming to the CF/Radial convention.
    • Visualization: Produces common radar plots such as Plan Position Indicator (PPI) and Range Height Indicator (RHI).
    • Corrections: Performs antenna coordinate corrections including reflectivity attenuation correction, velocity dealiasing, and specific (Kdp) and differential (PhiDP) phase corrections.
    • Gridding: Includes sophisticated mapping routines to create uniform Cartesian grids of radar fields from one or more radars. Grids can be plotted or saved to NetCDF.
  2. The Radar Object common data model

    main

    The core of Py-ART is the Radar object, which serves as a unified data model for weather radar data. This object abstracts the differences between various radar data formats (such as NEXRAD, ODIM, and UF) into a standardized interface.

    Key characteristics of the Radar object include:

    • Standardization: Provides a consistent way to access radar fields, coordinates, and metadata regardless of the original file format.
    • Interoperability: Designed to work seamlessly with the scientific Python ecosystem, including NumPy, SciPy, Matplotlib, and xarray.
    • Extensibility: Users can add custom fields, attributes, or processing steps directly to the Radar object.
  3. Comparing Py-ART with xradar to legacy radar data structures

    main

    Py-ART has evolved from legacy structures to a modern architecture, primarily through the adoption of xarray via the xradar extension. This shift moves the data representation from custom objects using dictionaries to a standardized, xarray-based model.

    | Feature | Legacy Radar Data Structures | Py-ART with xradar | | :--- | :--- | : | | Data Representation | Custom object with dictionaries | xarray-based, standardized | | Metadata Handling | Based on cfradial1 standards | Based on cfradial2 standards | | Performance | Limited scalability | Optimized for large datasets | | Multi-Dimensional Data | Limited support | Native support via xarray | | Interoperability | Minimal, package-by-package | Full integration with PyData ecosystem |

    Using Py-ART with xradar is recommended for handling large-scale or multi-dimensional datasets and for maintaining compatibility with modern data science workflows.

  4. Understand Py-ART versioning and compatibility

    main

    Py-ART follows semantic versioning.

    • Backward Compatibility: Any 1.x release is guaranteed to be backward compatible with an earlier 1.y release. This means code that is correct and works on 1.y will work on a future 1.x version.
    • Version Increments:
      • The x in 1.x.y is incremented when new features are released.
      • The y in 1.x.y is incremented for releases containing only bug fixes.
    • Breaking Changes: Backwards-incompatible changes are only permitted when moving to version 2.0. Such changes will be accompanied by FutureWarning where appropriate.
  5. Install Py-ART from source

    main

    If you need to install from source, you must first ensure the TRMM RSL library path is available. You can set this via the RSL_PATH environment variable. The path should point to the root TRMM RSL directory containing both lib and include subdirectories (default is /lib/local/trmm).

    If using CyLP, you must also set the COIN_INSTALL_DIR environment variable to the path of the coincbc directory.

    Global Installation

    $ export RSL_PATH=/path/to/rsl/
    $ python setup.py install

    User-level Installation

    $ export RSL_PATH=/path/to/rsl/
    $ python setup.py install --user

    Development Mode

    To install in editable mode using pip:

    $ pip install -e .

    In-place Compilation (No Installation)

    To use Py-ART without installing it, add the repository path to your PYTHONPATH and compile the extension in-place:

    $ python setup.py build_ext -i
  6. Set conda-forge as the priority channel

    main

    It is recommended to use the conda-forge channel for Py-ART, as it contains the most recent versions of required packages and helps avoid dependency conflicts by keeping packages within the same channel. You can set conda-forge as your priority channel or specify it during a single installation command.

    # Add conda-forge as the priority channel
    conda config --add channels conda-forge
    
    # OR, flag the channel for a specific installation
    conda install -c conda-forge numpy
  7. Create a Py-ART environment using the environment file

    main

    You can create a pre-configured conda environment using the official environment.yml file provided in the Py-ART repository. This creates an environment named pyart_env.

    # Create the environment from the file
    conda env create -f environment.yml
    
    # Activate the environment
    source activate pyart_env
    
    # Deactivate the environment when finished
    source deactivate pyart_env
  8. Install Py-ART using Conda

    main

    The recommended method for installing Py-ART is using conda with Python 3. It is highly recommended to create a new conda environment to avoid dependency conflicts.

    To create a new environment using the provided environment.yml file:

    conda env create -f environment.yml

    To create a basic environment and download optional dependencies as needed:

    conda create -n pyart_env -c conda-forge arm_pyart

    To install the latest version of Py-ART into your current environment:

    conda install -c conda-forge arm_pyart

    To update an existing installation of Py-ART to the latest release:

    conda update -c conda-forge arm_pyart
  9. Install Py-ART using an environment file

    main

    The most straightforward way to set up a Py-ART environment is to use the provided environment.yml file. This ensures all necessary dependencies are installed correctly.

    1. Download the environment.yml file from the repository.
    2. Create the environment using the following command:
      conda env create -f environment.yml
    3. Activate the environment named pyart_env:
      source activate pyart_env
    4. To stop using the environment, run:
      source deactivate pyart_env
    conda env create -f environment.yml
    source activate pyart_env
  10. Run Py-ART unit tests

    main

    To test the Py-ART installation, you must have pytest and open-radar-data installed. You can run the test suite from outside the source directory using the --pyargs flag.

    # Install testing dependencies
    conda install -c conda-forge pytest
    conda install -c conda-forge open-radar-data
    
    # Run the test suite
    pytest --pyargs pyart
  11. Install optional CyLP dependency

    main

    To use the CyLP package with Py-ART, you must first install its dependency coincbc and then set the COIN_INSTALL_DIR environment variable so the installer can locate it.

    1. Activate your pyart_env.
    2. Install coincbc from conda-forge:
      conda install -c conda-forge coincbc
    3. Export the path to your environment's directory to the COIN_INSTALL_DIR variable:
      export COIN_INSTALL_DIR=/path/to/your/anaconda3/envs/pyart_env
    4. Install the Python 3 compatible version of CyLP directly from GitHub using pip:
      pip install git+https://github.com/jjhelmus/CyLP.git@py3
    conda install -c conda-forge coincbc
    export COIN_INSTALL_DIR=/home/user/anaconda3/envs/pyart_env
    pip install git+https://github.com/jjhelmus/CyLP.git@py3