earthkit

earthkit is an open-source Python ecosystem led by ECMWF designed to accelerate weather and climate science workflows. It provides tools for simplifying various stages of the scientific pipeline, including:

• Data Access: Simplifying how users retrieve meteorological and climate data.
• Processing & Analysis: Tools for manipulating and analyzing scientific datasets.
• Visualisation: Capabilities for rendering and presenting scientific data.

For an interactive overview of the ecosystem, visit the earthkit website.

The earthkit ecosystem is composed of several specialized components designed for geospatial, meteorological, and hydrological workflows. Depending on your task, you may need one or more of the following packages:

• earthkit-data: A format-agnostic Python interface for geospatial data, specifically optimized for meteorology and climate science.
• earthkit-geo: Provides tools for geospatial computations.
• earthkit-hydro: Provides operations specifically for river networks.
• earthkit-meteo: Provides specialized meteorological computations.
• earthkit-plots: Provides visualization tools and templates.
• earthkit-transforms: Provides tools for the aggregation and transformation of geospatial data.

earthkit is a collection of multiple interoperable software components designed for earth sciences. It is built on top of established open-source Python libraries such as numpy, pandas, and matplotlib.

Key characteristics include:

• Interoperability: Components are designed to work together seamlessly.
• Operational Readiness: It leverages the robust software stack used in ECMWF production systems, including ecCodes and FDB.
• High-level Interface: Provides common tools to support diverse activities across research and operations.
• Scalability: Designed to be efficient for researchers while facilitating an easier transition to operational environments.

To install the full earthkit ecosystem, use pip with Python 3.10 or higher. Note that installing the earthkit package will install all available components by default. If you only require specific components, you can install them individually instead.

python3 -m pip install earthkit

Tests decorated with pytest.mark.no_cache_init require the pytest-forked plugin to run in a separate process. Use the --forked flag and the -m no_cache_init marker to execute them.

pytest --forked -v -m no_cache_init

Earthkit uses pre-commit hooks to perform quality control checks on every commit. If a check fails, the commit will be rejected. To install these hooks, run the following commands from the root of the repository:

pip install pre-commit
pre-commit install

To build the earthkit documentation locally, you must first install the necessary Python dependencies and then use make to generate the HTML files.

1. Navigate to the docs directory.
2. Install dependencies using pip install -r requirements.txt.
3. Run make html to build the site.
4. Open docs/_build/html/index.html in your browser to view the generated documentation.

You can install earthkit using conda from the conda-forge channel. This method is recommended if you want conda to manage the necessary binary dependencies for you.

conda install earthkit -c conda-forge

You can install the core earthkit package from PyPI using pip.

pip install earthkit

Notebook examples located in docs/examples are automatically included in the test suite. Some notebooks are skipped (e.g., those using remote data sources) based on a SKIP list defined in tests/documentation/test_notebooks.py. You can modify this list to change which notebooks are tested.

To run only the notebook tests:

pytest -v -m notebook