pygrib

repository·master·Indexed 43 days ago

https://github.com/jswhit/pygrib

A high-level Python interface for the ECMWF ECCODES C library used for reading and interacting with GRIB files. It provides capabilities for extracting data values and coordinates as numpy arrays, filtering messages by criteria, and modifying existing GRIB files. The package also includes CLI utilities such as cnvgrib1to2, cnvgrib2to1, grib_list, and grib_repack for format conversion and packing scheme management.

Tokens
2.8K
Snippets
17
Records
20
Agent score
51%

What's inside pygrib

  1. Overview of pygrib capabilities

    master

    pygrib provides a high-level Python interface to the ECMWF ECCODES C library for reading GRIB files.

    Key Capabilities:

    • Reading: Full support for reading GRIB files.
    • Writing: Capabilities are limited. You can modify the contents of an existing GRIB file, but you cannot create a new GRIB file from scratch.
  2. Open and iterate through GRIB files

    master

    Use pygrib.open() to create a GRIB message iterator. The returned object behaves like a Python file object, but offsets are measured in GRIB messages rather than bytes. You can use seek(), tell(), read(), and close() to navigate the file.

    • read(N): Returns a list containing the next N messages.
    • message(N): Returns the N-th message (1-indexed).
    • seek(N): Moves the pointer to the N-th message.
    • tell(): Returns the current message index.
    import pygrib
    
    # Open a GRIB file
    grbs = pygrib.open('sampledata/flux.grb')
    
    # Read the first message
    grb = grbs.read(1)[0]
    
    # Get the 2nd message directly
    grb = grbs.message(2)
    
    # Iterate through all messages
    for grb in grbs:
        print(grb)
    
    grbs.close()
  3. Install pygrib from source

    master

    To install from the GitHub repository, clone the repo and run pip install -e ..

    Requirement: You must set the $ECCODES_DIR environment variable to the directory containing the ECCODES C library headers and libraries (specifically include/grib_api.h and lib/libeccodes.so).

    If $ECCODES_DIR is not set, the installer will search common locations: $CONDA_PREFIX, /usr, /usr/local, and /opt/local.

    # After setting $ECCODES_DIR
    pip install -e .
  4. Install pygrib via Conda

    master

    If you are using Anaconda (recommended for Windows users), install pygrib from the conda-forge channel.

    Windows Troubleshooting: If you encounter an error stating that boot.def cannot be found, you must set the ECCODES_DEFINITION_PATH environment variable to: $CONDA_PREFIX/Library/share/eccodes/definitions.

    conda install -c conda-forge pygrib
  5. Install pygrib via pip or conda

    master

    You can install pygrib using standard Python package managers. Using pip is the easiest method, but conda is also supported via the conda-forge channel.

    # Using pip
    pip install pygrib
    
    # Using conda
    conda install -c conda-forge pygrib
  6. Modify and save GRIB messages

    master

    You can modify metadata keys in a GRIB message and then write the resulting binary string to a new file.

    1. Update keys using dictionary-style assignment (e.g., grb['key'] = value).
    2. Convert the message to a binary string using grb.tostring().
    3. Write the string to a new file using standard Python file I/O.
    import pygrib
    
    grbs = pygrib.open('sampledata/flux.grb')
    grb = grbs.read(1)[0]
    
    # Modify keys
    grb['forecastTime'] = 240
    grb.dataDate = 20100101
    
    # Export to binary string
    msg = grb.tostring()
    grbs.close()
    
    # Write to a new file
    with open('test.grb', 'wb') as grbout:
        grbout.write(msg)
  7. Verify installation and run tests

    master

    After installation, verify that the package is working by running the simple test script:

    cd test
    python test.py

    To run the full test suite, you must install pytest, matplotlib, pytest-mpl, and cartopy. Use the following command to execute the tests:

    export MPLBACKEND=agg
    cd test
    pytest test*py --mpl --mpl-baseline-path=baseline_images

    If you do not wish to install the heavy dependencies (matplotlib, pytest-mpl, cartopy), you can still run test.py and test_latlons.py.

  8. Build pygrib from source

    master

    To build from source, clone the repository and use pip install -e .. You must provide the ECCODES_DIR environment variable, which points to the directory containing include/grib_api.h and lib/libeccodes.so.

    If ECCODES_DIR is not specified, the installer will search common locations such as $CONDA_PREFIX, /usr, /usr/local, and /opt/local.

    git clone https://github.com/jswhit/pygrib
    cd pygrib
    ECCODES_DIR=path/to/eccodes pip install -e .
  9. Use the grib_repack CLI to change GRIB2 packing schemes

    master

    The grib_repack command (provided by the pygrib package) allows you to change the packing scheme of a GRIB2 file. It reads an input GRIB2 file, repacks the data using a specified scheme, and writes the result to a new output file.

    Usage: grib_repack -pN grib2filein grib2fileout

    Where:

    • -pN is the desired packing type (see Packing Type Reference).
    • grib2filein is the source GRIB2 file.
    • grib2fileout is the destination file for the repacked data.
    grib_repack -p3 grib2filein grib2fileout
  10. Convert a GRIB2 file to GRIB1 format using cnvgrib2to1

    master

    The cnvgrib2to1 command is a utility provided by the pygrib package used to convert meteorological data from the GRIB2 format to the GRIB1 format.

    Usage

    Run the command from your terminal by providing the source GRIB2 filename and the desired destination GRIB1 filename:

    cnvgrib2to1 <grib2filename> <grib1filename>

    Exit Status

    • 0: Success, or the program exited because the wrong number of command-line arguments was provided.
    • 1: The program failed during the conversion process.
    cnvgrib2to1 <grib2filename> <grib1filename>
  11. Open GRIB data from an in-memory buffer

    master

    Instead of an on-disk file, pygrib.open() can accept an io.BufferedReader instance. This is useful for processing GRIB data directly from network responses (e.g., using requests) without saving to disk first.

    import pygrib
    import requests
    import io
    
    fileurl = "https://example.com/data.grb"
    response = requests.get(fileurl)
    
    # Wrap the bytes in a BufferedReader
    buffer_io = io.BufferedReader(io.BytesIO(response.content))
    
    # Open via pygrib
    grbs = pygrib.open(buffer_io)
    grb = grbs.select(shortName='gh', level=500)[0]
    print(grb)
    grbs.close()