godal

repository·main·Indexed 41 days ago

https://github.com/airbusgeo/godal

Idiomatic Go bindings for the GDAL library designed to reduce Cgo overhead. It provides a Go-friendly error handling pattern and includes a VSI GDAL GCS plugin for accessing Google Cloud Storage files via the gs:// prefix. Key features include raster and vector dataset management, driver registration, and support for GDAL operations like Translate and Warp.

Tokens
14.2K
Snippets
52
Records
73
Agent score
49%

What's inside godal

  1. How godal works and its API design

    main

    Godal is an idiomatic Go wrapper around the GDAL library designed to minimize the overhead of Cgo calls. It follows these design principles:

    1. Idiomatic Error Handling: Function calls return a result and an error. The result is only valid if err == nil. The error message contains the root cause from GDAL.
    2. Reduced Cgo Overhead: Instead of exposing every individual GDAL C function, Godal groups frequently used calls into single Cgo functions. This allows you to retrieve complex structures (like dataset dimensions or band information) in a single call rather than multiple individual calls.
    3. Option Pattern: Unfrequently used or non-default parameters are passed using an options pattern (e.g., godal.Update()).
    4. VSI Support: Godal exposes a VSI handler that allows you to treat an io.ReaderAt as a filename that GDAL can open. This is useful for integrating cloud storage (like gs:// URIs via osio) directly into GDAL workflows.
    // Example of the idiomatic grouping pattern
    hDS, err := godal.Open(filename)
    if err != nil {
        panic(err)
    }
    
    // Instead of multiple C calls, use Structure() to get metadata at once
    structure := hDS.Structure()
    fmt.Printf("dataset size: %dx%dx%d\n", structure.SizeX, structure.SizeY, structure.NBands)
    
    for _, band := range hDS.Bands() {
        for o, ovr := range band.Overviews() {
            bstruct := ovr.Structure()
            fmt.Printf("overview %d size: %dx%d\n", o, bstruct.SizeX, bstruct.SizeY)
        }
    }
  2. Use `godal.RegisterGCSHandler()` instead of the VSI plugin

    main

    If you only need to access gs:// files from within a Go program using the godal package, it is better to use godal.RegisterGCSHandler() directly. This allows for better fine-tuning of the handler's configuration.

    Important: You must call godal.RegisterGCSHandler() before calling godal.RegisterAll() to prevent the VSI plugin from intercepting and registering the gs:// prefix.

  3. Authenticate with Google Cloud Storage for GDAL

    main

    Authentication is managed by the cloud.google.com/go/storage SDK.

    • On GCP compute instances: Authentication is transparent and requires no extra steps.
    • Outside of GCP: You must set up application default credentials by running:
    gcloud auth application-default login
  4. Install the VSI GDAL GCS plugin

    main

    The VSI GDAL plugin allows GDAL-linked programs (such as gdalinfo, gdal_translate, or qgis) to access files via the gs:// prefix using the Google Cloud Storage SDK.

    To install, build the plugin and copy the resulting shared object to your GDAL plugin directory:

    1. Build and install using make:
    make && sudo make install
    1. If automatic installation fails, manually copy gdal_gcs.so to $(GDAL_INSTALL_PREFIX)/lib/gdalplugins/.

    Once installed, any program using GDAL can open files like gs://bucket/path/to/cog.tif directly.

  5. Install godal

    main

    Godal requires GDAL version 3.0 or greater. You must ensure that the GDAL headers are installed on your system for the Go compiler to build the bindings.

    If your GDAL installation is in a non-standard location, you must set the PKG_CONFIG_PATH environment variable so the compiler can find the necessary configuration files.

    # Example for a non-standard GDAL installation
    export PKG_CONFIG_PATH=/opt/include/pkgconfig
  6. Manage raster band properties with Band

    main

    The Band type provides methods to interact with individual raster bands in a dataset. Key capabilities include:

    • Structure: Use Structure() to get dimensions (SizeX, SizeY), block sizes, scale, offset, and DataType.
    • NoData: Use NoData() to retrieve the nodata value, SetNoData() to set it, and ClearNoData() to remove it.
    • Scale/Offset: Use SetScaleOffset() and ClearScaleOffset() to manage data scaling.
    • Color: Use ColorInterp() and SetColorInterp() to manage color interpretation, and ColorTable()/SetColorTable() to manage color tables.
    • Masking: Use MaskFlags() to check mask flags, MaskBand() to get the existing mask band, or CreateMask() to generate a new one.
    • I/O: Use Read() and Write() (convenience wrappers for IO()) to move pixel data between the band and a Go buffer.
    • Statistics: Use GetStatistics() to retrieve cached values, ComputeStatistics() to calculate them, or SetStatistics() to manually set them.
    • Advanced Operations: Use Polygonize() to convert raster data to vector, FillNoData() to fill nodata areas, SieveFilter() to remove small features, and Histogram() to compute pixel distributions.
    // Example: Reading data from a band
    var buffer []float32 = make([]float32, 100*100)
    err := band.Read(0, 0, buffer, 100, 100)
    
    // Example: Getting band structure
    struct := band.Structure()
    fmt.Printf("Size: %dx%d\n", struct.SizeX, struct.SizeY)
  7. Manage dataset properties and operations with Dataset

    main

    The Dataset type is the primary entry point for interacting with raster datasets. Key capabilities include:

    • Bands: Use Bands() to retrieve all raster bands in the dataset.
    • Spatial Metadata: Use Bounds() to get the bounding box (optionally reprojected), Projection() for WKT, SpatialRef() for the spatial reference object, and GeoTransform()/SetGeoTransform() for affine transformations.
    • Dataset-wide Metadata: Use SetNoData() and SetScaleOffset() to apply settings to the entire dataset.
    • Transformations:
      • Translate(): Runs the equivalent of gdal_translate to convert formats or apply switches.
      • Warp(): Runs the equivalent of gdalwarp to reproject or warp a single dataset.
      • WarpInto(): Warps multiple source datasets into the existing dataset.
    • Overviews: Use BuildOverviews() to compute pyramid levels and ClearOverviews() to delete them.
    • I/O: Use Read(), Write(), and IO() to perform multi-band or single-band pixel operations.
    // Example: Getting dataset bounds
    bounds, err := ds.Bounds()
    
    // Example: Running gdal_translate via Translate
    newDS, err := ds.Translate("output.tif", []string{"-a_srs", "epsg:4326"}, godal.CreationOption("TILED=YES"))
  8. Override default error handling with ErrorHandler

    main

    By default, godal treats all GDAL messages with a severity of CE_Warning or higher as errors. You can override this behavior by providing a custom ErrorHandler function as an option to most godal functions.

    An ErrorHandler is a function with the following signature: type ErrorHandler func(ec ErrorCategory, code int, msg string) error

    Behavior Rules:

    • If the ErrorHandler returns nil, the parent godal function will not return an error. It is the responsibility of your handler to log the message if you want it recorded.
    • If the ErrorHandler returns an error, that specific error will be returned to the caller of the godal function.

    This allows you to filter out specific warnings or transform GDAL messages into custom Go error types.

    // Example: A handler that only returns an error if the message contains "critical"
    myHandler := func(ec ErrorCategory, code int, msg string) error {
        if strings.Contains(msg, "critical") {
            return fmt.Errorf("GDAL critical error: %s", msg)
        }
        // Return nil to ignore the message and prevent the godal function from failing
        return nil
    }
    
    // Use it with a godal function (e.g., Open)
    // Note: The exact function call depends on the specific godal API
    dataset, err := gdal.Open("path/to/file.tif", myHandler)
  9. Configure the GCS plugin internal cache

    main

    You can tune the internal cache performance of the GCS plugin using environment variables before starting your program. By default, the plugin caches 1000 blocks of 1Mb each.

    Use the following environment variables:

    • GODAL_BLOCKSIZE: Sets the size of each block. Can be a number of bytes or a value suffixed with Kb or Mb.
    • GODAL_NUMBLOCKS: Sets the number of blocks to cache.
  10. Register GDAL drivers

    main

    To use GDAL drivers, you must register them. You can register all available drivers, a specific subset, or individual drivers.

    • RegisterAll(): Registers all available raster and vector drivers.
    • RegisterInternalDrivers(): A shorthand for registering essential drivers (VRT, MEM, GTiff for raster; MEM, VRT, GeoJSON for vector). Note: registration order for VRT/GTiff may vary based on GDAL version.
    • RegisterRaster(drivers ...DriverName): Registers specific raster drivers.
    • RegisterVector(drivers ...DriverName): Registers specific vector drivers.
    • RegisterPlugin(name string, opts ...RegisterPluginOption): Registers a GDAL plugin.

    If a driver is not explicitly defined in the Go wrapper, you can pass a DriverName that matches the C function name GDALRegister_XXX (for raster) or RegisterOGRXXX (for vector) to attempt registration via the underlying binary.

    // Register everything
    godal.RegisterAll()
    
    // Or register essential drivers
    err := godal.RegisterInternalDrivers()
    
    // Or specific drivers
    err := godal.RegisterRaster(godal.GTiff, godal.VRT)
  11. Open a dataset with godal.Open()

    main

    Use godal.Open(filename, ...options) to open a dataset. By default, it opens in read-only mode. To open a dataset with write access, pass the godal.Update() option.

    // Read-only
    ds, err := godal.Open(filename)
    
    // Read-write
    ds, err := godal.Open(filename, godal.Update())
  12. Caveats and Limitations of the VSI GDAL GCS plugin

    main
    • OS Compatibility: This plugin does not work with musl and cannot be used on Alpine-based Linux distributions.
    • Performance: While any file format can be accessed, I/O performance is only reasonable for cloud-optimized formats, such as Cloud Optimized GeoTIFFs (COGs).