geozero

repository·main·Indexed 19 days ago

https://github.com/georust/geozero

A high-performance Rust library for zero-copy reading and writing of geospatial data formats, including GeoJSON, WKB, WKT, MVT, GDAL, FlatGeobuf, and GeoParquet. It provides a Processing API via the GeomProcessor trait to perform custom computations on geometries without intermediate representations, and includes a CLI tool for converting geospatial data between various formats.

Tokens
14.1K
Snippets
49
Records
65
Agent score
67%

What's inside geozero

  1. What is GeoZero?

    main

    GeoZero is a library designed for zero-copy reading and writing of geospatial data. It provides an API for accessing geospatial formats without requiring an intermediate representation, which improves performance and reduces memory overhead.

    GeoZero supports:

    • Geometry Types: OGC Simple Features, Circular arcs (SQL-MM Part 3), and TIN.
    • Dimensions: X, Y, Z, M, and T.
    • Core Abstractions: It uses traits to allow reading and converting to arbitrary formats or rendering geometries directly.
  2. Use the Processing API to perform custom geometry operations

    main

    The Processing API allows you to perform custom computations on geometries by implementing the GeomProcessor trait. Instead of converting a geometry into a full intermediate data structure, you can process coordinates, vertices, and geometry types on-the-fly. This is highly efficient for tasks like counting vertices, finding extrema (e.g., max height), rendering to a canvas, or building spatial indices.

    // Example: Implementing a vertex counter
    struct VertexCounter(u64);
    
    impl GeomProcessor for VertexCounter {
        fn xy(&mut self, _x: f64, _y: f64, _idx: usize) -> Result<()> {
            self.0 += 1;
            Ok(())
        }
    }
    
    let mut vertex_counter = VertexCounter(0);
    geometry.process(&mut vertex_counter, GeometryType::MultiPolygon)?;
  3. Use the geozero CLI for format conversion

    main

    The geozero CLI allows you to convert geospatial data between different formats. The basic usage pattern is to provide an input file (or URL) followed by an output file/format.

    Supported operations include converting local files (like GeoJSON to FlatGeobuf) or fetching data from a URL and converting it to a different format (like FlatGeobuf to SVG).

    # Convert GeoJSON to FlatGeobuf
    geozero cities.geojson cities.fgb
    
    # Convert FlatGeobuf to JSON with a spatial extent filter
    geozero --extent 8.8,47.2,9.5,55.3 countries.fgb countries.json
    
    # Fetch FlatGeobuf from a URL and convert to SVG with a spatial extent filter
    geozero --extent 8.522086,47.363333,8.553521,47.376020 https://pkg.sourcepole.ch/osm-buildings-ch.fgb buildings.svg
  4. Run the GeoZero benchmarks

    main

    To run the performance benchmarks for the geozero-bench package, you must install cargo-criterion, prepare the local test data, set up a PostGIS database, and start the required web services via Docker.

    Note: The benchmarks use various configurations including Shapefile (GDAL), FlatGeobuf (Rust driver), GeoPackage (SQLx/GDAL), GeoJSON (GDAL/HTTP), and PostGIS (SQLx/rust-postgres/rust-postgis).

    # Install cargo-criterion
    cargo install cargo-criterion
    
    # Prepare data
    cd tests/data
    make
    
    # Create PostGIS database
    make createdb
    make countries_table osm_buildings_table
    
    # Start web server
    cd ../..
    docker-compose up -d
    
    # Run benchmark
    export DATABASE_URL=postgresql://$USER@localhost/geozerobench?sslmode=disable
    cargo criterion
  5. Use GeoZero with PostGIS and rust-postgres

    main

    To work with PostGIS geometries in rust-postgres, enable the with-postgis-postgres feature. You can use wkb::Decode to read geometries from rows and wkb::Encode to insert them.

    Note: This requires the with-postgis-postgres feature.

    let mut client = Client::connect(&std::env::var("DATABASE_URL").unwrap(), NoTls)?;
    
    let row = client.query_one(
        "SELECT 'SRID=4326;POLYGON ((0 0, 2 0, 2 2, 0 2, 0 0))'::geometry",
        &[],
    )?;
    
    let value: wkb::Decode<geo_types::Geometry<f64>> = row.get(0);
    if let Some(geo_types::Geometry::Polygon(poly)) = value.geometry {
        assert_eq!(
            *poly.exterior(),
            vec![(0.0, 0.0), (2.0, 0.0), (2.0, 2.0), (0.0, 2.0), (0.0, 0.0)].into()
        );
    }
    
    // Insert geometry
    let geom: geo_types::Geometry<f64> = geo::Point::new(1.0, 3.0).into();
    let _ = client.execute(
        "INSERT INTO point2d (datetimefield,geom) VALUES(now(),ST_SetSRID($1,4326))",
        &[&wkb::Encode(geom)],
    );
  6. Install system dependencies for Ubuntu/Debian/Mint

    main

    If you are building or using GeoZero on Ubuntu, Debian, or Mint, you may need the following system libraries installed:

    apt-get install -y libgeos-dev libgdal-dev
    apt-get install -y libgeos-dev libgdal-dev
  7. Use GeoZero with PostGIS and SQLx

    main

    To work with PostGIS geometries in SQLx, enable the with-postgis-sqlx feature. You can use wkb::Decode for selecting and wkb::Encode for binding parameters in queries.

    Note: This requires the with-postgis-sqlx feature.

    let pool = PgPoolOptions::new()
        .max_connections(5)
        .connect(&env::var("DATABASE_URL").unwrap())
        .await?;
    
    let row: (wkb::Decode<geo_types::Geometry<f64>>,) =
        sqlx::query_as("SELECT 'SRID=4326;POLYGON ((0 0, 2 0, 2 2, 0 2, 0 0))'::geometry")
            .fetch_one(&pool)
            .await?;
    let value = row.0;
    if let Some(geo_types::Geometry::Polygon(poly)) = value.geometry {
        assert_eq!(
            *poly.exterior(),
            vec![(0.0, 0.0), (2.0, 0.0), (2.0, 2.0), (0.0, 2.0), (0.0, 0.0)].into()
        );
    }
    
    // Insert geometry
    let geom: geo_types::Geometry<f64> = geo::Point::new(10.0, 20.0).into();
    let _ = sqlx::query(
        "INSERT INTO point2d (datetimefield,geom) VALUES(now(),ST_SetSRID($1,4326))",
    )
    .bind(wkb::Encode(geom))
    .execute(&pool)
    .await?;
  8. Supported WKB Dialects

    main

    WkbWriter supports several WKB dialects, each with specific header requirements and metadata handling:

    • WkbDialect::Wkb: Standard OGC WKB.
    • WkbDialect::Ewkb: Extended WKB (used by PostGIS), supporting SRID and Z/M dimensions via bit flags.
    • WkbDialect::Geopackage: GeoPackage binary format, which includes a magic header (GP), versioning, flags, SRID, and an optional envelope.
    • WkbDialect::MySQL: MySQL-specific WKB format, which includes the SRID in the header.
    • WkbDialect::SpatiaLite: SpatiaLite WKB format, which includes specific header bytes and envelope information.
  9. How feature and property processing works

    main

    GeoZero uses a visitor-style pattern for processing geospatial data. Instead of loading entire datasets into memory, you provide a Processor (implementing FeatureProcessor, GeomProcessor, or PropertyProcessor) that the library calls as it iterates through the data.

    There are three main levels of abstraction:

    1. GeozeroDatasource: Represents a collection of features (e.g., a file or a database table). You call .process(processor) to iterate over all features.
    2. FeatureAccess: Represents a single feature. It combines geometry and properties. You can use .process(processor, idx) to process a specific feature's geometry and attributes.
    3. FeatureProperties: Provides methods to access the attributes of a feature.

    For high-performance, zero-copy access, implement the process_properties method in your PropertyProcessor. For convenience, the API provides helper methods like .property(name) and .properties() which use internal readers.

  10. Use the geozero CLI for geospatial data conversion

    main

    The geozero-cli tool allows you to convert geospatial data between various formats such as CSV, GeoJSON, FlatGeobuf (FGB), Parquet/GeoParquet, SVG, WKT, and more. The tool determines the input format based on the file extension of the input path and the output format based on the extension of the dest path.

    Supported Input Formats

    • CSV: Requires specifying the geometry column via --csv-geometry-column.
    • GeoJSON / JSON: Standard GeoJSON files.
    • GeoJSONL / JSONL: Line-delimited GeoJSON.
    • GeoParquet / Parquet: Uses GeoParquet metadata for spatial filtering.
    • FlatGeobuf (FGB): Supports local files and remote URLs (via HTTP/HTTPS).
    • WKT: Well-Known Text files.

    Supported Output Formats

    • CSV
    • GeoJSON / JSON
    • FlatGeobuf (FGB)
    • SVG: Generates an SVG visualization. If no extent is provided, the tool first computes the bounds of the input data.
    • WKT
  11. Generate Mapbox Vector Tile (MVT) layers with MvtWriter

    main

    The MvtWriter is a generator used to encode geometries and properties into MVT features and layers. It implements geozero processor traits, allowing you to stream data from a datasource (like GeoJSON) directly into an MVT layer.

    To use MvtWriter, you can either:

    1. Scale coordinates: Use MvtWriter::new(extent, left, bottom, right, top) to automatically transform map-space coordinates into integer tile-space coordinates based on the provided bounding box and extent.
    2. Use unscaled coordinates: Use MvtWriter::new_unscaled(extent) if your input data is already in the desired integer tile coordinate space.

    After processing your datasource, call .layer(name) to retrieve the finished MvtLayer. This layer can then be added to an MvtTile for final encoding.

    use geozero::{GeozeroDatasource, geojson::GeoJsonString, mvt::MvtWriter};
    
    let mut geojson = GeoJsonString(
        serde_json::json!({
            "type": "FeatureCollection",
            "features": [
                {
                    "type": "Feature",
                    "properties": {
                        "population": 100
                    },
                    "geometry": {
                        "type": "Point",
                        "coordinates": [1.0, 2.0]
                    }
                }
            ]
        })
        .to_string(),
    );
    
    // Create a writer with a 4096 extent (unscaled)
    let mut mvt_writer = MvtWriter::new_unscaled(4096).unwrap();
    
    // Process the datasource
    geojson.process(&mut mvt_writer).unwrap();
    
    // Obtain the finished MVT layer
    let mvt_layer = mvt_writer.layer("sample");
  12. SQLx Compile-time Verification with GeoZero

    main

    When using sqlx::query! macros, you can use type overrides to handle GeoZero types. This allows you to maintain compile-time type safety for geometry columns.

    Note: This requires the with-postgis-sqlx feature.

    let _ = sqlx::query!(
        "INSERT INTO point2d (datetimefield, geom) VALUES(now(), $1::geometry)",
        wkb::Encode(geom) as _
    )
    .execute(&pool)
    .await?;
    
    struct PointRec {
        pub geom: wkb::Decode<geo_types::Geometry<f64>>,
        pub datetimefield: Option<OffsetDateTime>,
    }
    let rec = sqlx::query_as!(
        PointRec,
        r#"SELECT datetimefield, geom as "geom!: _" FROM point2d"#
    )
    .fetch_one(&pool)
    .await?;
    assert_eq!(
        rec.geom.geometry.unwrap(),
        geo::Point::new(10.0, 20.0).into()
    );