DuckDB Spatial Extension

repository·v1.5-variegata·Indexed 20 days ago

https://github.com/duckdb/duckdb-spatial

A spatial extension for DuckDB providing support for geospatial data formats like ESRI Shapefiles and Parquet. It includes a comprehensive set of scalar, aggregate, macro, and table functions for geometric operations, including coordinate reference system transformations via ST_Transform, spatial joins using ST_Within, and data export to GeoJSONSeq using GDAL.

Tokens
25.6K
Snippets
79
Records
108
Agent score
71%

What's inside duckdb-spatial

  1. Coordinate Transformations via Embedded PROJ

    v1.5-variegata
    The extension includes an embedded version of the PROJ library. PROJ is used for coordinate transformations between different Coordinate Reference Systems (CRS). Because PROJ is embedded within the extension binary, users do not need to install the PROJ library separately on their system, and this functionality is compatible with WASM environments.
  2. Use the WKB_BLOB type for backward compatibility

    v1.5-variegata

    The WKB_BLOB type is used to represent geometries known to be in Well-Known Binary (WKB) format.

    Note: This type is deprecated. It is maintained solely for backward compatibility with previous versions of DuckDB Spatial. For new development, you should use the standard geometry types provided by the spatial extension instead of WKB_BLOB.

  3. Understand the Multi-tiered Geometry Type System

    v1.5-variegata

    The extension provides two distinct ways to handle spatial data, allowing you to choose between flexibility and performance:

    1. GEOMETRY Type (Flexible/Standard):

      • Supports standard subtypes: POINT, LINESTRING, POLYGON, MULTIPOINT, MULTILINESTRING, MULTIPOLYGON, and GEOMETRYCOLLECTION.
      • Internally represented as row-wise BLOBs using a format similar to PostGIS (double-aligned WKB).
      • Uses the GEOS library for spatial operations.
      • Best for columns containing mixed geometry types.
    2. Native 2D Types (High Performance):

      • Includes POINT_2D, LINESTRING_2D, POLYGON_2D, and BOX_2D.
      • Built on DuckDB STRUCT and LIST types, stored in a columnar fashion.
      • Optimized for OLAP workloads: leverages DuckDB's per-column statistics and efficient compression.
      • Allows spatial operations without de/serialization penalties (though function support is currently limited).
      • These types can be implicitly cast to GEOMETRY to use the full suite of spatial functions, though this incurs a de/serialization cost.

    Note on WKB: The extension includes a WKB_BLOB type, which is an alias for BLOB used to signify that the data contains valid WKB-encoded geometry.

  4. Set GDAL Layer Creation Options when exporting data

    v1.5-variegata

    When using the COPY command to export data via the GDAL driver, you can specify format-specific options using the LAYER_CREATION_OPTIONS parameter within the WITH clause. This is equivalent to using the -lco flag in the GDAL command-line interface.

    For example, when exporting to GeoJSON, you can enable options like WRITE_BBOX and RFC7946 to control the output structure.

    COPY (SELECT * from st_read('input.shp'))
    TO 'output.geojson'
    WITH (FORMAT GDAL, DRIVER 'GeoJSON', LAYER_CREATION_OPTIONS ('WRITE_BBOX=YES', 'RFC7946=YES'))
  5. Optimize spatial queries with PreparedGeosGeometry

    v1.5-variegata

    When performing multiple spatial predicate tests against the same geometry, use get_prepared() to create a PreparedGeosGeometry. This object pre-processes the geometry to significantly speed up subsequent predicate operations like contains, intersects, and within.

    Workflow

    1. Call geom.get_prepared() to create the prepared instance.
    2. Use the prepared instance to run multiple queries against different geometries.

    Supported Predicates

    PreparedGeosGeometry supports the same core predicates as GeosGeometry (e.g., contains, covers, crosses, disjoint, intersects, overlaps, touches, within) but at a higher performance level.

    // Assuming 'geom' is an existing GeosGeometry
    PreparedGeosGeometry prepared = geom.get_prepared();
    
    if (prepared.intersects(other_geom)) {
        // Perform fast intersection check
    }
  6. Check spatial relationships (Covers, CoveredBy, Crosses, Disjoint, Equals)

    v1.5-variegata

    Standard spatial predicate functions:

    • ST_CoveredBy: Returns true if geom1 is covered by geom2.
    • ST_Covers: Returns true if geom1 covers geom2.
    • ST_Crosses: Returns true if geom1 crosses geom2.
    • ST_Disjoint: Returns true if the geometries do not intersect.
    • ST_Equals: Returns true if the geometries are equal.
    BOOLEAN ST_CoveredBy (geom1 GEOMETRY, geom2 GEOMETRY)
    BOOLEAN ST_Covers (geom1 GEOMETRY, geom2 GEOMETRY)
    BOOLEAN ST_Crosses (geom1 GEOMETRY, geom2 GEOMETRY)
    BOOLEAN ST_Disjoint (geom1 GEOMETRY, geom2 GEOMETRY)
    BOOLEAN ST_Equals (geom1 GEOMETRY, geom2 GEOMETRY)
  7. Calculate perimeter on a spheroid

    v1.5-variegata

    Use ST_Perimeter_Spheroid to calculate the perimeter in meters using an ellipsoidal model of the Earth.

    Requirements:

    • Input must be in EPSG:4326 (WGS84) with [latitude, longitude] axis order.
    • Input must be a POLYGON, MULTIPOLYGON, or a GEOMETRYCOLLECTION containing polygons.

    Note: This is highly accurate but is the slowest perimeter calculation method.