DOCSEARCHbyAlgolia
Submit Docs

Apache Sedona Documentation

repository·master·Indexed 25 days ago

https://github.com/apache/sedona

A high-performance engine for large-scale geospatial analytics providing spatial extensions for distributed computing frameworks like Spark and Flink. It enables efficient processing of massive spatial datasets using SQL and programmatic APIs. The project includes Python and R wrappers, as well as specialized subprojects such as SedonaDB (a single-node analytical database) and SpatialBench (a geospatial SQL analytics benchmark suite).

Tokens
339.1K
Snippets
1.1K
Records
1.6K
Agent score
79%

What's inside Apache Sedona

  1. Overview of SedonaSQL operators in Flink

    master

    SedonaSQL supports the SQL/MM Part3 Spatial SQL Standard within Apache Flink. It provides four main categories of spatial operators to manipulate and query geometric data:

    1. Constructors: Used to create a Geometry object from raw input like WKT strings or coordinates.
    2. Functions: Used to perform spatial computations on one or more geometry columns (e.g., calculating distance).
    3. Aggregators: Used to perform spatial reductions on a column of geometries to return a single aggregated value (e.g., calculating a bounding envelope).
    4. Predicates: Used for spatial logic and filtering, returning a boolean value (e.g., checking if one geometry contains another).

  2. What is SpatialBench?

    master
    SpatialBench is a benchmarking framework specifically designed for spatial workflows, including spatial joins, distance queries, and point-in-polygon analyses. Unlike traditional benchmarking frameworks that focus on tabular data, SpatialBench targets the unique performance requirements of spatial queries. It currently comprises 12 queries designed to be unbiased and representative of real-world spatial data analysis tasks.

  3. Overview of SedonaDB

    master

    SedonaDB is a single-node spatial analytics engine written in Rust, built on Apache Arrow and DataFusion. It is designed for fast, interactive analytics and developer workflows with a lightweight, embedded-style deployment model.

    Key features include:

    • Native geometry and geography types
    • Built-in spatial indexing
    • Optimized spatial joins and nearest-neighbor queries
    • Support for GDAL/OGR compatible formats
    • GeoParquet 1.1 write support with bounding box metadata
    • Python and SQL interfaces

  4. What is Apache Sedona?

    master
    Apache Sedona is a cluster computing system designed for processing large-scale spatial data. It extends existing distributed computing engines—specifically Apache Spark, Apache Flink, and Snowflake—by providing distributed Spatial Datasets and Spatial SQL. This allows for efficient loading, processing, and analysis of massive spatial datasets across a cluster of machines.

  5. What is SedonaDB?

    master

    SedonaDB is a lightweight, single-node analytical database engine written in Rust that treats spatial data as a first-class citizen. It is a subproject of Apache Sedona designed for small-to-medium data analytics where the overhead of distributed systems (like Spark or Flink) is undesirable.

    Key Features:

    • Spatial Native: Full support for spatial types, joins, CRS (coordinate reference systems), and spatial functions.
    • Performance: Built on Apache Arrow and Apache DataFusion, providing a modern, vectorized query engine with built-in query optimizations, indexing, and data pruning.
    • Interoperability: Integrates with GeoArrow, GeoParquet, and GeoPandas.
    • Interfaces: Provides Pythonic and SQL interfaces, with additional APIs for R and Rust.

  6. Use Spatial Indexing functions in Apache Sedona for Flink

    master
    Apache Sedona provides a suite of spatial indexing functions for Flink that allow you to work with various indexing systems, including Bing Tiles, H3, S2, and GeoHash. These functions enable you to convert geometries to index identifiers (like cell IDs or quadkeys), find neighboring cells, or convert index identifiers back into geometric shapes (Polygons).

  7. Capabilities of Apache Sedona

    master

    Apache Sedona is a distributed computing engine for processing large-scale spatial data. It provides support for distributed spatial datasets, complex spatial objects, distributed spatial queries, and rich spatial analytics tools across multiple processing engines.

    Supported Processing Engines

    • Apache Spark: Supports Spatial RDD and Spatial DataFrame/SQL.
    • Apache Flink: Supports Spatial DataStream and Spatial Table/SQL.
    • Snowflake: Supports Spatial SQL.

    Supported Spatial Objects and Formats

    • Vector: Geometries and trajectories.
    • Raster: Raster images with Map Algebra support.
    • Input Formats: CSV, TSV, WKT, WKB, GeoJSON, Shapefile, GeoTIFF, ArcGrid, and NetCDF/HDF.

    Spatial Query and Indexing Support

    • Queries: Range queries, range join queries, distance join queries, and K-Nearest Neighbor (KNN) queries.
    • Indexing: R-Tree and Quad-Tree spatial indices.

    Analytics and Integration

    • Transformations: Coordinate Reference System (CRS) and Spatial Reference System (SRS) transformations.
    • Language Support: Scala, Java, Python, and R.
    • Python Ecosystem: Integration with Jupyter notebook, GeoPandas, and Shapely.
    • Visualization: Integration with Apache Zeppelin, KeplerGL, and DeckGL.

  8. Use the Apache Sedona Python wrapper

    master
    The apache-sedona Python package is a wrapper for Apache Sedona, a cluster computing system designed for processing large-scale spatial data. It extends Apache Spark and SparkSQL with Spatial Resilient Distributed Datasets (SRDDs) and SpatialSQL, enabling efficient loading, processing, and analysis of spatial data across a cluster.

  9. Available Apache Sedona modules and APIs

    master

    Apache Sedona provides several modules depending on your execution environment and preferred programming interface:

    • spark: Provides RDD, SQL, and DataFrame support using SpatialRDD and spatial DataFrames.
    • spark-shaded: A shaded version of the Spark module to avoid dependency conflicts.
    • python: Provides Python interfaces for SpatialRDD and spatial DataFrames.
    • Zeppelin: A plugin designed for Apache Zeppelin version 0.8.1 and above.

  10. Evaluate geospatial performance with Apache Sedona SpatialBench

    master
    Apache Sedona SpatialBench is a benchmarking mechanism designed to assess the performance of geospatial SQL analytics queries across different database systems. It allows for comparing SedonaDB against other tools like GeoPandas and DuckDB Spatial across various scale factors (e.g., SF1, SF10) and query types, including complex spatial joins and KNN queries.

  12. Available Apache Sedona modules

    master

    Apache Sedona provides several modules depending on your execution environment and preferred programming interface:

    • spark: Provides RDD, SQL, and DataFrame APIs via SpatialRDD and Spatial DataFrame.
    • spark-shaded: A shaded version of the Spark module.
    • python: Provides a Python interface for SpatialRDD and Spatial DataFrame.
    • Zeppelin: A plugin designed for Apache Zeppelin 0.8.1 and above.