h3-duckdb

repository·main·Indexed 42 days ago

https://github.com/isaacbrodsky/h3-duckdb

A DuckDB extension that adds support for the H3 discrete global grid system, enabling users to index points and geometries to hexagons directly within SQL. It implements the complete H3 API, supporting UBIGINT, BIGINT, and VARCHAR data types for H3 indexes, and provides functions for coordinate conversion, hierarchy traversal, and WKT/WKB geometry generation.

Tokens
4.1K
Snippets
8
Records
10
Agent score
54%

What's inside h3-duckdb

  1. H3 function data types and support

    main

    All H3 functions in this extension support the following input and output types:

    • UBIGINT (unsigned 64-bit integer)
    • BIGINT (signed 64-bit integer)
    • VARCHAR (string representation)

    Note: The unsigned API is preferred and is the default return type when the extension cannot explicitly detect which type to use. The signed and unsigned APIs are functionally identical.

  2. Run the built DuckDB extension

    main

    After building, you can run the bundled DuckDB shell:

    ./build/release/duckdb -unsigned

    Note: You must run DuckDB with the -unsigned flag to use the H3 extension if you are running a local build. Once inside the shell, load the extension using the path to the .duckdb_extension file:

    load 'build/release/extension/h3/h3.duckdb_extension';
  3. Update DuckDB C API headers

    main

    The C API headers in the duckdb_capi/ directory should correspond to the TARGET_DUCKDB_VERSION specified in the project's Makefile. To automatically synchronize these headers with the target version, run the following command from the root directory:

    make update_duckdb_headers

    Manual Updates: You can update headers manually, but ensure they meet these versioning requirements:

    1. They must be from the same DuckDB version as TARGET_DUCKDB_VERSION.
    2. They must not be from a later release than TARGET_DUCKDB_VERSION.
    3. Using headers from an older version is technically allowed but not recommended.
  4. Build the H3 extension from source

    main

    To build the extension from source, you need Git, CMake, and a C compiler. It is recommended to use ninja for faster builds via parallelism.

    Build steps

    1. Update submodules:
    git submodule update --init
    1. Build release version:
    GEN=ninja make release

    Alternatively, using standard make with manual parallelism (replace 4 with your CPU core count):

    CMAKE_BUILD_PARALLEL_LEVEL=4 make duckdb_release release
    1. Run tests:
    make test
    1. Update submodules to latest upstream:
    make update_deps
    git submodule update --init
    GEN=ninja make release
  5. Use H3 functions in SQL

    main

    The extension provides functions to work with H3 cell IDs. You can use VARCHAR strings, BIGINT (signed), or UBIGINT (unsigned) for H3 indexes. Using the integer API (UBIGINT or BIGINT) generally provides better performance.

    Example using a string index:

    SELECT h3_cell_to_latlng('822d57fffffffff');

    Example using an integer index (preferred for performance):

    SELECT h3_cell_to_latlng(586265647244115967);
  6. Reference: Full list of H3 functions

    main

    The extension implements the complete H3 API. Below is the list of available functions:

    FunctionDescription
    h3_latlng_to_cellConvert latitude/longitude coordinate to cell ID
    h3_latlng_to_cell_stringConvert latitude/longitude coordinate to cell ID (returns VARCHAR)
    h3_cell_to_latConvert cell ID to latitude
    h3_cell_to_lngConvert cell ID to longitude
    h3_cell_to_latlngConvert cell ID to latitude/longitude
    h3_cell_to_boundary_wktConvert cell ID to cell boundary WKT
    h3_cell_to_boundary_wkbConvert cell ID to cell boundary WKB
    h3_get_resolutionGet resolution number of cell ID
    h3_get_base_cell_numberGet base cell number of cell ID
    h3_get_index_digitGet specified indexing digit of a cell
    h3_string_to_h3Convert VARCHAR cell ID to UBIGINT
    h3_h3_to_stringConvert BIGINT or UBIGINT cell ID to VARCHAR
    h3_is_valid_cellTrue if this is a valid cell ID
    h3_is_valid_indexTrue if this is a valid cell/edge/vertex ID
    h3_is_res_class_iiiTrue if the cell's resolution is class III
    h3_is_pentagonTrue if the cell is a pentagon
    h3_get_icosahedron_facesList of icosahedron face IDs the cell is on
    h3_construct_cellCreate cell index from component parts
    h3_construct_cell_stringCreate cell index string from component parts
    h3_cell_to_parentGet coarser cell for a cell
    h3_cell_to_childrenGet finer cells for a cell
    h3_cell_to_children_sizeNumber of finer cells for a cell
    h3_cell_to_center_childGet the center finer cell for a cell
    h3_cell_to_child_posGet a sub-indexing number for a cell inside a parent
    h3_child_pos_to_cellConvert parent and sub-indexing number to a cell ID
    h3_compact_cellsConvert a set of single-resolution cells to the minimal mixed-resolution set
    h3_uncompact_cellsConvert a mixed-resolution set to a single-resolution set of cells
    h3_grid_diskFind cells within a grid distance
    h3_grid_disk_distancesFind cells within a grid distance, sorted by distance
    h3_grid_disk_unsafeFind cells within a grid distance, with no pentagon distortion
    h3_grid_disk_distances_unsafeFind cells within a grid distance, sorted by distance, with no pentagon distortion
    h3_grid_disk_distances_safeFind cells within a grid distance, sorted by distance
    h3_grid_ringFind cells exactly a grid distance away
    h3_grid_ring_unsafeFind cells exactly a grid distance away, with no pentagon distortion
    h3_max_grid_disk_sizeMaximum number of cells for a grid disk for size K
    h3_grid_path_cellsFind a grid path to connect two cells
    h3_grid_distanceFind the grid distance between two cells
    h3_cell_to_local_ijConvert a cell ID to a local I,J coordinate space
    h3_local_ij_to_cellConvert a local I,J coordinate to a cell ID
    h3_cell_to_vertexGet the vertex ID for a cell ID and vertex number
    h3_cell_to_vertexesGet all vertex IDs for a cell ID
    h3_vertex_to_latConvert a vertex ID to latitude
    h3_vertex_to_lngConvert a vertex ID to longitude
    h3_vertex_to_latlngConvert a vertex ID to latitude/longitude coordinate
    h3_is_valid_vertexTrue if passed a valid vertex ID
    h3_is_valid_directed_edgeTrue if passed a valid directed edge ID
    h3_origin_to_directed_edgesGet all directed edge IDs for a cell ID
    h3_directed_edge_to_cellsConvert a directed edge ID to origin/destination cell IDs
    h3_get_directed_edge_originConvert a directed edge ID to origin cell ID
    h3_get_directed_edge_destinationConvert a directed edge ID to destination cell ID
    h3_cells_to_directed_edgeConvert an origin/destination pair to directed edge ID
    h3_are_neighbor_cellsTrue if the two cell IDs are directly adjacent
    h3_directed_edge_to_boundary_wktConvert directed edge ID to linestring WKT
    h3_directed_edge_to_boundary_wkbConvert directed edge ID to linestring WKB
    h3_reverse_directed_edgeConvert a directed edge to one where origin and destination are swapped
    h3_get_hexagon_area_avgGet average area of a hexagon cell at resolution
    h3_cell_areaGet the area of a cell ID
    h3_get_hexagon_edge_length_avgAverage hexagon edge length at resolution
    h3_edge_lengthGet the length of a directed edge ID
    h3_get_num_cellsGet the number of cells at a resolution
    h3_get_res0_cellsGet all resolution 0 cells
    h3_get_res0_cells_stringGet all resolution 0 cells (returns VARCHAR)
    h3_get_pentagonsGet all pentagons at a resolution
    h3_get_pentagons_stringGet all pentagons at a resolution (returns VARCHAR)
    h3_great_circle_distanceCompute the great circle distance between two points (haversine)
    h3_versionReturns the version of the bound H3 library
    h3_cells_to_multi_polygon_wktConvert a set of cells to multipolygon WKT
    h3_cells_to_multi_polygon_wkbConvert a set of cells to multipolygon WKB
    h3_polygon_wkt_to_cellsConvert polygon WKT to a set of cells
    h3_polygon_wkt_to_cells_stringConvert polygon WKT to a set of cells (returns VARCHAR)
    h3_polygon_wkb_to_cellsConvert polygon WKB to a set of cells
    h3_polygon_wkb_to_cells_stringConvert polygon WKB to a set of cells (returns VARCHAR)
    h3_polygon_wkt_to_cells_experimentalConvert polygon WKT to a set of cells, new algorithm
    h3_polygon_wkt_to_cells_experimental_stringConvert polygon WKT to a set of cells, new algorithm (returns VARCHAR)
    h3_polygon_wkb_to_cells_experimentalConvert polygon WKB to a set of cells, new algorithm
    h3_polygon_wkb_to_cells_experimental_stringConvert polygon WKB to a set of cells, new algorithm (returns VARCHAR)
  7. Convert H3 data to Well-Known Binary (WKB) using WkbEncoder

    main

    The WkbEncoder class provides a stateful interface for constructing Well-Known Binary (WKB) representations of geometries. You build the geometry by calling specific lifecycle methods (e.g., StartPolygon, Point) and then retrieve the final binary string using Finish().

    Supported geometry types and methods:

    • Points: Point(double lng, double lat)
    • LineStrings: StartLineString(), EndLineString()
    • Polygons: StartPolygon(), EndPolygon()
    • MultiPolygons: StartMultiPolygon(uint32_t polygonCount), StartMultiPolygonPolygon(uint32_t loopCount), StartMultiPolygonLoop(), MultiPolygonEmpty(), EndMultiPolygonLoop(), EndMultiPolygonPolygon(), EndMultiPolygon()
    h3duckdb::WkbEncoder encoder;
    encoder.StartPolygon();
    encoder.Point(-122.41, 37.77);
    encoder.EndPolygon();
    std::string wkb = encoder.Finish();
  8. Convert H3 data to Well-Known Text (WKT) using WktEncoder

    main

    The WktEncoder class provides a stateful interface for constructing Well-Known Text (WKT) representations of geometries. It manages the necessary delimiters (like commas and parentheses) automatically. Use the lifecycle methods to define the geometry structure and call Finish() to get the resulting string.

    Supported geometry types and methods:

    • Points: Point(double lng, double lat)
    • LineStrings: StartLineString(), EndLineString()
    • Polygons: StartPolygon(), EndPolygon()
    • MultiPolygons: StartMultiPolygon(uint32_t polygonCount), StartMultiPolygonPolygon(uint32_t loopCount), StartMultiPolygonLoop(), MultiPolygonEmpty(), EndMultiPolygonLoop(), EndMultiPolygonPolygon(), EndMultiPolygon()
    h3duckdb::WktEncoder encoder;
    encoder.StartPolygon();
    encoder.Point(-122.41, 37.77);
    encoder.EndPolygon();
    std::string wkt = encoder.Finish();
  9. Available H3 SQL functions in DuckDB

    main

    The h3-duckdb extension provides a comprehensive suite of H3 geospatial indexing functions for use within DuckDB SQL queries. These functions are categorized into several functional groups: Indexing, Inspection, Hierarchy, Traversal, Directed Edges, Vertex operations, Miscellaneous geometry/metrics, and Region/Polygon operations.

    Note that functions are registered as either scalar functions (operating on single values) or function sets (often used for set-returning operations).

    // The following is a categorized list of available H3 functions:
    
    -- Indexing
    -- latlng_to_cell(lat, lng)
    -- latlng_to_cell_varchar(lat, lng)
    -- cell_to_lat(cell)
    -- cell_to_lng(cell)
    -- cell_to_latlng(cell)
    -- cell_to_boundary_wkt(cell)
    -- cell_to_boundary_wkb(cell)
    
    -- Inspection
    -- get_resolution(cell)
    -- get_base_cell_number(cell)
    -- string_to_h3(string)
    -- h3_to_string(cell)
    -- is_valid_index(cell)
    -- is_valid_cell(cell)
    -- is_res_class_iii(cell)
    -- is_pentagon(cell)
    -- get_icosahedron_faces(cell)
    -- get_index_digit(cell)
    -- construct_cell(cell)
    -- construct_cell_varchar(cell)
    
    -- Hierarchy
    -- cell_to_parent(cell)
    -- cell_to_children(cell)
    -- cell_to_children_size(cell)
    -- cell_to_center_child(cell)
    -- cell_to_child_pos(cell)
    -- child_pos_to_cell(child_pos, parent_cell)
    -- compact_cells(cells)
    -- uncompact_cells(cells)
    
    -- Traversal
    -- grid_disk(cell, k)
    -- grid_disk_distances(cell, k)
    -- grid_disk_unsafe(cell, k)
    -- grid_disk_distances_unsafe(cell, k)
    -- grid_disk_distances_safe(cell, k)
    -- grid_ring(cell, k)
    -- grid_ring_unsafe(cell, k)
    -- grid_path_cells(cell, path)
    -- grid_distance(cell1, cell2)
    -- max_grid_disk_size(k)
    -- cell_to_local_ij(cell)
    -- local_ij_to_cell(local_ij, parent_cell)
    
    -- Directed Edge
    -- are_neighbor_cells(cell1, cell2)
    -- cells_to_directed_edge(cells)
    -- is_valid_directed_edge(edge)
    -- get_directed_edge_origin(edge)
    -- get_directed_edge_destination(edge)
    -- directed_edge_to_cells(edge)
    -- origin_to_directed_edges(cell)
    -- directed_edge_to_boundary_wkt(edge)
    -- directed_edge_to_boundary_wkb(edge)
    -- reverse_directed_edge(edge)
    
    -- Vertex
    -- cell_to_vertex(cell)
    -- cell_to_vertexes(cell)
    -- vertex_to_lat(vertex)
    -- vertex_to_lng(vertex)
    -- vertex_to_latlng(vertex)
    -- is_valid_vertex(vertex)
    
    -- Misc
    -- get_hexagon_area_avg(res)
    -- cell_area(cell)
    -- get_hexagon_edge_length_avg(res)
    -- edge_length(cell)
    -- get_num_cells(cells)
    -- get_res0_cells(res)
    -- get_res0_cells_varchar(res)
    -- get_pentagons(res)
    -- get_pentagons_varchar(res)
    -- great_circle_distance(lat1, lng1, lat2, lng2)
    -- h3_version()
    
    -- Regions
    -- cells_to_multipolygon_wkt(cells)
    -- cells_to_multipolygon_wkb(cells)
    -- polygon_wkt_to_cells(wkt)
    -- polygon_wkt_to_cells_varchar(wkt)
    -- polygon_wkb_to_cells(wkb)
    -- polygon_wkb_to_cells_varchar(wkb)
    -- polygon_wkt_to_cells_experimental(wkt)
    -- polygon_wkt_to_cells_experimental_varchar(wkt)
    -- polygon_wkb_to_cells_experimental(wkb)
    -- polygon_wkb_to_cells_experimental_varchar(wkb)