PgSTAC

The src/pgstac-rs crate provides a high-performance Rust implementation for interacting with a PgSTAC database. It includes:

• A Read Engine: Handles search_plan and collection_search_plan, performing band stepping, hydration (converting EWKB to GeoJSON and merging fragments), keyset token minting, and STAC fields projection in Rust. This moves CPU-intensive tasks off the database.
• A Connection Pool: The PgstacPool (available via the pool feature) provides an async read API.
• Dump/Export Capabilities: A library and CLI tool (pgstac) for exporting data (available via export or cli features).
• Python Bindings: A Python wheel (pgstac) built with maturin (available via the python feature).

PgSTAC is a set of PostgreSQL SQL functions and schemas designed to build highly performant databases for Spatio-Temporal Asset Catalogs (STAC). It provides core data models and functions to enable a STAC API directly from a PostgreSQL database, supporting STAC Filters and CQL2 search.

Note that PgSTAC operates entirely within the database and does not provide an HTTP-facing API itself. To expose a STAC API, you can use backends like STAC FastAPI or Franklin.

By default, PgSTAC partitions data by collection. Collections can be further partitioned by year or month.

Crucial: Partitioning must be configured before loading any data. You configure it by setting the partition_trunc flag on a collection. Aim to keep each partition under a few hundred thousand rows to maintain performance.

-- Set a collection to be partitioned by month
UPDATE collections SET partition_trunc='month' WHERE id='<collection id>';

To automatically adjust collection spatial and temporal extents as items are ingested, set the configuration pgstac.update_collection_extent to true.

To minimize transaction overhead during ingestion, combine this with pgstac.use_queue, which defers the extent updates to a queue. This queue must be drained manually or via a scheduler.

How to drain the queue:

• CLI: Run pgstac runqueue (which executes CALL run_queued_queries()).
• SQL: Execute CALL run_queued_queries(); directly.
• Automation: Use pg_cron to schedule the queue processing.

# Manually run the queue
pgstac runqueue

The PgSTAC project consists of several specialized packages:

• pgstac (SQL): The core SQL schema and functions for STAC data models.
• pgstac-migrate: A focused migration package for managing database upgrades.
• pgstac (Rust CLI): A Rust-based CLI tool used for loading data, searching, querying, managing extensions, and maintenance.
• pypgstac-rs: A Rust-backed Python extension designed for read/write operations, specifically for use with stac-fastapi-pgstac.

PgSTAC requires PostgreSQL >= 13, PostGIS >= 3, and the btree_gist extension. For optimal performance, use PostGIS >= 3.1. All PgSTAC objects are installed into the pgstac schema, which must be included in your PostgreSQL search_path to function correctly.

-- Set search_path for a specific user
ALTER ROLE <user> SET SEARCH_PATH TO pgstac, public;

-- Or set it for the entire database
ALTER DATABASE <database> SET search_path TO pgstac, public;

For pgstac developers, adding or removing a promoted field is a high-impact change that requires synchronizing updates across six specific locations in the SQL schema files (src/pgstac/sql/003a_items.sql and src/pgstac/sql/002a_queryables.sql).

Failure to update all six locations will break the promotion lifecycle. The required locations are:

1. items TABLE DDL: The actual column declaration.
2. content_dehydrate(): The logic to extract the value from JSONB during ingest.
3. promoted_item_property_defs(): The metadata registry mapping the STAC property name to the SQL column.
4. promoted_properties_from_item(): The logic to rehydrate the column value back into the JSONB object during read.
5. promoted_items_column_list(): The ordered array used for auto-derived logic.
6. items_staging_dehydrate(): The enriched SELECT statement used for all staging branches.

Testing requires a running pgstac database.

1. From the root of the pgstac repository, start the server:

   scripts/server

2. In another terminal, run the tests:

   cargo test --manifest-path src/pgstac-rs/Cargo.toml

Each test runs in its own transaction and is rolled back automatically.

Customizing the test database connection: By default, tests connect to postgresql://username:password@localhost:5439/postgis. To use a different database, set the PGSTAC_RS_TEST_DB environment variable:

PGSTAC_RS_TEST_DB=postgresql://otherusername:otherpassword@otherhost:7822/otherdbname cargo test --manifest-path src/pgstac-rs/Cargo.toml

# Start server from repo root
scripts/server

# Run tests
cargo test --manifest-path src/pgstac-rs/Cargo.toml

PgSTAC uses three primary roles to manage access and security:

• pgstac_admin: The owner of all objects within the pgstac schema. Use this role for running migrations.
• pgstac_ingest: Has read/write privileges on all tables. Use this for data ingestion or when using the transactions extension with stac-fastapi-pgstac.
• pgstac_read: Has read-only access to items and collections, but can still write to logging tables.

You can use these roles directly by assigning a password, or grant them to an existing user.

-- To use a role directly with a password
ALTER ROLE pgstac_read LOGIN PASSWORD '<password>';

-- To grant permissions to an existing user
GRANT pgstac_read TO <user>;