Supabase Postgres Documentation

repository·develop·Indexed 23 days ago

https://github.com/supabase/postgres

A specialized PostgreSQL distribution bundling unmodified PostgreSQL with curated extensions. It is designed for production readiness and reproducibility via Nix and Docker. The documentation covers managing SQL migrations with dbmate-tool, building PostgreSQL from source, developing pgrx extensions, and using Supascan for machine baseline generation and system validation.

Tokens
40.2K
Snippets
113
Records
205
Agent score
83%

What's inside Supabase Postgres

  1. What is Supabase Postgres?

    develop

    Supabase Postgres is a batteries-included PostgreSQL distribution that provides unmodified PostgreSQL with a curated set of production-tested extensions pre-installed. It is designed to provide a production-ready setup for tasks such as full-text search, geospatial processing, time-series management, JSON validation, and security without requiring manual compilation of individual extensions.

    Core principles:

    • Unmodified PostgreSQL: Uses standard PostgreSQL core.
    • Curated Extensions: Includes well-maintained extensions.
    • Multi-version Support: Supports PostgreSQL 15, 17, and OrioleDB-17.
    • Ready for Production: Configured with sensible defaults for replication, security, and performance.
  2. Understand the Nix CI workflow steps

    develop

    The Nix artifacts are built via the Nix CI workflow (.github/workflows/nix-build.yml), which follows a 4-step dependency chain:

    1. Eval: Evaluates legacyPackages and checks flake outputs using nix-eval-jobs. It produces a JSON map of jobs for each architecture. This is implemented via the github_matrix.py script. Jobs are assigned to runners based on attributes like kvm (for self-hosted runners) or big-parallel (for 32vcpu ephemeral runners).
    2. Build: Instantiates build matrices for each architecture (aarch64 darwin, aarch64 linux, and x86_64 linux). It runs nix build ${job} and pushes results to the nix-postgres-artifacts S3 cache.
    3. Check: Runs automated tests using the JSON from the Eval step. Some tests require KVM virtualization and are routed to self-hosted runners.
    4. Images Build: Uses the artifacts from the Build step and the nix/packages/build-ami.nix script to generate AMI images based on Ubuntu Noble.
  3. Understand the Nix directory structure

    develop

    This project uses flake-parts to manage its Nix configuration, splitting the main flake.nix into specialized modules located in the nix/ directory. The root flake.nix acts as an entry point that imports these modules to define systems, inputs, and configurations.

    Directory Overview

    • nix/apps.nix: Definitions for applications accessible via nix run (tools, scripts, utilities).
    • nix/checks.nix: Build checks, integration tests, and validation to ensure Postgres packages build correctly.
    • nix/config.nix: Global constants (e.g., PostgreSQL default ports/users) and system-wide configurations.
    • nix/devShells.nix: Configurations for the development environment, including tool dependencies and environment variables.
    • nix/ext/: Definitions for PostgreSQL extensions (e.g., pgvector.nix, pgsodium.nix, pg_graphql.nix).
    • nix/fmt.nix: Code formatting configuration using treefmt.
    • nix/hooks.nix: Git hooks and pre-commit configurations.
    • nix/overlays/: Nixpkgs overlays for package customization.
    • nix/packages/: Custom package definitions (e.g., migrate-tool.nix, postgres.nix).
    • nix/postgresql/: Core PostgreSQL package definitions and patches.
    • nix/tests/: Test suites containing SQL files, expected outputs, and migrations.
  4. Understand the EBS-Surrogate file layout

    develop

    The ebssurrogate directory contains the configuration files and scripts used for the AMI build process:

    files/ directory

    • 70-ec2-nvme-devices.rules: Udev rules for NVMe devices.
    • cloud.cfg: Configuration for cloud-init.
    • ebsnvme-id: Utility for NVMe identification.
    • sources-arm64.cfg: apt/sources.list configuration for arm64.
    • sources.cfg: apt/sources.list configuration for amd64.
    • vector.timer: A systemd-timer used to delay vector execution.
    • zfs-growpart-root.cfg: Configuration for ZFS partition growth.

    scripts/ directory

    • chroot-bootstrap.sh: Installs grub and other required packages; configures target AMI settings.
    • surrogate-bootstrap.sh: Formats the disk, sets up the chroot environment, and executes Ansible tasks within that environment.
  5. Understand the Flake-Parts architecture

    develop
    This repository uses flake-parts to organize its Nix flake into a modular system. Instead of a single monolithic flake.nix, the configuration is split into specialized modules that handle specific concerns (e.g., packages, devShells, config, hooks). This allows for per-system evaluation, module composition, and type-safe configuration via the NixOS module system.
  6. How Nix overlays work in Supabase Postgres

    develop

    Overlays are a Nixpkgs feature used to modify the package namespace without changing upstream code. They serve two primary purposes:

    1. Adding new packages: Introducing new names to the namespace (e.g., adding foobar-1_2_3 alongside an existing foobar to provide a specific version).
    2. Globally overriding existing packages: Replacing an existing package name with a different version or configuration (e.g., overriding gdal with a version that has fewer features enabled).

    When you override an existing name, the change is global: any other package that depends on that name will now use the overridden version.

    final: prev: {
        gdal = prev.gdalMinimal;
    }
  7. Understand the perSystem context arguments

    develop

    When defining logic within a perSystem block, you have access to several special arguments that provide context for the specific architecture being evaluated:

    perSystem = {
      # Special arguments
      self',      # Current system's outputs
      inputs',    # Current system's inputs
      pkgs,       # nixpkgs for current system
      system,     # System string
      lib,        # nixpkgs lib
      config,     # Module config
      ...
    }: {
      # Your outputs
    };
  8. Simplify extension postBuild for slim builds

    develop

    For extensions that require complex post-build steps (like wrappers), the postBuild logic should be simplified when latestOnly is enabled. Specifically:

    • Skip migrations: Do not run create_migration_sql_files if latestOnly is true.
    • Simplify symlinks: When latestOnly is true, you only need to symlink the single latest library file to the main library name, rather than iterating through all previously packaged versions.
    • Control files: Always ensure the .control file is updated to point to the latestVersion.
  9. How Supascan validation categories work

    develop

    Supascan distinguishes between two types of specification files during validation:

    1. Critical Specs: These are mandatory requirements. If any check in these files fails, the validate command returns exit code 1.

      • Examples: service.yml, user.yml, group.yml, mount.yml, package.yml, files-security.yml, files-ssl.yml, files-postgres-config.yml, files-postgres-data.yml.
    2. Advisory Specs: These are informational. Failures in these files do not trigger a non-zero exit code but are reported in the summary for review.

      • Examples: kernel-param.yml, files-etc.yml, files-systemd.yml, and other files-*.yml categories.

    This allows teams to enforce strict security/configuration requirements while still monitoring general system state.

  10. Use self-references to access module outputs

    develop

    In the flake-parts architecture, modules can reference outputs from other modules using specific self-reference patterns. This allows you to access packages, configurations, or overlays defined elsewhere in the flake.

    • self'.packages.<name>: Reference a package for the current system.
    • self.supabase.defaults: Reference flake-level configuration.
    • self.overlays.default: Reference a flake-level overlay.
    • inputs'.<name>.packages.default: Reference packages from a specific flake input for the current system.
    # Reference current system's packages
    self'.packages."psql_15.bin"
    
    # Reference flake-level config
    self.supabase.defaults
    
    # Reference flake-level overlay
    self.overlays.default
    
    # Reference inputs for current system
    inputs'.nix-editor.packages.default
  11. Understand the Supabase Postgres directory structure

    develop

    The project is organized into several key directories:

    • nix/: The core build system using Nix. Contains PostgreSQL configurations, extension definitions (nix/ext/), overlays, and build tools.
    • ansible/: Infrastructure as Code for server configuration and AWS AMI deployment.
    • migrations/: Database schema migrations and upgrade tools.
    • tests/: Integration and system tests.
    • scripts/: Utility scripts for development and deployment.
    • Dockerfile-*: Specific Dockerfile definitions for different PostgreSQL versions.