PGHoard Documentation

PGHoard is a tool designed for PostgreSQL backup and recovery. Its core capabilities include:

Backup Capabilities

• Automatic periodic basebackups: Scheduled full backups of the database.
• Automatic transaction log (WAL/xlog) backups: Supports multiple methods including pg_receivewal (formerly pg_receivexlog), archive_command, or experimental PG native replication protocol support via walreceiver.
• Standalone Hot Backup support: Optional support for performing backups without a running standby.
• Cloud object storage support: Compatible with AWS S3, Google Cloud, OpenStack Swift, Azure, and Ceph.

Recovery and Restoration

• Point-in-time-recovery (PITR): Restore the database to a specific moment in time.
• Direct restoration: Restore directly from object storage with built-in compression and encryption.
• Standby initialization: Automatically configure and initialize a new standby from object storage backups as a replicating hot-standby.

Reliability and Monitoring

• Fault-resilience: Retries transfers during temporary object storage connectivity issues and verifies WAL file headers to prevent issues with recycled files.
• Maintenance tools: Includes an "Archive sync" tool to detect and fix holes in WAL backup streams, and an "Archive cleanup" tool to delete obsolete WAL files.
• Monitoring: Automatically manages history cleanup (deleting backups/WAL older than N days), updates statistics in a local file, and creates alert files on disk when problems occur.

Performance

• Parallelism: Uses parallel compression and encryption.
• Optimized Restore: Implements WAL pre-fetching during the restoration process.

The basebackup_mode setting determines how PostgreSQL data is extracted. Note that neither basic nor pipe modes support multiple tablespaces.

• basic (default): Runs pg_basebackup and waits for it to write an uncompressed tar file to disk before compressing and optionally encrypting it.
• pipe: Pipes data directly from pg_basebackup to PGHoard's compression/encryption pipeline, reducing temporary disk space requirements.
• local-tar: Bypasses pg_basebackup by reading files directly from $PGDATA. This mode supports user tablespaces. Requires pg_data_directory to be set. Use basebackup_threads (values 1 or 2) to optimize this mode.
• delta: Only uploads changed files by using file hashes and manifests. This is efficient for storage but requires a manifest for restoration.
• local-tar-delta-stats: Behaves like local-tar but collects metrics to help evaluate the efficiency of switching to delta mode.

PGHoard uses a JSON-formatted configuration file consisting of nested key-value pairs. The configuration defines global settings and specific backup_sites.

Key top-level sections include:

• backup_sites: Defines the database clusters to back up.
• transfer: Configures WAL and basebackup transfer parameters.
• compression: Configures compression algorithms and levels.
• stats / push_gateway: Configures monitoring and metrics.
• http_address / http_port: Configures the HTTP server for the daemon.

{
    "json_state_file_path": "/var/lib/pghoard/pghoard_state.json"
    "backup_sites": {
        "mycluster": {
            "nodes": [
                {
                    "host": "127.0.0.1",
                    "password": "secret",
                    "port": 5432,
                    "user": "backup",
                    "slot": "pghoard"
                }
            ],
            "basebackup_count": 5,
            "basebackup_mode": "delta",
            "object_storage": {
                "storage_type": "local",
                "directory": "/tmp/pghoard/backups"
            }
        }
    }
}