DOCSEARCHbyAlgolia
Submit Docs

django-pg-zero-downtime-migrations

repository·master·Indexed 20 days ago

https://github.com/tbicr/django-pg-zero-downtime-migrations

A Django PostgreSQL backend designed to achieve zero-downtime deployments by managing database locks, timeouts, and constraint dropping explicitly. It provides specialized backends for standard PostgreSQL and PostGIS, avoids transactions for migrations to prevent deadlocks, and offers safe alternatives for common schema changes like adding unique constraints or NOT NULL columns.

Tokens
3.9K
Snippets
7
Records
16
Agent score
19%

What's inside django-pg-zero-downtime-migrations

  1. How zero downtime migrations work and differ from standard Django

    master

    Unlike the standard Django backend, this backend does not use transactions for migrations (except for RunPython operations). This is because some SQL fixes cannot be run within a transaction, and avoiding transactions helps prevent deadlocks during complex migrations.

    Implications:

    • If a migration fails halfway, you must fix the database state manually.
    • Best Practice: Keep migration modules as small as possible.
    • Automation: Use ZERO_DOWNTIME_MIGRATIONS_IDEMPOTENT_SQL=True to allow rerunning failed migrations after manual fixes.
    • Safety: It provides additional guarantees to avoid stuck table locks by using explicit timeouts and explicit constraint dropping.

  2. Understand migration vs business logic lock types

    master

    To achieve zero downtime, you must distinguish between migration operations and concurrent business logic operations:

    Migration Locks

    • ACCESS EXCLUSIVE: Used for CREATE SEQUENCE, DROP SEQUENCE, CREATE TABLE, DROP TABLE, ALTER TABLE, and DROP INDEX.
    • SHARE: Used for CREATE INDEX.
    • SHARE UPDATE EXCLUSIVE: Used for CREATE INDEX CONCURRENTLY, DROP INDEX CONCURRENTLY, and ALTER TABLE VALIDATE CONSTRAINT.

    Business Logic Locks

    • ACCESS SHARE: Used by SELECT operations. Conflicts with ACCESS EXCLUSIVE.
    • ROW SHARE: Used by SELECT FOR UPDATE. Conflicts with ACCESS EXCLUSIVE and EXCLUSIVE.
    • ROW EXCLUSIVE: Used by INSERT, UPDATE, and DELETE. Conflicts with ACCESS EXCLUSIVE, EXCLUSIVE, SHARE ROW EXCLUSIVE, and SHARE.

  3. Understand Postgres table-level lock conflicts

    master
    Postgres uses various table-level locks that can conflict with each other. Understanding these conflicts is critical for zero-downtime migrations, as certain migration operations (like ALTER TABLE) require ACCESS EXCLUSIVE locks, which block all other operations including SELECT (which requires ACCESS SHARE).

  4. How to safely rename models or columns

    master

    Standard renames are unsafe because old and new code cannot operate on the same table simultaneously. Use an updatable view pattern:

    Renaming a Model (Table)

    1. In a transaction, rename the table and create an updatable view with the old name.
      • Old code uses the view (via the old name).
      • New code uses the table (via the new name).
    2. After deploying new code, drop the view.

    Renaming a Column

    1. In a transaction, rename the column, rename the table to a temporary name, and create an updatable view that contains both the old and new column names.
      • Old code uses the view with the old column name.
      • New code uses the view with the new column name.
    2. After deploying new code, drop the view and rename the table back to its original name.

  5. Configure the Django database backend for zero downtime migrations

    master

    To enable zero-downtime migrations, switch your Django DATABASES engine to the one provided by this package. For standard PostgreSQL, use django_zero_downtime_migrations.backends.postgres. For PostGIS, use django_zero_downtime_migrations.backends.postgis.

    Note: This backend only improves zero-downtime behavior for migrations involving schema changes and RunSQL operations. RunPython operations behave like the standard Django backend.

    DATABASES = {
    'default': {
        'ENGINE': 'django_zero_downtime_migrations.backends.postgres',
        # Use 'django_zero_downtime_migrations.backends.postgis' for PostGIS
        ...
    }
}

  6. Avoid downtime using Postgres timeouts

    master

    To prevent long-running queries or migrations from causing downtime due to lock waiting, use Postgres timeout settings:

    • To avoid downtime caused by long-running transactions/queries blocking your migration: Use SET lock_timeout TO '2s'. This prevents the migration from waiting indefinitely for a lock.
    • To avoid downtime caused by a long-running migration query blocking business logic: Use SET statement_timeout TO '2s'. This prevents the migration itself from running too long and holding locks.

  8. Zero downtime deployment flow requirements

    master

    To achieve true zero downtime using this package, your deployment must satisfy these requirements:

    1. Single Database: Use one database.
    2. High Availability: Multiple application instances must be running; the application must remain available even if one instance is restarted.
    3. Load Balancer: A balancer must exist in front of the instances.
    4. Schema Compatibility (Old App): The old application version must work correctly with both the old and the new database schema.
    5. Schema Compatibility (New App): The new application version must work correctly with the new database schema.

    Recommended Flow:

    1. Apply migrations.
    2. Disconnect an instance from the balancer, restart it, and reconnect it. Repeat this one-by-one for all instances.

  9. How to safely add a NOT NULL column

    master

    When adding a NOT NULL column, you must ensure old code (which doesn't know about the new column) can still perform inserts.

    Using db_default (Django 5.0+)

    This is the most robust method. The default is handled at the database level, so old code inserts will work fine.

    -- migration
ALTER TABLE tbl ADD COLUMN new_col integer DEFAULT 0 NOT NULL;

-- business logic (old code)
INSERT INTO tbl (old_col) VALUES (1); -- Works fine

    Using Django default (Django < 5.0)

    Standard Django default is applied in Python, not the DB. To emulate db_default behavior in older versions, use the setting ZERO_DOWNTIME_MIGRATIONS_KEEP_DEFAULT=True.

    -- migration
ALTER TABLE tbl ADD COLUMN new_col integer DEFAULT 0 NOT NULL;
ALTER TABLE tbl ALTER COLUMN new_col DROP DEFAULT;

-- business logic (old code)
INSERT INTO tbl (old_col) VALUES (1);  -- old code inserts fail
INSERT INTO tbl (old_col, new_col) VALUES (1, 1);  -- new code inserts work fine

  10. How to safely apply a NOT NULL constraint to an existing column

    master

    Applying SET NOT NULL triggers a full table scan, which takes an ACCESS EXCLUSIVE lock. To avoid this, use a CHECK constraint validation pattern:

    1. ALTER TABLE ADD CONSTRAINT CHECK (column IS NOT NULL) NOT VALID (Takes ACCESS EXCLUSIVE only for metadata update).
    2. ALTER TABLE VALIDATE CONSTRAINT (Takes SHARE UPDATE EXCLUSIVE while performing the full table scan).
    3. ALTER TABLE ALTER COLUMN SET NOT NULL (Takes ACCESS EXCLUSIVE only for metadata update, as it skips the scan if a valid check constraint exists).
    4. ALTER TABLE DROP CONSTRAINT (Cleans up the redundant check constraint).

  11. Safe alternatives for common Django migrations

    master

    Many standard Django migrations are unsafe for production because they take ACCESS EXCLUSIVE locks or perform long-running operations. Use these safe alternatives:

    OperationUnsafe ApproachSafe Alternative
    Add Unique/Primary KeyALTER TABLE ADD COLUMN PRIMARY KEYALTER TABLE ADD COLUMN $\rightarrow$ CREATE INDEX CONCURRENTLY $\rightarrow$ ALTER TABLE ADD CONSTRAINT ... USING INDEX
    Add Unique ConstraintALTER TABLE ADD COLUMN UNIQUEALTER TABLE ADD COLUMN $\rightarrow$ CREATE INDEX CONCURRENTLY $\rightarrow$ ALTER TABLE ADD CONSTRAINT ... USING INDEX
    Add NOT NULL ConstraintALTER TABLE ADD COLUMN ... SET NOT NULLALTER TABLE ADD CONSTRAINT CHECK (...) NOT VALID $\rightarrow$ ALTER TABLE VALIDATE CONSTRAINT $\rightarrow$ ALTER TABLE ALTER COLUMN SET NOT NULL
    Create IndexCREATE INDEXCREATE INDEX CONCURRENTLY
    Drop IndexDROP INDEXDROP INDEX CONCURRENTLY

  12. Configure zero downtime migration settings

    master

    Use the following settings to control how the backend handles locks, timeouts, and safety checks. It is recommended to set timeouts to prevent migrations from hanging and blocking application traffic.

    ZERO_DOWNTIME_MIGRATIONS_LOCK_TIMEOUT = '2s'
ZERO_DOWNTIME_MIGRATIONS_STATEMENT_TIMEOUT = '2s'
ZERO_DOWNTIME_MIGRATIONS_FLEXIBLE_STATEMENT_TIMEOUT = True
ZERO_DOWNTIME_MIGRATIONS_RAISE_FOR_UNSAFE = True