DOCSEARCHbyAlgolia
Submit Docs

py-pglite

repository·main·Indexed 20 days ago

https://github.com/wey-gu/py-pglite

A Python testing library for PGlite that provides an instant, zero-config, in-memory PostgreSQL environment. It supports real PostgreSQL features including JSONB, arrays, and extensions like pgvector. The library offers seamless integration with FastAPI, SQLAlchemy, and Django (via Lightweight/Socket or Full Integration/Backend patterns), and provides utilities like PGliteManager and PGliteConfig for database instance management.

Tokens
14.5K
Snippets
60
Records
66
Agent score
70%

What's inside py-pglite

  1. Understand Unix Socket vs TCP Socket modes

    main

    py-pglite supports two connection modes:

    Unix Socket Mode (Default)

    • Best for: Local testing and maximum performance.
    • Mechanism: Uses Unix domain sockets.
    • Note: Fastest for local testing.

    TCP Socket Mode

    • Best for: asyncpg compatibility, cloud-native testing, Docker containers with network isolation, and testing network-based tools.
    • Mechanism: Binds to a TCP port (defaults to 127.0.0.1:5432).
    • Requirement: asyncpg requires TCP mode.

    Important Constraints:

    • PGlite Socket supports only one active connection at a time.
    • SSL is not supported; always use sslmode=disable.
    • TCP mode binds to localhost by default for security.

  2. Migrate between Django py-pglite integration patterns

    main

    Since both patterns use standard Django models, you can switch between them by updating your configuration and test fixtures.

    Migrating from Lightweight/Socket to Full Integration

    1. Change the pytest fixture from configured_django to django_pglite_db.
    2. Update the DATABASES engine from django.db.backends.postgresql to py_pglite.django.backend.
    3. (Optional) Utilize enhanced JSON features.

    Migrating from Full Integration to Lightweight/Socket

    1. Change the pytest fixture from django_pglite_db to configured_django.
    2. Update the DATABASES engine from py_pglite.django.backend to django.db.backends.postgresql.
    3. (Optional) Simplify JSON usage if necessary.

  3. Integrate py-pglite with Django

    main

    There are two primary integration patterns for Django developers:

    1. Lightweight/Socket Pattern: Uses a standard PostgreSQL backend with a socket connection. It offers fast startup and minimal dependencies, following standard Django patterns.
    2. Full Integration/Backend Pattern: Uses a custom py-pglite backend. This provides enhanced features such as advanced JSON support and backend optimizations for a more production-like setup.

    To explore all Django patterns and documentation:

    pytest testing-patterns/django/ -v

  4. Quick Start with py-pglite

    main

    You can get a real PostgreSQL instance running in seconds with zero configuration. Use the instant demo to see the magic of starting a PostgreSQL instance immediately for testing.

    To run the instant demo:

    python quickstart/demo_instant.py

  5. Use the Full Integration/Backend pattern for Django integration

    main

    The Full Integration pattern uses the custom py_pglite.django.backend engine. This pattern is designed for comprehensive testing, providing backend optimizations and enhanced PostgreSQL JSON support (e.g., advanced JSON field queries).

    Configuration

    In your conftest.py, use the custom backend. The backend manages the connection automatically:

    DATABASES = {
    "default": {
        "ENGINE": "py_pglite.django.backend",
    }
}

    Usage Example

    When using the django_pglite_db fixture, you can leverage enhanced JSON capabilities:

    def test_django_with_backend(django_pglite_db):
    class Product(models.Model):
        name = models.CharField(max_length=100)
        metadata = models.JSONField(default=dict)
        tags = models.JSONField(default=list)

        class Meta:
            app_label = "example"

    product = Product.objects.create(
        name="Advanced Widget",
        metadata={"features": ["json", "backend"]},
        tags=["premium", "advanced"]
    )

    # Enhanced JSON queries work natively
    results = Product.objects.filter(tags__contains=["premium"])
    assert results.count() == 1

  6. Install py-pglite

    main

    Install the core framework-agnostic package or specific extras for your technology stack using pip.

    Core installation:

    pip install py-pglite

    Stack-specific installations:

    • SQLAlchemy + SQLModel: pip install py-pglite[sqlalchemy]
    • Django + pytest-django: pip install py-pglite[django]
    • Pure async client: pip install py-pglite[asyncpg]
    • Everything: pip install py-pglite[all]

    Extra features:

    • PGlite extensions (e.g., pgvector, fuzzystrmatch): pip install py-pglite[extensions]
    pip install py-pglite

  7. Use py-pglite with Django

    main

    Django integration offers two patterns depending on your needs:

    1. Lightweight/Socket Pattern: Minimal setup using a standard PostgreSQL backend via sockets. Best for speed and minimal dependencies.
    2. Full Integration/Backend Pattern: Uses a custom backend with enhanced features like improved JSON support. Best for advanced features.

    Refer to the Django patterns guide for detailed implementation details.

    # Lightweight/Socket Pattern
def test_django_socket_pattern(configured_django):
    Post.objects.create(title="Hello", content="World")
    assert Post.objects.count() == 1

# Full Integration/Backend Pattern
def test_django_backend_pattern(django_pglite_db):
    Post.objects.create(title="Hello", content="World", metadata={"tags": ["test"]})
    assert Post.objects.count() == 1

  8. Use py-pglite with any PostgreSQL client

    main

    You can extract connection details from a pglite_manager to use with standard clients like psycopg or asyncpg.

    For sync clients (e.g., psycopg): Extract the host, port, and database name from the engine URL.

    For asyncpg (Requires TCP mode): asyncpg requires use_tcp=True in the configuration. Additionally, you must pass server_settings={} in the asyncpg.connect call to ensure compatibility with PGlite.

    # Using with any client via manager
def test_any_client_works(pglite_manager):
    engine = pglite_manager.get_engine()
    host, port, database = str(engine.url.host), engine.url.port, engine.url.database
    # conn = psycopg.connect(host=host, port=port, dbname=database)

# Using with asyncpg (Requires TCP mode)
async def test_asyncpg_works(pglite_tcp_manager):
    config = pglite_tcp_manager.config
    conn = await asyncpg.connect(
        host=config.tcp_host,
        port=config.tcp_port,
        user="postgres",
        password="postgres",
        database="postgres",
        ssl=False,
        server_settings={}  # CRITICAL: Required for PGlite compatibility
    )
    result = await conn.fetchval("SELECT 1")
    await conn.close()

  9. Use the Lightweight/Socket pattern for Django integration

    main

    The Lightweight/Socket pattern uses the standard django.db.backends.postgresql engine and connects to PGlite via a direct socket. This is ideal for quick prototyping, basic ORM testing, and scenarios where you want minimal setup and fast startup times without extra dependencies.

    Configuration

    In your conftest.py, configure the DATABASES setting to point to the PGlite socket directory:

    DATABASES = {
    "default": {
        "ENGINE": "django.db.backends.postgresql",
        "HOST": socket_directory,  # The path to the PGlite socket
        # ... other settings
    }
}

    Usage Example

    When using the configured_django fixture, standard Django operations work as expected:

    def test_django_with_socket(configured_django):
    class Product(models.Model):
        name = models.CharField(max_length=100)
        price = models.DecimalField(max_digits=10, decimal_places=2)

        class Meta:
            app_label = "example"

    product = Product.objects.create(name="Widget", price=29.99)
    assert product.id is not None
    DATABASES = {
    "default": {
        "ENGINE": "django.db.backends.postgresql",
        "HOST": socket_directory,
    }
}

  10. SQLModel support in PGlite fixtures

    main

    The PGlite SQLAlchemy fixtures have built-in support for SQLModel. If sqlmodel is installed in your environment, the session fixtures (pglite_session and pglite_async_session) will automatically prefer SQLModel session types over standard SQLAlchemy sessions.

    Behavior with SQLModel:

    • Sync: Uses sqlmodel.Session.
    • Async: Uses sqlmodel.ext.asyncio.session.AsyncSession.
    • Schema Setup: Automatically calls SQLModel.metadata.create_all (with retry logic) to ensure your models are reflected in the PGlite instance before the test runs.

  11. Configure PGlite via PGliteConfig

    main

    Use PGliteConfig to customize the behavior of the database instance. Key options include:

    • timeout: Extended timeout for CI/CD environments.
    • log_level: Logging verbosity (e.g., "INFO").
    • cleanup_on_exit: Boolean to enable automatic cleanup.
    • work_dir: Custom directory for data storage.
    • use_tcp: Boolean to enable TCP mode (required for asyncpg).
    • tcp_host: The host to bind to (defaults to 127.0.0.1).
    • tcp_port: The port to bind to (defaults to 5432).
    • extensions: A list of PGlite extensions to enable.
    from py_pglite import PGliteConfig

config = PGliteConfig(
    timeout=60,
    log_level="INFO",
    cleanup_on_exit=True,
    work_dir=Path("./test-data"),
    use_tcp=True,
    tcp_host="127.0.0.1",
    tcp_port=5432,
    extensions=["pgvector"]
)

  12. How PGlite handles test isolation in Django

    main

    Because PGlite does not support the CREATE DATABASE command, the Django backend uses PostgreSQL schemas to provide test isolation.

    When running tests, the backend:

    1. Generates a unique test database name.
    2. Creates a corresponding PostgreSQL schema using CREATE SCHEMA IF NOT EXISTS "{schema_name}".
    3. Configures the connection to use the search_path option, setting it to {schema_name},public. This ensures that all queries are scoped to the isolated schema while still allowing access to the public schema.
    4. Automatically runs Django migrations within that schema.

    This approach allows multiple test databases to run concurrently using the same underlying PGlite instance by separating them into different logical schemas.