databases

repository·master·Indexed 26 days ago

https://github.com/encode/databases

An asyncio library providing simple support for PostgreSQL, MySQL, and SQLite. It enables the use of SQLAlchemy Core expression language for asynchronous database queries, making it suitable for integration with async web frameworks such as FastAPI, Starlette, and Sanic. The library includes a Database class for managing connections, transactions, and query execution, and supports integration with Alembic for migrations.

Tokens
4.6K
Snippets
16
Records
18
Agent score
87%

What's inside databases

  1. Quickstart: Create, insert, and query a database

    master

    This example demonstrates how to initialize a connection, create a table, insert multiple rows, and fetch results using aiosqlite.

    # Create a database instance, and connect to it.
    from databases import Database
    database = Database('sqlite+aiosqlite:///example.db')
    await database.connect()
    
    # Create a table.
    query = """CREATE TABLE HighScores (id INTEGER PRIMARY KEY, name VARCHAR(100), score INTEGER)"""
    await database.execute(query=query)
    
    # Insert some data.
    query = "INSERT INTO HighScores(name, score) VALUES (:name, :score)"
    values = [
        {"name": "Daisy", "score": 92},
        {"name": "Neil", "score": 87},
        {"name": "Carol", "score": 43},
    ]
    await database.execute_many(query=query, values=values)
    
    # Run a database query.
    query = "SELECT * FROM HighScores"
    rows = await database.fetch_all(query=query)
    print('High Scores:', rows)
  2. Connect and disconnect from a database

    master

    You can manage the database connection pool using an async context manager, which automatically handles connecting and disconnecting. Alternatively, you can use explicit .connect() and .disconnect() methods. If integrating with a web framework like Starlette, it is recommended to hook these methods into the framework's startup and shutdown events.

    # Using an async context manager
    async with Database(DATABASE_URL) as database:
        ...
    
    # Using explicit connect/disconnect
    database = Database(DATABASE_URL)
    await database.connect()
    ...
    await database.disconnect()
  3. Declare tables using SQLAlchemy core

    master

    To use SQLAlchemy core queries, you must declare your tables in your code using sqlalchemy.MetaData and sqlalchemy.Table. This allows you to keep your database schema in sync with your application code and enables the use of migration tools. You can use standard SQLAlchemy column types like sqlalchemy.Integer, sqlalchemy.String, sqlalchemy.Boolean, or sqlalchemy.JSON.

    import sqlalchemy
    
    metadata = sqlalchemy.MetaData()
    
    notes = sqlalchemy.Table(
        "notes",
        metadata,
        sqlalchemy.Column("id", sqlalchemy.Integer, primary_key=True),
        sqlalchemy.Column("text", sqlalchemy.String(length=100)),
        sqlalchemy.Column("completed", sqlalchemy.Boolean),
    )
  4. Create tables using SQLAlchemy schema compilation

    master

    Because databases does not use the SQLAlchemy engine internally, you cannot use metadata.create_all(). Instead, you must compile the CreateTable schema using the appropriate SQLAlchemy dialect and then execute the resulting SQL string via database.execute().

    Note: This method is recommended for local experimentation only. For production projects, use a migration tool like Alembic.

    from databases import Database
    import sqlalchemy
    
    database = Database("postgresql+asyncpg://localhost/example")
    
    # Establish the connection pool
    await database.connect()
    
    metadata = sqlalchemy.MetaData()
    dialect = sqlalchemy.dialects.postgresql.dialect()
    
    # Define your table(s)
    notes = sqlalchemy.Table(
        "notes",
        metadata,
        sqlalchemy.Column("id", sqlalchemy.Integer, primary_key=True),
        sqlalchemy.Column("text", sqlalchemy.String(length=100)),
        sqlalchemy.Column("completed", sqlalchemy.Boolean),
    )
    
    # Create tables
    for table in metadata.tables.values():
        # Set `if_not_exists=False` if you want the query to throw an
        # exception when the table already exists
        schema = sqlalchemy.schema.CreateTable(table, if_not_exists=True)
        query = str(schema.compile(dialect=dialect))
        await database.execute(query=query)
    
    # Close all connections in the connection pool
    await database.disconnect()
  5. Handle nested transactions using savepoints

    master

    Nested transactions are supported and are implemented using database savepoints. If an inner transaction fails (e.g., by raising an exception), you can suppress that exception to prevent it from affecting the outer transaction, allowing the outer transaction to continue.

    import contextlib
    
    async with databases.Database(database_url) as db:
        async with db.transaction() as outer:
            # Do something in the outer transaction
            ...
    
            # Suppress to prevent influence on the outer transaction
            with contextlib.suppress(ValueError):
                async with db.transaction():
                    # Do something in the inner transaction
                    ...
                    raise ValueError('Abort the inner transaction')
    
        # Observe the results of the outer transaction, without effects from the inner transaction.
        await db.fetch_all('SELECT * FROM ...')
  6. Install database drivers for databases

    master

    Install the specific async driver required for your database type using the following extras:

    • PostgreSQL: databases[asyncpg] or databases[aiopg]
    • MySQL: databases[aiomysql] or databases[asyncmy]
    • SQLite: databases[aiosqlite]

    Note: If you are using synchronous SQLAlchemy functions (like engine.create_all()) or Alembic migrations, you must also install a synchronous driver: psycopg2 for PostgreSQL or pymysql for MySQL.

    $ pip install databases[asyncpg]
    $ pip install databases[aiopg]
    $ pip install databases[aiomysql]
    $ pip install databases[asyncmy]
    $ pip install databases[aiosqlite]
  7. Enable test isolation with force_rollback

    master

    To ensure strict test isolation, you can configure the Database instance to run all connections within a transaction that automatically rolls back when the database is disconnected. This prevents test data from persisting between test cases.

    When integrating with a web framework, use a conditional check to use a dedicated test database URL with force_rollback=True during testing.

    # For strict isolation in all cases
    database = Database(DATABASE_URL, force_rollback=True)
    
    # Typical pattern for web frameworks
    if TESTING:
        database = Database(TEST_DATABASE_URL, force_rollback=True)
    else:
        database = Database(DATABASE_URL)
  8. Configure Alembic for database migrations

    master

    Since databases uses SQLAlchemy core, you can use Alembic for migrations. Note that migrations will use a standard synchronous database driver rather than the async drivers supported by databases.

    1. Initialize Alembic

    pip install alembic
    alembic init migrations

    2. Update alembic.ini

    Remove the default sqlalchemy.url line from your alembic.ini file:

    sqlalchemy.url = driver://user:pass@localhost/dbname

    3. Configure migrations/env.py

    In your migrations/env.py file, set the sqlalchemy.url configuration key using your application's DATABASE_URL and assign your table metadata to target_metadata.

    # migrations/env.py
    from myapp.settings import DATABASE_URL
    from myapp.tables import metadata
    
    # The Alembic Config object.
    config = context.config
    
    # Configure Alembic to use our DATABASE_URL and our table definitions.
    config.set_main_option('sqlalchemy.url', str(DATABASE_URL))
    target_metadata = metadata
  9. Configure connection options for SSL and pooling

    master

    PostgreSQL and MySQL backends allow configuring SSL and connection pool sizes via the connection URL query parameters or as keyword arguments in the Database constructor. Common options include ssl and pool size settings like min_size and max_size.

    # Using URL query parameters
    database = Database('postgresql+asyncpg://localhost/example?ssl=true')
    database = Database('mysql+aiomysql://localhost/example?min_size=5&max_size=20')
    
    # Using keyword arguments
    database = Database('postgresql+asyncpg://localhost/example', ssl=True, min_size=5, max_size=20)
  10. Configure MySQL dialect for Alembic migrations

    master
    When using MySQL with Alembic, you may need to explicitly specify the pymysql dialect because the default MySQL dialect might not support Python 3. If you are using the databases.DatabaseURL type, you can adjust the dialect using the .replace() method.
  11. Use the Database class API

    master

    The Database class provides the following primary async methods for interacting with your database:

    • await database.connect(): Establishes the connection to the database.
    • await database.execute(query=...): Executes a single query (e.g., CREATE TABLE, INSERT).
    • await database.execute_many(query=..., values=...): Executes a query against multiple sets of values (bulk insert).
    • await database.fetch_all(query=...): Executes a query and returns all resulting rows.