psqlpy Documentation

repository·main·Indexed 18 days ago

https://github.com/psqlpy-python/psqlpy

psqlpy is an asynchronous PostgreSQL driver for Python implemented in Rust, designed for high performance and type safety. It provides core components including ConnectionPool, ConnectionPoolBuilder, Connection, Transaction, Cursor, PreparedStatement, and Listener. The library includes performance benchmarks comparing its throughput (RPS) against asyncpg and psycopg, demonstrating significant gains particularly in external database environments.

Tokens
34.8K
Snippets
105
Records
142
Agent score
62%

What's inside psqlpy

  1. What is PSQLPy?

    main

    PSQLPy is an asynchronous PostgreSQL driver for Python implemented in Rust. It is designed to be a high-performance alternative inspired by Psycopg3 and AsyncPG.

    Key features include:

    • High Performance: Faster PostgreSQL interactions (provided queries are optimized).
    • Type Safety: Full support for MyPy and other Python type checkers, ensuring returned types match specifications.
    • Low Abstraction Overhead: Uses simple classes that map directly to PostgreSQL objects like transactions and cursors.
    • Comprehensive Documentation: Every component includes docstrings to reduce the need for external documentation lookups.
  2. Overview of PSQLPy core components

    main

    PSQLPy is composed of several key abstractions for managing PostgreSQL interactions:

    • ConnectionPool: Manages a collection of connections and provides them upon request.
    • ConnectionPoolBuilder: A chainable builder used to configure and initialize a ConnectionPool step-by-step.
    • Connection: Represents a single database connection. You can obtain one from a ConnectionPool or create one directly using the connect method.
    • Transaction: Represents a database transaction, which can be initiated from a Connection.
    • Cursor: An object used to execute queries and fetch results. Cursors can be created from a Connection, a Transaction, or a PreparedStatement.
    • PreparedStatement: Represents a PostgreSQL prepared statement for optimized query execution.
    • Listener: An object used to interface with PostgreSQL's LISTEN/NOTIFY functionality. Listeners can be created from a ConnectionPool.
    • QueryResult: Represents a collection of results returned from the database.
    • SingleQueryResult: Represents a single result row or object returned from the database.
    • Exceptions: The library provides custom exception types for handling database-specific errors.
  3. Understand PSQLPy performance benchmarks

    main

    PSQLPy benchmarks compare its Requests Per Second (RPS) against AsyncPG and PsycoPG 3 using the AioHTTP web framework. The benchmarks are categorized into two environments:

    1. Local Database: Used when the application and database reside on the same server. Benchmarks use 5 connections in a connection pool and 10 processes making requests.
    2. External Database: Used when the application and database are on different servers (typical production setup). Benchmarks use 40 connections and 100 processes. This environment typically shows the most significant performance gains.

    Key Performance Insights:

    • Same-server setup: PSQLPy provides approximately a 10% performance improvement over AsyncPG and PsycoPG 3.
    • Different-server setup: PSQLPy can provide up to a 3x boost in performance compared to other drivers.
  4. Important: Statement Preparation and PGBouncer Compatibility

    main

    By default, PSQLPy prepares all SQL statements. While this can improve performance, it may cause application errors when using connection poolers like PGBouncer in certain modes:

    • Transaction Pooling Mode
    • Statement Pooling Mode

    If you are using PGBouncer in either of these modes, you must disable statement preparation to prevent breaking your application. Detailed instructions on how to disable this behavior are available in subsequent sections of the documentation.

  5. Handle PSQLPy exceptions by component type

    main

    PSQLPy uses a hierarchical exception structure to allow developers to define specific error-handling behaviors. Exceptions are categorized into subclasses based on the component they originate from. This allows you to catch errors specifically related to connection management, transaction state, or cursor operations.

    Exceptions are grouped into the following subclasses:

    • ConnectionPool exceptions
    • Connection exceptions
    • Transaction exceptions
    • Cursor exceptions
  6. Understand the core components of PSQLPy

    main

    PSQLPy is built around several key abstractions that manage the lifecycle of a database interaction. Understanding these components is essential for managing connections and executing queries:

    • Connection pool: The primary entry point and main object in the library. It manages the lifecycle of connections, including initialization, creation, and storage. It must be started before any other operations.
    • Connection: Represents a single active database connection, typically retrieved from a Connection pool.
    • Transaction: Represents a database transaction, which is created from a Connection.
    • Cursor: Represents a database cursor, which is created from a Transaction.
    • Results: The data structure representing the data returned from the driver after executing queries.
    • Exceptions: Custom exceptions used for error handling (documentation for these is currently in development).
  7. Use the Listener object for PostgreSQL LISTEN/NOTIFY

    main

    The Listener object provides access to PostgreSQL's LISTEN and NOTIFY functionality. You can interact with notifications using two distinct patterns:

    1. Background Task Pattern: Register asynchronous callbacks for specific channels. The listen() method starts a non-blocking background task in the Rust event loop to handle incoming notifications.
    2. Async Iterator Pattern: Iterate directly over the Listener object to receive ListenerNotificationMsg objects as they arrive.

    To use either pattern, you must first create a Listener from a ConnectionPool and call await listener.startup().

    from psqlpy import ConnectionPool, Listener
    
    db_pool = ConnectionPool(dsn="postgres://postgres:postgres@localhost:5432/postgres")
    
    # Pattern 1: Background Task (Callbacks)
    async def main_callback():
        listener: Listener = db_pool.listener()
        await listener.startup()
        await listener.add_callback(channel="my_channel", callback=my_callback_func)
        listener.listen()
    
    # Pattern 2: Async Iterator
    async def main_iterator():
        listener: Listener = db_pool.listener()
        await listener.startup()
        async for listener_msg in listener:
            print(listener_msg)
  8. How Cursor objects work in PSQLPy

    main

    A Cursor object represents a server-side cursor in PostgreSQL.

    Important Lifecycle Note: Cursors always live inside a transaction. If you do not explicitly begin a transaction, one will be opened automatically to support the cursor.

    There are three primary ways to use a cursor:

    1. Pre-Initialization: Pass the query and parameters to connection.cursor() and call await cursor.start().
    2. Post-Initialization: Create a blank cursor with connection.cursor() and then call await cursor.execute(...) with the query and parameters.
    3. Async Context Manager: Use async with connection.cursor(...) as cursor: to handle the lifecycle automatically.
    # Async Context Manager pattern
    async with connection.cursor(
        querystring="SELECT * FROM users WHERE id > $1",
        parameters=[100],
        array_size=10,
    ) as cursor:
        result: QueryResult = await cursor.fetchone()
  9. How QueryResult and SingleQueryResult work

    main

    PSQLPy uses two primary result types to return data from the database to Python:

    1. QueryResult: Returned when a query produces multiple rows. Methods on this object return collections (like lists) of data.
    2. SingleQueryResult: Returned when a query is expected to produce exactly one row (e.g., via fetch_row). Methods on this object return a single data item (like a dict or a single class instance).

    Both result types support converting data into dictionaries, tuples, custom Python classes, or using a custom row_factory.