SQLx: The Rust SQL Toolkit

repository·main·Indexed 12 days ago

https://github.com/transact-rs/sqlx

An async, pure Rust SQL crate featuring compile-time checked queries without a DSL. It supports PostgreSQL, MySQL, MariaDB, and SQLite, and is runtime-agnostic, supporting tokio, async-std, and actix.

Tokens
41.9K
Snippets
129
Records
180
Agent score
97%

What's inside SQLx

  1. Best practices for using multiple schemas in Postgres

    main

    When working with multiple schemas (e.g., public, accounts, payments) in a single database, it is recommended to use schema-qualified names (e.g., accounts.users instead of just users) in your SQL queries and migrations.

    Why avoid changing search_path? While you can change the search_path to include multiple schemas to avoid prefixes, this can lead to errors during migrations. For example, if search_path is set to public,accounts,payments, and a migrator attempts to reference a table unqualified, it may conflict with or fail to find the intended _sqlx_migrations table if multiple schemas contain one.

  2. Querying with SQLx: Prepared vs Unprepared

    main

    SQLx supports two types of queries:

    1. Prepared (Parameterized) Queries: These use a Query or QueryAs struct. They are cached, use binary communication for speed, and prevent SQL injection via parameters. Use these for most application logic.
    2. Unprepared (Simple) Queries: These are treated as a &str. They are intended for database commands that do not support prepared statements, such as PRAGMA, SET, or BEGIN.

    Finalizers:

    • execute(): Returns the number of affected rows and drops results.
    • fetch(): Returns a stream-like type that iterates through rows.
    • fetch_one(): Returns a single row (errors if zero or multiple rows found).
    • fetch_optional(): Returns an Option<Row> (useful when zero rows are expected).
    • fetch_all(): Returns a Vec<Row>.
  3. Best practices for multi-tenant schema management

    main

    When managing multiple schemas within a single database (multi-tenancy), it is recommended to use schema-qualified names (e.g., accounts.users instead of just users).

    Why avoid changing search_path? While it is tempting to change the connection's search_path (e.g., SET search_path TO public, accounts, payments) to allow unqualified table names, this can lead to confusing errors. For example, if the migrator for the main application attempts to reference a table unqualified, it may fail or conflict with the _sqlx_migrations tables generated in the different schemas.

  4. How SQLx compile-time checked queries work

    main

    SQLx is not an ORM. It does not provide a DSL for building queries. Instead, it uses macros (like query!) that take raw SQL strings.

    To provide compile-time safety, SQLx connects to your development database during the compilation process. The database itself verifies the SQL syntax and returns information about the expected types. This means:

    1. You can use any syntax supported by your specific database (including extensions).
    2. The level of verification depends on the information the database driver can retrieve about the query.
    3. You must have a running development database available at compile time (unless using offline mode).
  5. How default host resolution works in PostgreSQL

    main

    If the connection URL does not specify a hostname and the PGHOST environment variable is not set, SQLx looks for a Unix domain socket in standard locations:

    • /var/run/postgresql/.s.PGSQL.{port} (Debian)
    • /private/tmp/.s.PGSQL.{port} (macOS via Homebrew)
    • /tmp/.s.PGSQL.{port} (Default)

    Important: This resolution depends on the port being correct. If your Postgres instance uses a non-default port, you must specify the port in the URL or via PGPORT. If no Unix domain socket is found, localhost is assumed as the fallback.

  6. Understand SQLx's Minimum Supported Rust Version (MSRV)

    main

    SQLx's MSRV is the second-to-latest stable release as of the beginning of the current release cycle (0.x.0). It remains at that version until the next major release (0.{x + 1}.0). This ensures compatibility with Rust versions that are at least six weeks old, allowing time for packaging systems to update.

    Note: It is not recommended to install Rust via operating system packages as they are often significantly outdated.

  7. Automatic Test Database Management with #[sqlx::test]

    main

    When the migrate feature is enabled, #[sqlx::test] can automatically create isolated test databases for every annotated function. This ensures tests are isolated from one another.

    Supported Function Signatures

    To activate this feature, change your test function signature to one of the following:

    • async fn(Pool<DB>) -> Ret (Pools share a single connection limit)
    • async fn(PoolConnection<DB>) -> Ret
    • async fn(PoolOptions<DB>, impl ConnectOptions<DB>) -> Ret (Use this to create a custom Pool, e.g., to set max_connections(1) to avoid exceeding server limits during parallel tests).

    Database Requirements

    • Postgres/MySQL: Requires DATABASE_URL to be set in your environment or a .env file so the driver can manage test databases.
    • SQLite: Does not require DATABASE_URL. It defaults to target/sqlx/test-dbs/<path>.sqlite where <path> is the function path.

    Cleanup Behavior

    • Success: Test databases are automatically cleaned up.
    • Failure/Panic: Databases are left in place to facilitate debugging.
    • Disk Space: Previously created test databases are deleted the next time a test binary using #[sqlx::test] is run.
    use sqlx::{pool::PoolOptions, postgres::PgConnectOptions, Postgres};
    
    #[sqlx::test]
    async fn basic_test(
        pool_options: PoolOptions<Postgres>,
        connect_options: PgConnectOptions,
    ) -> sqlx::Result<()> {
        let pool = pool_options
            .max_connections(1)
            .connect_with(connect_options)
            .await?;
    
        sqlx::query("SELECT 1").execute(&pool).await?;
    
        Ok(())
    }
  8. Compile-time SQL verification with macros

    main

    SQLx provides the sqlx::query! and sqlx::query_as! macros to verify your SQL syntax and types at compile time.

    sqlx::query!

    Returns an anonymous record type where each SQL column is a field.

    • Requirement: The DATABASE_URL environment variable must be set at build time to a database with the correct schema.
    • Usage: Parameters must be provided all at once.

    sqlx::query_as!

    Identical to query!, but allows you to map the results into a named struct.

    Offline Mode

    To compile without a live database, you can use the sqlx CLI to enable "offline mode," which caches query analysis results in a file.

    // Using query! (Anonymous record)
    let countries = sqlx::query!(
        "SELECT country, COUNT(*) as count FROM users GROUP BY country WHERE organization = ?",
        organization
    )
    .fetch_all(&pool)
    .await?;
    // countries[0].country
    
    // Using query_as! (Named struct)
    struct Country { country: String, count: i64 }
    
    let countries = sqlx::query_as!(Country, 
        "SELECT country, COUNT(*) as count FROM users GROUP BY country WHERE organization = ?",
        organization
    )
    .fetch_all(&pool)
    .await?;
  9. Quickstart with SQLx

    main

    To get started with SQLx, you need to create a connection pool using the appropriate options struct for your database (e.g., PgPoolOptions for Postgres, MySqlPoolOptions for MySQL, or SqlitePoolOptions for SQLite). You can then use sqlx::query_as to execute parameterized queries and map results to types.

    Note: For MySQL/MariaDB, use ? as a placeholder instead of $1.

    use sqlx::postgres::PgPoolOptions;
    
    #[tokio::main]
    async fn main() -> Result<(), sqlx::Error> {
        // Create a connection pool
        let pool = PgPoolOptions::new()
            .max_connections(5)
            .connect("postgres://postgres:password@localhost/test").await?;
    
        // Make a simple query to return the given parameter
        let row: (i64,) = sqlx::query_as("SELECT $1")
            .bind(150_i64)
            .fetch_one(&pool).await?;
    
        assert_eq!(row.0, 150);
    
        Ok(())
    }
  10. Configure TLS support for secure connections

    main

    To communicate securely with database servers over untrusted networks, you must enable a TLS feature.

    Available TLS Features:

    • tls-native-tls: Uses the OS-native TLS capabilities (SecureTransport on macOS, SChannel on Windows, or OpenSSL on other platforms).
    • tls-rustls: Uses the cross-platform rustls library. Supports TLS 1.2 and 1.3.
      • tls-rustls-ring: The default rustls provider using ring. Has fewer build-time dependencies.
      • tls-rustls-aws-lc-rs: Uses the aws-lc-rs provider. Enables additional cipher suites but has more complex build requirements.

    Precedence and Troubleshooting:

    • If multiple TLS features are enabled, tls-native-tls takes precedence.
    • If you encounter HandshakeFailure errors while using tls-rustls, your database server may not support TLS 1.2 or 1.3. Switching to tls-native-tls often resolves this.
    • If your connection configuration requires a TLS upgrade but no TLS feature is enabled, the connection attempt will fail with an error.
    # Example: Enabling native-tls
    sqlx = { version = "0.9.0", features = ["runtime-tokio", "postgres", "tls-native-tls"] }
  11. Enable offline mode for SQLx queries

    main

    To build your project without an active database connection (useful for CI/CD), you can use "offline mode". This involves saving query metadata to a .sqlx directory.

    Steps to enable offline mode

    1. Save metadata: Run cargo sqlx prepare. This saves metadata to a .sqlx directory in your current directory. For workspaces, use the --workspace flag to generate a single directory at the root.
    2. Check in metadata: Commit the .sqlx directory to version control.
    3. Build: Use cargo build as usual; SQLx will now use the metadata instead of connecting to a database.

    Verification and CI

    Use cargo sqlx prepare --check (or --check --workspace) in CI environments. This command exits with a non-zero status if the .sqlx metadata is out of sync with your current database schema or queries.

    Advanced Configuration

    • Force offline mode: If a DATABASE_URL is present, SQLx will try to connect to a database by default. To prevent this and force offline mode, set the SQLX_OFFLINE environment variable to true (you can add this to your .env file).
    • Include test/feature-flagged queries: To ensure queries inside tests or behind feature flags are captured, pass arguments through to cargo: cargo sqlx prepare -- --all-targets --all-features
    # Save metadata for the current crate
    cargo sqlx prepare
    
    # Save metadata for the entire workspace
    cargo sqlx prepare --workspace
    
    # Check if metadata is out of date (use in CI)
    cargo sqlx prepare --check
    
    # Prepare including all targets and features
    cargo sqlx prepare -- --all-targets --all-features
  12. Configure the database connection URL

    main

    All SQLx CLI commands require a database connection URL. You can provide this in two ways:

    1. Use the --database-url command line option for individual commands.
    2. Set the DATABASE_URL environment variable or include it in a .env file in your current working directory.

    Example .env for Postgres:

    DATABASE_URL=postgres://postgres@localhost/my_database
    # Postgres
    DATABASE_URL=postgres://postgres@localhost/my_database