deadpool

repository·main·Indexed 23 days ago

https://github.com/deadpool-rs/deadpool

An asynchronous connection pooler for Rust. Version 0.13.0 provides a dead simple async pool with various implementations including deadpool-postgres for PostgreSQL, deadpool-diesel for database backends (SQLite, Postgres, MySQL), deadpool-lapin for RabbitMQ/AMQP, deadpool-libsql for libsql, and deadpool-memcached for Memcached.

Tokens
33.2K
Snippets
69
Records
164
Agent score
78%

What's inside deadpool

  1. Key features and design principles of Deadpool

    main

    Deadpool is designed with several core principles:

    • Executor Agnostic: It is compatible with any executor because it uses the Drop trait to return objects to the pool and checks health upon retrieval rather than using background tasks.
    • Runtime Behavior: Creating a pool never fails. Errors only occur when calling Pool::get(). This ensures that temporary database unavailability doesn't cause application crashes during startup.
    • Performance: Uses minimal locking; a single Mutex is used for returning objects, and a Semaphore is used for retrieval to minimize contention.
    • Extensibility: Supports post_create, pre_recycle, and post_recycle hooks for custom logic.
    • Observability: Provides Metrics for objects and a status() method for pool insights.
    • Dynamic Resizing: The pool can be grown or shrunk at runtime.
  2. Use Deadpool for SQLite

    main

    Deadpool for SQLite provides an asynchronous connection pool for rusqlite. Because rusqlite is a synchronous library, this crate provides a wrapper that ensures connections are used correctly inside a separate thread via the interact() method. This allows you to perform blocking SQLite operations within an async context safely.

    use deadpool_sqlite::{Config, Runtime};
    
    #[tokio::main]
    async fn main() -> Result<(), Box<dyn std::error::Error>> {
        let mut cfg = Config::new("db.sqlite3");
        let pool = cfg.create_pool(Runtime::Tokio1).unwrap();
        let conn = pool.get().await.unwrap();
        let result: i64 = conn
            .interact(|conn| {
                let mut stmt = conn.prepare("SELECT 1")?;
                let mut rows = stmt.query([])?;
                let row = rows.next()?.unwrap();
                row.get(0)
            })
            .await??;
        assert_eq!(result, 1);
        Ok(())
    }
  3. What is deadpool-sync and when should I use it?

    main

    deadpool-sync provides helpers for creating connection pools for objects that do not support asynchronous operations and must be executed within a thread.

    Important Usage Note: This crate is primarily intended for library authors developing deadpool-* adapter crates. If you are building a binary application or using other libraries, you should generally not use deadpool-sync directly; instead, use the re-exports provided by the specific deadpool-* crate you are using.

  4. What is the Deadpool runtime abstraction?

    main

    The deadpool-runtime crate provides a Runtime enum designed to target multiple asynchronous runtimes. It is a lightweight abstraction that avoids boxed futures and only implements the specific functionality required by the deadpool-* ecosystem crates.

    Note for users: This crate is primarily intended for developers creating new deadpool-* adapter crates. If you are building a standard library or a binary application, you should generally not use this crate directly; instead, use the re-exports provided by the specific deadpool-* crate you are using (e.g., deadpool-postgres).

  5. How managed pools (connection pools) work

    main

    A managed pool is the standard way to use Deadpool for connection pooling. It requires implementing the deadpool::managed::Manager trait for your resource type. The manager defines how to create new objects and how to recycle existing ones (e.g., checking if a connection is still alive).

    Key components of the Manager trait:

    • type Type: The type of object being pooled.
    • type Error: The error type returned by the manager.
    • async fn create(&self) -> Result<Self::Type, Self::Error>: Logic to instantiate a new resource.
    • async fn recycle(&self, object: &mut Self::Type, metrics: &deadpool::managed::Metrics) -> deadpool::managed::RecycleResult<Self::Error>: Logic to validate a resource before it is reused.
    use deadpool::managed;
    
    #[derive(Debug)]
    enum Error { Fail }
    
    struct Computer {}
    
    impl Computer {
        async fn get_answer(&self) -> i32 {
            42
        }
    }
    
    struct Manager {}
    
    impl managed::Manager for Manager {
        type Type = Computer;
        type Error = Error;
    
        async fn create(&self) -> Result<Computer, Error> {
            Ok(Computer {})
        }
    
        async fn recycle(&self, _: &mut Computer, _: &managed::Metrics) -> managed::RecycleResult<Error> {
            Ok(())
        }
    }
    
    type Pool = managed::Pool<Manager>;
    
    #[tokio::main]
    async fn main() {
        let mgr = Manager {};
        let pool = Pool::builder(mgr).build().unwrap();
        let mut conn = pool.get().await.unwrap();
        let answer = conn.get_answer().await;
        assert_eq!(answer, 42);
    }
  6. How to handle connection recycling methods

    main

    Deadpool uses a recycling method to check if a connection is still valid before returning it from the pool.

    • RecyclingMethod::Fast (Default): Relies on tokio_postgres::Client::is_closed. In rare cases of unreliable networks, this might return a connection that has actually disconnected but hasn't been noticed by the client yet.
    • RecyclingMethod::Verified: Performs a test query to verify the connection is alive. This is slower but more reliable.

    You can set this via ManagerConfig::recycling_method or, if using the config crate, via the environment variable PG__MANAGER__RECYCLING_METHOD=Verified.

  7. How unmanaged pools work

    main

    An unmanaged pool is used when you want to pool objects without implementing a Manager trait. It is slightly faster than a managed pool because it skips the create and recycle lifecycle steps, leaving object management entirely to the user. You initialize an unmanaged pool by providing a collection of pre-existing objects.

    use deadpool::unmanaged::Pool;
    
    struct Computer {}
    
    impl Computer {
        async fn get_answer(&self) -> i32 {
            42
        }
    }
    
    #[tokio::main]
    async fn main() {
        let pool = Pool::from(vec![
            Computer {},
            Computer {},
        ]);
        let s = pool.get().await.unwrap();
        assert_eq!(s.get_answer().await, 42);
    }
  8. Compare managed and unmanaged pools

    main

    Deadpool offers two distinct pool types:

    Managed Pool (deadpool::managed::Pool)

    • Behavior: Automatically creates and recycles objects as needed.
    • Use Case: Ideal for database connection pools where the pool manages the lifecycle of connections.
    • Requirement: Must enable the managed feature in Cargo.toml.

    Unmanaged Pool (deadpool::unmanaged::Pool)

    • Behavior: Objects must be created by the user and added to the pool manually, or the pool can be initialized from an existing collection of objects.
    • Use Case: Useful when you already have a set of objects you want to distribute or manage.
    • Requirement: Must enable the unmanaged feature in Cargo.toml.
  9. Perform SQLite operations with interact()

    main

    To execute SQLite queries using a connection from the pool, use the interact() method. This method takes a closure containing your synchronous rusqlite code and executes it on a separate thread to avoid blocking the async runtime. The result of the closure is returned when you .await the interact() call.

    let result: i64 = conn
        .interact(|conn| {
            let mut stmt = conn.prepare("SELECT 1")?;
            let mut rows = stmt.query([])?;
            let row = rows.next()?.unwrap();
            row.get(0)
        })
        .await??;
  10. Configure Deadpool for Redis

    main

    To use deadpool-redis, you can create a connection pool using a Config object. The configuration can be initialized from a single URL or via environment variables. Once the pool is created, you can retrieve connections using .get().await and execute commands using the re-exported redis crate API.

    use std::env;
    use deadpool_redis::{redis::{cmd, FromRedisValue}, Config, Runtime};
    
    #[tokio::main]
    async fn main() {
        let mut cfg = Config::from_url(env::var("REDIS__URL").unwrap());
        let pool = cfg.create_pool(Some(Runtime::Tokio1)).unwrap();
        {
            let mut conn = pool.get().await.unwrap();
            cmd("SET")
                .arg(&["deadpool/test_key", "42"])
                .query_async::<()>(&mut conn)
                .await.unwrap();
        }
        {
            let mut conn = pool.get().await.unwrap();
            let value: String = cmd("GET")
                .arg(&["deadpool/test_key"])
                .query_async(&mut conn)
                .await.unwrap();
            assert_eq!(value, "42".to_string());
        }
    }