Underway

repository·main·Indexed 41 days ago

https://github.com/maxcountryman/underway

A durable background workflow engine for Rust built on top of PostgreSQL. Underway allows developers to model complex, multi-step business processes as typed workflows that are resilient to crashes and restarts by persisting state and side-effect intentions directly in the database. It features a builder for sequential steps, a trait-based Activity system for replay-safe side effects, and the ability to enqueue or schedule workflows atomically within SQL transactions.

Tokens
11.3K
Snippets
27
Records
35
Agent score
48%

What's inside Underway

  1. Overview of Underway

    main
    Underway is a library for running durable background workflows using PostgreSQL as the orchestration and queueing layer. It allows you to model business logic as typed Rust steps, ensuring that workflow progress and side-effect intentions are persisted. This enables automatic recovery from restarts, crashes, or deployments without requiring an external broker.
  2. Implement durable side effects with Activities

    main

    To perform side effects that are replay-safe and durable, implement the Activity trait. Activities are registered with the workflow via .activity(). Inside a workflow step, you can interact with activities using:

    • InvokeActivity::call: Executes an activity and waits for its result.
    • InvokeActivity::emit: Triggers an activity (fire-and-forget style) without waiting for a return value.

    Activities must define a NAME, an Input type, and an Output type. Errors within an activity can be categorized as retryable using ActivityError::retryable to signal to the runtime that the task should be attempted again.

    use serde::{Deserialize, Serialize};
    use sqlx::PgPool;
    use underway::{Activity, ActivityError, InvokeActivity, Transition, Workflow};
    
    #[derive(Clone)]
    struct LookupEmail {
        pool: PgPool,
    }
    
    impl Activity for LookupEmail {
        const NAME: &'static str = "lookup-email";
    
        type Input = i64;
        type Output = String;
    
        async fn execute(&self, user_id: Self::Input) -> underway::activity::Result<Self::Output> {
            let email = sqlx::query_scalar::<_, String>("select concat('user-', $1::text, '@example.com')")
                .bind(user_id)
                .fetch_one(&self.pool)
                .await
                .map_err(|err| ActivityError::retryable("db_error", err.to_string()))?;
    
            Ok(email)
        }
    }
    
    struct TrackSignupMetric;
    
    impl Activity for TrackSignupMetric {
        const NAME: &'static str = "track-signup-metric";
    
        type Input = String;
        type Output = ();
    
        async fn execute(&self, email: Self::Input) -> underway::activity::Result<Self::Output> {
            println!("tracking signup metric for {email}");
            Ok(())
        }
    }
    
    #[derive(Deserialize, Serialize)]
    struct Signup {
        user_id: i64,
    }
    
    #[tokio::main]
    async fn main() -> Result<(), Box<dyn std::error::Error>> {
        let pool = PgPool::connect(&std::env::var("DATABASE_URL")?).await?;
        underway::run_migrations(&pool).await?;
    
        let workflow = Workflow::builder()
            .activity(LookupEmail { pool: pool.clone() })
            .activity(TrackSignupMetric)
            .step(|mut cx, Signup { user_id }| async move {
                let email: String = LookupEmail::call(&mut cx, &user_id).await?;
                TrackSignupMetric::emit(&mut cx, &email).await?;
                Transition::complete()
            })
            .name("signup-side-effects")
            .pool(pool)
            .build()
            .await?;
    
        workflow.enqueue(&Signup { user_id: 42 }).await?;
        workflow.runtime().run().await?;
        Ok()
    }
  3. Build and run a typed workflow

    main

    You can define a multi-step workflow using the Workflow::builder(). Each step is defined using .step(), which takes a closure that receives a context and an input type, returning a Transition. Transitions can move to the next step using Transition::next(input) or finish the workflow using Transition::complete(). Once built, you can enqueue an initial input using .enqueue() and start the execution engine with .runtime().

    use serde::{Deserialize, Serialize};
    use sqlx::PgPool;
    use underway::{Transition, Workflow};
    
    #[derive(Deserialize, Serialize)]
    struct ResizeImage {
        asset_id: i64,
    }
    
    #[derive(Deserialize, Serialize)]
    struct PublishImage {
        object_key: String,
    }
    
    #[tokio::main]
    async fn main() -> Result<(), Box<dyn std::error::Error>> {
        let pool = PgPool::connect(&std::env::var("DATABASE_URL")?).await?;
        underway::run_migrations(&pool).await?;
    
        let workflow = Workflow::builder()
            .step(|_cx, ResizeImage { asset_id }| async move {
                let object_key = format!("images/{asset_id}.webp");
                Transition::next(PublishImage { object_key })
            })
            .step(|_cx, PublishImage { object_key }| async move {
                println!("Publishing {object_key}");
                Transition::complete()
            })
            .name("image-pipeline")
            .pool(pool)
            .build()
            .await?;
    
        workflow.enqueue(&ResizeImage { asset_id: 42 }).await?;
    
        let runtime_handle = workflow.runtime().start();
        runtime_handle.shutdown().await?;
        Ok()
    }
  4. Enqueue and schedule workflows atomically in a transaction

    main

    Underway allows you to enqueue or schedule workflows within an existing SQL transaction using the *_using API. This ensures that your business logic updates (e.g., updating a database row) and the workflow trigger (e.g., enqueuing a cleanup task) succeed or fail together.

    • enqueue_using(&mut *tx, &input): Adds a workflow to the queue within the transaction.
    • schedule_using(&mut *tx, &cron_expression, &input): Schedules a workflow to run at a specific time (using cron syntax) within the transaction.
    use serde::{Deserialize, Serialize};
    use sqlx::PgPool;
    use underway::{Transition, Workflow};
    
    #[derive(Deserialize, Serialize)]
    struct TenantCleanup {
        tenant_id: i64,
    }
    
    #[tokio::main]
    async fn main() -> Result<(), Box<dyn std::error::Error>> {
        let pool = PgPool::connect(&std::env::var("DATABASE_URL")?).await?;
        underway::run_migrations(&pool).await?;
    
        let workflow = Workflow::builder()
            .step(|_cx, TenantCleanup { tenant_id }| async move {
                println!("Running cleanup for tenant {tenant_id}");
                Transition::complete()
            })
            .name("tenant-cleanup")
            .pool(pool.clone())
            .build()
            .await?;
    
        let nightly = "0 2 * * *[UTC]".parse()?;
        let tenant_id = 7;
    
        let mut tx = pool.begin().await?;
    
        sqlx::query("update app_tenant set cleanup_enabled = true where id = $1")
            .bind(tenant_id)
            .execute(&mut *tx)
            .await?;
    
        let input = TenantCleanup { tenant_id };
        workflow.enqueue_using(&mut *tx, &input).await?;
        workflow.schedule_using(&mut *tx, &nightly, &input).await?;
    
        tx.commit().await?;
        Ok()
    }
  5. How the Scheduler singleton works

    main

    The Scheduler ensures that schedules are dispatched at most once per queue using PostgreSQL advisory locks.

    1. When run() is called, it attempts to acquire an advisory lock named {queue_name}-scheduler.
    2. If the lock is already held by another process/instance, run() returns Ok(()) immediately without starting the loop.
    3. If the lock is acquired, the scheduler enters a loop that:
      • Fetches the current schedule from the database.
      • Calculates the time until the next execution.
      • Sleeps until that time (or until a shutdown signal is received).
      • Enqueues the task.
    4. The lock is released when the scheduler loop terminates or is interrupted.
  6. Use Activities for durable side effects

    main

    For side effects that need to be durable and replay-safe (like sending emails or calling external APIs), use Activities.

    1. Implement the Activity trait: Define a NAME, Input type, and Output type. The execute method handles the logic and returns an underway::activity::Result.
    2. Register the activity: Add it to the Workflow::builder using .activity(YourActivity).
    3. Invoke the activity: Inside a workflow step, use InvokeActivity::call (request/response, suspends step until done) or InvokeActivity::emit (fire-and-forget).

    Activities allow you to handle errors specifically as retryable or fatal.

    use serde::{Deserialize, Serialize};
    use sqlx::PgPool;
    use underway::{Activity, ActivityError, InvokeActivity, Transition, Workflow};
    
    struct ResolveEmail;
    
    impl Activity for ResolveEmail {
        const NAME: &'static str = "resolve-email";
        type Input = i64;
        type Output = String;
    
        async fn execute(&self, user_id: Self::Input) -> underway::activity::Result<Self::Output> {
            if user_id <= 0 {
                return Err(ActivityError::fatal("invalid_user", "user_id must be positive"));
            }
            Ok(format!("{user_id}@example.com"))
        }
    }
    
    #[derive(Deserialize, Serialize)]
    struct Input { user_id: i64 }
    
    # async fn run_with_activity(pool: PgPool) -> Result<(), Box<dyn std::error::Error>> {
    let workflow = Workflow::builder()
        .activity(ResolveEmail)
        .step(|mut cx, Input { user_id }| async move {
            let _email: String = ResolveEmail::call(&mut cx, &user_id).await?;
            Transition::complete()
        })
        .name("resolve-email-workflow")
        .pool(pool)
        .build()
        .await?;
    
    workflow.runtime().run().await?;
    # Ok(())
    //}
  7. How the Runtime orchestrates workflows

    main

    The Runtime is the high-level entrypoint for executing workflows. It manages the coordination between three critical components:

    1. Workflow Worker: Processes workflow steps.
    2. Workflow Scheduler: Manages the scheduling of workflow tasks.
    3. Activity Worker: Handles the execution of activities called by or emitted from workflows.

    This orchestration is necessary because workflow steps often suspend while waiting for an activity to complete. The Runtime ensures that both the workflow execution and the activity execution run concurrently so that suspended steps can be resumed once their requested activities finish.

    use serde::{Deserialize, Serialize};
    use sqlx::PgPool;
    use underway::{Activity, ActivityError, InvokeActivity, Transition, Workflow};
    
    #[derive(Deserialize, Serialize)]
    struct FetchUser {
        user_id: i64,
    }
    
    struct LookupEmail;
    
    impl Activity for LookupEmail {
        const NAME: &'static str = "lookup-email";
    
        type Input = i64;
        type Output = String;
    
        async fn execute(&self, user_id: Self::Input) -> underway::activity::Result<Self::Output> {
            if user_id <= 0 {
                return Err(ActivityError::fatal(
                    "invalid_user",
                    "user_id must be positive",
                ));
            }
    
            Ok(format!("user-{user_id}@example.com"))
        }
    }
    
    #[tokio::main]
    async fn main() -> Result<(), Box<dyn std::error::Error>> {
        let pool = PgPool::connect(&std::env::var("DATABASE_URL")?).await?;
        let workflow = Workflow::builder()
            .activity(LookupEmail)
            .step(|mut cx, FetchUser { user_id }| async move {
                let email: String = LookupEmail::call(&mut cx, &user_id).await?;
                println!("Got email {email}");
                Transition::complete()
            })
            .name("lookup-email-workflow")
            .pool(pool)
            .build()
            .await?;
    
        workflow.runtime().run().await?;
        Ok()
    }
  8. Core Underway Concepts

    main

    Underway is built on several interconnected abstractions:

    • Tasks: The lowest-level unit of work with defined input and behavior.
    • Workflows: A sequence of Task steps where each step provides input to the next.
    • Runtime: The execution engine that orchestrates workers, schedulers, and activity-workers to resume suspended workflows.
    • Activities: Handlers for durable side effects invoked from within workflow steps.
    • Queues: The interface for managing the lifecycle (enqueuing/dequeuing) of tasks in the database.
    • Workers: The components that poll queues and execute the task logic.
  9. Build and run a basic Workflow

    main

    A Workflow is a series of sequential steps. You can build one using Workflow::builder(), defining steps that receive input and return a Transition.

    • Transition::complete() signals the end of the workflow.
    • Transition::next(input) signals the workflow should proceed to the next step using the provided input.

    Once built, you can enqueue an initial input to start the workflow and use the runtime() to execute it.

    use serde::{Deserialize, Serialize};
    use sqlx::PgPool;
    use underway::{Transition, Workflow};
    
    #[derive(Deserialize, Serialize)]
    struct ResizeImage { asset_id: i64 }
    
    #[derive(Deserialize, Serialize)]
    struct PublishImage { object_key: String }
    
    # async fn run_workflow(pool: PgPool) -> Result<(), Box<dyn std::error::Error>> {
    let workflow = Workflow::builder()
        .step(|_cx, ResizeImage { asset_id }| async move {
            let object_key = format!("images/{asset_id}.webp");
            Transition::next(PublishImage { object_key })
        })
        .step(|_cx, PublishImage { object_key }| async move {
            println!("Publishing {object_key}");
            Transition::complete()
        })
        .name("image-pipeline")
        .pool(pool)
        .build()
        .await?;
    
    workflow.enqueue(&ResizeImage { asset_id: 42 }).await?;
    workflow.runtime().run().await?;
    # Ok(())
    //}
  10. Run Underway migrations

    main

    Before using Underway, you must run its migrations to set up the necessary database schema. The run_migrations function creates a dedicated underway schema and applies migrations within it. This ensures that Underway's internal tables are isolated from your application's schema.

    To use this, pass a sqlx::PgPool or any type that implements Acquire for Postgres to the function.

    use sqlx::PgPool;
    
    # async fn setup() -> Result<(), Box<dyn std::error::Error>> {
        let pool = PgPool::connect(&std::env::var("DATABASE_URL")?).await?;
        underway::run_migrations(&pool).await?;
        Ok(())
    //}
  11. Enqueue and schedule workflows within a transaction

    main

    To ensure atomicity between your application state and your background workflows, Underway provides *_using methods. This allows you to enqueue a task or schedule a cron job as part of a single Postgres transaction.

    • workflow.enqueue_using(&mut transaction, &input): Adds a task to the queue immediately upon commit.
    • workflow.schedule_using(&mut transaction, &cron_string, &input): Schedules a task to run at a specific time (using cron syntax) upon commit.
    use serde::{Deserialize, Serialize};
    use sqlx::PgPool;
    use underway::{Transition, Workflow};
    
    #[derive(Deserialize, Serialize)]
    struct TenantCleanup { tenant_id: i64 }
    
    # async fn transactional_enqueue(pool: PgPool) -> Result<(), Box<dyn std::error::Error>> {
    let workflow = Workflow::builder()
        .step(|_cx, TenantCleanup { tenant_id }| async move {
            println!("Running cleanup for tenant {tenant_id}");
            Transition::complete()
        })
        .name("tenant-cleanup")
        .pool(pool.clone())
        .build()
        .await?;
    
    let nightly = "0 2 * * *[UTC]".parse()?;
    let tenant_id = 7;
    let input = TenantCleanup { tenant_id };
    
    let mut tx = pool.begin().await?;
    
    // Perform app logic
    sqlx::query("update app_tenant set cleanup_enabled = true where id = $1")
        .bind(tenant_id)
        .execute(&mut *tx)
        .await?;
    
    // Enqueue/Schedule as part of the same transaction
    workflow.enqueue_using(&mut *tx, &input).await?;
    workflow.schedule_using(&mut *tx, &nightly, &input).await?;
    
    tx.commit().await?;
    # Ok(())
    //}
  12. Configure the PostgreSQL service via Docker Compose

    main

    The underway project uses a PostgreSQL database. You can use the provided docker-compose.yaml to spin up a local instance. The service is named postgres and uses the postgres:18.1 image.

    By default, the following environment variables and port mappings are configured:

    • User: postgres
    • Password: postgres
    • Database Name: underway
    • Port: 5432 (mapped from host to container)
    services:
      postgres:
        image: postgres:18.1
        environment:
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: postgres
          POSTGRES_DB: underway
        ports:
          - "5432:5432"