repository·main·Indexed 41 days ago
https://github.com/microsoft/duroxideA lightweight, embeddable durable execution runtime for Rust (v0.1.30) that allows async code to survive process crashes, restarts, and deployments by recording orchestration steps. It features a strict separation between deterministic orchestrations and non-deterministic activities, supporting patterns like function chaining, the Saga pattern for compensation, and sub-orchestrations.
The profiling setup consists of the following components:
Dockerfile: Contains the environment with Rust, perf, and flamegraph installed.docker-compose.yml: Provides an optional, simplified orchestration for the profiling environment.results/: The directory where generated SVG flamegraphs are stored (this directory is gitignored).RESULTS.md: A document containing the latest performance optimization results.For LLM Users:
If you are using an AI agent to analyze performance, refer to prompts/duroxide-profile.md for specific instructions on profiling and analysis guidance.
Durable execution allows your code to continue running through process crashes, restarts, and machine failures. It provides the illusion that code never stopped, even if it takes days or weeks to complete.
To achieve this, Duroxide uses a replay engine. For this to work, your orchestration code must be deterministic: given the same history of events, it must always make the same decisions. This allows Duroxide to recreate the state of your code by replaying the execution history after a restart.
A Provider is a storage backend used by Duroxide to persist orchestration state. It acts as a database adapter with specific semantics for durable execution.
Responsibilities of the Provider:
What the Provider is NOT:
Key Principle: The provider is "dumb storage." All orchestration logic is contained within the runtime.
Duroxide is a durable execution framework for Rust designed to build async workflows that survive crashes, restarts, and failures. It achieves this through automatic replay, state persistence, and recovery.
The ecosystem is organized into three distinct layers:
duroxide): The foundation providing the runtime, replay engine, a bundled SQLite provider, and the Client API.Provider trait. This allows you to swap storage (e.g., from SQLite to PostgreSQL) without changing your application logic.toygres).The Lightweight Filesystem Provider is a specialized Duroxide provider designed for environments where minimal code size, low memory footprint, and transactional safety are critical. It is optimized for single-runtime process ownership of the data directory.
The Management API distinguishes between removing an entire orchestration and trimming its history using two distinct concepts:
delete_instance, delete_instance_bulk): Permanently removes an orchestration instance and ALL its associated data (executions, history events, and queue messages). Use this to clean up terminated or failed orchestrations.prune_executions, prune_executions_bulk): Removes old executions from a long-running orchestration (e.g., ContinueAsNew chains) while keeping the instance alive. Use this to manage storage growth for active, long-running orchestrations without stopping them.Duroxide uses structured error types including ProviderError, ErrorDetails, and ClientError.
Note on Error Sources: In current versions, ProviderError may store the operation and message but might not preserve the full underlying error source chain (i.e., std::error::Error::source() may return None). When debugging, you may need to rely on the flattened message string.
Note on Error Classification: Some parts of the system (like the SQLite provider or client delete paths) may rely on substring matching in error messages (e.g., checking if a message contains "not found" or "database is locked") to classify errors. This behavior is implementation-dependent and may change between library versions.
Duroxide implements activity cancellation by using a cancel_requested flag directly on the activity's row in the worker queue. This replaces the previous ExecutionState model. The behavior depends on when the cancellation is requested:
ActivityExecute item, sees cancel_requested=true, and immediately acks the work item without executing it.cancel_requested=true. The worker signals the cancellation token to the activity handler. After a grace period, the worker aborts and acks.is_completion_already_in_history, cancelled_source_ids, or execution ID mismatch).Note on Retries: The system assumes one worker_queue row per scheduled activity ID. Retries occur via re-delivery of the same row. Cancellation of a scheduled activity ID cancels all its attempts.
The durable_async! macro implements an 'inline task scope' model.
ctx.begin_scope() to generate a unique scope_id and pushes it onto the OrchestrationContext scope stack.durable_async! block is automatically registered to the current scope_id via ctx.register_activity_in_scope(event_id).DurableFuture created by the macro is a 'loser' in a select2() operation, the orchestration engine retrieves all activity IDs associated with that scope_id and marks them as cancelled. This ensures that a group of sequential steps is cleaned up atomically if the group is no longer needed.In a cluster with multiple runtimes, settings are applied locally to each runtime. There is no global coordination for session capacity.
When a session migrates from Worker A to Worker B (e.g., because Worker A's lock expired), the session adopts the lock and idle characteristics of Worker B going forward. This means the session's lifecycle is governed by the settings of the worker currently holding it.
Setting force=true in delete_instance performs a database-only deletion. It does not send signals to running code or kill active Tokio tasks.
Key behaviors:
ctx.is_cancelled(). Well-behaved activities should check this signal to terminate gracefully.instance_locks to prevent a race condition where a completing turn recreates the deleted instance.