Workflow Orchestration Library for Go

A Step is the fundamental building block of a workflow. It is a function that implements business logic by processing the current workflow state.

Each step:

1. Receives the current Run state (containing your business data).
2. Executes logic (e.g., database updates, API calls).
3. Returns the next Status to transition to, or an error to trigger a retry.

Steps are type-safe, using a ConsumerFunc signature that binds a specific data type and status type to the step logic.

type ConsumerFunc[Type any, Status StatusType] func(
    ctx context.Context,
    r *Run[Type, Status]
) (Status, error)

A Run is an individual instance of a workflow processing specific data. Calling Trigger() creates a new Run.

A Run contains your business data (Object), a ForeignID (your unique identifier for the business entity), and a RunID (unique to that specific run).

type Run[Type any, Status StatusType] struct {
    WorkflowName string
    ForeignID    string    // Your unique identifier for this business entity
    RunID        string    // Unique identifier for this specific run
    RunState     RunState  // Lifecycle state (Initiated, Running, Completed, etc.)
    Status       Status    // Your workflow-specific status
    Object       *Type     // Your business data
    CreatedAt    time.Time
    UpdatedAt    time.Time
}

The workflow system uses a hierarchical configuration model. Settings follow this priority (from lowest to highest):

1. Global Defaults: Set via workflow.WithDefaultOptions() during .Build(). These apply to all steps.
2. Step-Specific Overrides: Set via .WithOptions() on an individual step. These override any global defaults for that specific step.

This allows you to define a sane baseline for the whole workflow while fine-tuning critical or high-latency steps individually.

Workflow uses Go generics to provide compile-time guarantees for your workflow data. When defining a step function, the *workflow.Run[T, S] parameter ensures that r.Object is typed as your specific business object T, and the return value must match your status type S. This prevents runtime type mismatches during state transitions.

// Your IDE knows exactly what data flows where
func processPayment(ctx context.Context, r *workflow.Run[Order, OrderStatus]) (OrderStatus, error) {
    // r.Object is typed as *Order, OrderStatus is your enum
    // Compiler catches mismatches before they cause runtime errors
}

An Event is the mechanism by which workflow steps communicate. When a step completes and returns a new Status, the workflow persists the Run, publishes an event to the EventStreamer, and the next step's consumer receives that event to process the Run.

This architecture provides:

• Loose coupling: Steps do not call each other directly.
• Reliability: Events are persisted and retried on failure.
• Observability: All state changes are observable.
• Scalability: Steps can run on different machines.

type Event struct {
    ID        int64             // Unique event identifier
    ForeignID string            // Links to workflow instance
    Type      int               // Status the workflow moved to
    Headers   map[Header]string // Metadata
    CreatedAt time.Time         // Event timestamp
}

When designing systems that rely on hooks, be aware of the following constraints:

1. Eventual Consistency: Hooks are asynchronous and may experience delays.
2. No Ordering Guarantees: If multiple hooks are registered, they may execute in any order.
3. No Return Values: Hooks cannot modify the workflow outcome or return values to the workflow engine.
4. Memory Usage: Hook consumers are responsible for maintaining their own event processing state.
5. Error Isolation: Errors occurring within a hook do not cause the parent workflow to fail, though they may accumulate.

Workflows are implemented as state machines where States are your Status enum values, Transitions are defined by your step functions, and Data (the Object) flows through the states.

Unlike traditional state machines, Workflow state machines support cycles (returning to previous states), carry data that is modified during transitions, and are distributed and durable.

b := workflow.NewBuilder[Order, OrderStatus]("order-processing")

// Define valid transitions
b.AddStep(OrderCreated, processPayment, PaymentProcessed, PaymentFailed)
b.AddStep(PaymentProcessed, reserveInventory, InventoryReserved, InventoryFailed)
b.AddStep(InventoryReserved, fulfillOrder, OrderFulfilled)

// Cycles are allowed - orders can be retried
b.AddStep(PaymentFailed, retryPayment, PaymentProcessed, OrderCancelled)

Role-Based Scheduling ensures that only one instance of a specific process runs at a time in a distributed environment. Each step creates a unique role string following the pattern: workflow-name-status-consumer-shard-of-total-shards.

Sharding

To handle high-throughput steps, you can shard across multiple processes using WithOptions(workflow.ParallelCount(n)). This distributes events across shards using consistent hashing on the ForeignID.

b.AddStep(OrderCreated, processPayment, PaymentProcessed).
    WithOptions(workflow.ParallelCount(5)) // Run 5 parallel instances

Workflow is designed to be infrastructure agnostic by using pluggable adapters. You can start with in-memory adapters for development and switch to production-grade infrastructure (like Kafka, SQL, and Rink) by simply changing the arguments passed to Build().

// Development: Everything in-memory
```go
wf := b.Build(memstreamer.New(), memrecordstore.New(), memrolescheduler.New())

// Production: Battle-tested infrastructure

wf := b.Build(kafkastreamer.New(), sqlstore.New(), rinkrolescheduler.New())