DOCSEARCHbyAlgolia
Submit Docs

Workflow Core Documentation

repository·master·Indexed 26 days ago

https://github.com/danielgerlag/workflow-core

A lightweight, embeddable workflow engine for .NET Standard designed for long-running processes with state tracking. It features a pluggable architecture for persistence (supporting MongoDB, SQL Server, PostgreSQL, etc.) and concurrency. The engine supports defining workflows via JSON or YAML using WorkFlowCore.DSL, implementing Saga transactions with compensation, and configuring error handling. Extensions include WorkflowCore.AI.AzureFoundry for agentic workflows, LLM chat completions, and RAG, as well as WorkflowCore.Users for human-in-the-loop tasks.

Tokens
51.7K
Snippets
174
Records
219
Agent score
89%

What's inside Workflow Core

  1. Understand the enhanced test log format

    master

    When using enhanced reporting, the GitHub Actions console logs will transition from a simple summary to a detailed per-test breakdown.

    Enhanced log features include:

    • Individual execution status (Passed/Failed) per test case.
    • Execution duration (e.g., [2 ms]).
    • Detailed error messages (e.g., Assert.Equal() Failure).
    • Full stack traces for failed tests.
    • A final Test Run Summary containing totals for Passed, Failed, and Skipped tests.

  2. Understand Workflow Core performance characteristics

    master

    Workflow Core performance varies based on hardware configuration and concurrency levels. Performance testing was conducted using a single node with the MemoryPersistenceProvider.

    Key findings from testing version 3.7.0 indicate:

    • Throughput (Workflows per second): Scales with hardware. On a 32 vCPU configuration, throughput can reach approximately 529 workflows per second. On an 8 vCPU configuration, it reaches approximately 341 workflows per second.
    • Latency: Latency (Mean, P50, and P99) increases as the number of concurrent workflow copies increases. Higher hardware configurations (e.g., 32 vCPU) generally maintain lower latency compared to lower configurations (e.g., 8 vCPU) under the same load.

  3. Available Persistence Providers

    master

    Workflow Core requires persistence to store workflow state between steps. While MemoryPersistenceProvider is the default (intended for testing/demo), production environments should use one of the following available NuGet packages:

    • MongoDB
    • SQL Server
    • PostgreSQL
    • Sqlite
    • Amazon DynamoDB
    • Cosmos DB
    • Azure Table Storage
    • Redis
    • Oracle

  5. Define workflows using JSON or YAML

    master

    You can define workflows externally using JSON or YAML formats. To use this feature, you must install the WorkFlowCore.DSL NuGet package. Each definition requires an Id, a Version, and a list of Steps containing Id, StepType (the fully qualified type name), and optionally NextStepId.

    {
  "Id": "HelloWorld",
  "Version": 1,
  "Steps": [
    {
      "Id": "Hello",
      "StepType": "MyApp.HelloWorld, MyApp",
      "NextStepId": "Bye"
    },
    {
      "Id": "Bye",
      "StepType": "MyApp.GoodbyeWorld, MyApp"
    }
  ]
}
    Id: HelloWorld
Version: 1
Steps:
- Id: Hello
  StepType: MyApp.HelloWorld, MyApp
  NextStepId: Bye
- Id: Bye
  StepType: MyApp.GoodbyeWorld, MyApp

  6. Use WorkflowCore.Testing with xUnit

    master

    To write tests using xUnit:

    1. Create a class that inherits from WorkflowTest<TWorkflow, TData>.
    2. Call the Setup() method in the class constructor.
    3. Use helper methods such as StartWorkflow(), WaitForWorkflowToComplete(), GetStatus(), GetData(), and UnhandledStepErrors to assert workflow behavior.

    Available helper methods:

    • StartWorkflow()
    • WaitForWorkflowToComplete()
    • WaitForEventSubscription()
    • GetStatus()
    • GetData()
    • UnhandledStepErrors
    public class xUnitTest : WorkflowTest<MyWorkflow, MyDataClass>
{
    public xUnitTest()
    {
        Setup();
    }

    [Fact]
    public void MyWorkflow()
    {
        var workflowId = StartWorkflow(new MyDataClass() { Value1 = 2, Value2 = 3 });
        WaitForWorkflowToComplete(workflowId, TimeSpan.FromSeconds(30));

        GetStatus(workflowId).Should().Be(WorkflowStatus.Complete);
        UnhandledStepErrors.Count.Should().Be(0);
        GetData(workflowId).Value3.Should().Be(5);
    }
}

  7. Setup and use the Workflow Host

    master

    The workflow host is responsible for executing workflows, polling the persistence provider, and publishing events.

    Setup

    Use the AddWorkflow extension method on IServiceCollection during application startup. By default, it uses MemoryPersistenceProvider and SingleNodeConcurrencyProvider.

    Usage

    1. Retrieve IWorkflowHost from your IServiceProvider.
    2. Call RegisterWorkflow<TWorkflow>() to register your workflow definitions.
    3. Call Start() to begin the execution thread pool.
    4. Use StartWorkflow(id, version, data) to initiate a new instance.
    // Setup
services.AddWorkflow();

// Usage
var host = serviceProvider.GetService<IWorkflowHost>();            
host.RegisterWorkflow<HelloWorldWorkflow>();
host.Start();

host.StartWorkflow("HelloWorld", 1, null);

Console.ReadLine();
host.Stop();

  8. Use Activities to wait for external work

    master

    An Activity allows a workflow to pause and wait for an item to appear on an external queue of work. You can define an activity in a workflow, pass parameters to it, and map its eventual result back to your workflow data.

    To process these activities, you must implement a worker that calls GetPendingActivity to retrieve the activity and its associated data, and then calls SubmitActivitySuccess to provide the result back to the workflow engine.

    public class ActivityWorkflow : IWorkflow<MyData>
{
    public void Build(IWorkflowBuilder<MyData> builder)
    {
        builder                
            .StartWith<HelloWorld>()
            .Activity("activity-1", (data) => data.Value1)
                .Output(data => data.Value2, step => step.Result)
            .Then<PrintMessage>()
                .Input(step => step.Message, data => data.Value2);
    }
}

// Worker implementation example:
var activity = host.GetPendingActivity("activity-1", "worker1", TimeSpan.FromMinutes(1)).Result;

if (activity != null)
{
    Console.WriteLine(activity.Parameters);
    host.SubmitActivitySuccess(activity.Token, "Some response data");
}

  9. Use a master compensation step for an entire saga

    master

    Instead of defining individual compensation steps for every task, you can specify a single master compensation step at the saga level. This step will be triggered if any part of the saga fails.

    builder
    .StartWith(context => Console.WriteLine("Begin"))
        .Saga(saga => saga
            .StartWith<Task1>()
            .Then<Task2>()
            .Then<Task3>()
    )
        .CompensateWith<UndoEverything>()
    .Then(context => Console.WriteLine("End"));

  11. Implement Saga transactions with compensation steps

    master

    You can implement Saga transactions using the Fluent API to define compensation steps for individual components. If a step within the saga fails, the defined compensation steps are triggered in reverse order to undo the completed actions.

    builder
    .StartWith<SayHello>()
        .CompensateWith<UndoHello>()
    .Saga(saga => saga
        .StartWith<DoTask1>()
            .CompensateWith<UndoTask1>()
        .Then<DoTask2>()
            .CompensateWith<UndoTask2>()
        .Then<DoTask3>()
            .CompensateWith<UndoTask3>()
    )
    .Then<SayGoodbye>();