DOCSEARCHbyAlgolia
Submit Docs

Microsoft Conductor

repository·main·Indexed 43 days ago

https://github.com/microsoft/conductor

A CLI-driven orchestration tool for building repeatable, deterministic multi-agent workflows using YAML definitions. It supports LLM providers like Anthropic Claude and GitHub Copilot, featuring capabilities for parallel processing, Human-in-the-Loop workflows, Model Context Protocol (MCP) tool execution, and OpenTelemetry tracing. Version 0.1.37 includes a web dashboard for real-time DAG visualization and a validator for output grading.

Tokens
199.1K
Snippets
496
Records
802
Agent score
57%

What's inside Conductor

  1. What is Conductor?

    main

    Conductor is a CLI tool designed for defining and running multi-agent workflows. It allows you to orchestrate complex flows—such as code review pipelines, research-then-synthesize loops, or plan-then-implement cycles—using the GitHub Copilot SDK and Anthropic Claude.

    Key characteristics include:

    • Repeatable: Workflows follow the same path for the same inputs.
    • Deterministic: Routing is handled via Jinja2 templates and expression evaluation rather than LLM calls, ensuring the first matching condition wins without extra token costs.
    • Source-controlled: Workflows are defined in plain YAML files, making them easy to version control and diff in pull requests.

  2. Overview of Conductor MCP Toolsets

    main

    When running conductor mcp serve, the server exposes several toolsets to the MCP host. These toolsets allow for workflow execution, run management, introspection, and diagnostics.

    ToolsetDefaultToolsBacking
    workflowsonN generatedbg_runner.launch_background
    runsonconductor_await_run, conductor_run_status, conductor_cancel_run, conductor_list_runsfleet/records + fleet/summary + /api/stop
    introspectoffconductor_run_events, conductor_node_detail, conductor_plan_treeread_event_log_full, derive_step_detail, WorkflowConfig
    diagnoseoffconductor_doctor, conductor_validate_workflow, conductor_run_logsproviders/diagnostics.gather(), cli/validate, ResourceLinks
    discoveryautoconductor_find_workflow, conductor_run_workflowreplaces workflows above the cap

  4. Conductor Dashboard Deep-Link Specification

    main

    The Conductor web dashboard (launched via conductor run --web) supports deep-linking into specific nodes of a workflow graph using URL query parameters. This allows external tools to generate links that open the UI focused on a specific agent or subworkflow.

    Key Behaviors

    • Inline Expansion: Targets are surfaced inline. The deep-link expands subworkflow containers along the path so the node appears within the full root graph and is centered, rather than re-rooting the view.
    • Transitive Search: If only an agent parameter is provided, the dashboard performs a transitive search through the root workflow and all sub-workflows/for_each iterations.
    • Resolution Priority: For subworkflow paths, segments are matched using an Exact slot key first, falling back to the Bare parent-agent name.
    • Execution: Deep-linking occurs exactly once per page load after the workflow graph has been populated via WebSocket.

  5. Explore the Conductor project structure

    main

    The Conductor repository is organized into several functional modules:

    • conductor.cli: Contains the Typer-based CLI commands: run (execute workflow), validate (check schema), and init (scaffold new workflows).
    • conductor.config: Handles YAML parsing, Pydantic schema definitions, and validation.
    • conductor.engine: Manages the workflow execution loop, context accumulation, routing, and execution limits.
    • conductor.providers: Implements the AgentProvider protocol and the provider factory for different LLM SDKs (e.g., copilot.py).
    • conductor.executor: Handles single agent execution, Jinja2 prompt rendering, and output parsing.
    • conductor.gates: Manages human-in-the-loop interactions.
    • examples/: Contains sample workflow definitions like simple-qa.yaml and research-assistant.yaml.

  6. Conductor Roadmap and Feature Phases

    main

    The development of Conductor is organized into four distinct phases, moving from a core MVP to advanced features and production polish:

    Phase 1: Core MVP

    Focuses on the foundational project structure, configuration loading/validation, template rendering, and basic workflow execution with context management. Includes the conductor run command.

    Phase 2: Routing & Loops

    Introduces a conditional routing engine (using simpleeval), support for loop-back patterns, safety limits (max_iterations, timeout), and a dry-run mode.

    Phase 3: Advanced Features

    Adds human-in-the-loop gates with a Rich UI, tool support/SDK pass-through, additional CLI commands (validate, init, templates), and verbose logging.

    Phase 4: Polish

    Focuses on production readiness, including comprehensive error handling, retry logic with exponential backoff, context trimming strategies, lifecycle hooks, and PyPI distribution.

  7. Dependencies for Dynamic Parallel Agents

    main

    Dynamic Parallel Agents rely on several internal Conductor components and external Python libraries:

    Internal Dependencies

    • WorkflowContext: Used for array resolution and managing execution context.
    • AgentExecutor: Responsible for executing individual for-each instances.
    • TemplateRenderer: Used to render templates with injected loop variables.
    • Router: Used for evaluating routes within a for-each group.

    External Dependencies

    • Python 3.10+: Required for Union type syntax and copy.deepcopy functionality.
    • asyncio: Used for concurrent batch execution.
    • Pydantic 2.x: Used for schema validation and field aliasing.

  8. Overview of the Conductor Fleet Manager (conductor fleet)

    main

    The Fleet Manager is a high-level management layer designed to manage multiple concurrent Conductor workflow runs via a Terminal User Interface (TUI) command: conductor fleet.

    Key Capabilities

    • Run Enumeration: Lists all active workflows, including both foreground (fg) and background (--web-bg) runs.
    • Observability: Provides a glanceable view of workflow name, current step, total elapsed time, time on current step, status (e.g., running, at-gate), token usage, and cost.
    • Dashboard Access: Allows users to open the existing per-run web dashboard directly from the TUI.
    • Discovery: Enables browsing of configured registries, workflows, and available providers/models.
    • Workflow Launching: Launches new runs from file paths or registry references using a generated form based on the workflow's input: schema.

    Status Vocabulary

    Workflows transition through an explicit state machine:

    • running: The workflow is actively executing.
    • at-gate: The workflow is paused at a human gate (requires user intervention).
    • paused: The workflow is in a paused state.
    • completed: The workflow finished successfully.
    • failed: The workflow encountered an error.

  9. What is Conductor and how does it work?

    main

    Conductor is a CLI-first, YAML-based orchestration tool designed for managing complex multi-agent AI workflows. Instead of writing custom code to chain agent calls, you define workflows declaratively in YAML files.

    Core Capabilities:

    • Declarative Workflows: Define agent sequences, conditional routing, and loops in version-controlled YAML.
    • Context Management: Pass data between agents using configurable strategies like accumulate, last_only, or explicit.
    • Conditional Routing: Use output-based logic to route workflows to different agents using first-match semantics.
    • Human-in-the-loop: Include human_gate agent types to pause execution for interactive CLI approvals.
    • Safety Controls: Prevent runaway processes with iteration limits (default: 10) and timeouts (default: 600s).
    • Provider Abstraction: While it integrates natively with the GitHub Copilot SDK, the architecture is designed to support other LLM providers in the future.

    Note: Conductor is a CLI-only tool and does not provide a GUI or web interface.

  10. What is the ACA provider and when to use it

    main

    The aca provider (AcaRuntimeProvider) is an experimental, opt-in provider designed to run an agent's entire runtime—including the agentic loop and its tool execution—inside an isolated Azure Container Apps (ACA) dynamic session.

    Unlike default providers that execute file/shell tools on the host machine, the aca provider relocates these side effects into a Hyper-V isolated sandbox.

    Use cases for the aca provider:

    • Untrusted/Model-generated code: Running shell commands or scripts generated by an LLM without risking the host environment.
    • Coding agents: Providing a disposable, isolated workspace for agents to clone, edit, and run code.
    • Multi-tenant/Concurrent runs: Ensuring that parallel agent executions do not share the same host filesystem or environment.

  11. Understand Conductor Workflow Definitions

    main

    Conductor workflows are defined in YAML using the WorkflowDef schema. A valid definition includes name, description, version, entry_point, input, metadata, and runtime settings.

    Inputs are defined using InputDef, which supports the following types:

    • string
    • number
    • boolean
    • array
    • object

    Each InputDef can specify if it is required (defaults to True), provide a default value, and include a description.

    Outputs are defined in WorkflowConfig.output as a dictionary of Jinja2 template strings (dict[str, str]).

  12. Prevent infinite loops in file references using cycle detection

    main

    When implementing external file references (e.g., via a !file tag), use a set of resolved absolute path strings to detect circular dependencies.

    Detection Logic:

    1. Resolve the target path to an absolute path using Path.resolve() (this canonicalizes symlinks to prevent evasion).
    2. Check if the absolute path exists in the current resolution set.
    3. If it exists, raise a ConfigurationError containing the cycle chain.
    4. If it does not exist, add the path to the set, parse the file, and remove the path from the set once parsing is complete.

    This approach supports nested chains (e.g., A → B → C) by tracking the traversal stack.