DOCSEARCHbyAlgolia
Submit Docs

Dagu Documentation

website·Indexed 33 days ago

https://docs.dagu.sh/

Documentation for Dagu, a workflow engine for automating DAGs. It covers writing workflows with YAML specifications, managing resource limits, scheduling via cron, and utilizing various step types including SQL (PostgreSQL, SQLite), LLM integrations, Browser and Computer automation, and containerized execution via Docker or Podman.

Tokens
246.7K
Snippets
1.3K
Records
1.7K
Agent score
63%

What's inside Dagu

  1. Overview of Dagu Runtime Profiles

    Runtime profiles are named sets of variables and secrets that can be selected when starting or enqueueing a DAG run. They allow the same DAG to run against different environments (e.g., dev, staging, prod) without changing the DAG YAML. Profiles modify the variables and secrets supplied to a run but do not create separate schedulers, execution environments, or worker pools.

  2. Overview of Mailbox step actions

    Dagu allows interaction with existing mailboxes via IMAP (for reading/organizing) and SMTP (for sending). The primary actions available are:

    • mail.search: Find emails in a folder, including sender, subject, text, and attachments.
    • mail.organize: Mark emails as read, unread, flagged, or unflagged, and move them to folders, archive, or trash.
    • mail.send: Send emails from a configured account or reply to received emails.

  3. Overview of Dagu MCP Tools

    Dagu exposes three primary MCP tools to allow external agents or clients to interact with the system:

    • dagu_read: Used to retrieve the current state of DAGs, Markdown Wiki pages, DAG runs, logs, and reference resources.
    • dagu_change: Used to preview or apply changes to DAG YAML specifications and workspace-aware Wiki pages.
    • dagu_execute: Used to manage the lifecycle of DAG runs, including starting, enqueuing, retrying, or stopping them.

  4. Overview of Dagu Incident Routing

    Incident routing is used to open and resolve incidents in operational management systems (like PagerDuty and SolarWinds Incident Response) when workflow failures require an operational response. Unlike standard notifications (Slack, Teams, Email), incident routing supports a full lifecycle: triggering an incident on final failure, deduplicating repeated failures, and automatically resolving the incident when the DAG recovers.

    License Requirement: Incident routing requires an active Dagu self-host license or trial.

  5. Overview of DAG run artifacts

    DAG run artifacts are arbitrary files written by a DAG run. They are distinct from logs and step outputs. Artifact storage is automatically enabled if a DAG references ${context.paths.artifacts_dir}, uses an artifact action, or uses stdout.artifact / stderr.artifact. It can also be enabled explicitly in the DAG configuration.

  6. Overview of Persistent State in Dagu

    Persistent state is used to store small JSON values that must survive across DAG runs. It is ideal for cursors, checkpoints, last-seen IDs, previous API responses, and simple change detection.

    Key constraints and characteristics:

    • Values must be JSON-serializable.
    • Values are normalized, versioned, and hashed before being written.
    • Maximum size is 1 MiB after normalization. For larger files or documents, use artifacts or external storage.

    Comparison with other Dagu mechanisms:

    • Step outputs: Lifetime is the current DAG run; used for passing values between steps.
    • DAG run outputs: Lifetime is the current DAG run history; used for structured run results.
    • Artifacts: Lifetime is the current DAG run history; used for files, reports, and large payloads.
    • Persistent state: Lifetime is across DAG runs; used for cursors and checkpoints.

  7. Overview of Dagu Runtime Profiles

    Runtime profiles are named sets of environment variables and secrets selected at run time. They allow the same DAG to run against different contexts (e.g., dev, staging, prod) without hardcoding environment values into the DAG YAML. Profiles are configuration overlays and do not provide execution isolation; for independent execution boundaries, separate Dagu deployments should be used.

  8. Overview of Dagu Webhooks

    Webhooks allow external callers to trigger a specific DAG using the endpoint POST /api/v1/webhooks/{fileName}. Each DAG can have at most one webhook configuration. While management of webhooks uses the standard authenticated API, trigger requests use a specific webhook token instead of API keys or session JWTs.

  9. Overview of Dagu Workspaces

    Workspaces are used to organize workflows and runs within a single Dagu installation, typically to separate them by team, business function, or product (e.g., finance, platform-ops). They provide scoping for access control, Wiki pages, managed secrets, profile defaults, notifications, and incident routing.

    Important Limitation: Workspaces are NOT environment boundaries. They share the same scheduler, execution environment, coordinator, workers, queues, storage, and Git Sync configuration. To isolate development, staging, and production environments, you must run separate Dagu deployments.

  10. Overview of Dagu API Keys

    API keys allow scripts, CI jobs, external tools, and other Dagu servers to interact with the Dagu API without interactive login. They require Builtin Authentication to be enabled. Each key consists of a name, an optional description, a role, workspace limits, accepted surfaces (REST API or MCP), and an attribution class (user-owned or service account). The secret is displayed only once upon creation.

  11. Overview of Dagu Git Sync

    Git Sync synchronizes DAG definitions, scripts, templates, binaries, Wiki pages, and Wiki attachments between a Git repository and Dagu storage directories. It assigns every managed file an itemId and kind (e.g., dag, file, doc, doc-asset) used by the Web UI, CLI, and REST API. DAG and Wiki page IDs omit their file extensions (.yaml, .yml, .md), while supporting files retain their full relative path and extension.

  12. Define worker capabilities using labels

    Worker labels are key-value pairs that describe a worker's capabilities. The coordinator uses these labels to match tasks to workers via the worker_selector field in DAG definitions. All workers must also be configured with worker.coordinators to connect to the coordinator.