repository·main·Indexed 12 days ago
https://github.com/windmill-labs/windmillAn open-source developer platform that turns scripts in Python, TypeScript, and Go into APIs, background jobs, workflows, and user interfaces. It includes a high-performance backend composed of specialized Rust crates (windmill-api, windmill-queue, windmill-worker, and windmill-audit) and a comprehensive AI Evals benchmark runner for testing production prompts and LLM generation modes.
The Windmill Debug Module is a Debug Adapter Protocol (DAP) implementation designed to enable step-through debugging within Windmill's Monaco editor. It allows developers to set breakpoints, inspect variables, and view stack traces for scripts written in Python and TypeScript/Bun.
Communication is handled via WebSockets using the DAP protocol between the Monaco editor frontend and language-specific debug backends.
The Windmill backend is composed of several specialized crates that handle API requests, job queuing, execution, and auditing. Understanding these components is essential for understanding how the platform processes workflows and manages state.
The wmill SDK provides a high-level interface for interacting with the Windmill API, managing jobs, handling resources, and interacting with the workspace S3 storage.
Importing:
import wmillCore Capabilities:
// on <asset-uri>). You can now develop, preview, and run these pipelines locally from your working directory without deploying them to a workspace. This enables a local edit → preview → run loop using either a headless CLI or a browser live-preview.In Windmill, a data pipeline is not a single runnable flow. Instead, it is a Directed Acyclic Graph (DAG) composed of independent scripts deployed separately. These scripts form a pipeline by:
schedule, webhook, or when an upstream asset is produced).Pipelines are visualized and managed at /pipeline/<folder>. Every node in a pipeline is a standard workspace script that includes specific pipeline annotations in its header comments.
A Windmill Raw App follows a specific directory structure for frontend, backend, and configuration files:
raw_app.yaml: Main app configuration (summary, path, data settings).index.tsx / App.tsx: Frontend entry points and main components.package.json: Frontend dependencies.wmill.ts: Auto-generated backend type definitions (DO NOT EDIT).backend/: Directory containing server-side scripts (runnables).sql_to_apply/: Local development folder for SQL migrations.AGENTS.md: AI agent instructions.DATATABLES.md: Database schemas.index.css: Styles.A Windmill Flow is composed of several types of modules (steps). The primary types are:
main function. The default language is bun if not specified. Supported languages include deno, python3, go, bash, rust, sql variants, etc.path (e.g., f/scripts/my_script).flow_input.iter.value.Each backend runnable is identified by a unique key and falls into one of four types:
| Type | Description |
|---|---|
inline | Custom code stored within the app itself. The language is determined by the file extension. Requires a main function. |
script | A reference to an existing workspace script via its path. |
flow | A reference to an existing workspace flow via its path. |
hubscript | A reference to a hub script via its path. |
For script, flow, and hubscript types, the referenced item's input/output schema defines the runnable's surface. You can use staticInputs (a Record<string, any>) to pre-fill arguments that should not be overridable by the frontend.
Windmill supports a 'Workflow-as-Code' pattern where you define complex logic using TypeScript. To ensure reliability during replays (e.g., after a failure or suspension), you must use step() to checkpoint results.
workflow(fn): Marks an async function as a workflow entry point. The function must be deterministic. Any external state (time, random, API calls) must be wrapped in a step().step(name, fn): Executes fn and checkpoints the result. On a replay, the cached JSON-encoded result is returned instead of re-running fn.taskScript(path, options): Creates a task that dispatches to a separate Windmill script.taskFlow(path, options): Creates a task that dispatches to a separate Windmill flow.parallel(items, fn, options): Processes items in parallel with optional concurrency control.import * as wmill from 'windmill-client'
wmill.workflow(async () => {
const data = await wmill.step("fetch-data", async () => {
// This external call is checkpointed
return await someExternalApiCall();
});
const processed = await wmill.step("process", async () => {
return data.map(item => item.id);
});
const pipeline = wmill.taskFlow("f/etl/pipeline");
await pipeline({ input: processed });
});import * as wmill from 'windmill-client'
wmill.workflow(async () => {
const data = await wmill.step("fetch-data", async () => {
return await someExternalApiCall();
});
const processed = await wmill.step("process", async () => {
return data.map(item => item.id);
});
const pipeline = wmill.taskFlow("f/etl/pipeline");
await pipeline({ input: processed });
});To ensure nodes in a pipeline connect correctly, asset paths must match exactly. Windmill uses specific normalization rules for S3 assets:
s3:///x. This resolves to path /x. This is the recommended way to ensure connectivity across different languages (TS, Python, SQL).s3://secondary/key names a storage called secondary. This is treated as a different object and a different node than the default storage.writeS3File({s3:"/x"})write_s3_file(S3Object(s3="/x"))COPY ... TO 's3:///x'All the above resolve to the same canonical path /x in the default storage. Avoid using the no-slash form s3://x if you intend to connect to the default workspace storage, as it will look for a storage named x.