Woodpecker CI Documentation

repository·main·Indexed 26 days ago

https://github.com/woodpecker-ci/woodpecker

A lightweight, extensible CI/CD engine designed for minimal RAM usage. This documentation covers installation, the Woodpecker Go client (v3), and the admin CLI for managing server settings, including log levels, organizations, plugin registries, global secrets, and user accounts.

Tokens
94.2K
Snippets
174
Records
542
Agent score
92%

What's inside Woodpecker CI

  1. Overview of Woodpecker Extensions

    main

    Woodpecker allows you to replace internal logic with external extensions by using pre-defined HTTP endpoints. There are currently three types of extensions available:

    • Configuration extension: Modifies or generates pipeline configurations on the fly.
    • Registry extension: Retrieves registry credentials from an external service.
    • Secret extension: Retrieves secrets from an external service.
  2. Understand workflow ordering and status

    main

    Status Reporting

    Each workflow reports its own status back to your Git forge.

    Ordering

    Queued workflows within the same group start in the order their pipelines were created. This ensures that even if a later pipeline's dependencies finish faster, it will not overtake an earlier pipeline, guaranteeing that workflows like deployments happen in the correct commit order.

  3. Understand Woodpecker terminology and architecture

    main

    To effectively use Woodpecker, it is important to understand its core components and how they relate to each other:

    Core Components

    • Server: The central component that handles webhooks from forges, orchestrates agents, sends status back to forges, and serves the API and Web UI.
    • Agent: A component that executes pipelines (specifically workflows) using a specific backend (e.g., Docker, Kubernetes, or local). Agents connect to the server via GRPC.
    • Forge: The hosting platform (e.g., GitHub, GitLab) where your repositories are hosted.
    • CLI: A terminal tool used to administer the server, execute pipelines locally for debugging/testing, and lint pipelines.

    Execution Hierarchy

    1. Pipeline: A sequence of workflows triggered by an Event (e.g., a push from a forge or a manual trigger).
    2. Workflow: A sequence of Steps and Services defined in a YAML file. Each workflow has its own isolated Workspace.
    3. Step: Individual commands, actions, or tasks within a workflow.
    4. Service: A step executed from the start to the end of a workflow that can be accessed by name via the network from other steps in the same workflow.

    Key Concepts

    • Workspace: A shared folder between all steps of a workflow containing the repository and data generated by previous steps.
    • Plugin: Extensions providing pre-defined actions or commands for a step, configurable via settings.
    • Matrix: A configuration option to execute workflows for each value in a defined matrix.
    • Extension: Allows replacing internal services like secrets storage or config fetchers.
    • Status: The outcome of a step or workflow, determined by the command exit code.
  4. Understand Plugin Isolation and Workspace behavior

    main

    Plugins operate with specific constraints to ensure they perform only their intended functions:

    • Workspace Access: Plugins share the build workspace, which is mounted as a volume. The workspace base is always mounted at /woodpecker, though the working directory is dynamically adjusted for the plugin.
    • Restrictions: You cannot use commands or entrypoint in a plugin step; doing so will cause the step to fail.
    • Environment Variables: While you can use the environment key, doing so changes how the container is treated. The container will no longer be treated as a standard plugin, meaning it cannot access secrets via plugin filters and will not be privileged unless explicitly defined.
  5. Understand Woodpecker components

    main

    Woodpecker is composed of three main components:

    • server: Provides the user interface, processes forge webhooks, serves the API, and analyzes pipeline YAML configurations.
    • agent: Executes workflows using a specific backend (Docker, Kubernetes, or local) and connects to the server via GRPC. You can run multiple agents to fine-tune job limits and backends.
    • autoscaler (optional): Automatically spins up and destroys VMs on cloud providers to process pending builds.

    To increase parallel workflow capacity, you can either add more agents or set the WOODPECKER_MAX_WORKFLOWS environment variable on an existing agent.

  6. Understand Woodpecker System Architecture

    main

    Woodpecker consists of three primary components that interact to provide CI/CD capabilities:

    1. Server: The central orchestrator. It handles REST API requests, manages the database (store), connects to forges (GitHub, GitLab, etc.), and maintains a queue of work. It provides a WebUI and communicates with agents via gRPC.
    2. Agent: A remote worker that connects to the server via gRPC. It polls the server's queue for new work, executes pipeline steps using the engine, and streams logs and execution state back to the server.
    3. CLI: A command-line tool used to interact with the server (managing users, secrets, repos, etc.) or to execute pipelines locally for testing.

    The Engine (found in the pipeline package) acts as the shared kernel that parses configuration files, enriches them with forge metadata, and prepares them for execution.

  7. Follow Woodpecker naming conventions

    main

    When configuring Woodpecker or writing code/scripts, adhere to these preferred naming conventions:

    • URLs vs Links: Environment variables ending in *_LINK should be referred to as *_URL. When writing code, use the URL() method instead of Link().
    • Pipelines: Use the term pipelines (formerly referred to as builds).
    • Steps: Use the term steps (formerly referred to as jobs).
    • Advanced Variables: Use the prefix WOODPECKER_EXPERT_ for advanced environment variables that are typically not required for standard user configuration.
  8. Handle service initialization delays

    main

    Service containers often require time to initialize before they can accept connections. To prevent connection errors, implement a wait mechanism (like sleep) or a retry/backoff logic in your pipeline steps that depend on the service.

    steps:
      - name: test
        image: golang
        commands:
          - sleep 15
          - go get
          - go test
    
     services:
       - name: database
         image: mysql
  9. Manually deploy the website

    main

    To manually build and push the website to the deployment target, use the pnpm deploy command with the required environment variables. Note that standard deployments are typically handled via CI to woodpecker-ci.org.

    GIT_USER=woodpecker-bot USE_SSH=true DEPLOYMENT_BRANCH=main pnpm deploy
  10. Use `when` for conditional step execution

    main

    The when block allows you to execute a step only if specific conditions are met. If multiple conditions are listed in a when block, a step executes if at least one condition is true. Within a single condition, all sub-conditions must be true.

    Supported Conditions:

    • repo: Match the repository name.
    • branch: Match the git branch (supports doublestar patterns).
    • event: Match the build event (e.g., push, pull_request, tag, release, cron, manual).
    • ref: Match the git reference (e.g., refs/tags/v*).
    • status: Match the workflow status (success or failure). Default is success.
    • platform: Match the execution platform (e.g., linux/amd64).
    • matrix: Match a specific matrix permutation.
    • instance: Match the Woodpecker instance hostname.
    • path: Match changed files using include or exclude glob patterns (applies to push and pull_request).
    • evaluate: Execute a step based on a boolean expression using the expr syntax (accesses CI_ environment variables).
    • cron: Filter based on the name of a cron job (requires event: cron).
    steps:
      - name: prettier
        image: woodpeckerci/plugin-prettier
        when:
          - event: pull_request
            repo: test/test
          - event: push
            branch: main