Runner.Server

repository·main·Indexed 42 days ago

https://github.com/christopherhx/runner.server

A local emulator for GitHub Actions and Azure Pipelines that allows developers to run workflows and test runner configurations locally. It includes Runner.Server (backend) and Runner.Client (workflow scheduler), supporting matrix parsing, secret management, and local artifact/cache hosting. Also provides an experimental implementation for Azure Pipelines and a VSCode extension for validating and expanding Azure Pipeline YAML files.

Tokens
29.3K
Snippets
90
Records
139
Agent score
52%

What's inside Runner.Server

  1. Capabilities and Support for Container Hooks

    main

    The implementation of Container Hooks provides the following capabilities for self-hosted runners:

    • Non-Docker Scenarios: The runner supports non-Docker environments for self-hosted runners, allowing administrators to customize their Docker invocations.
    • Documentation and Examples: Official documentation and an open-source repository containing examples are provided for managing Docker hooks.
    • Telemetry: The system includes telemetry specifically designed to troubleshoot support issues related to these hooks.
  2. Understand how the runner uses Node.js

    main

    The runner does not use the node executable found in your system $PATH. Instead, it uses its own built-in copies of Node.js located within the runner root directory. JavaScript-based Actions are executed using these specific versions based on the version requested in the action's metadata.

    Node.js locations:

    • <runner_root>/externals/node20/
    • <runner_root>/externals/node24/
  3. Verify Git connectivity to GitHub

    main

    The Git Connection Check ensures that the git executable can successfully access GitHub.com or your GitHub Enterprise Server. The check is performed by executing git ls-remote against a known repository. To assist in debugging, the runner sets the environment variables GIT_TRACE=1 and GIT_CURL_VERBOSE=1 during this check, which forces git to produce detailed debug logs.

    # For GitHub.com
    git ls-remote --exit-code https://github.com/actions/checkout HEAD
    
    # For GitHub Enterprise Server
    git ls-remote --exit-code https://ghes.me/actions/checkout HEAD
  4. Understand `action_status` in the GitHub context

    main

    To support conditionals within composite actions, a new field action_status is available in the GitHub context. This field mimics the job context status but specifically tracks the status of the current composite action.

    • Purpose: Allows main steps within a composite action to use functions like success() to check if the current composite action is still in a successful state.
    • Scope: This field is specifically relevant for composite actions. In a regular workflow, success() checks the job_status. In a composite action's main step, success() checks if action_status == success.
  5. How Container Hooks work in the Runner

    main

    Container Hooks are high-level abstractions that map to runner actions rather than specific low-level container commands (like docker build). This design allows the runner to be extensible and flexible, supporting different environments like Docker or Kubernetes without forcing users to implement a large number of low-level hooks.

    Instead of mirroring every container lifecycle event, the runner provides a few key hooks that represent major job phases. This approach simplifies the implementation for users and allows the runner to manage state (like container IDs) internally, which is particularly useful for environments like Kubernetes where traditional container IDs might not be the primary way to interact with workloads.

  6. Understand the Container Hook Input Format

    main

    The runner sends a JSON object to your hook handler via stdin. This object contains the command to execute, the arguments, and a state object for persistence.

    Input Fields:

    • command: The specific hook command the runner expects you to invoke.
    • responseFile: The file path where you must write your output (if the command requires returning data).
    • args: A JSON object containing specific arguments for the command.
    • state: A JSON object used to maintain state across different hook invocations.
    {
      "command": "job_cleanup",
      "responseFile": "/users/thboop/runner/_work/{guid}.json",
      "args": {},
      "state": 
      {
          "id": "82e8219701fe096a35941d869cf8d71af1d943b5d3bdd718850fb87ac3042480"
      }
    }
  7. Use Context and Commands within Runner Hooks

    main

    When your hook scripts run, they have access to specific job context and GitHub Actions features to allow for dynamic setup and reporting.

    Available Context

    • Default Environment Variables: Standard variables like GITHUB_REPOSITORY, GITHUB_REF, etc., are available. Note that step-specific variables like GITHUB_ACTION will not be set.
    • Webhook Payload: You can access the full webhook event payload by reading the file path stored in GITHUB_EVENT_PATH.

    Supported GitHub Actions Commands

    Hooks support several workflow commands to interact with the job execution:

    • Warnings/Annotations: You can use commands to issue warnings to users (e.g., notifying them of a deprecated runner pool).
    • Environment Files: You can use ::add-path: and ::set-env: to dynamically modify the runner environment for the duration of the job.

    Unsupported Commands

    • ::save-state: is not supported.
    • ::set-output: is not supported (as hooks do not have a specific id like traditional steps).
  8. Configure multi-line problem matchers

    main

    Multi-line matchers are used when an error spans multiple lines (e.g., a header line containing the filename followed by several error lines).

    To implement this, provide an array of patterns in the pattern property. The last pattern in the array can include a "loop": true property, which tells the engine to continue searching for more matches using that pattern.

    Example: ESLint Stylish Mode

    In this pattern, the first regex captures the filename, and the second regex (with loop: true) captures the individual error details.

    {
      "owner": "eslint-stylish",
      "pattern": [
        {
          "regexp": "^([^\\s].*)$",
          "file": 1
        },
        {
          "regexp": "^\\s+(\\d+):(\\d+)\\s+(error|warning|info)\\s+(.*)\\s\\s+(.*)$",
          "line": 1,
          "column": 2,
          "severity": 3,
          "message": 4,
          "code": 5,
          "loop": true
        }
      ]
    }
  9. How the `env` context works in the runner

    main

    The env context allows users to reference workflow variables defined in YAML files for use in action inputs, displayName, and if conditions.

    Lifecycle of the env context:

    1. Job Start: The context is initialized with environment variables defined at the workflow or job level in the YAML file.
    2. Runtime Updates: The context is updated when a step uses the ::set-env:: command to set variables at the runner level.
    3. Step Execution: The context is updated with the specific env block of a step immediately before that step runs.

    Important Constraints:

    • The env context is only available in the runner. It cannot be used in any server evaluation parts (similar to the runner context).
    • To prevent confusion in containerized environments (like job containers or container actions), the env context does not include environment variables from the runner machine itself (e.g., $USER from the host machine).
    env:
      env1: 10
      env2: 20
      env3: 30
    jobs:
      build:
        env:
          env1: 100
          env2: 200
        runs-on: ubuntu-latest
        steps:
          - run: |
              echo ${{ env.env1 }}  # 100
              echo $env1            # 100
              echo $env2            # 200
              echo $env3            # 30
            if: env.env2 == 200     # true
            name: ${{ env.env1 }}_${{ env.env2 }} # 100_200
            env:
              env1: 1000