runitor

repository·main·Indexed 43 days ago

https://github.com/bdd/runitor

A command wrapper and task scheduler that reports execution status and logs to Healthchecks.io or private instances. It functions as a 'dead man's switch' for periodic tasks, supporting one-shot execution, interval-based scheduling, and cron expressions to ensure alerts are triggered if jobs fail or terminate unexpectedly.

Tokens
3.8K
Snippets
16
Records
19
Agent score
57%

What's inside runitor

  1. Use runitor as a task scheduler

    main

    Runitor can act as a long-running process that executes a command at specified intervals, serving as a lightweight alternative to cron or systemd.timer.

    • Interval-based: Use the -every <duration> flag (e.g., -every 2h24m).
    • Cron-based: Use the -at <cron> flag (e.g., -at "00 02,14 * * *").

    Triggering an immediate run

    If runitor is running in periodic mode, you can force it to execute the command immediately and reset its interval timer by sending a SIGALRM signal to the process:

    pkill -ALRM runitor
    runitor -slug db-backup -at "00 02,14 * * *" -- /script/db-backup
  2. Run a command with runitor

    main

    The basic usage pattern for runitor is to wrap a command. Runitor executes the command, captures its output, and reports the result to a Healthchecks.io instance (or your private instance).

    Syntax: runitor [flags] -- command [args...]

    Note that the -- separator is used to distinguish runitor's own flags from the command you wish to execute.

    runitor -- ls -la
  3. Install runitor

    main

    You can install runitor using one of the following methods:

    Download Signed Release Binaries

    Download the latest binaries for your platform from the GitHub releases page.

    To verify the integrity of your download, use the SHA256 checksum manifest. The manifest is signed with an SSH key available at https://bdd.fi/x/runitor.pub. You can use ssh-keygen and sha256sum to verify the signatures.

    Build Locally from Source

    If you have Go 1.19 or newer installed, run:

    go install bdd.fi/x/runitor/cmd/runitor@latest

    The binary will be located at $GOPATH/bin/runitor or $HOME/go/bin/runitor.

    To cross-compile for a different operating system or architecture, set the GOOS and GOARCH environment variables:

    GOOS=plan9 GOARCH=arm go install bdd.fi/x/runitor/cmd/runitor@latest

    Use a Container Image

    Runitor is available on Docker Hub as runitor/runitor. Images are based on Alpine, Debian, and Ubuntu and support x86_64, arm64v8, and armv7 architectures.

    go install bdd.fi/x/runitor/cmd/runitor@latest
  4. Publish a rebuild for vulnerability fixes

    main

    When a Go update introduces or fixes vulnerabilities (CRITICAL or HIGH), a new build can be released based on the original release tag. This process involves creating a dedicated git worktree for the release, setting a specific GOTOOLCHAIN version, and running the release script.

    Steps:

    1. Use nix develop with the unstable nixpkgs input to prepare the environment.
    2. Create a git worktree at build/worktree/${RELEASE}.
    3. Set the GOTOOLCHAIN environment variable to the target Go version.
    4. Execute ./scripts/mkrel with the build identifier.
    5. Clean up the worktree and environment variables after completion.
    nix develop --override-input nixpkgs github:NixOS/nixpkgs/nixpkgs-unstable
    
    RELEASE=vX.Y.Z
    RELBUILD=${RELEASE}-build.N
    
    WORKTREE=$(git rev-parse --show-toplevel)/build/worktree/${RELEASE}
    git worktree remove ${WORKTREE}
    git worktree add ${WORKTREE} ${RELEASE}
    
    GOTOOLCHAIN=go1.X.Y
    
    export WORKTREE GOTOOLCHAIN
    ./scripts/mkrel ${RELBUILD}
    
    # Cleanup
    git worktree remove $WORKTREE
    unset WORKTREE GOTOOLCHAIN
  5. Sign unsigned commits via rebase

    main

    To sign a series of commits that have not yet been pushed to a public remote, you can use an interactive rebase with the --exec flag to amend each commit with a signature.

    Warning: This rewrites git history. Only perform this on commits that have not been pushed.

    Steps:

    1. Identify the SHA1 of the last signed commit using git log --show-signature.
    2. Add your SSH key to the agent.
    3. Run git rebase using the --exec flag to automate the git commit --amend --no-edit -S command for every commit in the range.
    ssh-add -K
    git rebase --exec 'git commit --amend --no-edit -S' -i 0cafe42
  6. Build and publish OCI images

    main

    After a rebuild is prepared, you must build and publish the OCI images. This requires setting up multi-architecture support via binfmt and configuring container authentication.

    Steps:

    1. Install arm and arm64 support using tonistiigi/binfmt.
    2. Prepare the container authentication file in ${XDG_RUNTIME_DIR}/containers/auth.json using pass.
    3. Run the mkoci scripts to build, tag_rt_shorts, tag_default_rt, and push.
    4. Securely delete the authentication file using shred.
    sudo podman run --privileged --rm tonistiigi/binfmt --install arm,arm64
    mkdir -m 0700 -p ${XDG_RUNTIME_DIR}/containers
    pass show host/$(hostname -s)/containers/auth.json > ${XDG_RUNTIME_DIR}/containers/auth.json
    
    RELEASE=vX.Y.Z
    RELBUILD=${RELEASE}-build.N
    ./scripts/mkoci build
    ./scripts/mkoci tag_rt_shorts
    ./scripts/mkoci tag_default_rt
    ./scripts/mkoci push
    
    shred -u ${XDG_RUNTIME_DIR}/containers/auth.json
  7. Handle identification via UUID or Slug/Key

    main

    Runitor identifies which check to report to using one of two methods:

    1. UUID: Provide a unique check ID via -uuid or CHECK_UUID.
    2. Slug + Ping Key: Provide a check slug via -slug or CHECK_SLUG AND a project ping key via -ping-key or PING_KEY.

    If you use a slug, you must also provide a ping key. If you use the -create flag, you must use the Slug + Ping Key method.

  8. Configure runitor via environment variables

    main

    Runitor can be configured using the following environment variables:

    • HC_API_URL: The URL of the Healthchecks API (defaults to https://hc-ping.com).
    • PING_KEY: The ping key used for authentication. You can use the file: prefix to point to a file containing the key (e.g., file:/path/to/key).
    • CHECK_SLUG: The slug of the check to use.
    • CHECK_UUID: The UUID of the check to use.
  9. How RunConfig controls execution behavior

    main

    The RunConfig struct defines the operational parameters for a single run. Key behaviors include:

    • Output Capture: Controlled by Quiet (no stdout) and Silent (no stdout or stderr).
    • Ping Customization: NoStartPing prevents the initial 'start' signal. NoOutputInPing prevents command output from being sent in the success/failure pings. NoRunId prevents generating a unique UUID for each run.
    • Error Handling: OnSuccess, OnNonzeroExit, and OnExecFail allow you to map different command outcomes to specific PingType values (e.g., PingTypeSuccess, PingTypeFail, PingTypeLog).
    • Payload Limits: PingBodyLimit determines the maximum size of the command output sent to the API. If the output exceeds this, it is truncated, and a notice is added to the body.
  10. Use 'file:' prefix for secrets

    main

    To avoid passing sensitive information like PING_KEY or CHECK_UUID directly in the command line or environment variables, you can use the file: prefix. When a value starts with file:, runitor reads the content of the file at the specified path and uses it as the value, trimming any whitespace.

    # Instead of passing the key directly:
    runitor -ping-key "my-secret-key" "ls"
    
    # Pass a path to a file containing the key:
    runitor -ping-key "file:/run/secrets/ping_key" "ls"
  11. Example: Repository Maintenance with Slug and Ping Key

    main

    This example demonstrates running a maintenance script every 2 hours and 24 minutes using a check slug and a ping key loaded from a file.

    export PING_KEY=file:/run/secrets/hc_prod_pingkey
    runitor -slug git-repo-maintenance \
        -every 2h24m -- \
        /script/git-maint
    # (Using per-project ping key, and check slugs.)
    export PING_KEY=file:/run/secrets/hc_prod_pingkey
    runitor -slug git-repo-maintenance \
        -every 2h24m -- \
        /script/git-maint
  12. Example: Certificate Renewal with UUID

    main

    This example shows how to use a specific check UUID to run a dehydrated certificate renewal command.

    export CHECK_UUID=8116e449-d71c-4112-8f5d-a66f60902091
    runitor -- dehydrated --cron --config example.com.conf
    # (Using per check UUIDs.)
    export CHECK_UUID=8116e449-d71c-4112-8f5d-a66f60902091
    runitor -- dehydrated --cron --config example.com.conf