DOCSEARCHbyAlgolia
Submit Docs

Gofer

website·Indexed 31 days ago

https://clintjedwards.com/gofer

A tool designed to run short-lived jobs easily using a client-server model. Gofer deploys as a single static binary and manages workloads as pipelines—directed acyclic graphs of tasks wrapped in Docker containers. It includes a CLI for interaction, a scheduler for container orchestration, and support for extensions such as Cron, GitHub webhooks, and interval-based triggers.

Tokens
24.2K
Snippets
138
Records
200
Agent score
49%

What's inside Gofer

  1. Overview of Gofer automation engine

    Gofer is a streamlined automation engine designed to execute short-lived, containerized jobs (such as code linting, build automation, testing, port scanning, and ETL operations). It acts as a scheduling intermediary between event triggers and container orchestrators like Nomad, Kubernetes, or Local Docker. Key features include DAG support for complex workflows, built-in object and secret storage, and an extensible architecture via gRPC plugins.

  2. Overview of the Gofer Github Extension

    The Github extension enables Gofer pipelines to run based on Github webhook events, allowing for event-driven workloads. To use this extension, Gofer must have the 'external events' feature enabled to accept webhooks from Github's servers. For security, it is recommended to limit the IP addresses that can access Gofer's external events endpoint to only those used by Github.

  3. Overview of Gofer automation engine

    Gofer is an opinionated automation engine designed to execute custom scripts in containerized environments. It is deployed as a single static binary and uses declarative configurations written in programming languages (such as Go or Rust) rather than YAML. It supports scheduling and running short-term jobs—such as code linting, build automation, testing, port scanning, and ETL operations—on Nomad, Kubernetes, or Local Docker.

  4. Overview of Gofer provided extensions

    Gofer includes several pre-written extensions to trigger pipeline runs based on time or external events. The interval and cron extensions are included by default, while the github extension must be added manually.
    NameImageDefaultDescription
    intervalghcr.io/clintjedwards/gofer/extensions/interval:latestYesTriggers a run after a predetermined amount of time.
    cronghcr.io/clintjedwards/gofer/extensions/cron:latestYesUsed for nuanced, long-term intervals (e.g., yearly).
    githubghcr.io/clintjedwards/gofer/extensions/github:latestNoTriggers runs based on branch, tag, or release activity.

  5. Overview of Gofer automation engine

    Gofer is an opinionated automation engine designed to execute custom scripts in containerized environments. It is deployed as a single static binary and uses declarative configurations written in programming languages (like Go or Rust) rather than YAML. It supports scheduling tasks on Nomad, Kubernetes, or Local Docker. Common use cases include code linting, build automation, testing, port scanning, and ETL operations.

  6. Overview of Gofer Container Scheduling

    Gofer executes containers referenced in pipeline configurations using a container orchestrator known as a "scheduler". Currently, the only supported scheduler is local Docker, which is intended for development work and small deployments.

  7. Overview of Gofer Extensions

    Extensions add additional functionality to Gofer pipelines by allowing a pipeline to subscribe to an extension. For example, the interval extension can trigger a pipeline to run every few minutes by sending an API request to Gofer when a timer expires.

  8. Configure Gofer CLI using flags, environment variables, and files

    The Gofer CLI supports three methods of configuration. When multiple sources are used, the hierarchy of precedence (from lowest to highest) is: Configuration File < Environment Variables < Flags. Values provided via flags will override environment variables, which in turn override values in the configuration file.

  9. Understand Gofer's architecture and core workflow

    Gofer operates as a client-server model acting as a scheduling middleman between event-driven triggers and a container orchestration system (such as local Docker, Kubernetes, or Nomad).

    The general operational workflow is as follows:

    1. Gofer connects to a configured container orchestrator.
    2. Gofer launches configured extensions (which are Docker containers) that listen for events.
    3. Users define pipelines via configuration files specifying the order and type of containers to run.
    4. A pipeline run is triggered either manually or automatically by an extension.
    5. Gofer schedules the containers defined in the pipeline configuration.
    6. Gofer collects logs, exit codes, and performance summaries from the container runs for the user.

  10. Understand Gofer core concepts

    Gofer uses a hierarchical structure to manage short-lived jobs:

    • Pipeline: A collection of tasks that can be run together, defined via a configuration file and registered via CLI or API.
    • Run: A single execution of a pipeline, triggered manually (CLI/API) or automatically via extensions.
    • Task: The smallest unit in Gofer; an abstraction over a single container. It defines the container image, execution order relative to other containers, and required variables/secrets.
    • Task Run: The actual execution of a single task container, used to examine logs, results, and details.
    • Extension: Long-running docker containers that extend pipeline functionality (e.g., triggering runs based on events).

  11. Understand Gofer Task and Task Run concepts

    A Task is the smallest unit of work in Gofer, acting as an abstraction over a single container. Tasks define the container image to use, execution order relative to other containers, and the required variables or secrets. A Task Run is the actual execution of a specific task container; it is the entity used to inspect logs, results, and execution details.

  12. Implement Extension Health Checks and Info

    The Info method serves as a health check endpoint. It should return basic metadata about the extension, including its name, a link to its documentation, and a list of currently registered pipelines (formatted as namespace/pipeline).