DOCSEARCHbyAlgolia
Submit Docs

grlx DevOps Automation Engine

repository·master·Indexed 42 days ago

https://github.com/gogrlx/grlx

A high-performance, low-resource DevOps automation engine written in Go. It utilizes a client-server architecture consisting of a central 'farmer' control plane and 'sprout' agents on managed nodes, using TLS-encrypted communication and YAML-based 'recipes' for fleet configuration management.

Tokens
16.9K
Snippets
89
Records
111
Agent score
53%

What's inside grlx

  2. Use the New API for secure path operations on Linux

    master

    The New API provides hardened path operations that use modern Linux kernel features like openat2 (on kernels 5.6+) to restrict lookups and prevent race attacks. These APIs are specifically designed to be safer than the legacy SecureJoin by operating on file descriptors rather than just strings.

    Key differences from the old API:

    • Strictness: Unlike SecureJoin, the new API functions (OpenInRoot, MkdirAll) will error out immediately if they encounter a dangling symlink or a non-existent path. They do not allow partial resolution of dangling symlinks.
    • Platform Support: These APIs are only supported on Linux.
    • Security: They protect against various race attacks and malicious /proc mounts.

  3. How grlx architecture works

    master

    grlx is a fleet configuration management system composed of three main components:

    1. farmer: A daemon that runs on a central management server. It contains an embedded NATS messaging Pub-Sub server and an API server. It acts as the control plane.
    2. sprout: A daemon installed on all managed fleet nodes (referred to as 'sprouts'). Sprouts subscribe to messages from the farmer over the bus.
    3. grlx CLI: The command-line utility used to control the farmer. It can be run locally on the management server or remotely via a secure, TLS-encrypted API.

    Communication is secure-by-default using TLS encryption (elliptic curve) and public-key cryptography for sprout authentication.

  4. Understand the grlx state application lifecycle

    master

    When applying state, grlx follows a specific sequence of operations to ensure safe and ordered configuration management. Understanding this lifecycle helps in debugging where a process might fail:

    1. Root Path & State Check: Determines the root path and verifies if a state file exists; errors out if not found.
    2. Property Refresh: Refreshes properties/pillars for each target.
    3. Template Rendering: Renders files using Golang template rendering (referred to as rendering magic).
    4. Namespace Validation: Ensures a unique state namespace exists to allow for safe requisite inclusions.
    5. Dependency Management: Builds a dependency graph and checks for cycles.
    6. Dry Run: Performs a test run of all recipes to determine necessary changes.
    7. Execution: Runs recipes in order, utilizing parallel execution (via workgroups/goroutines).
    8. Output Collection: Collects output and returns it to the server.

    Output Format Note:

    • The output includes a job id at the bottom.
    • Errors are colorized in the stream and are explicitly listed at the bottom for each target and in the final summary.

  5. Manage state with Recipes

    master
    In grlx, jobs are typically defined as Recipes. A recipe is a YAML document that describes the desired state of a sprout after the recipe is applied (or 'cooked'). The grlx CLI is the primary method for creating and managing these jobs.

  6. Understand the grlx topology

    master

    grlx operates as a one-to-many client-server system consisting of three main components:

    1. Farmer: The central control server. It manages all other systems. Only Linux is officially supported for the farmer.
    2. Sprout: The agent installed on systems you wish to manage. Sprouts communicate with the farmer. Currently, only Linux is officially supported, though Windows and macOS support is planned.
    3. grlx CLI: The command-line utility used by developers or administrators to issue commands. It can be run from a local machine or a jump server and does not need to run on the farmer itself. It supports Linux and macOS (Windows support is unofficial).

    Traffic Flow: All traffic flows in one direction: from the cli and sprout to the farmer.

  7. Understand the grlx recipe execution lifecycle

    master

    When running recipes in grlx, the system follows a specific execution pipeline to ensure safe and ordered configuration management. The lifecycle consists of the following stages:

    1. Initialization: Determines the root path and verifies the existence of a state file (errors out if missing).
    2. Property Refresh: Refreshes sprout properties for every target.
    3. Template Rendering: Renders files using Golang template rendering (referred to as rendering magic).
    4. Namespace Validation: Ensures a unique state namespace is present to allow for safe requisite inclusions.
    5. Dependency Resolution: Builds a dependency graph and checks for cycles to prevent infinite loops.
    6. Dry Run/Test: Performs a test run of all recipes to identify necessary changes.
    7. Parallel Execution: Executes recipes in the correct order using goroutines for parallel processing.
    8. Output Collection: Collects all outputs and returns them to the server.

  8. Install and configure the grlx sprout

    master

    The sprout should run as the root user to allow it to configure system files.

    1. Place the sprout binary in /usr/local/bin/grlx-sprout.
    2. Place the sample systemd service file (grlx-sprout.service) in /etc/systemd/system/grlx-sprout.service.
    3. Create a configuration file at /etc/grlx/sprout with the following format:
    farmerinterface: <DOMAIN_OR_IP_HERE>
farmerurl: https://<DOMAIN_OR_IP_HERE>:5405
    1. Enable and start the service:
    systemctl daemon-reload && systemctl enable --now grlx-sprout
    farmerinterface: <DOMAIN OR IP HERE>
farmerurl: https://<DOMAIN OR IP HERE>:5405

  9. Install the sprout on fleet nodes

    master

    The sprout is the agent installed on every node you wish to manage.

    If you need to use custom ports, you can set the FARMER_BUS_PORT and FARMER_API_PORT environment variables. When installing via the bootstrap script, you can use FARMERINTERFACE to specify the interface (e.g., localhost).

    # or, just run as root instead of sudo
# FARMER_BUS_PORT and FARMER_API_PORT variables are available in case you chose to use different ports.
curl -L https://bootstrap.grlx.dev/latest/sprout | FARMERINTERFACE=localhost sudo -E bash

  10. Run tests in grlx

    master

    Tests can be run for the entire repository or scoped to specific packages and individual test functions using the standard go test tool.

    go test ./...                # Run all tests
go test ./path/to/package    # Run tests for specific package
go test -run TestName ./path/to/package # Run single test
go test -v ./...            # Run tests with verbose output

  12. Install and configure the grlx CLI

    master

    To set up the grlx command line utility and authenticate with a farmer:

    1. Place the grlx binary in your PATH.
    2. Run grlx auth and select 'n' to reject the public key initially.
    3. Edit your local config (typically ~/.config/grlx/grlx) to point to your farmer's interface and URL.
    4. Generate a new private key by running grlx auth privkey. Press Enter to pin the farmer's TLS certificate.
    5. Retrieve your public key by running grlx auth pubkey.
    6. On the farmer, add your public key to the configuration under the pubkeys section:
    pubkeys:
    admin:
        - <YOUR_PUBKEY_HERE>
    1. Restart the farmer service: systemctl restart grlx-farmer.
    2. Validate the connection by running grlx version from your CLI machine.
    grlx auth
grlx auth privkey
grlx auth pubkey
# On the farmer
systemctl restart grlx-farmer
# Validate
grlx version