DOCSEARCHbyAlgolia
Submit Docs

Colmena Documentation

repository·main·Indexed 25 days ago

https://github.com/nix-community/colmena

Colmena is a stateless NixOS deployment tool written in Rust that acts as a thin wrapper over Nix commands to enable parallel deployment to multiple NixOS nodes. It supports traditional Nix configurations and Nix Flakes, offering features such as node tagging, local deployment via `colmena apply-local`, secrets management through `deployment.keys`, and ad hoc evaluation of configuration values. Version 0.5.0-pre.

Tokens
13.4K
Snippets
24
Records
93
Agent score
80%

What's inside Colmena

  2. Overview of Colmena features

    main

    Colmena provides several key features for managing NixOS deployments:

    • Node Tagging: Allows you to target a specific subset of nodes for deployment using tags.
    • Local Deployment: Enables deploying configurations to the host machine where Colmena is currently running.
    • Secrets Management: Provides a mechanism to deploy sensitive files separately from your main Nix configuration.
    • Ad Hoc Evaluation: Allows you to evaluate Nix expressions while maintaining access to your existing Colmena configuration.
    • Parallelism Control: Gives you control over how many deployment processes run in parallel.
    • Remote Builds: Supports building system profiles directly on the remote machines being managed.

  4. Build for foreign architectures by executing builds remotely

    main
    Instead of using local emulation, if your remote nodes have sufficient resources, you can execute builds directly on them. This avoids the performance overhead of QEMU emulation on the local machine. Refer to the Remote Builds documentation for specific implementation details.

  5. Use secret dependencies with Key Services

    main

    For every secret defined in deployment.keys.<name>, Colmena automatically creates a systemd service named ${name}-key.service.

    This service is only active when the corresponding secret file is present on the node. You can use this to create service dependencies, ensuring that services requiring secrets only attempt to start once the secrets are successfully deployed.

  6. Configure a Colmena Hive (Non-Flakes)

    main

    A Colmena configuration (a "hive") is typically defined in a .nix file. It consists of three main parts:

    1. meta: Global metadata. Use nixpkgs to pin the Nixpkgs version (accepts a path, a lambda like import <nixpkgs>, or an attribute set). You can also use nodeNixpkgs to provide different Nixpkgs versions per node. If your host allows remote builds, you can specify a machinesFile.
    2. defaults: A function that receives pkgs and returns a NixOS module applied to all hosts.
    3. Hosts: Individual host definitions. Hosts can reference other nodes via the nodes parameter (e.g., nodes.host-b.config.time.timeZone).

    Common deployment options within a host include:

    • deployment.targetHost: Override the default SSH connection target.
    • deployment.targetPort: Override the SSH port.
    • deployment.targetUser: Specify the SSH user.
    • deployment.tags: A list of strings used for filtering deployments via the CLI.
    • deployment.replaceUnknownProfiles: Boolean to control if unknown remote profiles are replaced during apply.
    {
  meta = {
    nixpkgs = <nixpkgs>;
    nodeNixpkgs = {
      node-b = ./another-nixos-checkout;
    };
  };

  defaults = { pkgs, ... }: {
    environment.systemPackages = with pkgs; [ vim wget curl ];
  };

  host-a = { name, nodes, ... }: {
    networking.hostName = name;
    time.timeZone = nodes.host-b.config.time.timeZone;
  };

  host-b = {
    deployment.targetHost = "host-b.mydomain.tld";
    deployment.targetPort = 1234;
    deployment.tags = [ "web" "infra-lax" ];
    time.timeZone = "America/Los_Angeles";
  };
}

  7. Use native Nix distributed builds for remote execution

    main

    Instead of using Colmena-specific logic, you can use Nix's native distributed build feature. When enabled, Nix transparently forwards builds to configured builders and automatically copies the results back to the local machine once complete.

    Configuration: Builders can be configured:

    1. Globally via your Nix configuration.
    2. Within your Colmena configuration using the meta.machinesFile option.

  11. Perform ad hoc evaluation of configuration values

    main

    You can extract values from your Colmena configuration (the nodes attribute set) to use in other programs by evaluating a .nix file containing a lambda. The lambda receives the standard Colmena arguments: nodes, pkgs, and lib. The result must be a JSON-serializable value.

    To evaluate a file, use colmena eval <file.nix>. To evaluate an expression directly from the command line without a file, use the -E flag.