spot

repository·master·Indexed 19 days ago

https://github.com/umputun/spot

A deployment and configuration management tool that uses YAML or TOML playbooks to define tasks and targets. Spot enables concurrent command execution on remote hosts via SSH and includes built-in commands for script execution, file manipulation (copy, sync, delete, line), and service waiting. It supports both full playbooks with multiple target sets and tasks, and simplified playbooks for single-environment use cases.

Tokens
11.5K
Snippets
38
Records
49
Agent score
68%

What's inside spot

  1. Configure Task-level options and error handling

    master

    In a full playbook, tasks can define several optional fields to control execution flow and error recovery:

    • on_error: A command to execute on the local host if the task fails. Use the {SPOT_ERROR} variable to include the error message.
    • user: The SSH user for this specific task (overrides the playbook-level user).
    • targets: A list of host names, groups, tags, or addresses. Can be overridden via the -t CLI flag.
    • tags: A list of tags for filtering. When using the -n flag, Spot matches the name exactly; if no match is found, it treats the value as a tag and runs all tasks with that tag.

    Note: Every task must have a unique name field. Playbooks missing a name field will fail immediately.

    - name: deploy-things
      tags: ["deploy"]
      on_error: "curl -s localhost:8080/error?msg={SPOT_ERROR}"
      options: {ignore_errors: true, no_auto: true, only_on: [host1, host2]}
      commands:
        - name: wait
          script: sleep 5s
  2. Core concepts of Spot

    master

    Spot is a lightweight deployment and configuration management tool designed for simplicity and predictability. Unlike declarative tools (like Ansible), Spot uses an imperative approach: a playbook is simply a direct list of commands to execute against a list of remote targets.

    Key Mental Models

    • Predictability: Spot strictly follows your instructions without attempting to interpret or guess your intentions. If you provide a command, Spot runs that command.
    • Playbook Structure: A playbook consists of two main parts:
      1. Targets: A list of hosts, inventory files, or inventory URLs.
      2. Tasks: A list of commands to execute on those targets.
    • Control Modes:
      • Dry mode: Allows you to preview changes before execution.
      • Verbose mode: Provides detailed execution information.
      • Debug mode: Provides maximum detailed logs for deep investigation.
    • Execution Patterns:
      • Concurrent Execution: Tasks can run on multiple hosts simultaneously to speed up deployment.
      • Rolling Updates: You can implement controlled updates using user-defined wait commands between tasks.
  3. Compare Full vs Simplified Playbooks

    master

    When choosing between a full and simplified playbook, consider the following differences:

    FeatureFull PlaybookSimplified Playbook
    TargetingMultiple target sets (targets)Single target set (targets or target)
    TasksMultiple tasks (tasks)Single task (task)
    Target Typeshosts, groups, namesList of names or host addresses
    Error HandlingSupports on_error per taskNot supported
    User OverridesSupports user per taskNot supported
    FormatYAML or TOMLYAML or TOML
  4. Core concepts of Spot

    master

    Understanding the hierarchy of Spot's execution model:

    • Playbook: A YAML or TOML file defining the automation logic. It contains a list of tasks and targets.
    • Task: A named set of commands. Tasks can be triggered by name or by a tag. They can be executed concurrently across multiple hosts.
    • Command: The individual actions within a task (e.g., script, copy, sync, delete, echo, wait).
    • Target: The destination for tasks. A target can be a specific hostname, a group of hosts defined in an inventory, or an inventory file/URL.
    • Inventory: A collection of targets (hosts/groups) used to define where tasks should run.
  5. How target selection priority works

    master

    Spot follows a specific order of precedence to determine which hosts to run tasks on:

    1. CLI --target flag: If provided, Spot follows this search order:
      • Match against a target name defined in the playbook.
      • Match against a group name in the inventory.
      • Match against tags in the inventory.
      • Match against a hostname in the inventory.
      • Match against a host address in the playbook.
      • Use the value as a direct host address.
    2. Task-level targets: If no CLI --target is set, Spot checks the targets list defined within the specific task.
    3. Default target: If neither is set, Spot uses the default target.
  6. Define targets in a playbook

    master

    Targets define the remote hosts where tasks are executed. In a full playbook, you can define multiple target sets using several types:

    • hosts: A list of objects specifying host, user, port, and an optional name.
    • groups: A list of group names from your inventory.
    • tags: A list of tags from your inventory.
    • names: A list of host names from your inventory.

    Target types can be combined within a single target definition. Spot deduplicates the final host list based on host+ip+user.

    Note: Simplified playbooks only support a single, anonymous target type that combines hosts and names into a simple list of strings.

    targets:
      prod:
        hosts: [{host: "h1.example.com", user: "test"}, {"h2.example.com", "port": 2222, name: "h2"}]
      staging:
        groups: ["staging"]
      dev:
        groups: ["dev", "staging"]
        names: ["host1", "host2"]
      all-servers:
        groups: ["all"]
    
    tasks:
      - name: task1
        targets: ["dev", "host3.example.com:2222"]
        commands:
          - name: command1
            script: echo "Hello World"
  7. Pass variables between commands and tasks

    master

    Spot provides two ways to share data between commands:

    1. Shell Export: In a script command, use the standard export VAR=val syntax. Subsequent commands in the same task will have access to these variables.
    2. Explicit Registration (register): Use the register option to capture variables. Registered variables are automatically populated into the environment of all subsequent tasks in the playbook.

    Dynamic Variable Names: You can use template substitution in register to create host-specific or environment-specific variable names using ${SPOT_REMOTE_ADDR} or environment variables.

    Important: Spot performs literal string replacement for variables (e.g., $VAR) before the shell runs. To preserve literal dollar signs (like in hashes or passwords), wrap the value in single quotes.

    # Using register to pass variables between tasks
    tasks:
      - name: set_register_var
        commands:
          - name: some command
            script: |
              export len=$(echo "content" | wc -c)
            register: [len]
    
      - name: use_register_var
        commands:
          - name: some command
            script: "echo len is $len"
  8. Create a full playbook in Spot

    master

    A full playbook is the most powerful configuration format in Spot. It allows you to define multiple target sets (e.g., prod, staging, dev) and multiple tasks (e.g., deploy-things, docker), each with its own set of commands and configurations. This enables executing different sets of commands across different environments within a single file. Full playbooks support YAML or TOML formats.

    Key features of full playbooks:

    • Multiple Target Sets: Define environments using hosts, groups, or names.
    • Multiple Tasks: Group commands into named tasks that can be filtered via tags.
    • Task-level Error Handling: Use the on_error field to execute a hook (like a curl command) when a task fails.
    • Task-level User Overrides: Specify a different user for a specific task.
    • Target Types: Supports hosts (with optional user, name, and port), groups (from inventory), and names (from inventory).
    user: umputun
    ssh_key: keys/id_rsa
    ssh_shell: /bin/bash
    ssh_temp: /tmp
    local_shell: /bin/bash
    inventory: /etc/spot/inventory.yml
    
    targets:
      prod:
        hosts:
          - {host: "h1.example.com", user: "user2", name: "h1"}
          - {host: "h2.example.com", port: 2222}
      staging:
        groups: ["dev", "staging"]
      dev:
        names: ["devbox1", "devbox2"]
    
    tasks:
      - name: deploy-things
        tags: ["deploy"]
        on_error: "curl -s localhost:8080/error?msg={SPOT_ERROR}"
        commands:
          - name: wait
            script: sleep 5s
          - name: copy configuration
            copy: {"src": "testdata/conf.yml", "dst": "/tmp/conf.yml", "mkdir": true}
          - name: sync things
            sync: {"src": "testdata", "dst": "/tmp/things"}
          - name: some command
            script: |
              ls -laR /tmp
              echo all good
  9. Use secrets in Spot playbooks

    master

    Spot allows you to use encrypted secrets in your playbook files to handle sensitive information like passwords or API keys. Secrets are decrypted at runtime and passed into commands as environment variables.

    To use secrets:

    1. Define a secret provider via CLI options or environment variables.
    2. In your playbook, list the secret keys under options.secrets at either the task level (applies to all commands in that task) or the command level (applies only to that specific command).

    When running with --verbose or --dbg, secrets are automatically masked with **** in the output to prevent accidental exposure.

    tasks:
      - name: access sensitive data
        commands:
          - name: read api response
            script: |
              curl -s -u ${user}:${password} https://api.example.com  
              curl https://api.example.com -H "Authorization: Bearer ${token}"
        options:
          secrets: [user, password, token]
  10. Install the latest development version of Spot

    master

    To use the latest development version of Spot, you can install it using go install or by building from source. Note that you must have Go 1.16+ installed on your machine.

    Using go install

    Run the following commands to install both the spot CLI and the secrets utility:

    go install github.com/umputun/spot/cmd/spot@master
    go install github.com/umputun/spot/cmd/secrets@master

    Using git and make

    Clone the repository, navigate to the directory, and build it using make:

    git clone github.com/umputun/spot
    cd spot
    make build

    After building with make, the binaries will be located at:

    • spot/.bin/spot
    • spot/.bin/sport-secrets
  11. Enable Editor Autocompletion for Spot Playbooks and Inventories

    master

    You can enable autocompletion and validation for Spot YAML files using JSON schemas.

    Schema URLs:

    • Playbook: https://raw.githubusercontent.com/umputun/spot/master/schemas/playbook.json
    • Inventory: https://raw.githubusercontent.com/umputun/spot/master/schemas/inventory.json

    Per-file (Inline)

    Add this comment to the top of your YAML file for editors supporting yaml-language-server (VSCode, Neovim, Zed):

    # yaml-language-server: $schema=https://raw.githubusercontent.com/umputun/spot/master/schemas/playbook.json

    VSCode (Per-project)

    Create .vscode/settings.json:

    {
      "yaml.schemas": {
        "https://raw.githubusercontent.com/umputun/spot/master/schemas/playbook.json": ["spot.yml", "*.spot.yml"],
        "https://raw.githubusercontent.com/umputun/spot/master/schemas/inventory.json": ["inventory.yml"]
      }
    }

    Zed (Per-project)

    Create .zed/settings.json:

    {
      "lsp": {
        "yaml-language-server": {
          "settings": {
            "yaml": {
              "schemas": {
                "https://raw.githubusercontent.com/umputun/spot/master/schemas/playbook.json": ["spot.yml", "*.spot.yml"],
                "https://raw.githubusercontent.com/umputun/spot/master/schemas/inventory.json": ["inventory.yml"]
              }
            }
          }
        }
      }
    }
  12. Run Spot playbooks

    master

    Spot executes tasks defined in a playbook file (YAML or TOML). By default, it looks for a file named spot.yml and executes the default target with a concurrency of 1.

    Basic Execution

    • Run default playbook: spot
    • Run specific playbook: spot -p my_playbook.yml
    • Run a specific task: spot --task=deploy-things
    • Run a specific task for a specific target: spot --task=deploy-things --target=prod
    • Run in dry-run mode (prints commands without executing them): spot -p spot.yml -t prod --dry
    • Run ad-hoc commands on specific hosts: spot "ls -la /tmp" -t dev1.example.com -t dev2.example.com
    spot --task=deploy-things --target=prod