Consul Template

repository·main·Indexed 26 days ago

https://github.com/hashicorp/consul-template

A tool that renders configuration files from dynamic data sources like Consul and Vault, designed for service discovery and secret management in distributed systems. It supports template watching, an exec mode for process supervision, and various CLI flags for managing Consul and Vault connectivity, SSL/TLS, and logging.

Tokens
13.7K
Snippets
47
Records
75
Agent score
82%

What's inside consul-template

  1. Understand the Consul Template templating language

    main

    Consul Template uses the Go Template format. In addition to standard Go template functions, Consul Template provides several specialized categories of functions to interact with Consul, Nomad, secrets, and local files.

    Key function categories include:

    • API Functions: For interacting with Consul services, keys, nodes, and partitions.
    • Scratch: For managing temporary variables during template execution.
    • Helper Functions: For string manipulation, encoding/decoding, and data parsing (JSON, YAML, TOML).
    • Math Functions: For basic arithmetic and comparisons.
    • Nomad Functions: For accessing Nomad-specific service data.
    • Debugging Functions: For inspecting data structures during template development.
  2. Run an application in Exec Mode

    main

    Exec mode allows Consul Template to spawn and manage the lifecycle of a child process. This is ideal for containers or schedulers like Nomad and Kubernetes. The child process is spawned only after all templates have been rendered at least once.

    Key Behaviors

    • Lifecycle Management: If the child process dies, Consul Template also dies. Consul Template does not supervise the process; supervision should be handled by your scheduler or init system.
    • Foreground Requirement: The child process must remain in the foreground.
    • Signal Handling:
      • Consul Template proxies most signals to the child process.
      • When a template changes, Consul Template sends a configurable reload_signal to the child process. If no reload signal is defined, it will kill the process and spawn a new instance.
      • When Consul Template is stopped gracefully, it sends a configurable kill_signal (default is SIGTERM) to the child process.
    • Command Execution: Commands are run using sh -c. For complex shell-like commands, ensure sh is on your PATH. For simpler usage, provide a single command with arguments.
    • Limitations: You can only have one exec command per Consul Template process, though individual templates can still have their own reload commands.
    $ consul-template \
        -template "/tmp/config.ctmpl:/tmp/server.conf" \
        -exec "/bin/my-server -config /tmp/server.conf"
  3. Use Once Mode to wait for dependencies

    main

    In Once mode, Consul Template waits for all dependencies to be rendered before completing. If a template specifies a dependency (a request) that does not exist in Consul, Once mode will wait until Consul returns data for that dependency. Note that an empty response (e.g., no healthy services found) is considered a valid response and will not cause the process to continue waiting.

    Important: Once mode implicitly disables any wait/quiescence timers specified in configuration files or via the command line.

    To enable Once mode, use the -once flag or set it in your configuration file.

  4. Reload Configuration and Templates using SIGHUP

    main

    When running Consul Template as a system service, changes made to configuration files or templates on disk are not automatically propagated to the running process. This design allows for pre-flight validation and simultaneous updates.

    To trigger a reload of all configurations and templates from disk, send a SIGHUP signal to the running consul-template process.

    Example using kill:

    kill -HUP <pid>
  5. Configure Consul Template with an HCL configuration file

    main

    Consul Template supports configuration via HCL (HashiCorp Configuration Language) or JSON files. Use the -config flag to specify the path to your configuration file.

    Precedence Rules:

    1. Commands specified on the CLI take highest precedence.
    2. The right-most configuration file provided via -config takes precedence over earlier ones.
    3. If a directory is provided to -config, all files in that directory are merged in lexical order (recursively, but symbolic links are not followed).

    Security Note: For security, use the CONSUL_TOKEN or VAULT_TOKEN environment variables instead of putting plain-text tokens in your configuration files.

    $ consul-template -config "/my/config.hcl"
  6. Handle multi-phase template evaluation errors

    main

    Consul Template performs multiple passes to resolve nested dependencies. During the first pass, some queries (like service "foo") may return empty results. If your template attempts to access an index on an empty result immediately, it will trigger an error.

    Incorrect pattern (will fail on first pass):

    {{ with index (service "foo") 0 }}
    # ...
    {{ end }}

    Correct pattern (safe for all passes): Wrap the index access inside a with block for the service itself to ensure the slice is not empty before indexing.

    {{ with service "foo" }}
    {{ with index . 0 }}
    {{ .Node }}{{ end }}
    {{ end }}
  7. Prevent Consul Template from exiting on command failure

    main

    By default, Consul Template will exit with a non-zero status if an optional command exits with a non-zero status. This allows process managers like Upstart or God to manage the service.

    To keep Consul Template running even if the command fails, append || true to your command and wrap it in a shell execution.

    Example CLI usage:

    $ consul-template \
      -template "in.ctmpl:out.file:/bin/bash -c 'service nginx restart || true'"
  8. Enable De-Duplication Mode to reduce Consul queries

    main

    De-Duplication mode allows multiple instances of Consul Template rendering the same template to share work. It uses leader election on a per-template basis so only one node performs the queries. Results are shared via compressed data passed through the Consul K/V store.

    Security Note: Vault data is not stored in the compressed template. Consul Template will still request secrets from Vault on each iteration to maintain security boundaries.

    Requirement: If your templates use local functions like env, the environment variables must be consistent across all machines participating in the de-duplication, otherwise the template will fail to resolve.

    To enable De-Duplication mode, use the -dedup flag or use the deduplicate configuration block.

  9. Use command line flags to render templates

    main

    You can use the consul-template CLI to render templates directly without a configuration file.

    Common patterns include:

    • Rendering a single template to a destination file.
    • Rendering multiple templates in a single process, optionally specifying a command to execute whenever a template changes.
    • Specifying custom Consul or Vault addresses.
    • Using -exec to spawn and monitor a child process as a supervisor after templates are rendered.
  10. Handle signals in Exec mode using trap or exec

    main

    When running shell commands in Exec mode, the shell process itself might intercept signals intended for the managed process. Use one of these two patterns to ensure correct signal propagation:

    Option 1: Use trap to ignore signals in the shell This allows the shell to catch the signal (e.g., HUP) and do nothing, letting the underlying process handle it (though the shell might still exit if not careful). command = "trap '' HUP; /usr/sbin/nginx -c /etc/nginx/nginx.conf"

    Option 2: Use exec to replace the shell (Recommended) This replaces the shell process with the target process, keeping the same PID and allowing the target to receive signals directly. command = "exec /usr/sbin/nginx -c /etc/nginx/nginx.conf"

  11. Configure Docker permissions for Consul Template

    main

    When using the Alpine Docker image, the consul-template user has a UID of 100 and a GID of 1000.

    If you mount an external volume to render shared templates, you must ensure the consul-template user has write permissions to that directory. This applies to:

    • /consul-template/config: used for adding configuration when using the image as a parent.
    • /consul-template/data: exported as a VOLUME for rendering shared results.

    If building a custom image based on the official one, you can override these values using --build-arg parameters.