HashiCorp envconsul

repository·main·Indexed 24 days ago

https://github.com/hashicorp/envconsul

A tool that launches subprocesses with environment variables automatically populated from HashiCorp Consul or Vault. It follows 12-factor app principles, allowing applications to read standard environment variables while remaining unaware of the configuration source. envconsul supports polling for configuration changes to restart processes, HCL/JSON configuration, and advanced environment variable transformations such as sanitization and upcasing.

Tokens
6.2K
Snippets
18
Records
43
Agent score
83%

What's inside envconsul

  1. Customize Vault secret environment variable names

    main

    You can control how Vault secrets are mapped to environment variables using several methods:

    1. Disable path prefixing: Set no_prefix = true in the secret block.
    2. Apply key transformations: Use the -upcase flag to uppercase all keys.
    3. Use format strings: Use the format option in the secret block. The {{ key }} placeholder is replaced by the key name. This is processed by the Go formatter.
    4. Per-key configuration: Define specific key blocks within a secret block to apply unique format or name settings to individual keys.

    Note: Per-key format only works if there is no format defined at the secret level.

    secret {
      path   = "secret/passwords"
      format = "creds_{{ key }}"
    
      key {
        name   = "username"
        format = "readonly_user_{{ key }}"
      }
      key {
        name   = "password"
        format = "custom_prefix_{{ key }}"
      }
    }
  2. Understand Envconsul signal handling

    main

    Envconsul manages several signals to control its own lifecycle and the lifecycle of the child process:

    • kill_signal: The signal Envconsul listens for to shut itself down. This can be different from the signal sent to the child.
    • reload_signal: The signal Envconsul listens for to reload its configuration. If set to an empty string, reloading is disabled.
    • exec.kill_signal: The signal Envconsul sends to the child process to initiate a graceful shutdown.
  3. Install envconsul via pre-compiled binaries

    main

    To install envconsul using pre-compiled binaries:

    1. Download a released version from the [envconsul releases] page (available as zip or tarball).
    2. Extract the binary using unzip or tar.
    3. Move the binary into your $PATH to make it available globally.
  4. Configure Envconsul with an HCL file

    main

    Envconsul can be configured using HashiCorp Configuration Language (HCL) or JSON. Use the -config flag to specify the configuration file(s).

    Precedence Rules:

    1. CLI arguments take precedence over configuration files.
    2. Vault secrets always take precedence over Consul prefixes (for security).
    3. When multiple configuration files are provided via -config, the right-most file takes highest precedence. If a directory is provided, files are merged in lexical order.

    To use a configuration file:

    $ envconsul -config "config.hcl"
  5. Install envconsul from source (Go)

    main

    To build envconsul from source, ensure you have Go and common build tools installed.

    1. Clone the repository:
      git clone https://github.com/hashicorp/envconsul.git
      cd envconsul
    2. Build using the development make target:
      make dev
    3. Alternatively, build for a specific platform and architecture using the following pattern:
      make darwin/amd64 # or linux/amd64 or windows/amd64, etc

    The resulting binary will be located in pkg/OS_ARCH.

    $ git clone https://github.com/hashicorp/envconsul.git
    $ cd envconsul
    $ make dev
    $ make darwin/amd64
  6. Configure the child process execution (exec)

    main

    The exec { ... } block defines how the child process is managed. Key options include:

    • command: The command to execute.
    • splay: A random delay before killing the command to prevent thundering herds during reloads.
    • env: Configuration for the child's environment:
      • pristine: If true, the child does not inherit the parent's environment.
      • custom: Additional environment variables to inject.
      • allowlist: Glob patterns of variables to include.
      • denylist: Glob patterns of variables to exclude.
    • kill_signal: The signal sent to the child for graceful termination.
    • kill_timeout: Duration to wait for graceful termination before a hard kill (default 30s).
  7. Configure Vault connection settings

    main

    The vault { ... } block defines how Envconsul interacts with Vault. Key options include:

    • address: The Vault leader address (requires protocol, e.g., https://...).
    • namespace: Vault Enterprise namespace (can also be set via VAULT_NAMESPACE).
    • token: The Vault token (can also be set via VAULT_TOKEN).
    • unwrap_token: If true, treats the provided token as a wrapped token.
    • renew_token: Automatically renews the top-level Vault token. Note: secrets specified as a prefix are always renewed regardless of this setting.
    • k8s_auth_role_name, k8s_service_account_token_path, k8s_service_account_token, k8s_service_mount_path: Options for Kubernetes authentication.
    • retry and ssl: Reuses the same configuration structures as the Consul section.
  8. Configure Vault integration

    main

    To pull secrets from Vault into your environment, you must provide Vault connection details. This can be done via a configuration file or command-line flags. The data must be "flat" (keys and values must be strings or string-like); envconsul will error if it encounters a map.

    vault {
      address     = "https://vault.service.consul:8200"
      token       = "abcd1234" # May also be specified via the envvar VAULT_TOKEN
      renew_token = true
    
      ssl {
        enabled = true
        verify  = true
        cert    = "/path/to/client/cert.pem"
        ca_cert = "/path/to/ca/cert.pem"
      }
    }
  9. Configure Consul prefix watching

    main

    The prefix { ... } block defines which Consul paths to watch. You can specify multiple prefix blocks. Key options include:

    • path: The path in Consul to watch (required).
    • format: A custom formatter using Go template syntax (e.g., custom_{{ key }}).
    • no_prefix: If true, does not prefix keys with their parent folder.
  10. How Runner handles environment variable updates

    main

    The Runner maintains an internal environment state. When a dependency change is detected via the watcher, the following sequence occurs:

    1. Data Collection: The runner collects data from all dependencies (KV, Vault, or Services).
    2. Environment Construction: It builds a new environment map. If the configuration is not set to Pristine, it starts with the current process's environment (os.Environ()).
    3. Transformation:
      • Prefixing: Keys are prefixed with the dependency path (e.g., path_key).
      • Formatting: Custom templates can be applied to keys.
      • Sanitization: Non-alphanumeric characters (except _) can be replaced with underscores.
      • Casing: Keys can be forced to uppercase.
    4. Filtering: The environment is filtered through Allowlist and Denylist rules defined in the configuration.
    5. Custom Overrides: Any Custom environment variables defined in the config are applied last, taking precedence over all other sources.
    6. Process Restart: If the new environment differs from the previous one, the existing child process is stopped and a new one is spawned with the updated cmdEnv.
  11. Configure Consul connection settings

    main

    The consul { ... } block defines how Envconsul interacts with the Consul agent or server. Key options include:

    • address: The address of the Consul agent (default 127.0.0.1:8500).
    • token: The ACL token for Consul. Can also be set via CONSUL_TOKEN env var.
    • auth: A block for basic authentication (enabled, username, password).
    • retry: Configures exponential back-off behavior (enabled, attempts, backoff, max_backoff).
    • ssl: Configures SSL/TLS (enabled, verify, cert, key, ca_cert, ca_path, server_name).
    • max_stale: Maximum interval to allow "stale" data from followers instead of the leader.