vals

repository·main·Indexed 20 days ago

https://github.com/helmfile/vals

A tool for managing configuration values and secrets by replacing URI-like references in YAML/JSON documents with actual values from backends such as Vault, AWS (SSM, Secrets Manager, S3, KMS), GCP, Azure, and others. It provides a CLI for evaluating templates, managing environment variables, and decoding Kubernetes secrets, and can be integrated into Go applications or GitOps workflows.

Tokens
19.7K
Snippets
74
Records
86
Agent score
73%

What's inside vals

  1. What vals is not designed for

    main

    To maintain its focus on value composition, vals has explicit non-goals:

    • Complex String-Interpolation / Template Functions: vals is not a full-fledged YAML templating engine. It is intended for composing sets of values to be consumed by other engines (like Jsonnet or CUE), rather than performing complex data transformations or function-based manipulations.
    • YAML Merging: Merging multiple YAML files is out of scope. For complex merging logic, use tools like Jsonnet, Sprig, or CUE.
  2. Understand vals expression syntax

    main

    vals identifies and replaces expressions following this URI-like pattern:

    ref+<BACKEND>://<PATH>[?<PARAMS>][#<FRAGMENT>][+]

    Components

    • ref+<BACKEND>: The identifier for the backend provider (e.g., ref+vault, ref+aws).
    • <PATH>: The backend-specific path to the secret.
    • ?<PARAMS>: (Optional) Key-value pairs used as query parameters (e.g., ?proto=http). Parameters are separated by & and keys/values by =.
    • #<FRAGMENT>: (Optional) A path-like expression used to extract a specific value from the retrieved secret. vals parses the secret as YAML/JSON and traverses it using this fragment. The native type (string, number, boolean, or object/array) is preserved.
    • +: (Optional) An explicit end-of-expression marker. This is useful for simple string interpolation where you want to ensure the expression stops at a specific point (e.g., foo ref+SECRET1+ bar).
  3. Authenticate to HashiCorp Vault

    main

    The auth_method (or VAULT_AUTH_METHOD environment variable) determines how vals authenticates to Vault.

    Supported Methods:

    • token: Requires a VAULT_TOKEN. If not set, it checks VAULT_TOKEN_FILE or ~/.vault-token.
    • approle: Requires role_id and secret_id (via query params or VAULT_ROLE_ID/VAULT_SECRET_ID env vars).
    • kubernetes: Used when running inside a K8s cluster. Requires a Kubernetes role (role_id or VAULT_ROLE_ID). The login path can be customized via VAULT_KUBERNETES_MOUNT_POINT (default /kubernetes). Custom JWT token paths can be set via VAULT_KUBERNETES_JWT_TOKEN_PATH.
    • userpass: Requires a username (e.g., via VAULT_USERNAME) and a password. The password can be provided via VAULT_PASSWORD_ENV (takes precedence) or VAULT_PASSWORD_FILE.
  4. Authenticate Infisical via GitHub Actions OIDC

    main

    To use Infisical with GitHub Actions via OIDC, your workflow must request an ID token and export it as INFISICAL_AUTH_JWT.

    Requirements:

    1. Set permissions to include id-token: write and contents: read.
    2. Use curl to request the token from the GitHub OIDC endpoint.
    3. Export the token to the $GITHUB_ENV.
    4. Set INFISICAL_AUTH_METHOD to OIDC_AUTH.
    permissions:
      id-token: write
      contents: read
    
    jobs:
      deploy:
        runs-on: ubuntu-latest
        steps:
          - uses: actions/checkout@v4
    
          - name: Request OIDC token
            run: |
              TOKEN=$(curl -sS -H "Authorization: Bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" \
                "${ACTIONS_ID_TOKEN_REQUEST_URL}&audience=https://github.com/${{ github.repository_owner }}" \
                | jq -r '.value')
              echo "::add-mask::$TOKEN"
              echo "INFISICAL_AUTH_JWT=$TOKEN" >> "$GITHUB_ENV"
    
          - name: Resolve secrets with vals
            env:
              INFISICAL_AUTH_METHOD: OIDC_AUTH
              INFISICAL_OIDC_AUTH_IDENTITY_ID: ${{ vars.INFISICAL_OIDC_AUTH_IDENTITY_ID }}
            run: vals eval -f refs.yaml
  5. Discriminate between config and secrets for GitOps

    main

    To support GitOps workflows, vals allows you to distinguish between non-sensitive configuration and sensitive secrets using specific URI prefixes. This enables you to evaluate and review configuration values while keeping secrets masked in your output.

    Usage Pattern

    1. Use ref+<value uri> for non-sensitive configuration values.
    2. Use secretref+<value uri> for sensitive secrets.
    3. Run vals eval --exclude-secret to generate a manifest where configuration values are resolved to their actual content, but secrets remain as un-evaluated URIs. This output is safe to commit to Git.

    This approach allows you to review the impact of configuration changes in your pull requests without accidentally committing actual secret values.

    # Input configuration
    myconfigvalue: ref+awsssm://myconfig/value
    mysecretvalue: secretref+awssecrets://mysecret/value

    Output of vals eval --exclude-secret

    myconfigvalue: MYCONFIG_VALUE
    mysecretvalue: secretref+awssecrets://mysecret/value
  6. Integrate vals with Helm and GitOps

    main

    You can use vals to create GitOps-friendly manifests. By using ref+<BACKEND> URIs in your Helm values, you can commit manifests to version control that contain references instead of actual secrets.

    Workflow

    1. Generate Manifests: Use helm template and pipe the output through vals ksdecode to convert Kubernetes data fields to stringData (which allows vals to replace the values).
    2. Render Secrets: Before applying to the cluster, pipe the manifests through vals eval to inject the real secrets.
    3. Apply: Use kubectl apply on the rendered output.

    Example Workflow:

    # 1. Generate safe manifests
    helm template mysql-1.3.2.tgz --set mysqlPassword='ref+vault://secret/data/foo#/mykey' | vals ksdecode -o yaml -f - | tee manifests.yaml
    
    # 2. Replace refs with actual secrets
    cat manifests.yaml | vals eval -f - | tee all.yaml
    
    # 3. Deploy
    kubectl apply -f all.yaml
    helm template mysql-1.3.2.tgz --set mysqlPassword='ref+vault://secret/data/foo#/mykey' | vals ksdecode -o yaml -f - | tee manifests.yaml
  7. Build vals from source

    main

    To build the standard binary with all providers included:

    go build -o vals ./cmd/vals

    Custom Builds (Reducing Binary Size)

    By default, all providers are compiled into the binary. To create a smaller binary containing only the providers you need, use the custom_providers build tag along with specific provider tags.

    Example: Build with only Vault, AWS, GCP, and Azure providers:

    go build -tags "custom_providers,vault,aws,gcp,azure" -o vals ./cmd/vals

    Available Provider Tags: vault, openbao, aws, gcp, azure, terraform, gitlab, sops, onepassword, doppler, k8s, hcpvaultsecrets, conjur, bitwarden, scaleway, infisical, oci, httpjson, pulumi, secretserver, servercore, keychain, yandex.

    Note: Utility providers (echo, file, envsubst, exec) are always included. Use all_providers to explicitly include everything.

    go build -tags "custom_providers,vault,aws,gcp,azure" -o vals ./cmd/vals
  8. Install vals

    main

    You can install vals using various package managers or by downloading the binary directly.

    Package Managers

    • macOS/Linux (Homebrew): brew install vals
    • Arch Linux: sudo pacman -S vals
    • Alpine Linux Edge: apk add vals
    • macOS (MacPorts): sudo port install vals
    • Nix / NixOS: nix profile install nixpkgs#vals
    • Windows (Scoop): scoop install vals

    Manual Installation

    Download the latest executable for your platform from the GitHub releases page and add it to your PATH.

  9. Retrieve Terraform state from GitLab

    main

    Use the tfstategitlab provider to access Terraform state stored in GitLab. Authentication requires both a username and a token (via HTTP Basic Auth).

    URI Format: ref+tfstategitlab://{gitlab_host}/api/v4/projects/{project_id}/terraform/state/{state_name}/RESOURCE_NAME[?gitlab_user=GITLAB_USER&gitlab_token=GITLAB_TOKEN&gitlab_scheme=http|https]

    Authentication Details:

    • gitlab_user defaults to the GITLAB_USER environment variable.
    • gitlab_token defaults to the GITLAB_TOKEN environment variable.
    • Credentials are sent in the Authorization header and are not exposed in the URL.
    • Both user and token are required; providing only one will result in a 401 error.
    # Example: Using environment variables (GITLAB_USER and GITLAB_TOKEN)
    echo 'foo: ref+tfstategitlab://my-gitlab.com/api/v4/projects/xx/terraform/state/xxx/output.my_output' | vals eval -f -
    
    # Example: Providing credentials via URL parameters
    echo 'foo: ref+tfstategitlab://my-gitlab.com/api/v4/projects/xx/terraform/state/xxx/output.my_output?gitlab_user=username&gitlab_token=token' | vals eval -f -
  10. Use the Servercore secret manager provider

    main

    Retrieve secrets from Servercore Secrets Manager. The provider expects the API to return a base64-encoded string in the version.value field, which it then decodes and attempts to parse as JSON (falling back to YAML).

    Authentication: Set the following environment variables:

    • SERVERCORE_USERNAME
    • SERVERCORE_PASSWORD
    • SERVERCORE_ACCOUNT_ID
    • SERVERCORE_PROJECT_NAME

    URI Formats:

    • ref+servercore://SECRET_NAME: Returns the secret value as a string.
    • ref+servercore://SECRET_NAME#/key/in/secret: Returns the value at the specified leaf key path within the decoded JSON/YAML secret.
  11. Configure HashiCorp Vault backend

    main

    Use the ref+vault:// URI scheme to retrieve secrets from HashiCorp Vault. You can specify the path to the KV backend and the specific field key using a fragment identifier (#).

    Supported Query Parameters:

    • address: Vault server address (defaults to VAULT_ADDR).
    • token_file: Path to a file containing the Vault token.
    • token_env: Name of the environment variable containing the Vault token.
    • namespace: Vault namespace (defaults to VAULT_NAMESPACE).
    • auth_method: Authentication method (defaults to token). Supported: approle, token, kubernetes, userpass.
    • role_id: Used for approle or kubernetes (defaults to VAULT_ROLE_ID).
    • secret_id: Used for approle (defaults to VAULT_SECRET_ID).
    • version: Specific version of the secret to retrieve.
    • decode: Transformation for the retrieved value. Use base64 to decode base64-encoded binary data (e.g., certificates). Defaults to raw.
    # Basic usage with default env vars
    ref+vault://PATH/TO/KVBACKEND#/fieldkey
    
    # Using a specific address and token from an env var
    ref+vault://mykv/foo?address=https://vault1.example.com:8200&token_env=VAULT_TOKEN_VAULT1#/bar
    
    # Using Kubernetes authentication
    ref+vault://mykv/foo?auth_method=kubernetes&role_id=my-kube-role#/bar
    
    # Using userpass authentication with password from an env var
    ref+vault://mykv/foo?auth_method=userpass&username=some-user&password_env=VAULT_PASSWORD#/bar
  12. Retrieve secrets from Azure Key Vault

    main

    The azurekeyvault provider retrieves secrets from Azure Key Vault. The path specifies the vault and the secret name, with an optional version.

    URI Format: ref+azurekeyvault://VAULT-NAME/SECRET-NAME[/VERSION]

    Authentication: Vals uses the azidentity Go module. It attempts authentication in this order:

    1. Environment Variables
    2. Workload Identity
    3. Managed Identity
    4. Azure CLI
    5. Azure Developer CLI

    To force a specific authentication method, set the AZKV_AUTH environment variable to default, workload, managed, cli, or devcli.

    # Example: Accessing a secret
    echo 'foo: ref+azurekeyvault://my-vault/secret-a' | vals eval -f -
    
    # Example: Accessing a specific version of a secret
    echo 'foo: ref+azurekeyvault://my-vault/secret-a/ba4f196b15f644cd9e949896a21bab0d' | vals eval -f -
    
    # Example: Accessing a secret in a non-default Azure cloud
    echo 'foo: ref+azurekeyvault://gov-cloud-test.vault.usgovcloudapi.net/secret-b' | vals eval -f -