konf

repository·main·Indexed 19 days ago

https://github.com/nil-go/konf

A lightweight, zero-dependency configuration loader for Go that decouples application logic from configuration sources. It supports loading settings from files, environment variables, and cloud providers (AWS, Azure, GCP) via a unified API. Key features include support for hot-reloading with OnChange callbacks, configuration precedence management, and a debugging tool (Explain) to trace value resolution.

Tokens
21.7K
Snippets
118
Records
129
Agent score
62%

What's inside konf

  1. Configure observability and logging

    main

    Konf uses slog.Default() for logging configuration changes and watching status. You can customize this behavior using the following options:

    • konf.WithLogHandler: Change the logger used by the library.
    • konf.WithOnStatus: Register a callback to monitor the status of configuration loading/watching (e.g., for recording metrics).
  2. Implement custom configuration providers

    main

    You can extend konf by implementing the following interfaces:

    • Static Providers: Implement the Loader interface (e.g., for fs).
    • Dynamic Providers: Implement both the Loader and Watcher interfaces (e.g., for appconfig).
  3. Quickstart: Initialize and use konf

    main

    To use konf, you typically perform two distinct steps:

    1. Setup (Application Entrypoint): Early in your application (e.g., in main()), create a konf.Config instance, load your desired sources (files, environment variables, etc.), and call konf.SetDefault(config) to make the configuration globally accessible.
    2. Consumption (Application Logic): Throughout the rest of your application, use the package-level konf functions to read values without needing to know the underlying source.

    This decoupling allows you to change your configuration source (e.g., moving from a local file to AWS AppConfig) without changing your business logic.

    // 1. Setup in main()
    func main() {
        var config konf.Config
    
        // Load from embed FS
        if err := config.Load(fs.New(config, "config/config.json")); err != nil {
            // handle error
        }
    
        // Load from environment variables
        if err := config.Load(env.New(env.WithPrefix("server"))); err != nil {
            // handle error
        }
    
        // Watch for changes
        go func() {
          if err := config.Watch(ctx); err != nil {
            // handle error
          }
        }()
    
        // Make it global
        konf.SetDefault(config)
    }
    
    // 2. Consumption in application logic
    func (app *appObject) Run() {
        serverConfig := struct {
            Host string
            Port int
        }{
            Host: "localhost",
            Port: "8080",
        }
    
        // Read configuration into a struct
        if err := konf.Unmarshal("server", &serverConfig); err != nil {
            // handle error
        }
    
        // Register a callback for when "server" config changes
        konf.OnChange(func() {
          // Reconfigure application
        }, "server")
    }
  4. Blob provider requirements and behavior

    main

    Permissions

    To access blobs, the identity used must have the Storage Blob Data Reader role.

    Change Detection Logic

    • Polling: The provider uses ETags to detect if the blob has changed. If the ETag matches the last known version, no reload occurs.
    • Events: Only Microsoft.Storage.BlobCreated events trigger a reload. Other event types are ignored.
    • Data Format: By default, the provider expects the blob content to be JSON. You can customize the unmarshaling behavior using Options (if available in the package).
  5. How configuration precedence and merging works

    main

    The Config object manages a collection of providers. When you call Load(loader), the new values are merged into the existing configuration state.

    1. Merging: konf uses a merge strategy where newer loaders overwrite existing keys from older loaders.
    2. Path Resolution: When you Unmarshal or Explain a path, konf traverses the merged map to find the most specific value.
    3. Immutability during Read: While Load updates the configuration, Unmarshal and sub operations use atomic pointer swaps. This ensures that reading configuration is non-blocking and consistent even while new configuration is being loaded.
  6. Use the Azure Blob Storage provider

    main

    The azblob package provides a Blob provider that loads configuration from Azure Blob Storage. It supports both periodic polling and event-driven updates via Cloud Events.

    To use it, call New(endpoint, container, blob, opts...). By default, it uses azidentity.DefaultAzureCredential for authentication, which requires the Storage Blob Data Reader role to access the blob.

    import "github.com/nil-go/nil-go/konf/provider/azblob"
    
    // endpoint, container, and blob are your Azure storage details
    provider := azblob.New("https://mystorage.blob.core.windows.net", "my-container", "config.json")
    
    // Load the configuration once
    config, err := provider.Load()
  7. Use the PFlag provider to load configuration from command-line flags

    main

    The pflag package allows you to load configuration from flags defined by the [spf13/pflag] library. It converts flags into a nested map[string]any based on a delimiter (defaulting to .).

    Key Behaviors:

    • Prefix Filtering: You can specify a prefix so that only flags starting with that prefix are loaded.
    • Nesting: Flag names like parent.child.key="1" are parsed into nested maps: {parent: {child: {key: "1"}}}.
    • Smart Merging: To prevent overriding values set by other providers, PFlag skips flags that have not been explicitly changed on the command line AND have a zero default value (e.g., 0, "", false, or []).
    • Conflict Prevention: If a key already exists in the konf instance, PFlag will only merge the flag value if it was explicitly set in the command line.
    // Example usage concept
    // Assuming you have a konf instance and pflag defined
    provider := pflag.New(myKonf, pflag.WithPrefix("app."))
    config, err := provider.Load()
    if err != nil {
        log.Fatal(err)
    }
    // config will contain flags like --app.database.port
  8. Use the GCS provider to load configuration

    main

    The gcs package allows you to load configuration from a Google Cloud Storage (GCS) bucket and object. It supports both periodic polling and event-driven updates via Pub/Sub.

    Prerequisites

    The target GCS object requires the following IAM role:

    • roles/storage.objectViewer

    Basic Usage

    Use gcs.New(uri) where the URI follows the format gs://bucket/object or bucket/object.

    import "github.com/nil-go/nil-go/konf/provider/gcs"
    
    // Create a new GCS provider
    provider := gcs.New("gs://my-bucket/config.json")
    
    // Load the configuration once
    values, err := provider.Load()
    if err != nil {
        panic(err)
    }
  9. Use GCP Secret Manager as a configuration provider

    main

    The SecretManager provider allows you to load and watch configuration stored in Google Cloud Platform (GCP) Secret Manager.

    Prerequisites

    • The target GCP project must have the roles/secretmanager.viewer role assigned.
    • If running on GCP, the provider can automatically detect the Project ID via metadata.

    Key Features

    • Automatic Loading: Fetches secret values using the latest version of each secret.
    • Polling: Periodically checks for changes in secret ETags.
    • Event-Driven Updates: Can be integrated with Pub/Sub to trigger updates immediately when secrets change.
    • Hierarchical Keys: Supports mapping secret names to nested configuration keys using a custom splitter (defaults to splitting by -).
    import "github.com/nil-go/konf/provider/secretmanager"
    
    // Create a new provider
    provider := secretmanager.New()
    
    // Load configuration once
    config, err := provider.Load()
    if err != nil {
        panic(err)
    }
  10. Watch for configuration changes in GCS

    main

    The Watch method allows you to react to configuration changes in GCS. It supports two modes of change detection:

    1. Polling (Default): Periodically checks the object for changes.
    2. Pub/Sub Notifications: If you have configured Pub/Sub notifications for Cloud Storage, you can trigger updates manually by calling OnEvent with the appropriate attributes.

    Watching with a callback

    Pass a context and an onChange function that is executed whenever the configuration is updated.

    Triggering updates via Pub/Sub

    To use event-driven updates, call OnEvent(attributes) with the following map keys:

    • bucketId: The name of the bucket.
    • objectId: The name of the object.
    • eventType: Must be OBJECT_FINALIZE to trigger a reload. Other event types are ignored.
    ctx, cancel := context.WithCancel(context.Background())
    // ... handle cancellation ...
    
    err := provider.Watch(ctx, func(newConfig map[string]any) {
        fmt.Printf("Config updated: %v\n", newConfig)
    })
    
    // If receiving Pub/Sub events, call OnEvent:
    // err := provider.OnEvent(map[string]string{
    //     "bucketId": "my-bucket",
    //     "objectId": "config.json",
    //     "eventType": "OBJECT_FINALIZE",
    // })
  11. Integrate Cobra with the pflag provider

    main

    To use cobra commands as a configuration source, use the pflag loader with the pflag.WithFlagSet option.

    config.Load(kflag.New(&config, kflag.WithFlagSet(yourCobraCmd.Flags())))
  12. Debug configuration with konf.Explain

    main

    If you are loading configuration from multiple sources (e.g., a mix of local files and cloud providers), use Config.Explain() to understand how konf resolved a specific value. It provides a trace of which loaders contributed to the final value and blurs sensitive information like passwords or API keys.

    // Example output:
    // config.nest has value [map] is loaded by map.
    // Here are other value(loader)s:
    //   - env(env)