operator-rs

repository·main·Indexed 41 days ago

https://github.com/stackabletech/operator-rs

A Rust library providing shared helper methods, structs, and enums for building Kubernetes Operators, designed to be used alongside the kube-rs crate. It includes utilities such as config-utils for configuration templating and escaping (supporting .properties and XML), containerdebug for collecting system metrics from Docker containers, k8s-version for parsing Kubernetes API versions, and the stackable-versioned crate for managing struct versioning and field lifecycles.

Tokens
53.8K
Snippets
157
Records
209
Agent score
50%

What's inside operator-rs

  1. Overview of Container Support Helper

    main

    The containerdebug tool is designed to assist support engineers by collecting and printing detailed environment information from within a running Docker container. It captures a wide range of system metrics to aid in debugging containerized environments.

    Collected information includes:

    • CPU: Core count and architecture.
    • Memory: Free, total, and available memory, including CGroup limits and swap information.
    • OS/Kernel: Host name, kernel version, operating system name, and CPU architecture.
    • Storage: Mount points and available space on those points.
    • Network: Network interfaces (names and IP addresses), reverse DNS lookups (IP $\rightarrow$ hostnames), and forward DNS lookups (hostname $\rightarrow$ IPs).
  2. Overview of Stackable operator-rs

    main

    The operator-rs library is a Rust-based toolkit designed to simplify the development of Kubernetes Controllers and Operators. It provides a collection of helper methods, structs, and enums specifically intended to be used alongside the kube-rs crate.

    It serves as the foundational shared library for the Stackable Data Platform, ensuring consistency and interoperability across various data application operators (such as Apache Kafka, Spark, and Trino) running on Kubernetes.

  3. Version structs using the versioned macro

    main

    The stackable-versioned crate allows you to declare multiple versions for a struct (and eventually enums) using the #[versioned] attribute macro. This is primarily used to support Kubernetes API versioning by declaring a sequence of version names on a single data type.

    You can also annotate individual fields within the struct to track their lifecycle, including when they were added, when they changed (including name changes), or when they were deprecated.

    use stackable_versioned::versioned;
    
    #[versioned(
        version(name = "v1alpha1"),
        version(name = "v1beta1"),
        version(name = "v1"),
        version(name = "v2"),
        version(name = "v3")
    )]
    struct Foo {
        /// My docs
        #[versioned(
            added(since = "v1alpha1"),
            changed(since = "v1beta1", from_name = "gau"),
            deprecated(since = "v2", note = "not required anymore")
        )]
        deprecated_bar: usize,
        baz: bool,
    }
  4. Use templating with config-utils

    main

    The config-utils utility supports templating to fill configuration files with values from environment variables or external files. This process involves replacing placeholders in your configuration with actual data, while automatically applying the appropriate escaping for the file format (e.g., XML escaping).

    Placeholder Syntax

    • Environment Variables: Use ${env:VARIABLE_NAME} to inject the value of an environment variable.
    • Files: Use ${file:ENCODING:PATH} to inject the contents of a file. For example, ${file:UTF-8:example-password} reads the file example-password using UTF-8 encoding.
    • Nested Templating: You can nest placeholders. For example, ${env:${env:ENV_TEST_PASSWORD_ENV_NAME}} allows the name of the environment variable to be determined by another environment variable.

    Supported Formats

    config-utils automatically detects the file type based on the extension, but you can also specify it manually via CLI arguments. Currently supported formats are:

    • .properties files
    • XML files
    # Example setup
    export EXAMPLE_USERNAME=my-user
    echo 'example-password <123>!' > example-password
    
    # Create a template file
    cat > example.xml << 'EOF'
    <credentials>
      <username>${env:EXAMPLE_USERNAME}</username>
      <password>${file:UTF-8:example-password}</password>
    </credentials>
    EOF
    
    # Run templating
    config-utils template example.xml
  5. How to use operator-rs for Kubernetes Operator development

    main

    To build a Kubernetes Operator using operator-rs, you should integrate it into a Rust project that already utilizes the kube-rs crate.

    While this repository does not host its own standalone documentation, you can learn how to implement its patterns by examining the source code of the official Stackable operators. These operators use operator-rs to implement shared logic for managing data applications on Kubernetes.

    Recommended learning path:

    1. Use kube-rs as your primary Kubernetes client/controller framework.
    2. Incorporate operator-rs to access shared helper types and logic.
    3. Reference official operator implementations (e.g., kafka-operator, spark-k8s-operator) to see real-world usage of the library's abstractions.
  6. Configure log output for containerdebug

    main

    By default, containerdebug writes log-style output to stdout. To redirect these logs to files for ingestion into log aggregation systems (like Vector), set the CONTAINERDEBUG_LOG_DIRECTORY environment variable.

    When outputting to files, the logs are written in JSON format and include automatic log rotation.

    export CONTAINERDEBUG_LOG_DIRECTORY=/path/to/logs/directory
  7. Configure product logging in Custom Resource Definitions

    main

    Logging configuration is defined within Custom Resource Definitions (CRDs) to control how different containers within a product emit logs. You can globally enable or disable the Vector log agent and specify per-container logging configurations.

    There are two primary modes for container log configuration:

    1. Automatic: The operator manages loggers, console appenders, and file appenders based on provided thresholds.
    2. Custom: The operator uses a specific log configuration provided via a Kubernetes ConfigMap.

    Custom configurations take precedence over automatic ones.

    {
      "enableVectorAgent": true,
      "containers": {
        "product": {
          "loggers": {
            "ROOT": { "level": "INFO" }
          },
          "console": { "level": "DEBUG" },
          "file": { "level": "WARN" }
        },
        "vector": {
          "custom": { "configMap": "my-logging-config" }
        }
      }
    }
  8. Versioned item attributes for fields and variants

    main

    When using the stackable-versioned macros, you can manage the lifecycle of struct fields or enum variants using versioned attributes. These attributes allow you to track when an item was added, when its name or type changed, and when it was deprecated.

    Supported Actions

    1. added: Marks the version in which the field or variant was introduced.

      • since: The version when it was added.
      • default_fn: (Optional) The path to a function used to provide a default value for this item. Defaults to ::std::default::Default::default.
    2. changed: Marks a modification to an existing item.

      • since: The version when the change occurred.
      • from_name: (Optional) The previous name of the item. This must not include the deprecation prefix.
      • from_type: (Optional) The previous type of the item. If provided, you can use upgrade_with and downgrade_with.
      • upgrade_with: (Optional) A path to a function used to upgrade the value from the old type to the new type. Only valid if from_type is set.
      • downgrade_with: (Optional) A path to a function used to downgrade the value from the new type to the old type. Only valid if from_type is set.
    3. deprecated: Marks the version in which the item should no longer be used.

      • since: The version when it was deprecated.
      • note: (Optional) A descriptive note about the deprecation.
      • Requirement: When an item is marked as deprecated, its name must include the configured deprecation prefix.

    Usage Rules

    • Chronology: added < changed < deprecated.
    • Exclusivity: You cannot use added and deprecated (or added and changed) in the same version.
    • Naming: If an item is deprecated, it must use the deprecation prefix. If it is not deprecated, it must not use the prefix.
    • No direct #[deprecated]: Do not use the standard Rust #[deprecated] attribute on items. Instead, use the macro-provided #[versioned(deprecated(since = "..."))] syntax.
    // Example of versioning a field
    struct MyStruct {
        #[versioned(added(since = "0.1.0"))]
        new_field: String,
    
        #[versioned(changed(since = "0.2.0", from_name = "old_field", from_type = "u32"))]
        current_field: String,
    
        #[versioned(deprecated(since = "0.3.0", note = "Use current_field instead"))]
        deprecated_field_name: String,
    }
  9. Understand JdbcDatabaseConnectionDetails

    main

    The JdbcDatabaseConnectionDetails struct represents the resolved connection parameters required to connect to a database via JDBC. It is used by the operator to inject the necessary configuration into application containers.

    Fields:

    • driver: The Java class name of the driver (e.g., org.postgresql.Driver).
    • connection_url: The JDBC connection URL (excluding user/password).
    • username_env: An optional EnvVar that mounts the credentials Secret and provides the username.
    • password_env: An optional EnvVar that mounts the credentials Secret and provides the password.

    You can use the add_to_container method to automatically inject the required username and password environment variables into a ContainerBuilder instance.

  10. Use ObjectLabels for standard Stackable metadata

    main

    The ObjectLabels<'a, T> struct provides a recommended set of labels to apply to objects created by Stackable operators. This ensures consistency across managed resources.

    Fields include:

    • owner: A reference to the K8s object owning the resource (e.g., HdfsCluster).
    • app_name: The name of the managed app (e.g., zookeeper).
    • app_version: The version of the app (e.g., 3.0.0-stackable23.11).
    • operator_name: The DNS-style name of the managing operator (e.g., zookeeper.stackable.tech).
    • controller_name: The specific controller inside the operator.
    • role: The role of the object.
    • role_group: The role group of the object.
  11. How versioned modules work in stackable-versioned-macros

    main

    The stackable-versioned-macros crate provides a mechanism for versioning multiple containers (structs or enums) simultaneously within a single module block. This allows developers to manage multiple API versions (e.g., v1alpha1, v1) without creating conflicting definitions.

    Core Concepts

    • Versioned Modules: A macro-driven module block that contains definitions for different versions of the same data structures.
    • Containers: The actual data structures (structs or enums) that are versioned. The macro identifies these within the module block and generates version-specific implementations.
    • Submodules: Used to manage imports or re-exports for specific versions. Submodules must be defined as mod blocks within the versioned module and can only contain use statements.
    • Preserving the Module: By default, the macro may erase the wrapping module. However, if the preserve_module option is enabled, the macro generates a public module that wraps the versioned submodules, making them accessible via the original module name.

    Constraints

    • Submodule Naming: Submodule names must match the names defined in the version attributes.
    • Submodule Content: Submodules are restricted to use statements only. Any other items (structs, functions, etc.) inside a submodule will trigger a compilation error.
    • Disallowed Items: Any items placed inside the versioned module that are not recognized as containers (structs/enums) or submodules (mod blocks with use statements) are disallowed and must be moved outside the macro block.
  12. Use the Duration struct for timeouts and retries

    main

    The Duration struct is a wrapper around std::time::Duration designed for use in Stackable operators. It implements Deref and DerefMut, meaning you can call all standard std::time::Duration methods directly on a Duration instance.

    It is recommended to use Duration instead of std::time::Duration whenever dealing with human-readable configuration values like timeouts, retries, or lifetimes.

    use stackable_shared::time::duration::Duration;
    use std::time::Duration as StdDuration;
    
    let dur: Duration = "10s".parse().unwrap();
    // Use std::time::Duration methods via Deref
    let secs = dur.as_secs();