Kubetail Documentation

repository·main·Indexed 23 days ago

https://github.com/kubetail-org/kubetail

A real-time logging dashboard for Kubernetes that enables developers to view merged, chronological logs from multi-container workloads via a web UI or terminal. The ecosystem includes a Rust-based Cluster Agent for node-specific log information, the RGKL (RipGrep for Kubernetes Logs) search package, a React-based Dashboard UI, and a Cluster API.

Tokens
29.1K
Snippets
57
Records
185
Agent score
82%

What's inside Kubetail

  1. Overview of RipGrep for Kubernetes Logs (RGKL)

    main
    RGKL is a Rust package designed to search Kubernetes log files. It achieves this by using ripgrep as a library to perform high-performance grep operations on a file-by-file basis. The package handles the parsing of Kubernetes log files and leverages ripgrep's engine to identify and return matching lines.
  2. What is the Kubetail Cluster Agent?

    main
    The Kubetail Cluster Agent is a small Rust-based gRPC service designed to run on every node in a Kubernetes cluster. It responds to node-specific requests from Kubetail Cluster API instances. Its primary current function is to return real-time information about container log files, such as file size and the timestamp of the last event.
  3. Overview of Kubetail Dashboard UI

    main

    The Kubetail Dashboard UI is a React-based static website designed to interface with the Kubetail Dashboard server. It provides a clean user interface for querying Kubernetes data.

    Key technical details:

    • Data Fetching: Uses Apollo Client for GraphQL queries to the server and standard fetch() for REST API authentication requests.
    • Language: Written in TypeScript for high type safety.
    • Development Tooling: Uses vite for development.
    • Deployment: In production, it is deployed as a static website hosted directly by the Kubetail Dashboard server.
  4. Overview of Kubetail Dashboard Server

    main

    The Kubetail Dashboard Server is a Go-based HTTP server that hosts the Kubetail dashboard web application frontend and manages its backend services.

    Key Responsibilities:

    • Frontend Hosting: Serves the dashboard web app (packaged as a static site in the website directory in production).
    • Backend Proxying: Proxies frontend requests to the user's Kubernetes API and the Kubetail API running inside the cluster.
    • Authentication: Provides custom authentication functionality.
    • API Interfaces: Responds to Kubernetes-related queries via a GraphQL endpoint (powered by gqlgen) and handles other requests via a REST API.

    Technical Stack:

    • Web Framework: Gin
    • Kubernetes Integration: Kubernetes Go-client
  5. Overview of Kubetail Rust Packages

    main

    The crates workspace contains the Rust-based modules used within the Kubetail ecosystem. The primary modules are:

    • cluster_agent: The Kubetail Cluster Agent.
    • rgkl: RipGrep for Kubernetes Logs (RGKL).

    For specific details on how to use or configure these individual modules, refer to their respective directory READMEs.

  6. What is Kubetail and how does it work?

    main

    Kubetail is a real-time logging dashboard for Kubernetes designed to tail logs across multi-container workloads.

    Core Mechanics:

    • Direct API Access: Kubetail uses your cluster's Kubernetes API to fetch logs directly. This means data stays within your possession and no external log forwarding is required by default.
    • Workload Merging: It merges logs from all containers within a workload (e.g., a Deployment or DaemonSet) into a single, chronological timeline.
    • Lifecycle Tracking: It tracks container lifecycle events via the Kubernetes API to keep the log timeline in sync as containers are created, stopped, or replaced.
    • Filtering: Users can filter logs by workload type (Deployment, CronJob, etc.), time ranges, node properties (AZ, CPU architecture, Node ID), and via grep-style searches.
  7. Develop the Rust-based Cluster Agent

    main

    By default, the development environment uses a pre-built image for the Rust-based cluster-agent. To work on the Rust source code, set the KUBETAIL_RUST_DEV_MODE environment variable when running tilt up.

    Docker Mode

    Compiles the Rust code inside a Docker container. No local Rust toolchain is required.

    KUBETAIL_RUST_DEV_MODE=docker tilt up

    Local Mode

    Compiles the Rust code on your machine using cross-compilation. This is faster for rebuilds but requires local setup.

    Dependencies:

    • rustup
    • protobuf
    • A cross-compiler toolchain:
      • macOS: brew install FiloSottile/musl-cross/musl-cross
      • Linux (Ubuntu): apt-get install musl-tools

    Setup Steps:

    1. Add the required Rust target:
      # x86_64
      rustup target add x86_64-unknown-linux-musl
      
      # aarch64
      rustup target add aarch64-unknown-linux-musl
    2. For macOS users, configure the linker in ~/.cargo/config.toml:
      [target.x86_64-unknown-linux-musl]
      linker = "x86_64-linux-musl-gcc"
      
      [target.aarch64-unknown-linux-musl]
      linker = "aarch64-linux-musl-gcc"
    3. Run Tilt in local mode:
      KUBETAIL_RUST_DEV_MODE=local tilt up
  8. Generate a self-signed Certificate Authority (CA) for development

    main

    To set up a local development environment using TLS, you must first generate a Certificate Authority (CA). This involves creating a 4096-bit RSA private key and then using that key to create a self-signed CA certificate valid for 10 years (3650 days).

    # 1. Generate the CA private key
    openssl genpkey \
      -algorithm RSA \
      -out ca.key \
      -pkeyopt rsa_keygen_bits:4096
    
    # 2. Create the self-signed CA cert (10 years valid)
    openssl req -x509 \
      -new -nodes \
      -key ca.key \
      -config csr.cnf \
      -days 3650 \
      -out ca.crt
  9. Run tests and linting for Kubetail modules

    main

    You can run tests and quality checks using either the standard Go toolchain or the project's Makefile.

    Using Go Toolchain

    To run the test suite directly:

    go test ./...

    Using Makefile

    From the project root, use the following commands:

    • Run tests for all modules (including dashboard): make modules-test
    • Run linter: make modules-lint
    • Run code vetting: make modules-vet
    • Run all Dashboard checks at once: make modules-all
    # Run tests for all modules including dashboard
    make modules-test
    
    # Run linter
    make modules-lint
    
    # Run code vetting
    make modules-vet
    
    # Run all Dashboard checks at once
    make modules-all
  10. Development commands for Kubetail Rust packages

    main

    If you are contributing to the Rust packages in the crates workspace, use the following Cargo commands to maintain code quality and build the project:

    Linting and Formatting

    • Check formatting: Ensures code adheres to the project's style guidelines.
    • Run Clippy: Runs the Rust linter and treats warnings as errors to ensure high code quality.

    Testing and Building

    • Run tests: Executes the test suite for the workspace.
    • Run builder: Compiles the workspace in release mode for optimized performance.
    # Run linter
    cargo fmt --all -- --check
    
    # Run clippy
    cargo clippy --all -- -D warnings
    
    # Run tests
    cargo test
    
    # Run builder
    cargo build --release