DOCSEARCHbyAlgolia
Submit Docs

kubectl-ai

repository·main·Indexed 27 days ago

https://github.com/googlecloudplatform/kubectl-ai

An intelligent Kubernetes management interface that translates natural language user intent into precise Kubernetes operations. It supports various LLM providers including Gemini, OpenAI, Azure OpenAI, Anthropic, and local options like Ollama and LlamaCPP. The project includes the gollm Go library for LLM integration, MCP server configuration for tool extension, and utilities for deploying LLM serving endpoints on GKE or KinD.

Tokens
21.2K
Snippets
49
Records
149
Agent score
92%

What's inside kubectl-ai

  1. Build and deploy LLM serving endpoints

    main
    The modelserving directory contains the necessary components to build, containerize, and deploy Large Language Model (LLM) serving endpoints using Kubernetes. It includes Kubernetes manifests, Dockerfiles, and development scripts to automate the workflow from downloading model weights to deploying on GKE or KinD.

  2. Quick Start: Integrate Permiflow and Resend

    main

    Follow these steps to set up a security automation workflow that scans RBAC using Permiflow and sends reports via Resend:

    1. Start the Permiflow MCP server using the HTTP transport: permiflow mcp --transport http --http-port 8080

    2. Execute kubectl-ai with the MCP client enabled to run the orchestrated command: kubectl-ai --mcp-client --quiet "scan rbac and send report to admin@company.com from sec@company.com"

    # 1. Start the Permiflow MCP server
permiflow mcp --transport http --http-port 8080

# 2. Execute kubectl-ai with MCP client enabled
kubectl-ai --mcp-client --quiet "scan rbac and send report to admin@company.com from sec@company.com"

  3. Configure GKE access for kubectl-ai

    main

    To run kubectl-ai against a GKE cluster from a container, you must first set up authentication on your host machine:

    1. Create Application Default Credentials (ADC): Run gcloud auth application-default login to save credentials to ~/.config/gcloud.
    2. Configure kubectl: Run gcloud container clusters get-credentials <cluster-name> --location <location> to update your ~/.kube/config file.
    gcloud auth application-default login

gcloud container clusters get-credentials <cluster-name> --location <location>

  4. Secure the kubectl-ai HTTP MCP Endpoint with OAuth 2.1

    main

    The streamable-http endpoint is unauthenticated by default. To secure it for non-localhost use, you can configure it as an OAuth 2.1 Resource Server. This requires providing an --mcp-auth-issuer (the authorization server's URL) and an --mcp-auth-audience (the server's resource identifier). The server will verify Bearer tokens against the issuer's JWKS, ensuring the aud claim matches the configured audience.

    kubectl-ai --mcp-server --mcp-server-mode streamable-http --http-port 9080 \
  --mcp-auth-issuer https://authgate.corp \
  --mcp-auth-audience https://kubectl-ai.corp/mcp

  5. Use model serving development tasks

    main

    Use the scripts in dev/tasks to manage the lifecycle of your model server.

    • download-model: Fetches required model weights (e.g., Gemma 3 12B IT).
    • build-images: Downloads model weights and builds the Docker image using the Dockerfiles in images/.
    • deploy-to-gke or dev/tasks/deploy-to-kind: Builds the images and deploys the Kubernetes manifests to Google Kubernetes Engine (GKE) or a local KinD cluster. After deployment, use kubectl get svc to locate the service and its endpoint.
    • run-local: Runs the model server locally for testing, bypassing Kubernetes entirely.

  8. Verify sandboxed tool execution in GKE

    main

    When using the kubectl-ai hosted UI, the agent executes commands inside isolated sandbox pods within the computer namespace. You can verify that these pods are being created and running by checking the computer namespace:

    kubectl get pods -n computer

    Pods named kubectl-ai-sandbox-* indicate active command execution. You can test this by asking the agent to run a command like uname -a and checking if the output matches the sandbox image (e.g., bitnami/kubectl).

  9. Use kubectl expect to poll for CEL expressions

    main

    The kubectl expect extension allows you to poll a Kubernetes object and wait until a specific Common Expression Language (CEL) expression evaluates to true. This is useful for waiting for specific states in resources during automation or evaluations.

    kubectl expect StatefulSet/mysql 'self.status.replicas >= 1'

  10. Use local LLM providers (Ollama or llama.cpp)

    main

    You can run kubectl-ai with local models using ollama or llama.cpp.

    To use ollama with a specific model (e.g., gemma3:12b-it-qat), use the --llm-provider ollama flag. If your Ollama server is remote, set the OLLAMA_HOST environment variable. Some models may require the --enable-tool-use-shim flag to enable tool calling.

    # Set remote host if necessary
export OLLAMA_HOST=http://192.168.1.3:11434/

# Run with Ollama
kubectl-ai --llm-provider ollama --model gemma3:12b-it-qat --enable-tool-use-shim

# Discover available local models
kubectl-ai models
    kubectl-ai --llm-provider ollama --model gemma3:12b-it-qat --enable-tool-use-shim

  11. Configure MCP servers in mcp.yaml

    main

    MCP server configurations are stored in ~/.config/kubectl-ai/mcp.yaml. If the file does not exist, a default configuration containing the sequential-thinking server is created automatically.

    You can define both local (stdio-based) and remote (HTTP-based) servers in this file.

    servers:
  - name: sequential-thinking
    command: npx
    args:
      - -y
      - "@modelcontextprotocol/server-sequential-thinking"
  - name: cloudflare-documentation
    url: https://docs.mcp.cloudflare.com/mcp

  12. Enable custom tools via CLI flag

    main

    Use the --custom-tools-config flag to enable tools. You can provide a path to a single YAML file or a directory containing multiple YAML files.

    Running from a Local Binary

    Pass the path to your local tools directory:

    ./kubectl-ai --custom-tools-config=<path-to-tools-directory> "your prompt here"

    Running with Docker Image

    Using Built-in Tools

    To use the default tool configurations baked into the image, point to the internal path:

    docker run --rm -it your-kubectl-ai-image:latest \
  --custom-tools-config=/etc/kubectl-ai/tools \
  "list all pull requests on GitHub"

    Using a Local Tools Directory

    To use custom tools from your host machine, mount the directory into the container and point the flag to the mount point:

    docker run --rm -it \
  -v /path/to/your/local/tools:/my-custom-tools \
  your-kubectl-ai-image:latest \
  --custom-tools-config=/my-custom-tools \
  "your prompt here"