DOCSEARCHbyAlgolia
Submit Docs

docker-otel-lgtm

repository·main·Indexed 23 days ago

https://github.com/grafana/docker-otel-lgtm

An OpenTelemetry backend bundled into a single Docker image containing the OpenTelemetry Collector, Prometheus, Tempo, Loki, Pyroscope, and Grafana. Designed for development, demo, and testing environments, it includes examples for eBPF profiling, Java JSON logging in Kubernetes, Spring Boot instrumentation, and OpenTelemetry eBPF Instrumentation (OBI) for zero-code traces and metrics.

Tokens
7.5K
Snippets
24
Records
41
Agent score
84%

What's inside docker-otel-lgtm

  1. What is OBI (OpenTelemetry eBPF Instrumentation)?

    main
    OBI (formerly Grafana Beyla) is a language-agnostic instrumentation method that uses Linux eBPF to hook into kernel-level events like network I/O and function calls. It automatically generates OpenTelemetry traces and metrics for HTTP/gRPC services without requiring any code changes, OpenTelemetry SDKs, or language-specific agents within the target applications.

  2. How scheduled releases work

    main
    Releases for this project are automatically published on a weekly basis via a scheduled GitHub Actions workflow. The workflow runs every Friday at 09:00 UTC. A new release is published only if changes have been detected in the docker/ directory since the latest release. The version number is automatically incremented to the next minor or patch version based on changes to the components installed in the container image.

  3. OBI limitations and capabilities

    main

    When choosing between OBI and traditional SDK-based instrumentation, keep the following limitations in mind:

    • Protocol Support: OBI instruments network protocols (HTTP/gRPC) only; it cannot instrument application-internal logic.
    • Customization: Unlike SDK instrumentation, you cannot add custom spans or attributes.
    • Telemetry Types: OBI generates traces and metrics, but it does not generate logs.
    • Deployment: Requires a privileged container with access to the host kernel.
    • Platform: Linux only.

  4. Quick start with the OBI example

    main

    To run the OBI demonstration, which automatically instruments 5 different language applications (Java, Go, Python, .NET, and Node.js), use Docker Compose. The setup requires a Linux host with kernel 5.8+ and Docker support for privileged containers and pid: "host".

    docker compose up --build

  5. Run the Java OTLP JSON logging Kubernetes example

    main

    This example demonstrates how to export application logs using OTLP logging within a Kubernetes environment. To run the full demonstration, follow these steps:

    1. Build the Docker image: Use the provided build.sh script.
    2. Deploy to Kubernetes: Apply the manifests located in the k8s/ directory using kubectl apply -f k8s/ (e.g., using a local cluster like k3d.sh).
    3. Generate traffic: Execute the traffic generator script located at ../../../generate-traffic.sh to produce logs.
    4. View logs in Grafana:
      • Access the Grafana instance at http://127.0.0.1:3000.
      • Use credentials: User admin / Password admin.
      • Navigate to the Explore tab.
      • Select Loki as the data source to query the exported logs.

  6. Enable OBI (eBPF auto-instrumentation)

    main

    OpenTelemetry eBPF Instrumentation (OBI) uses eBPF to automatically generate traces and RED metrics for HTTP/gRPC services with zero code changes.

    Requirements

    • Linux kernel 5.8+ with BTF support.
    • If running docker run directly (instead of using the provided scripts), you must manually add the --pid=host and --privileged flags.

    Enabling OBI

    Add ENABLE_OBI=true to your .env file or pass it as an environment variable.

    Targeting specific applications

    By default, OBI monitors common ports (80, 443, 8080-8099, 3000-3999, 5000-5999). You can override this behavior using OBI_TARGET or OTEL_EBPF_OPEN_PORT.

    VariablePurpose
    OBI_TARGETFriendly language target: java, python, node, dotnet, ruby, or any regular expression
    OTEL_EBPF_OPEN_PORTOverride ports to monitor (native OBI environment variable)
    OTEL_EBPF_AUTO_TARGET_EXEExecutable name pattern (native OBI environment variable, set automatically by OBI_TARGET)
    # Monitor all Java processes
ENABLE_OBI=true OBI_TARGET=java ./run-lgtm.sh

# Monitor all Python processes
ENABLE_OBI=true OBI_TARGET=python ./run-lgtm.sh

# Monitor a specific executable by name
ENABLE_OBI=true OBI_TARGET=myapp ./run-lgtm.sh

# Monitor specific ports
ENABLE_OBI=true OTEL_EBPF_OPEN_PORT=8080,9090 ./run-lgtm.sh

# Using mise
mise run lgtm-obi

  7. Run the Java example with a standalone Dockerfile

    main

    If you want to run the instrumented Java application independently of the provided Docker Compose setup, build and run the image manually. You must provide OpenTelemetry environment variables to point the application to your OTLP collector endpoint.

    1. Build the image:
    docker build -t java-rolldice .
    1. Run the container:

    On macOS or Windows (Docker Desktop): Use host.docker.internal to resolve the host machine's network.

    docker run -p 8080:8080 \
  -e OTEL_SERVICE_NAME=rolldice \
  -e OTEL_EXPORTER_OTLP_ENDPOINT=http://host.docker.internal:4318 \
  java-rolldice

    On Linux: You must manually add the host.docker.internal mapping using --add-host.

    docker run -p 8080:8080 \
  --add-host=host.docker.internal:host-gateway \
  -e OTEL_SERVICE_NAME=rolldice \
  -e OTEL_EXPORTER_OTLP_ENDPOINT=http://host.docker.internal:4318 \
  java-rolldice
    docker build -t java-rolldice .

# macOS / Windows (Docker Desktop)
docker run -p 8080:8080 \
  -e OTEL_SERVICE_NAME=rolldice \
  -e OTEL_EXPORTER_OTLP_ENDPOINT=http://host.docker.internal:4318 \
  java-rolldice

  9. Integrate with AI tools via MCP

    main

    The stack supports Model Context Protocol (MCP) integration, allowing AI coding tools to query logs, metrics, traces, and dashboards.

    Enable Tempo MCP

    To enable the Tempo MCP server, set the TEMPO_EXTRA_ARGS environment variable to include --query-frontend.mcp-server.enabled=true when running the container.

    Retrieve MCP Configuration

    After starting the container, retrieve the MCP configuration JSON by executing cat /etc/lgtm/mcp.json inside the container. You can then paste this JSON into your AI tool's MCP configuration.

    Note: Traces can also be queried via the client-side Grafana MCP server using uvx mcp-grafana.

  10. Configure Grafana MCP server in Claude Code

    main

    To allow Claude Code to access Grafana data (Dashboards, PromQL, LogQL), you must provide a service account token and the Grafana URL. This requires uvx to run the mcp-grafana process.

    1. Extract the service account token from the running container.
    2. Use the claude mcp add command to register the server.
    # Get the service account token
TOKEN=$(docker exec lgtm cat /tmp/grafana-sa-token)

# Add the Grafana MCP server (requires uvx)
claude mcp add grafana \
  -e GRAFANA_URL=http://localhost:3000 \
  -e GRAFANA_SERVICE_ACCOUNT_TOKEN="$TOKEN" \
  -- uvx mcp-grafana

  11. Set up the Tempo MCP server

    main

    To enable the Tempo MCP server inside the container, add the following environment variable to your .env file before starting the container:

    TEMPO_EXTRA_ARGS=--query-frontend.mcp-server.enabled=true

    Then start the container using the provided script:

    ./run-lgtm.sh
    echo 'TEMPO_EXTRA_ARGS=--query-frontend.mcp-server.enabled=true' >> .env
./run-lgtm.sh

  12. Run the Java JSON logging with Logback Kubernetes example

    main

    This example demonstrates how to export application logs using JSON logging in a Kubernetes environment. To run the full demonstration, follow these steps:

    1. Build the Docker image: Execute the build.sh script.
    2. Deploy to Kubernetes: Apply the manifests located in the k8s/ directory using kubectl apply -f k8s/ (e.g., using k3d.sh).
    3. Generate traffic: Run the traffic generator using ../../../generate-traffic.sh to produce logs.
    4. Access Grafana: Log in to http://127.0.0.1:3000 using the credentials:
      • User: admin
      • Password: admin
    5. View Logs: Navigate to the Explore tab and select Loki as the data source to view the JSON-formatted logs.