Claude Code Observability Stack

repository·main·Indexed 19 days ago

https://github.com/colemurray/claude-code-otel

A comprehensive observability solution for monitoring Claude Code usage, performance, and costs. It utilizes an OpenTelemetry pipeline consisting of an OpenTelemetry Collector, Prometheus for time-series metrics, Loki for event and log aggregation, and Grafana for visualization. The stack allows for tracking token consumption, development impact, tool performance, and API reliability via gRPC or HTTP OTLP exporting.

Tokens
6.7K
Snippets
17
Records
24
Agent score
67%

What's inside claude-code-otel

  1. How the Observability Stack Architecture Works

    main

    The stack follows a standard OpenTelemetry pipeline to ingest, store, and visualize Claude Code data:

    1. Claude Code: Generates telemetry data.
    2. OpenTelemetry Collector: Receives metrics and logs via gRPC (port 4317) or HTTP (port 4318).
    3. Storage Layer:
      • Prometheus: Stores time-series metrics.
      • Loki: Aggregates and stores events and logs.
    4. Grafana: Provides the visualization layer for querying both Prometheus and Loki to create dashboards.
    Claude Code → OpenTelemetry Collector → Prometheus (metrics) + Loki (events/logs)
                                         ↓
                                  Grafana (visualization & analysis)
  2. Analyze Claude Code usage and costs

    main

    Use the exported metrics to build dashboards for the following purposes:

    Usage & Productivity

    • Adoption: Track claude_code.session.count over time.
    • Token Consumption: Break down claude_code.token.usage by type (input/output), model, or user.account_uuid.
    • Development Impact: Monitor claude_code.lines_of_code.count, claude_code.commit.count, and claude_code.pull_request.count to measure productivity.

    Cost Management

    • Trend Analysis: Use claude_code.cost.usage to track spending across teams or individuals.
    • Optimization: Identify high-cost sessions by segmenting cost metrics by model or session.id.

    Note: Cost metrics are approximations. Use your API provider's console (Anthropic, AWS, or Google Cloud) for official billing data.

  3. Analyze Claude Code tool and API performance

    main

    Use event data to monitor the reliability and performance of the Claude Code environment:

    Tool Performance

    Analyze claude_code.tool_result events to determine:

    • Most frequently used tools.
    • Tool success rates.
    • Average execution time (duration_ms).
    • Error patterns by tool name.

    API Reliability

    Analyze claude_code.api_request and claude_code.api_error events to track:

    • Request latency (duration_ms).
    • Error rates and specific error messages.
    • Model-specific performance (e.g., comparing Sonnet vs Haiku latency/errors).
    • Retry attempts via the attempt attribute in error events.
  4. Configure OpenTelemetry via Managed Settings (Administrator)

    main

    Administrators can centrally manage OpenTelemetry settings for all users by using a managed settings file. Environment variables defined here have high precedence and cannot be overridden by individual users. This is ideal for distribution via MDM (Mobile Device Management).

    File Locations:

    • macOS: /Library/Application Support/ClaudeCode/managed-settings.json
    • Linux: /etc/claude-code/managed-settings.json
    {
      "env": {
        "CLAUDE_CODE_ENABLE_TELEMETRY": "1",
        "OTEL_METRICS_EXPORTER": "otlp",
        "OTEL_LOGS_EXPORTER": "otlp",
        "OTEL_EXPORTER_OTLP_PROTOCOL": "grpc",
        "OTEL_EXPORTER_OTLP_ENDPOINT": "http://collector.company.com:4317",
        "OTEL_EXPORTER_OTLP_HEADERS": "Authorization=Bearer company-token"
      }
    }
  5. Quick Start: Enable and configure OpenTelemetry for Claude Code

    main

    Claude Code supports OpenTelemetry (OTel) for metrics and events. To enable telemetry, you must set the CLAUDE_CODE_ENABLE_TELEMETRY environment variable to 1. You can then configure exporters (like otlp, prometheus, or console) and OTLP endpoints via environment variables.

    For debugging, you can reduce the export intervals using OTEL_METRIC_EXPORT_INTERVAL (default 60000ms) and OTEL_LOGS_EXPORT_INTERVAL (default 5000ms).

    # 1. Enable telemetry
    export CLAUDE_CODE_ENABLE_TELEMETRY=1
    
    # 2. Choose exporters
    export OTEL_METRICS_EXPORTER=otlp       # Options: otlp, prometheus, console
    export OTEL_LOGS_EXPORTER=otlp          # Options: otlp, console
    
    # 3. Configure OTLP endpoint
    export OTEL_EXPORTER_OTLP_PROTOCOL=grpc
    export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
    
    # 4. Set authentication (if required)
    export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer your-token"
    
    # 5. For debugging: reduce export intervals
    export OTEL_METRIC_EXPORT_INTERVAL=10000  # 10 seconds
    export OTEL_LOGS_EXPORT_INTERVAL=5000     # 5 seconds
    
    # 6. Run Claude Code
    claude
  6. Configure Claude Code for Telemetry

    main

    After starting the stack, you must configure your Claude Code environment variables to enable telemetry and point the OpenTelemetry exporters to the local collector. Use the following configuration to enable gRPC-based OTLP exporting.

    # Enable telemetry
    export CLAUDE_CODE_ENABLE_TELEMETRY=1
    
    # Configure exporters
    export OTEL_METRICS_EXPORTER=otlp
    export OTEL_LOGS_EXPORTER=otlp
    export OTEL_EXPORTER_OTLP_PROTOCOL=grpc
    export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
    
    # Run Claude Code
    claude
  7. Quick Start: Install and Start the Observability Stack

    main

    To set up the Claude Code Observability Stack, use the provided Makefile to manage the Docker-based services. This will start the OpenTelemetry Collector, Prometheus, Loki, and Grafana.

    # Start all services
    make up
    
    # Check status
    make status
  8. Control Metrics Cardinality

    main

    To manage storage requirements and query performance in your metrics backend, you can control which attributes are included in metrics using the following environment variables. Lower cardinality reduces costs and improves performance but provides less granular data.

    Environment VariableDescriptionDefault ValueExample to Disable
    OTEL_METRICS_INCLUDE_SESSION_IDInclude session.id attribute in metricstruefalse
    OTEL_METRICS_INCLUDE_VERSIONInclude app.version attribute in metricsfalsetrue
    OTEL_METRICS_INCLUDE_ACCOUNT_UUIDInclude user.account_uuid attribute in metricstruefalse
  9. Configure metric attribute inclusion

    main

    You can control which metadata is attached to exported metrics using the following environment variables:

    • OTEL_METRICS_INCLUDE_SESSION_ID: Controls inclusion of the session.id attribute.
    • OTEL_METRICS_INCLUDE_VERSION: Controls inclusion of the app.version attribute.
    • OTEL_METRICS_INCLUDE_ACCOUNT_UUID: Controls inclusion of the user.account_uuid attribute.
  10. Configure Advanced Telemetry via Environment Variables

    main

    You can fine-tune the telemetry behavior using several OpenTelemetry and Claude-specific environment variables. Use these to control export intervals, privacy, and data cardinality.

    # Core telemetry
    CLAUDE_CODE_ENABLE_TELEMETRY=1
    
    # Exporter configuration
    OTEL_METRICS_EXPORTER=otlp,prometheus    # Multiple exporters
    OTEL_LOGS_EXPORTER=otlp
    
    # Protocol and endpoints
    OTEL_EXPORTER_OTLP_PROTOCOL=grpc
    OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
    OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer token"
    
    # Export intervals
    OTEL_METRIC_EXPORT_INTERVAL=60000        # 1 minute (production)
    OTEL_LOGS_EXPORT_INTERVAL=5000           # 5 seconds
    
    # Privacy controls
    OTEL_LOG_USER_PROMPTS=1                   # Enable prompt content logging
    
    # Cardinality control
    OTEL_METRICS_INCLUDE_SESSION_ID=true
    OTEL_METRICS_INCLUDE_VERSION=false
    OTEL_METRICS_INCLUDE_ACCOUNT_UUID=true