TokenTelemetry

repository·main·Indexed 42 days ago

https://github.com/vasihemanth/tokentelemetry

A local, open-source observability dashboard for tracking token usage, LLM costs, tool calls, and session traces for AI coding and autonomous agents. It provides a unified view of agents such as Claude, Codex, and Gemini without sending data to the cloud. Version 1.0.0.

Tokens
100.4K
Snippets
235
Records
489
Agent score
54%

What's inside TokenTelemetry

  1. What is TokenTelemetry?

    main

    TokenTelemetry is a free, open-source, 100% local observability dashboard designed for AI coding agents. It provides a unified view of token usage, LLM costs, tool calls, session traces, and reasoning steps by reading the log files that agents already write to your machine.

    Key characteristics:

    • Local-first: All data stays in ~/.tokentelemetry/ on your machine. Logs, prompts, and costs never leave your machine.
    • Zero-config: Requires no SDK, no API keys, and no code changes. It works by watching existing agent log files.
    • Privacy: Does not modify agent logs. Outbound traffic is limited to optional hourly GitHub version checks and anonymous product telemetry (feature usage only, never code or prompts).

    Supported agents include Claude Code, Codex, Gemini CLI, Cursor, Copilot, Qwen, OpenCode, Vibe, Antigravity, Grok Build, Cline, SmallCode, Pi, Muse Code, Prime Agent, DeepSeek Harness, Qoder, ZCode, Kimi Code, and Hermes Agent.

  2. Overview of TokenTelemetry capabilities

    main

    TokenTelemetry is a local-first dashboard designed to scan coding-agent data directories on a user's machine without network access. It provides observability into agentic workflows by reporting on:

    • Sessions & Activity: Detailed session logs, subagent delegation, and per-agent summaries.
    • Resource Usage: Token consumption, cost analysis, and environmental impact (power/CO2).
    • Project Context: Projects, skills, MCP (Model Context Protocol) inventories, and plugin inventories.
    • Agent State: Specialized views for autonomous agents like Hermes, including kanban, memory, and soul states.
  3. Overview of TokenTelemetry features

    main

    The dashboard provides several specialized views for monitoring AI agent activity:

    • Dashboard: Live KPI strip (sessions, tokens, projects, cost) with 15-second auto-refresh, agent/model distribution charts, and recent activity.
    • Analytics: Date-range filters, per-agent/model breakdowns, token area charts, cache-efficiency metrics, and a Delegation & Ecosystem section (subagent spend, skills, MCP servers).
    • Traces: Replayable session traces including events, messages, tool calls, artifacts, and plan-mode output.
    • Summarization: One-click LLM-generated narratives or structured briefs (What / Tools / Why / Next) for sessions.
    • Projects: Per-working-directory cards featuring a 365-day heatmap, streaks, tool usage, and configuration overlays.
    • Hermes Agent: A dedicated /hermes hub for monitoring gateway health, 38 source platforms, scheduled jobs, latency, skills, and memory.
    • Desktop App: A native window version of the dashboard available via a local Electron shell.
  4. Overview of the Hermes Agent dashboard

    main

    The Hermes Agent dashboard is located at /hermes within TokenTelemetry. It is designed for autonomous agent frameworks (like Nous Research's Hermes) that operate across various platforms (CLI, Telegram, Discord, Slack, etc.) rather than just coding tasks.

    Core Metrics

    • Gateway Health: Status and uptime.
    • Source Breakdown: Monitoring across 38 platforms (e.g., CLI, Telegram, Discord, Slack, Feishu, DingTalk, cron, webhook).
    • Performance: Per-API-call latency and cache-hit percentages (parsed from agent.log).
    • Cost Monitoring: Detection of cost anomalies and reasoning-token waste.
    • Subagent Tracking: Inline delegate_task cards showing child session summaries, tokens, duration, and tool traces.

    Dashboard Sub-pages

    1. Skills: Tracks 90 loaded skills, platform conditions, and invocation counts.
    2. Tools: Tool registry with call counts and success rates.
    3. Profiles: Active agent profiles and configurations.
    4. Soul: Agent persona/soul configuration.
    5. Memory: Progress tracking for MEMORY.md and USER.md including character limits and edit history.
    6. Gateway: Connection health, latency trends, and error rates by provider.
    7. Schedules: Cron job registry, run history, and next-run countdowns.
  5. Analyze Qoder usage and spend via credits

    main

    Qoder does not record token counts (e.g., input_tokens and output_tokens are always 0 in the usage object). Instead, it uses a credits system.

    To calculate spend, use the credits field in the assistant record's usage object.

    Key fields in the usage object:

    • credits: The actual amount charged.
    • original_credits: Typically $2\times$ the credits value (likely the list price).
    • context_usage_ratio: A fraction of an unknown denominator; token counts cannot be reconstructed from this value.

    Important: When calculating total session costs, ensure you include both Root sessions and Subagent transcripts, as delegation (subagents) can account for a significant portion (e.g., ~49%) of total spend.

    {
      "input_tokens": 0,
      "output_tokens": 0,
      "cache_read_input_tokens": 0,
      "cache_creation_input_tokens": 0,
      "server_tool_use": {
        "web_search_requests": 0,
        "web_fetch_requests": 0
      },
      "service_tier": "standard",
      "inference_geo": "",
      "iterations": [],
      "speed": "standard",
      "credits": 1.5205821428571429,
      "original_credits": 3.0411642857142858,
      "billable": true,
      "request_id": "<uuid>",
      "context_usage_ratio": 0.026787
    }
  6. Navigate session data using the five trace tabs

    main

    Traces are organized into five functional tabs for different auditing needs:

    1. Events: A chronological timeline of the session (start, turns, tool calls, results, end). Events are color-coded:
      • Amber: Reasoning / thinking blocks.
      • Sky blue: Tool calls and results.
      • Emerald: Assistant responses.
      • Default: User turns and system events.
    2. Messages: The raw conversation history (user messages, system prompts, and assistant responses) as seen by the model.
    3. Tools: A grouped view of every tool call. Shows tool name (e.g., Bash, Edit, Read, WebSearch), arguments, results, timing, and success/failure status.
    4. Artifacts: Captured outputs such as published pages, screenshots, browser recording thumbnails, and documents (e.g., task.md, plan.md).
    5. Plans: Structured plan-mode output (e.g., Claude Code's /plan command) shown as formatted text with timestamps.
  7. Understand Loops, Goals, and Workflows

    main

    TokenTelemetry categorizes agent activity into three distinct types to provide accurate cost and token attribution. Understanding these helps you interpret how much money and tokens were spent on automated vs. manual tasks:

    TypeDescriptionExample Agents
    LoopA prompt running on a schedule (e.g., every hour).Claude Code, Grok Build, Cline
    GoalAn objective an agent works toward across multiple turns.Codex, Claude Code, Grok Build, Antigravity
    WorkflowA single prompt that fans out into many subagents.Claude Code

    Note: All data is read from local agent storage; nothing is uploaded to a network to build these views.

  8. Understand and use Artifacts in TokenTelemetry

    main

    Artifacts are deliverables produced by agents that go beyond raw code, such as hosted result pages, task lists, walkthroughs, screenshots, and browser recordings. TokenTelemetry collects these from local agent storage and organizes them in two ways:

    • Per project: The Artifacts tab in a project workspace lists all deliverables from every session within that project.
    • Per session: The Artifacts tab in a specific session trace lists everything produced during that session, including raw files.

    Important Privacy Note: Everything is read from local agent storage. Nothing is uploaded or fetched remotely.

  9. Understand the `/goal` telemetry design

    main

    The /goal telemetry feature tracks autonomous agent objectives—when a user provides a goal and the agent works toward it across multiple turns. Unlike the /loop telemetry (which tracks recurring timers), /goal telemetry is designed to handle multiple goals per session and varying levels of agent-reported evidence.

    Key characteristics:

    • Multiple Goals: A single session can contain a list of goals (sess["goals"]).
    • Honesty Flag: The state_source field distinguishes between reported (the agent explicitly stated the status, e.g., Codex) and inferred (TokenTelemetry guessed the status from logs, e.g., Claude).
    • Variable Evidence: Because different agents implement /goal differently, the evidence object contains agent-specific metadata.
  10. Understand the Qoder filesystem structure

    main

    Qoder (Alibaba's AI coding IDE) stores data in two primary locations. The CLI harness uses a structure similar to Claude Code, while the Electron app uses a SQLite database.

    CLI Data (~/.qoder):

    • projects/<slugged-cwd>/<uuid>.jsonl: The primary session transcripts.
    • projects/<slugged-cwd>/<uuid>/state.json: Encrypted session state.
    • shell-snapshots/: Shell session snapshots.
    • .models/: Contains model information; catalog-v6 is encrypted.

    App Data (~/Library/Application Support/com.qoder.app.stable/):

    • main.sqlite: A session store that mirrors CLI session IDs but provides additional metadata like session titles and durations.
    • qoder-data.v1.json: General app data.

    Note: When accessing main.sqlite, it must be opened in read-only mode because the application may hold a large Write-Ahead Log (WAL) file while running.

  11. Hermes Agent data sources

    main

    For the Hermes Agent (located at ~/.hermes/ or $HERMES_HOME), TokenTelemetry extracts data from multiple sources to populate the dashboard:

    • Tokens and Cost: From sessions.db or session JSON.
    • Latency and Cache Hits: From agent.log (captures per-API-call latency and cache-hit percentages).
    • Configuration: From the skills/, memory/, and soul/ directories.
    • Schedules: From cron schedule files (used for the Schedules sub-page).