DOCSEARCHbyAlgolia
Submit Docs

tokscale

repository·main·Indexed 25 days ago

https://github.com/junhoyeo/tokscale

A high-performance toolset for tracking and visualizing AI token usage and costs across various AI coding agents. It features a Rust-based TUI, a web dashboard, and a CLI (tokscale-cli) that supports a wide range of clients including Claude Code, Cursor IDE, GitHub Copilot CLI, and Zed. Key capabilities include real-time pricing via LiteLLM, detailed token breakdowns (input, output, cache, reasoning), and task-attributed reports.

Tokens
51.1K
Snippets
92
Records
307
Agent score
86%

What's inside tokscale

  1. Overview of supported platforms and data locations

    main

    Tokscale monitors and analyzes token consumption across a wide variety of AI clients and tools. It identifies usage by locating specific database files, JSONL logs, or configuration directories on your local system.

    Key supported clients include:

    • Claude Code: ~/.claude/projects/ and ~/.claude/transcripts/
    • GitHub Copilot CLI: ~/.copilot/otel/*.jsonl (can be configured via COPILOT_OTEL_FILE_EXPORTER_PATH)
    • Cursor IDE: Usage is cached at ~/.config/tokscale/cursor-cache/usage*.csv (requires desktop auto-login or cookie pasting)
    • Zed Agent: ~/.local/share/zed/threads/threads.db (macOS: ~/Library/Application Support/Zed/threads/threads.db; Windows: %LOCALAPPDATA%/Zed/threads/threads.db)
    • Cline: VS Code globalStorage tasks (e.g., ~/.config/Code/... on Linux)
    • Goose: ~/.local/share/goose/sessions/sessions.db (can be configured via GOOSE_PATH_ROOT)
    • Trae IDE: Syncs via tokscale trae sync to ~/.config/tokscale/trae-cache/sessions/*.json
    • Warp: Syncs via tokscale warp sync to ~/.config/tokscale/warp-cache/usage.json

    Tokscale uses LiteLLM pricing data to provide real-time cost calculations, supporting tiered pricing models and cached token discounts.

  2. Overview of Tokscale

    main

    Tokscale is a high-performance CLI tool and visualization dashboard designed to track token usage and costs for AI coding assistants across multiple platforms.

    Key features include:

    • A native Rust-based TUI (Terminal User Interface) for monitoring models, daily summaries, and statistics.
    • A web-based frontend featuring a 3D contributions graph.
    • Support for tracking usage across various AI platforms.

    Version 2 introduced a native Rust TUI and cross-platform support.

  3. Overview of tokscale Features

    main

    tokscale is a high-performance tool for tracking LLM usage across various platforms. Key features include:

    • Interactive TUI Mode: A Ratatui-powered terminal UI with 6 views (Overview, Models, Daily, Hourly, Stats, Agents) and optional Minutely view via minutelyTabEnabled.
    • Multi-platform support: Tracks usage for a wide range of tools including Claude Code, Cursor IDE, Copilot CLI, Gemini CLI, Zed, and many others.
    • Real-time pricing: Fetches current pricing from LiteLLM with a 1-hour disk cache and automatic fallback to OpenRouter.
    • Detailed breakdowns: Tracks input, output, cache read/write, and reasoning tokens.
    • Native Rust core: Uses Rust for 10x faster parsing and aggregation.
    • Task-attributed reports: Uses LLM-powered summarization to group sessions by task (supports Apple FM, Claude, Codex, Gemini, Kiro).
    • Web visualization: Provides interactive 2D and 3D contribution graphs.
    • Export to JSON: Allows exporting data for external tools.

  4. Overview of Tokscale supported platforms

    main
    Tokscale collects and analyzes token consumption from various AI clients and platforms to provide a unified view of usage and costs. It supports a wide range of tools including IDEs (Cursor, Trae, Zed), CLI agents (Claude Code, Gemini CLI, GitHub Copilot CLI), and various open-source agents (OpenCode, OpenClaw, Hermes Agent).

  6. Understand Tokscale pricing sources and fallback logic

    main

    Tokscale provides real-time model pricing by aggregating data from multiple sources with a specific fallback hierarchy:

    1. LiteLLM: Primary source for real-time pricing.
    2. OpenRouter: Dynamic fallback for models not yet in LiteLLM (fetches from OpenRouter's endpoints API).
    3. Cursor Model Pricing: Hardcoded overrides for very recent models (e.g., gpt-5.3-codex) sourced from Cursor's documentation.

    Pricing Details Included:

    • Input tokens
    • Output tokens
    • Cache read tokens (discounted)
    • Cache write tokens
    • Reasoning tokens (e.g., for o1 models)
    • Model-specific tiered pricing (e.g., for usage above 200k or 272k tokens)

    Note on Sakana Fugu: fugu router model cost is unpriced as it depends on the underlying orchestrated model; Fugu Ultra cost is estimated from Sakana's published rates.

  7. Supported AI clients and data locations

    main

    Tokscale monitors and analyzes token consumption by reading data from various AI clients and IDEs. The following table lists supported clients and their respective data storage locations:

    ClientData Location
    OpenCode~/.local/share/opencode/opencode.db (1.2+, all channels including opencode-stable.db) or/and ~/.local/share/opencode/storage/message/ (legacy/unmigrated)
    Claude Code~/.claude/projects/ and ~/.claude/transcripts/
    OpenClaw~/.openclaw/agents/ (+ legacy: .clawdbot, .moltbot, .moldbot)
    Codex CLI~/.codex/sessions/
    Sakana Fuguvia Codex — ~/.codex/sessions/*.jsonl (model_provider: sakana)
    GitHub Copilot CLI~/.copilot/otel/*.jsonl (+ COPILOT_OTEL_FILE_EXPORTER_PATH)
    Hermes Agent$HERMES_HOME/state.db and $HERMES_HOME/profiles/*/state.db (fallback: ~/.hermes/...)
    Gemini CLI$GEMINI_CLI_HOME/tmp/*/chats/*.json (fallback: ~/.gemini/tmp/*/chats/*.json)
    Cursor IDECursor API export cached at ~/.config/tokscale/cursor-cache/usage*.csv
    Amp (AmpCode)~/.local/share/amp/threads/
    Codebuff~/.config/manicode/ (+ manicode-dev, manicode-staging; override via CODEBUFF_DATA_DIR)
    Droid (Factory Droid)~/.factory/sessions/
    Pi~/.pi/agent/sessions/ and ~/.omp/agent/sessions/
    Kimi CLI / Kimi Codekimi-cli: ~/.kimi/sessions/; kimi-code: ~/.kimi-code/sessions/ (override via KIMI_CODE_HOME)
    Qwen CLI~/.qwen/projects/
    Roo Code~/.config/Code/User/globalStorage/rooveterinaryinc.roo-cline/tasks/ (+ server: ~/.vscode-server/data/User/globalStorage/rooveterinaryinc.roo-cline/tasks/)
    Kilo / Kilo CLIKilo: ~/.config/Code/User/globalStorage/kilocode.kilo-code/tasks/ (+ server: ~/.vscode-server/data/User/globalStorage/kilocode.kilo-code/tasks/); Kilo CLI: ~/.local/share/kilo/kilo.db
    Mux~/.mux/sessions/
    Crush$XDG_DATA_HOME/crush/projects.json (fallback: ~/.local/share/crush/projects.json)
    Goose~/.local/share/goose/sessions/sessions.db (override via GOOSE_PATH_ROOT)
    Google AntigravityCached via tokscale antigravity sync to ~/.config/tokscale/antigravity-cache/sessions/*.jsonl
    Antigravity CLI~/.gemini/antigravity-cli/conversations/*.db (override via GEMINI_CLI_HOME)
    Trae IDE / SoloCached via tokscale trae sync to ~/.config/tokscale/trae-cache/sessions/*.json
    Warp / OzCached via tokscale warp sync to ~/.config/tokscale/warp-cache/usage.json
    Grok Build$GROK_HOME/sessions/*/*/updates.jsonl (fallback: ~/.grok/sessions/*/*/updates.jsonl)
    Zed Agent~/.local/share/zed/threads/threads.db (macOS: ~/Library/Application Support/Zed/threads/threads.db; Windows: %LOCALAPPDATA%/Zed/threads/threads.db)
    Kiro~/.kiro/sessions/cli/*.json, ~/.local/share/kiro-cli/data.sqlite3, and Kiro IDE globalStorage snapshots
    ClineVS Code globalStorage tasks (Linux: ~/.config/Code/...; macOS: ~/Library/Application Support/Code/...; Windows: %APPDATA%\Code\...)
    gajae-code (gjc)~/.gjc/agent/sessions/ (override via GJC_CODING_AGENT_DIR, GJC_CONFIG_DIR, PI_CONFIG_DIR)
    Jcode~/.jcode/sessions/session_*.json (override via JCODE_HOME)
    MiMo Code~/.local/share/mimocode/mimocode.db
    Junie~/.junie/sessions/*/events.jsonl
    Command Code~/.commandcode/projects/**/*.jsonl
    ZCode~/.zcode/cli/db/db.sqlite and ~/.zcode/projects/**/*.jsonl
    OpenCodeReview~/.opencodereview/sessions/**/*.jsonl
    CodeBuddy~/.codebuddy/projects/**/*.jsonl
    WorkBuddy~/.workbuddy/projects/**/*.jsonl
    Devin CLI~/.local/share/devin/cli/sessions.db
    Devin DesktopmacOS: ~/Library/Application Support/Devin/User/acp-events/; Linux: ~/.config/Devin/User/acp-events/; Windows: %APPDATA%\Devin\User\acp-events/
    SyntheticRe-attributed via hf: model prefix or synthetic provider (e.g., Octofriend: ~/.local/share/octofriend/sqlite.db)

  8. Understand Tokscale pricing data sources and caching

    main

    Tokscale uses a multi-layered approach to fetch real-time model pricing, ensuring coverage for both established and newly released models:

    1. LiteLLM: Primary source for real-time pricing.
    2. OpenRouter: Dynamic fallback for models not yet present in LiteLLM.
    3. Cursor Model Docs: Hardcoded overrides for the latest models (e.g., gpt-5.3-codex) not found in LiteLLM or OpenRouter. These overrides are checked before fuzzy matching and yield to upstream sources once they become available.
    4. Sakana Fugu: Fugu Ultra costs are estimated based on Sakana's pay-as-you-go rates. The fugu router model does not have a fixed price as its cost is the variable rate of the underlying orchestrated model.

    Caching: To ensure fast startup, pricing data is cached on disk with a 1-hour TTL:

    • LiteLLM cache: ~/.config/tokscale/cache/pricing-litellm.json
    • OpenRouter cache: ~/.config/tokscale/cache/pricing-openrouter.json

    Pricing details included:

    • Input tokens
    • Output tokens
    • Cache read tokens (discounted)
    • Cache write tokens
    • Reasoning tokens (for models like o1)
    • Tiered pricing per model (e.g., for 200k or 272k+ tokens)

  9. Identify Tokscale primary product surfaces

    main

    Tokscale's primary user-facing surfaces include:

    • Public user profiles: Accessible at /u/[username]
    • Profile embed dialog: For configuring and previewing widgets.
    • Public README widgets: Available via /api/embed/[username]/svg
    • Global leaderboard: Accessible at /leaderboard
    • Group directory: Accessible at /leaderboard?view=groups
    • Group detail: Accessible at /groups/[slug]
    • Group workflows: Create and join flows via /groups/new and /groups/join/[token]

  10. Understand Tokscale pricing data sources and fallback logic

    main

    Tokscale uses a multi-layered approach to ensure accurate model pricing:

    1. Primary Source: Real-time pricing is fetched from the LiteLLM pricing database.
    2. Dynamic Fallback: For models not yet in LiteLLM, Tokscale automatically fetches pricing from OpenRouter's endpoint API.
    3. Cursor Model Pricing: For the newest models (e.g., gpt-5.3-codex) missing from both LiteLLM and OpenRouter, Tokscale uses hardcoded pricing derived from Cursor model documentation. This takes precedence over upstream sources and fuzzy matching.
    4. Sakana Fugu Pricing: Fugu Ultra costs are estimated based on Sakana's pay-as-you-go rates. The fugu routing model is intentionally unpriced as its cost fluctuates with the underlying models it orchestrates.

    Pricing details included:

    • Input Tokens
    • Output Tokens
    • Cache Read Tokens (discounted)
    • Cache Write Tokens
    • Reasoning Tokens (e.g., for o1 models)
    • Model-specific tiered pricing (e.g., for usage above 200k or 272k tokens)

  11. Quickstart: Run Tokscale via npx, bunx, or deno

    main
    You can run Tokscale immediately without permanent installation using one of the following commands. By default, it provides a full interactive TUI experience. To run in 'light mode' (which only renders tables), use the --light flag.

  12. Quickstart: Login, Submit, and View data

    main

    Follow these steps to start using Tokscale:

    1. Login: Run tokscale login to authenticate via GitHub, or create an API token in Settings for CI/headless environments.
    2. Submit: Run tokscale submit to upload your usage data.
    3. View: Visit the web platform to see your profile and leaderboards.