9Router AI Routing Proxy

repository·master·Indexed 10 days ago

https://github.com/decolua/9router

An AI routing proxy designed to reduce token consumption and costs for developers. It features automatic fallback between subscription-based, cheap, and free AI models, RTK (Real-Time Kompression) for tool output compression, and format translation between OpenAI, Claude, Gemini, and other providers. It integrates with AI coding tools like Claude Code, Cursor, and Cline via an OpenAI-compatible endpoint.

Tokens
82.5K
Snippets
230
Records
395
Agent score
97%

What's inside 9Router

  1. 9Router Technical Stack

    master

    9Router is built using the following technologies:

    • Runtime: Node.js 20+
    • Framework: Next.js 16
    • UI: React 19 + Tailwind CSS 4
    • Database: LowDB (JSON-based)
    • Streaming: Server-Sent Events (SSE)
    • Authentication: OAuth 2.0 (PKCE) + JWT + API Keys
  2. Format Conversion between AI Providers

    master

    9Router acts as a translation layer, allowing any CLI tool that supports custom OpenAI endpoints to communicate with various AI providers. It performs seamless format conversion between:

    OpenAIClaudeGeminiCursorKiroVertexAntigravityOllamaOpenAI Responses.

    Workflow:

    1. Your CLI tool sends a request in OpenAI format.
    2. 9Router receives the request and converts it to the provider's native format.
    3. The provider processes the request and returns a response.
    4. 9Router converts the response back to the format expected by your tool.
  3. Create a Model Combo for automatic fallback

    master

    Combos allow you to chain models so that 9Router automatically falls back to a secondary model when your primary quota is exhausted. This maximizes value by using free tiers first.

    How to create a combo:

    1. Go to the Dashboard $\rightarrow$ Combos $\rightarrow$ Create New.
    2. Name your combo (e.g., premium-coding).
    3. Add models in order of priority (e.g., Free $\rightarrow$ Subscription $\rightarrow$ Cheap Backup).
    4. Use the combo name in the CLI.

    Example Strategy:

    1. gc/gemini-3-flash-preview (Free primary)
    2. cc/claude-opus-4-5 (Subscription)
    3. cx/gpt-5.2-codex (Subscription backup)
    Dashboard → Combos → Create New
    
    Name: premium-coding
    Models:
      1. gc/gemini-3-flash-preview (FREE, use first)
      2. cc/claude-opus-4-5-20251101 (Subscription)
      3. cx/gpt-5.2-codex (Subscription backup)
    
    Use in CLI: premium-coding
  4. Understand 9Router pricing and billing

    master

    9Router is free, open-source software and does not charge any fees, send invoices, or handle credit card information. It acts as a local proxy/router.

    How Billing Works

    • 9Router Software: Always FREE.
    • Providers: You pay your AI providers (e.g., Claude, OpenAI, Anthropic) directly via their own subscription or API models.
    • Dashboard Costs: The costs displayed in the 9Router dashboard are estimated savings trackers, not actual bills. They show what you would have paid if you were using paid APIs directly, helping you visualize the value of using free models or existing subscriptions.

    Provider Categories

    CategoryExamplesPayment Method
    Token SaverRTK (built-in)FREE (saves 20-40% tokens)
    SubscriptionClaude Code, Codex, GitHub Copilot, CursorPay provider directly
    CheapGLM, MiniMax, KimiPay provider directly (pay-as-you-go)
    FREEKiro AI, OpenCode Free, Vertex AIGenuinely free within tier limits
  5. Understand 9Router Format Translation

    master

    9Router acts as a translation layer between different LLM API formats. This allows you to use any CLI tool that supports OpenAI-compatible endpoints to interact with various providers.

    Supported Formats:

    • OpenAI ↔ Claude ↔ Gemini ↔ Cursor ↔ Kiro ↔ Vertex ↔ Antigravity ↔ Ollama ↔ OpenAI Responses
  6. Use RTK Token Saver for input compression

    master

    RTK (Real-Time Kompression) automatically detects tool outputs (like git diff, grep, ls, tree, etc.) and applies lossless compression before the request reaches the LLM. This can save 20-40% of input tokens.

    • Auto-detection: No configuration is required; RTK inspects the first 1KB of tool_result to select the appropriate filter.
    • Safety: If compression fails or increases the size, RTK silently reverts to the original text.
    • Availability: It runs before format translation and is enabled by default. You can toggle it in Dashboard → Endpoint settings.
  7. Monitor real-time quota tracking

    master

    9Router tracks token consumption and quota status for all connected providers. You can monitor usage, reset countdowns, and cost estimations via the dashboard.

    Key tracking features:

    • Token consumption: Tracks input/output tokens per request.
    • Reset countdown: Shows time remaining until quota refreshes.
    • Usage stats: Provides daily, weekly, and monthly reports.
    • Cost estimation: Projects spending for paid tiers.
    • Quota alerts: Notifications when quotas are running low.

    To view quota: Navigate to DashboardProvidersQuota Tracking.

  8. Understand the 9Router Architecture and Request Lifecycle

    master

    9Router acts as a local AI routing gateway that provides an OpenAI-compatible API surface (/v1/*). It sits between developer clients (like Claude Code, Codex CLI, or Continue) and upstream AI providers (like OpenAI, Anthropic, or Gemini).

    Core Capabilities

    • OpenAI-compatible API: Use standard OpenAI SDKs/CLIs by pointing them to the 9Router endpoint.
    • Request Translation: Automatically converts requests between different provider formats.
    • Fallback Mechanisms:
      • Model Combo Fallback: If a requested model fails, 9Router can iterate through a sequence of models defined in a 'combo'.
      • Account Fallback: If one API key or OAuth account hits a rate limit or error, 9Router automatically tries the next available account for that provider.
    • Usage Tracking: Logs costs and token usage locally.
    • Local Persistence: Stores all provider connections, keys, and settings in a local JSON database.
    sequenceDiagram
        autonumber
        participant Client as CLI/SDK Client
        participant Route as /api/v1/chat/completions
        participant Chat as src/sse/handlers/chat
        participant Core as open-sse/handlers/chatCore
        participant Model as Model Resolver
        participant Auth as Credential Selector
        participant Exec as Provider Executor
        participant Prov as Upstream Provider
        participant Stream as Stream Translator
        participant Usage as usageDb
    
        Client->>Route: POST /v1/chat/completions
        Route->>Chat: handleChat(request)
        Chat->>Model: parse/resolve model or combo
        ...
        Core->>Prov: upstream API call
        Prov-->>Exec: SSE/JSON response
        Exec-->>Core: response + metadata
        Core->>Stream: translate/normalize stream to client format
        Stream-->>Client: SSE chunks / JSON response
        Stream->>Usage: extract usage + persist history/log
  9. How 9Router routing and tiers work

    master

    9Router is an AI model router that uses a 3-tier fallback system to maximize subscription value and minimize costs. It intelligently routes requests across providers based on availability and cost:

    1. Tier 1: Subscription (Maximize First) - Uses existing paid quotas (e.g., Claude Code, OpenAI Codex, Gemini CLI, GitHub Copilot) to ensure you get value from services you already pay for.
    2. Tier 2: Cheap (Backup) - Uses ultra-cheap alternatives (e.g., GLM-4.7, MiniMax M2.1) when subscription quotas are exhausted.
    3. Tier 3: Free (Emergency) - Uses free models (e.g., iFlow, Qwen, Kiro) as a zero-cost fallback when all other tiers are limited.

    This system allows for automatic fallback, real-time quota tracking, and significant cost savings compared to direct API usage.

  10. Reasoning level resolution for GPT-5.6 Codex models

    master

    The 9router reasoning level resolution logic is provider-scoped. This ensures that specific reasoning overrides for OpenAI Codex (cx/) do not unintentionally affect other providers like Kiro (kr/) even if they use similar model names.

    Resolution Logic for cx/ provider

    1. Parsing: The system recognizes (max) and (ultra) as discrete reasoning levels via the suffix parser.
    2. Translation:
      • If the requested level is supported by the specific Codex model, it is preserved.
      • For gpt-5.6-luna, an ultra request is converted to max (the highest supported level for Luna).
      • For any other unsupported max or ultra requests, the system converts them to xhigh as a safe fallback.
    3. Execution: The Codex executor resolves virtual models (like -review variants) to their base models before dispatching, ensuring the reasoning level is correctly applied to the upstream Codex Responses endpoint.

    Note: Do not use hyphen-suffixes like gpt-5.1-codex-max for reasoning overrides; max in that context is part of the actual model identifier. Always use the parenthesized syntax (max) or (ultra) for reasoning level overrides.

  11. How Smart Routing and the 3-Tier Fallback System work

    master

    9Router uses a 3-tier fallback system to automatically route requests through the best available provider based on quota, cost, and availability. This ensures continuous service even when primary subscriptions are exhausted or rate-limited.

    The 3 Tiers

    1. Tier 1: SUBSCRIPTION (Primary) - Maximizes value from existing paid services (e.g., Claude Code, OpenAI Codex, Gemini CLI, GitHub Copilot).
    2. Tier 2: CHEAP (Backup) - Ultra-low-cost providers used when subscription quotas run out (e.g., GLM-4.7, MiniMax M2.1, Kimi K2).
    3. Tier 3: FREE (Emergency) - Zero-cost providers for unlimited coding when budgets are reached (e.g., iFlow, Qwen, Kiro).

    Routing Logic

    When a request is made, 9Router follows this sequence:

    1. Check Quota: Verifies if the primary provider has remaining quota.
    2. Check Cost Tier: Follows the hierarchy: Subscription $\rightarrow$ Cheap $\rightarrow$ Free.
    3. Check Reset Timing: Considers when quotas will reset.
    4. Check Provider Health: Automatically skips providers returning errors (e.g., 503 Service Unavailable).
    Request $\rightarrow$ 9Router $\rightarrow$ Check Tier 1 (Subscription)
                         $\downarrow$ quota exhausted
                         Check Tier 2 (Cheap)
                         $\downarrow$ budget limit
                         Check Tier 3 (Free)
                         $\downarrow$
                         Response
  12. Manage 9Router data and backups

    master

    9Router stores its state in ~/.9router. The directory structure is as follows:

    • db.json: Main database containing providers, combos, and settings.
    • logs/: Application logs.
    • cache/: Temporary cache files.

    To back up your data, copy the directory to a backup location. To restore, copy the backup back to the default location.

    # Backup
    cp -r ~/.9router ~/.9router.backup
    
    # Restore
    cp -r ~/.9router.backup ~/.9router