DOCSEARCHbyAlgolia
Submit Docs

ccflare

repository·main·Indexed 21 days ago

https://github.com/snipeship/ccflare

A multi-provider native proxy for Anthropic and OpenAI featuring routing by URL prefix, load-balancing across multiple accounts, and account failover on rate limits. It includes a management API for account and configuration control, built-in observability via a web dashboard and TUI, and support for OAuth accounts via a CLI.

Tokens
87.4K
Snippets
280
Records
373
Agent score
76%

What's inside ccflare

  1. Overview of the ccflare Database Layer

    main

    ccflare uses SQLite as its exclusive runtime datastore, utilizing Bun's native SQLite bindings. The database layer is responsible for managing configured accounts, request summaries, full request/response payload documents, and short-lived OAuth authentication sessions.

    Key architectural components include:

    • Repository Pattern: Each domain (Accounts, Requests, etc.) has a dedicated repository that owns its SQL and maps rows to application shapes.
    • Async Write Queue: An AsyncDbWriter handles background persistence to prevent blocking request forwarding.
    • Migration System: A normalization-oriented system that ensures schema consistency and upgrades older layouts to the current runtime model.

  2. ccflare HTTP API Overview

    main

    ccflare provides a RESTful HTTP API designed for:

    • Managing accounts
    • Monitoring usage
    • Proxying requests to Anthropic and OpenAI

    Base URL: http://localhost:8080 (default) Authentication: The API requires no authentication by default. Content Type: All API responses are returned in JSON format with Content-Type: application/json.

  3. Overview of the ccflare Provider Layer

    main

    The provider layer in ccflare provides a consistent interface for interacting with multiple upstream APIs. It abstracts away provider-specific complexities, allowing the proxy and runtime server to remain provider-agnostic.

    Key responsibilities of the packages/providers package include:

    • Provider registration and lookup
    • Constructing provider-specific URLs
    • Preparing authentication headers
    • Orchestrating OAuth flows and refresh tokens
    • Parsing rate-limit signals and usage data (tokens, cache hits, etc.)

  4. Understand ccflare core behaviors

    main

    When using ccflare as a proxy, keep the following behaviors in mind:

    • No Authentication Required: API endpoints do not require authentication from the client; ccflare manages OAuth tokens internally for proxying to Claude.
    • Automatic Failover: If a request fails or an account is rate limited, ccflare automatically tries the next available account. If no accounts are available, requests are forwarded without authentication as a fallback.
    • Token Refresh: Access tokens are automatically refreshed when they expire.
    • Session Affinity: The session strategy maintains sticky sessions for consistent routing within a time window.
    • Rate Limit Tracking: ccflare extracts rate limit information (including reset times and remaining requests) from provider responses and stores it per account.
    • Provider Filtering: Accounts are automatically filtered by provider during request selection to ensure compatibility.

  5. Understand the AsyncDbWriter and Runtime Write Paths

    main

    To avoid blocking the high-performance proxy, ccflare uses an AsyncDbWriter as a background write queue. Most non-critical writes are offloaded to this worker.

    Typical Write Paths:

    1. API/TUI: Account CRUD, authentication sessions, and maintenance.
    2. Proxy: Request metadata and account state updates (via AsyncDbWriter).
    3. Worker (Post-processor): Payloads and usage summaries (via AsyncDbWriter).
    4. Token/Rate-Limit Updates: Writing refreshed tokens or rate-limit status back to accounts.

  6. Scale ccflare from SQLite to PostgreSQL

    main

    For horizontal scaling across multiple application instances, you must migrate from the default SQLite database to a shared database like PostgreSQL.

    In a scaled architecture:

    1. Use a Load Balancer (Nginx/HAProxy) to distribute traffic.
    2. Use a shared database (PostgreSQL) for persistent data.
    3. Use a shared session store (Redis) to maintain session state across instances.
    4. Use Service Discovery (Consul/etcd) for managing instances.

    Example PostgreSQL schema for accounts and requests tables is required to support features like rate limiting, token tracking, and session management.

  7. How ccflare Load Balancing works

    main

    ccflare uses a session-based load balancing strategy. This is the only supported strategy because it maintains conversation context with 5-hour sessions, which maximizes prompt cache efficiency and avoids triggering Claude's anti-abuse systems.

    Warning: Other strategies like round-robin, least-requests, or weighted have been removed to ensure account safety.

  8. Understand the Database and Persistence model

    main

    The @ccflare/database package manages all state via SQLite. It uses a repository pattern to interact with the data.

    Primary Repositories:

    • AccountRepository: Manages account configuration and live state.
    • RequestRepository: Stores request summaries and full payloads.
    • AuthSessionRepository: Manages short-lived OAuth session state.
    • AnalyticsRepository & StatsRepository: Handles observability data.
    • StrategyRepository: Manages load balancing strategies.

    Performance Note: To maintain high throughput, ccflare uses an AsyncDbWriter to perform non-blocking database writes, ensuring that persistence does not block the hot request path.

  9. How ccflare routing works

    main

    ccflare acts as a native proxy by routing requests based on URL prefixes. It strips the /v1/{provider} prefix exactly once before forwarding the request to the upstream provider.

    Provider-prefixed routes

    • http://localhost:8080/v1/anthropic/* $\rightarrow$ https://api.anthropic.com/*
    • http://localhost:8080/v1/openai/* $\rightarrow$ https://api.openai.com/*

    Compatibility routes

    The /v1/ccflare/* route allows you to keep a client-facing schema while selecting a provider family via the model field prefix. This allows cross-provider model mapping.

    • openai/<model-id> $\rightarrow$ prefers codex, then openai family.
    • anthropic/<model-id> $\rightarrow$ prefers claude-code, then anthropic family.

    Example mapping:

    • A request to /v1/ccflare/openai/chat/completions with "model":"anthropic/claude-sonnet-4" will route to an Anthropic provider.

  10. Understand the ccflare system architecture

    main

    ccflare is a Bun/TypeScript-based proxy system designed to manage and balance traffic across multiple provider-native HTTP and WebSocket APIs (like OpenAI or Anthropic).

    Core Functions:

    • Proxying: Forwards requests to providers while balancing load across configured accounts.
    • Persistence: Stores request/account/auth-session state in SQLite.
    • Management: Provides a REST API, a Web Dashboard, and a Terminal UI (TUI) for controlling the system.

    Key Architectural Layers:

    • Runtime Server: The orchestration layer that bootstraps the server, handles routing, and manages the lifecycle.
    • API Layer (@ccflare/api): The control plane for managing accounts, OAuth, stats, and logs.
    • Proxy Layer (@ccflare/proxy): The data plane that handles request forwarding, retries, and load balancing.
    • Provider Layer (@ccflare/providers): Contains the specific logic for different providers (e.g., openai, anthropic).

  11. Understand ccflare's Session-Based Load Balancing

    main

    ccflare uses a Session-Based Strategy to distribute requests across multiple Claude OAuth accounts. This strategy is designed to mimic natural human behavior by maintaining 'sticky sessions' with individual accounts for a configurable duration (default: 5 hours).

    Why use this strategy?

    • Rate Limit Avoidance: Minimizes frequent account switching which can trigger anti-abuse systems.
    • High Availability: Automatically filters out rate-limited or paused accounts and provides an ordered list of fallback accounts for automatic failover.
    • Safety: Other strategies (like round-robin) have been removed because they create suspicious patterns that can lead to account bans.

    How it works

    1. Filtering: Accounts are filtered by provider compatibility and availability (not paused, not currently rate-limited).
    2. Selection: The system looks for an account with an active session within the session_duration_ms window.
    3. Ordering: The selected active account is placed first in the list, followed by other available accounts as fallbacks.
    // The strategy returns an ordered list: [activeAccount, ...fallbacks]
// If no active session exists, it starts a new one with the first available account.

  12. Understand the ccflare runtime architecture

    main

    ccflare separates concerns into a control plane and a data plane to maintain low latency while providing high observability:

    • runtime-server: The entry point that routes all incoming traffic.
    • api: Manages the control plane (management endpoints, account setup, and database interactions).
    • proxy: Manages the data plane (forwarding provider-native requests to upstreams).
    • providers: Contain the logic for provider-specific authentication and parsing.
    • database: A SQLite backend used for persisting state, analytics, and logs.
    • post-processor worker: Handles heavy lifting like usage extraction and payload parsing in the background to keep the request path responsive.