CliRelay Documentation

CliRelay is a unified proxy server designed for AI CLI tools. It allows you to use existing AI subscriptions (like Gemini, Claude, OpenAI, etc.) through a single, managed API layer.

By pointing your AI coding tools (e.g., Claude Code, Gemini CLI, OpenAI Codex, Ollama, etc.) to the CliRelay endpoint, you gain access to advanced features like:

• Unified Endpoint: Access multiple providers via http://localhost:8317.
• Smart Load Balancing: Round-robin or fill-first scheduling across multiple API keys.
• Auto Failover: Automatic switching to backup channels when quotas are hit or errors occur.
• Request Logging: Full capture of requests, responses, and token usage in PostgreSQL.
• API Key Management: Create and manage keys with custom quotas and rate limits.
• Management Panel: A web-based UI at /manage for visual configuration.

Architecture Overview:

• Upstream: Connects to providers like Google Gemini, OpenAI, Anthropic, Vertex, Bedrock, etc.
• Proxy Core: Acts as the intermediary at :8317.
• Clients: AI tools like Claude Code, Gemini CLI, or any OpenAI-compatible client connect to CliRelay.

┌───────────────────────┐         ┌──────────────┐         ┌────────────────────┐
│   AI Coding Tools     │         │              │         │  Upstream Providers │
│                       │         │              │ ──────▶ │  Google Gemini      │
│  Claude Code          │ ──────▶ │   CliRelay   │         │  OpenAI / Codex     │
│  Gemini CLI           │         │   :8317      │         │  Anthropic Claude   │
│  OpenAI Codex         │         │              │         │  Qwen / iFlow       │
│  Amp CLI / IDE        │         └──────────────┘         │  Antigravity/xAI    │
│  Any OAI-compatible   │                                  │  Vertex / Bedrock   │
└───────────────────────┘                                  │  OpenCode/Cline     │
                                                           │  Ollama / Amp       │
                                                           └────────────────────┘

CliRelay provides several key capabilities for managing AI traffic:

Multi-Provider Proxy Engine

• Unified Endpoint: A single endpoint (default http://localhost:8317) handles multiple providers including Gemini, Claude, Codex, Qwen, Vertex, Bedrock, and more.
• Smart Load Balancing: Uses polling or priority-based scheduling across multiple API keys.
• Routing: Supports grouping channels and custom path namespaces for teams or specific business logic.
• Automatic Failover: Automatically switches to backup channels when a quota is exhausted or an error occurs.
• Multimodal & Streaming: Supports text, images, function calling, and SSE streaming.
• OpenAI Compatibility: Works with any upstream service following the OpenAI Chat Completions protocol.

Request Logging & Monitoring

• Full Capture: Logs timestamps, models, tokens (input/output/reasoning/cache), latency, status, and source channels to PostgreSQL.
• Message Storage: Stores compressed request/response bodies with configurable retention policies.
• Health Scoring: Provides a real-time 0–100 health score based on success rates, latency, and error patterns.
• Real-time Monitoring: Uses WebSockets to push system stats (CPU, memory, network I/O, etc.).

API Key & Permission Management

• CRUD Operations: Create, edit, and delete API keys with custom names and enable/disable toggles.
• Quota & Rate Limiting: Set maximum token/request quotas and per-key rate limits (RPM/RPH).
• Access Control: Assign keys to specific users/teams and restrict their access to certain channel groups or models.
• Self-Service: Provides a public query page for end-users to check their own usage and logs without logging in.

Channel Management

• Provider Organization: Manage channels by type (Gemini, Claude, Vertex, etc.).
• Customization: Set custom names, proxy URLs, headers, and model alias mappings.
• Latency Tracking: Monitors average latency_ms per channel.
• Model Control: Exclude specific models from certain channels (e.g., blocking expensive models on backup keys).

When performing migrations in CliRelay, the following technical rules and safety constraints apply:

Data Copying Rules

• No SELECT *: Every table uses a stable column list.
• Ordering: Data is sorted by primary key or unique key. Tables without auto-incrementing primary keys are sorted by a combination of primary key columns.
• Large Objects: Large BLOB tables (like request_log_content) are copied in separate batches.
• Dependencies: Foreign key tables are copied following dependency order (parent tables before child tables).
• Dialect Safety: PostgreSQL schema names are quoted safely; MySQL tables and columns use backticks.

Cutover and Configuration Rules

• YAML Configuration: If the database is managed via config.yaml, the backend will attempt to write the new configuration directly.
• Environment Variables: If the database is configured via environment variables, the backend cannot modify them; it will instead provide instructions on how to update the deployment configuration.
• Service Restarts: If the service does not support hot-swapping connections, the system will mark the migration as "requires restart" and will not restart the service automatically.
• Safety: Any failed cutover attempt will not delete the source database.

To correctly interact with CliRelay, distinguish between these key concepts:

• Upstream Model ID: The original model name used by the provider (e.g., the real name used by an API key or OAuth account).
• External Model ID: The model value sent in a client request. This is what users see in /models or management tables. It may be the original name, an alias, or a prefixed name.
• Alias: A mapping (name -> alias) where a user requests an alias and the execution layer maps it back to the original name.
• Prefix Model: A model ID generated by adding a prefix to the model name (e.g., prefix/model).
• Default Path: Public API paths without custom prefixes, such as /v1/chat/completions or /v1/images/generations.
• Custom Path: Paths generated by channel groups or path-routes, such as /{group}/v1/... or multi-segment custom paths.
• Path Family: A group of interfaces sharing the same protocol (e.g., OpenAI v1, Claude Messages, Gemini v1beta).

The watcher.AuthUpdate struct is the data contract used to communicate credential changes.

Contract Details

• Action: Can be add, modify, or delete.
• ID: The unique identifier for the credential.
• Auth: For add or modify actions, this contains a full clone of the Auth object. For delete actions, this field can be omitted.

Watcher Logic

• Snapshot Comparison: The watcher maintains a currentAuths snapshot. When a file or configuration event occurs, it rebuilds the snapshot, compares it to the old one, and generates a minimal list of AuthUpdate objects.
• Event Merging: Updates are merged based on the credential ID. If a single credential undergoes multiple changes in a short period (e.g., a write followed by a delete), the watcher only emits the final state (e.g., a single delete action).
• Asynchronous Dispatch: The watcher uses an internal asynchronous loop. Producers append events to an in-memory buffer and wake the dispatch goroutine, ensuring that file event threads are not blocked even if the channel is temporarily full.