FreeLLMAPI

The router uses a fallback chain to ensure high availability. You can rank this chain using one of six selectable strategies via the Dashboard:

• priority: Uses your manually defined order.
• balanced: A mix of speed and reliability.
• smartest: Prioritizes model capability.
• fastest: Prioritizes low latency.
• reliable: Prioritizes providers with the highest uptime/health.
• custom: Allows for your own specific weight mix.

Key Routing Features:

• Automatic Fallover: If a provider fails (429, 5xx, or timeout), the router skips it, applies a cooldown, and retries (up to 20 attempts within a wall-clock budget).
• Unified Models: Multiple providers serving the same logical model (e.g., GLM-4.7 on Cloudflare and Z.ai) are collapsed into a single entry in /v1/models for seamless in-group failover.
• Model Profiles: You can save and switch between different fallback-chain configurations (e.g., a coding chain, a long-context chain, or a vision chain) from the dashboard.
• Sticky Sessions: To prevent hallucinations caused by mid-conversation model switches, the router attempts to keep a conversation talking to the same model for 30 minutes.

Instead of using a specific model name, you can use the auto prefix to let the router select a model based on specific performance profiles. This allows you to steer requests without changing dashboard settings.

Available Profiles:

• auto:smart: Favors highest-intelligence models.
• auto:fast: Favors throughput and low time-to-first-byte.
• auto:cheap: Budget-leaning (currently uses the balanced blend).
• auto:reliable: Favors models with recent high success rates.
• auto:balanced: The default (reliability first, then split between speed and intelligence).

Note: Profiles are case-insensitive and support common synonyms (e.g., auto:fastest, auto:smartest, auto:budget).

Named Profiles: You can route through a specific named fallback chain created in your dashboard using auto:<profile-name>. If the profile name is unknown, the API returns a 400 error.

# Example using the 'fast' profile
curl http://localhost:3001/v1/chat/completions \
  -H "Authorization: Bearer freellmapi-your-unified-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto:fast",
    "messages": [{"role": "user", "content": "hi"}]
  }'

The Premium service provides a live, signed model catalog that keeps your local router up-to-date with the latest providers, model families, and quota changes.

Key Features:

• Automatic Updates: The router pulls a signed catalog from freellmapi.co twice daily, applying updates to your local database without touching your custom providers or enable/disable choices.
• Security: Every download is verified against a pinned Ed25519 key before application.
• Coverage: Tracks approximately 29 providers, 251 model families, and 358 provider/model endpoints.
• Unified Access: A single fla_ key covers all your running routers (Desktop, Homelab, Raspberry Pi, etc.).

Activation and Management:

• Activate via the Premium section in the dashboard.
• Manage or cancel billing at freellmapi.co/manage.
• The router software remains MIT-licensed and free; Premium only covers the live catalog feed.

FreeLLMAPI acts as an OpenAI-compatible proxy that routes requests to various free-tier LLM providers (e.g., Google, Groq, Cerebras, OpenRouter, HF).

The Request Flow:

1. Client Request: An OpenAI SDK, curl, or any OpenAI-compatible client sends a request to the Express proxy (default port :3001) using a freellmapi-... bearer token.
2. Routing Logic: The Router selects the highest-priority model that satisfies two conditions:
   • It has a healthy API key.
   • It is currently under its rate limits (RPM/RPD/TPM/TPD).
3. Execution: The router decrypts the provider key, calls the provider's SDK, and streams tokens back to the client.
4. Error Handling: If a provider returns a 429 (Rate Limit) or 5xx (Server Error), the router puts that key on cooldown and automatically retries the request using the next model in the fallback chain.

Core Components:

• Router: Manages model selection per request.
• Rate-limit ledger: Tracks in-memory counters (RPM, RPD, TPM, TPD) backed by SQLite.
• Provider adapters: Implement the Provider base class for chatCompletion() and streamChatCompletion().
• Health service: Periodically probes keys to ensure they are valid and active.
• Storage: Uses SQLite with AES-256-GCM encryption for API keys.

┌──────────────────┐   Bearer freellmapi-…   ┌─────────────────────────┐
│  OpenAI SDK /    │ ──────────────────────▶ │  Express proxy (:3001)  │
│  curl / any      │ ◀────────────────────── │  /v1/chat/completions   │
│  OpenAI client   │      streamed tokens    └────────────┬────────────┘
└──────────────────┘                                      │
                                                          ▼
                             ┌────────────────────────────────────────────────┐
                             │  Router                                        │
                             │   1. Pick highest-priority model that          │
                             │      (a) has a healthy key and                 │
                             │      (b) is under all its rate limits.         │
                             │   2. Decrypt key, call provider SDK.           │
                             │   3. On 429/5xx → cooldown + retry next model. │
                             └────────────────────────────────────────────────┘