RelayPlane Proxy

Complexity-Based Routing automatically classifies incoming requests into simple, moderate, or complex tiers based on prompt length, token patterns, and tool usage. Each tier can be mapped to a specific model to optimize for cost or performance.

Classification Logic:

• Simple: Short prompts, basic Q&A, or simple code tasks.
• Moderate: Multi-step reasoning, code reviews, or analysis with context.
• Complex: Architecture decisions, large codebases, tasks with many tools, or long prompts containing evaluation/comparison language.

Classification happens locally; prompt content is never sent to the classifier.

{
  "routing": {
    "complexity": {
      "enabled": true,
      "simple": "claude-3-5-haiku-latest",
      "moderate": "claude-sonnet-4-20250514",
      "complex": "claude-opus-4-20250514"
    }
  }
}

The directory packages/proxy within the turleydesigns/relayplane-workflows repository is the single source of truth for both the @relayplane/proxy npm package and the github.com/RelayPlane/proxy GitHub repository.

Edits made to files within packages/proxy/ (such as README.md) are automatically propagated to both destinations via a CI/CD workflow. There is no requirement for manual copying or syncing between these destinations; the automation handles the build, versioning, npm publishing, and GitHub mirroring.

A proposed mechanism for external clients to influence routing is via the X-RelayPlane-Complexity HTTP header.

Implementation Details

• Accepted Values: simple, moderate, or complex.
• Behavior: If the header is present, the value should be validated and fed into the local complexity variable before tier selection and into resolvePolicy.
• Caching Warning: Because response cache lookups currently hash the request body and not the headers, any implementation using this header must ensure the override is part of the response cache identity to prevent a complex request from incorrectly reusing a simple-tier cached response.

RelayPlane supports a credentialPool to manage multiple accounts (e.g., Account A and Account B) for automatic failover and load balancing.

Account Priority and Setup

• Account A (Priority 1): Typically the primary login located at ~/.claude/.credentials.json. It is treated as refresh:false and is not managed by RelayPlane.
• Account B (Priority 2): A secondary account located in a dedicated file at ~/.relayplane/account-b.credentials.json. It is treated as refresh:true, meaning RelayPlane will manage its OAuth refresh loop.

Configuration Integration

To enable pooling, you must merge a credentialPool array into your RelayPlane configuration file. When configuring paths within the JSON, use the absolute path to your $HOME directory.

RelayPlane supports Anthropic subscription tokens (starting with sk-ant-oat*) via the x-api-key header.

Key Authentication Details:

• Header Usage: All Anthropic token types (sk-ant-api* and sk-ant-oat*) must be sent via the x-api-key header. The proxy does not use Authorization: Bearer for Anthropic requests.
• Subscription Validation: The proxy automatically forwards user-agent and x-app headers required by Anthropic to validate Claude Max subscriptions.
• Configuration: No special configuration is required for standard passthrough of these tokens. For advanced hybrid authentication (using MAX tokens for expensive models and standard keys for cheap ones), refer to the Configuration reference.

Cascade Mode starts with the cheapest model in a provided list and escalates to more expensive models only when the response indicates uncertainty, refusal, or an error. This provides a safety net while minimizing costs.

escalateOn options:

• uncertainty: Triggers on hedging language (e.g., "I'm not sure").
• refusal: Triggers when the model refuses to assist.
• error: Triggers on outright request failure.

maxEscalations: Caps the number of retries with more expensive models (Default: 1).

{
  "routing": {
    "mode": "cascade",
    "cascade": {
      "enabled": true,
      "models": [
        "claude-3-5-haiku-latest",
        "claude-sonnet-4-20250514",
        "claude-opus-4-20250514"
      ],
      "escalateOn": "uncertainty",
      "maxEscalations": 2
    }
  }
}

Once testing is complete, you can enable pooling for your live service and interactive Claude Code sessions.

1. Update Live Config: Merge the credentialPool block into your production config at $HOME/.relayplane/config.json.
2. Restart Service: Restart the relayplane-proxy system service.
3. Route Interactive Sessions: To make your interactive Claude Code sessions use the proxy (enabling pooling and failover), set the ANTHROPIC_BASE_URL environment variable in your shell profile or tmux environment.

Note: To roll back, remove the credentialPool key from the config and restart the proxy. The proxy will return to single-token behavior.

# 4b. Restart the live service:
sudo systemctl restart relayplane-proxy

# 4c. Point interactive Claude Code sessions at the proxy:
export ANTHROPIC_BASE_URL=http://localhost:4100

If you are an Anthropic Max plan user using OAuth tokens (sk-ant-oat*), auto-routing to Haiku requires an additional configuration step. Anthropic does not support OAuth for the Haiku model.

To ensure the proxy can route simple requests to Haiku, you must provide a standard API key (sk-ant-api03-*) via the ANTHROPIC_API_KEY environment variable. Without this, the proxy cannot use Haiku and will fall back to Sonnet for simple tasks.

RelayPlane reads its configuration from ~/.relayplane/config.json. You can override this default location by setting the RELAYPLANE_CONFIG_PATH environment variable.

All configuration fields are optional and use a deep merge strategy with sensible defaults. A minimal configuration structure looks like this:

{
  "enabled": true,
  "modelOverrides": {},
  "routing": {
    "mode": "cascade",
    "cascade": { "enabled": true },
    "complexity": { "enabled": true }
  }
}