DOCSEARCHbyAlgolia
Submit Docs

Nexus Router Documentation

website·Indexed 32 days ago

https://nexusrouter.com/docs

Documentation for Nexus, a router and proxy for LLMs and Model Context Protocol (MCP) servers. It provides a single OpenAI-compatible API to manage multiple AI model providers (OpenAI, Anthropic, Google), featuring token-based rate limiting, header transformation, OAuth2 authentication, and OpenTelemetry integration for monitoring and tracing.

Tokens
25.3K
Snippets
151
Records
186
Agent score
67%

What's inside Nexus

  1. Implement MCP Role-Based Access Control (RBAC)

    Nexus RBAC allows restricting access to MCP servers and specific tools based on user group membership. Access is governed by these principles:

    • Deny takes precedence: Users in a denied group are blocked regardless of allow rules.
    • Empty allow list blocks all: Setting allow = [] prevents all access without requiring client identification.
    • Tool rules override server rules: Tool-specific settings take precedence over server-level settings.
    • No rules means open access: Access is unrestricted if no RBAC configuration is provided.

  2. NexusRouter LLM routing capabilities

    NexusRouter provides a unified access layer for LLMs with the following capabilities:

    • Multi-Protocol Support: Native endpoints for both OpenAI and Anthropic protocols.
    • Unified Access: Ability to route to any provider regardless of the protocol used by the client.
    • Client Compatibility: Compatible with OpenAI, Anthropic, and Claude Code clients.
    • Model Management: Support for custom model aliases and mixing models from different vendors.
    • Traffic Control: Token rate limiting per user/model and real-time streaming via SSE.
    • Request Manipulation: Header transformation (forward, insert, remove, rename) and token forwarding allowing users to provide their own API keys.

  3. Analyze HTTP Request spans

    The root span for all incoming HTTP requests is named using the format {method} {path} (e.g., POST /mcp). It contains metadata about the request and response, as well as client identification for rate limiting.

    Attributes:

    • http.request.method: HTTP method (GET, POST, etc.)
    • http.route: Request route pattern
    • http.response.status_code: Response status code
    • http.request.body.size: Request body size in bytes
    • http.response.body.size: Response body size in bytes
    • client.id: Client identifier from rate limiting
    • client.group: Client group for rate limiting
    • url.path: Full URL path
    • url.query: Query parameters
    • user_agent.original: Client user agent string

  4. Configure basic Nexus server network and health settings

    Configure the fundamental network behavior and health check endpoint for the Nexus server in the nexus.toml file. The listen_address defines where the server binds, while the health section manages the monitoring endpoint used by load balancers and orchestrators like Kubernetes.
    [server]
listen_address = "127.0.0.1:8000" # Optional, defaults to 127.0.0.1:8000

[server.health]
enabled = true # Default: true
path = "/health" # Default: /health
listen = "0.0.0.0:8001" # Optional: separate address for health endpoint

  5. Install Nexus via quick install script

    The fastest way to install Nexus on Linux (x86_64, aarch64), macOS (Intel, Apple Silicon), or Windows (via WSL) is using the official installation script. The script detects the OS/architecture, downloads the binary, and installs it to ~/.nexus/bin.
    curl -fsSL https://nexusrouter.com/install | bash

  6. Configure Codex CLI to route through Nexus

    Edit the Codex configuration file at ~/.codex/config.toml to point to the Nexus OpenAI-compatible endpoint.

    Critical Requirement: Nexus serves the endpoint at /llm/openai/, but Codex expects a /v1 suffix. The base_url must end with /v1 (e.g., http://127.0.0.1:6000/llm/openai/v1).

    [model_providers.nexus]
name = "Nexus AI router"
base_url = "http://127.0.0.1:6000/llm/openai/v1"
wire_api = "chat"
query_params = {}

  7. CORS Production Best Practices

    To ensure security and stability in production environments, follow these guidelines:

    1. Avoid using wildcard origins ("*"); specify exact origins instead.
    2. Only expose the minimum necessary headers via expose_headers.
    3. Use HTTPS for all allowed origins.
    4. Regularly validate and update the list of allowed origins.
    5. Log and monitor CORS failures to identify misconfigurations or attacks.
    6. Test CORS settings in real browsers rather than relying solely on CLI tools.

  8. Query traces for performance and error analysis

    Use the following filter strategies in your OTLP-compatible backend (e.g., Jaeger, Tempo) to debug Nexus operations:
    • Slow Requests: Filter duration > 1s, group by http.route
    • Failed Requests: Filter http.response.status_code >= 500 or error = true
    • Token Usage: Filter span.name = "LLM*", sum gen_ai.usage.total_tokens
    • Rate Limits: Filter rate_limit.blocked = true, group by client.id and rate_limit.scope

  9. Run Nexus in Docker for Claude Code integration

    When running Nexus in a Docker container, map port 6000 and mount the nexus.toml configuration file. Configure Claude Code to point to the container's exposed port.
    # docker-compose.yml
services:
  nexus:
    image: ghcr.io/grafbase/nexus:latest
    ports:
      - "6000:6000"
    volumes:
      - ./nexus.toml:/etc/nexus.toml
    environment:
      - ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
    export ANTHROPIC_BASE_URL="http://localhost:6000/llm/anthropic"
export ANTHROPIC_MODEL="anthropic/claude-3-5-sonnet-20241022"

  10. Implement token forwarding for MCP servers

    Token forwarding allows Nexus to pass user identity to downstream MCP servers. This is recommended for Single Sign-On (SSO) implementations, maintaining user context across services, creating audit trails, and isolating data in multi-tenant systems. To enable this, ensure OAuth2 is enabled on Nexus and the server configuration is set to type = "forward".

  11. Combine OAuth2 authentication with token forwarding

    You can require OAuth2 authentication for access to the Nexus API while still allowing users to provide their own provider keys. In this scenario, the client must send both the Authorization: Bearer <token> header for Nexus access and the X-Provider-API-Key header for the LLM provider.
    curl -X POST http://localhost:8000/llm/v1/chat/completions \
  -H "Authorization: Bearer <oauth-token>" \
  -H "X-Provider-API-Key: sk-user-api-key" \
  -H "Content-Type: application/json" \
  -d '{"model": "openai/gpt-4", "messages": [...]}'

  12. Deploy Nexus using Docker

    Nexus is available as a Docker image. You can pull the stable tag (recommended), a specific version (e.g., 0.2.0), or the latest development build. The server typically listens on port 8000 and requires a nexus.toml configuration file mounted to /etc/nexus.toml.
    # Pull stable image
docker pull ghcr.io/grafbase/nexus:stable

# Run with config file and environment variables
docker run -p 8000:8000 \
  -v $(pwd)/nexus.toml:/etc/nexus.toml \
  -e GITHUB_TOKEN=${GITHUB_TOKEN} \
  -e OPENAI_API_KEY=${OPENAI_API_KEY} \
  ghcr.io/grafbase/nexus:stable