Plexus LLM Gateway

Plexus is an LLM gateway that sits in front of multiple LLM providers (OpenAI, Anthropic, Gemini, and OpenAI-compatible providers). It provides a single, consistent API surface for clients, allowing you to decouple your client code from specific provider implementations.

Key Capabilities:

• Protocol Translation: Handles requests in OpenAI Chat Completions, Anthropic Messages, and Gemini native formats, including streaming and tool use.
• Policy-Aware Routing: Map model aliases to one or more targets using selection strategies like random, in_order, cost, performance, latency, usage, quota, or e2e_performance.
• Resilience: Supports exponential cooldowns, provider failover, stalled stream detection, and vision fallthrough for non-vision targets.
• Observability: Provides a dashboard to inspect request logs, tokens, cost, latency, throughput, and provider health.
• Security: Supports AES-256-GCM encryption at rest for sensitive data (API keys, OAuth tokens, etc.) when an ENCRYPTION_KEY is provided.

When making image requests, you can influence how Plexus selects and routes to providers using the following routing intent parameters:

• provider.only: Restricts eligible requests to a specific set of configured Plexus provider names. An empty set results in an error.
• provider.ignore: Excludes specific configured Plexus provider names from consideration.
• provider.order: Ranks eligible providers in a preferred order. Providers not listed in the order remain valid fallback candidates.
• provider.allow_fallbacks:
  • true (default): Allows the system to try fallback providers if the primary target fails.
  • false: Limits the request strictly to the primary eligible target.
• provider.sort: A routing preference (e.g., by price or latency) based on Plexus ranking data. Note that this is a routing preference, not an image generation parameter.
• provider.options: Scoped by the configured provider name. These options are passed to the specific target transformer and are not shared across providers.

Plexus normalizes all image provider responses into a single unified model.

Response Structure:

• created: Unix timestamp (seconds).
• data[]: An array of image objects. Each image contains:
  • bytes or base64: The image data.
  • url: A usable URL (if provided by the upstream).
  • media_type: The actual media type when known.
  • revised_prompt: An optional field containing the revised prompt.
• usage: (Optional) Token usage details.

Usage Normalization: Token fields from OpenAI and Gemini usageMetadata are normalized to Plexus internal names. For OpenRouter-shaped responses, these are projected to:

• prompt_tokens
• completion_tokens
• total_tokens (including reported cost if available).

Image Token Details: If a target reports image tokens, they are mapped to:

• input_image_tokens (via input_tokens_details.image_tokens)
• output_image_tokens (via output_tokens_details.image_tokens)

In Plexus, /v1/images/generations and /v1/images/edits are unified into a single dispatch loop. The system uses a UnifiedImageGenerationRequest to handle both.

Whether a request is treated as a generation or an edit is determined by the presence of input_references. For multipart edits, the uploaded image is mapped to input_references[0].

Key behaviors:

• Masks: Masks remain an explicit field at every target. While the OpenAI multipart path appends the mask, other transformers like Gemini, OpenRouter, and Codex will reject a mask with an explicit error rather than silently dropping it.
• Response Formats: For /v1/images/edits, if response_format is omitted, the system defaults to the provider's own default (e.g., url for dall-e-2) instead of forcing b64_json.
• MIME Types: If an upload's declared MIME type is not image/*, Plexus sniffs the magic bytes to determine the type rather than rejecting it.

The tool uses static analysis to detect Fastify route registrations.

Route Detection

It looks for patterns like:

• fastify.get('/path', handler)
• mgmt.post('/path', handler)
• app.put('/path', handler)

Note: Dynamic paths (e.g., fastify.get(path, handler)) or variable methods (e.g., fastify[method](path, handler)) will not be detected.

Path Normalization

To compare code to documentation, the tool normalizes path parameters:

Tag Assignment

When generating new files, tags are automatically assigned based on the path prefix:

• /v0/management/config/* $\rightarrow$ Management — Config
• /v0/management/quotas/* $\rightarrow$ Management — Quotas (Provider)
• /v0/management/oauth/* $\rightarrow$ Management — OAuth
• /v1/* $\rightarrow$ Inference
• /mcp/* $\rightarrow$ MCP