Apache APISIX

The ai-proxy plugin simplifies access to Large Language Model (LLM) and embedding model providers. It acts as a transformer, converting APISIX plugin configurations into the specific request formats required by various providers, including:

• OpenAI
• DeepSeek
• Azure
• AIMLAPI
• Anthropic
• OpenRouter
• Gemini
• Vertex AI
• Amazon Bedrock
• Other OpenAI-compatible APIs

Additionally, the plugin can log LLM request metadata (such as token usage, model name, and time to first response) into the access log. These logs are compatible with logging plugins like http-logger and kafka-logger. Note that these logs are sent to the access log and do not affect error.log.

The ai-proxy-multi plugin is an extension of the ai-proxy plugin designed to simplify access to Large Language Models (LLMs) and embedding models. It acts as a proxy that transforms plugin configurations into the specific request formats required by various providers, including OpenAI, DeepSeek, Azure, AIMLAPI, Anthropic, OpenRouter, Gemini, Vertex AI, and Amazon Bedrock.

Key features include:

• Advanced Reliability: Adds load balancing, retries, fallbacks, and health checks to LLM requests.
• Observability: Supports logging LLM-specific metadata in the access log, such as token usage, model name, and time to first response. These logs are compatible with logging plugins like http-logger and kafka-logger.

The lago plugin enables API monetization by reporting usage data to Lago Self-hosted or Lago Cloud via the Lago REST API. It is particularly useful for billing scenarios like AI token consumption or standard API call metering.

Key features:

• Collects request/response context (event codes, transaction IDs, subscription IDs).
• Serializes data into Event JSON objects.
• Uses a batching mechanism to send up to 100 events at once to meet Lago's requirements.
• Integrates with APISIX features like the consumer and request-id plugins to dynamically map usage to specific customers and subscriptions.

The request-validation plugin validates incoming requests before they are forwarded to upstream services. It uses JSON Schema to define validation rules for both request headers and the request body.

To use this plugin, you must provide at least one of header_schema or body_schema in the plugin configuration.

The tcp-logger plugin sends log data as JSON objects to an external TCP server. It supports both real-time sending and batch processing to improve efficiency. When using the batch processor, logs are sent every 5 seconds or when the queue reaches 1000 entries (unless configured otherwise).

curl http://127.0.0.1:9180/apisix/admin/routes/5 -H "X-API-KEY: $admin_key" -X PUT -d '
{
      "plugins": {
            "tcp-logger": {
                 "host": "127.0.0.1",
                 "port": 5044,
                 "tls": false,
                 "batch_max_size": 1,
                 "name": "tcp logger"
            }
       },
      "upstream": {
           "type": "roundrobin",
           "nodes": {
               "127.0.0.1:1980": 1
           }
      },
      "uri": "/hello"
}'

The ai-lakera-guard plugin integrates Apache APISIX with the Lakera Guard API (v2) to perform ML-based security scanning of LLM traffic. It inspects requests for prompt injection, jailbreak, PII leakage, content-policy violations, and malicious links.

Key Requirements:

• Must be used with either the ai-proxy or ai-proxy-multi plugin to ensure the plugin can extract chat content in a protocol-aware way.
• Requests that do not pass through ai-proxy/ai-proxy-multi (e.g., plain HTTP traffic) cannot be inspected and are handled based on the fail_mode setting.

Core Logic:

• The plugin relies on a project_id to determine which detectors and thresholds to apply via the Lakera project policy.
• It can scan the request (input), the response (output), or both (both).

The opa plugin integrates Apache APISIX with Open Policy Agent (OPA) to enable unified policy enforcement. When a client request hits a protected Route, APISIX sends request metadata to the OPA engine. OPA evaluates the request against policies written in Rego and returns a decision.

To work correctly, your OPA policy must return an object containing an allow field. You can also optionally include reason, headers, and status_code in the OPA response to customize the authorization outcome.

The ai-rag plugin enhances Large Language Model (LLM) outputs by using Retrieval-Augmented Generation (RAG). It retrieves relevant documents from external data sources to improve the accuracy and contextual relevance of LLM responses.

Currently, the plugin supports:

• Azure OpenAI for generating embeddings.
• Azure AI Search for performing vector searches.