Bella OpenAPI

repository·develop·Indexed 42 days ago

https://github.com/lianjiatech/bella-openapi

A high-performance AI API gateway providing unified access to LLM chat, text embedding, ASR, TTS, and image generation. It features integrated billing, rate limiting, and a four-layer hierarchical metadata structure (Category, Endpoint, Model, Channel) for flexible routing. Includes a web frontend (bella-openapi-web v0.1.0) and SDK callbacks for streaming responses, WebSockets, and SSE.

Tokens
40.9K
Snippets
101
Records
174
Agent score
53%

What's inside Bella OpenAPI

  1. Overview of Bella OpenAPI capabilities

    develop

    Bella OpenAPI is an AI-focused API gateway that provides a unified interface for various AI capabilities, similar to OpenRouter but with broader service offerings. It integrates billing, rate limiting, and resource management into a single platform.

    Key capabilities include:

    • Text Processing: Chat Completion (supporting OpenAI and Anthropic API formats) and Text Embedding.
    • Voice Services: Real-time Speech Recognition (streaming and one-shot), Audio Transcription (offline), Text-to-Speech (with streaming), and Real-time Conversation (voice input to voice output).
    • Image Services: Text-to-Image and Image-to-Image (coming soon).
    • Testing: Built-in Mock capability for unit and stress testing.
  2. Core Capabilities of the /apikey Page

    develop

    The API Key management page provides the following functional capabilities:

    • List Management: View API Key lists, pagination, and asynchronous balance fetching for each key.
    • Search: Search functionality with a 500ms debounce and clear input option.
    • Creation: Create new keys and view/copy the generated API Key via a success dialog.
    • Editing: Inline editing of names and service names, and updating security levels.
    • Row Actions: Manage sub-AKs, reset keys, transfer ownership, copy AK codes, and delete keys.
    • Transfer Flow: Search for users, provide a mandatory transfer reason, and complete the transfer.
    • Localization: Support for access via locale-specific routes.
  3. Overview of RocksDB Full Log Storage implementation

    develop
    The RocksDB Full Log Storage feature provides complete storage for Large Language Model (LLM) request/response logs. While the existing Elasticsearch (ES) pipeline continues to store truncated logs for efficient searching, RocksDB persists the full, non-truncated log content to Persistent Volume Claims (PVC). This allows users to view complete request/response payloads via the frontend when the truncated version is insufficient.
  4. Trace SSE connection lifecycles

    develop

    Server-Sent Events (SSE) connections are managed via SseHelper. To ensure observability, the traceId is included in all four lifecycle stages of an SSE connection. This allows you to correlate streaming events (like chat completions) with the original HTTP request.

    Lifecycle Logs include [traceId=xxx]:

    1. Creation: When the SSE connection is successfully established.
    2. Completion: When the connection finishes normally.
    3. Timeout: When the connection reaches its timeout limit.
    4. Error: When an exception occurs during the connection.
  5. Understand the ApiKey permission model

    develop

    The ApiKey permission system determines access based on a triplet: (caller identity, relationship between caller and target AK, operation type).

    Instead of scattered logic, permissions are centralized in a permission matrix. The core mechanism uses a single entry point, checkPermission(targetCode, operation), where business logic only needs to specify the operation type.

    Core Abstractions

    • AkOperation: The specific action being performed (e.g., QUERY, RESET, UPDATE_QUOTA).
    • AkRelation: The relationship between the caller and the target AK (OWNER, MANAGER, SAME_ORG, or UNRELATED).
    • rolecode: Defines the administrative level and path permissions:
      • all: Super administrator (/**).
      • console: Console administrator (/v*/**, /console/**).
      • high: High-level personal AK.
      • low: Basic personal AK.
    • ownerType: The ownership category of the AK (system, org, person, or console).
    // Example of the core operation types used in permission checks
    public enum AkOperation {
        QUERY,                 // Query AK info
        RESET,                 // Reset keys
        RENAME,                // Rename AK
        UPDATE_ROLE,           // Modify role/path permissions (Admin only)
        UPDATE_MANAGER,        // Modify manager (Owner/Manager can do this)
        CERTIFY,               // Security certification
        UPDATE_QUOTA,          // Modify quota
        UPDATE_QPS,            // Modify QPS limits
        CHANGE_STATUS,         // Enable/Disable
        CREATE_CHILD,          // Create child AK
        TRANSFER,              // Transfer ownership (Owner only)
        VIEW_TRANSFER_HISTORY, // Query transfer history
        BIND_SERVICE           // Bind service
    }
  6. How model name normalization works

    develop

    Bella OpenAPI implements a normalization logic to handle model name format discrepancies between external systems (like Hermes) and the internal database. Specifically, it allows underscore-separated version numbers (e.g., claude_opus_4_7) to match dot-separated names in the database (e.g., claude_opus_4.7).

    Key Logic: "Search-then-Replace" (先查后替)

    To prevent side effects and performance degradation, the system follows a strict fallback strategy:

    1. Initial Search: The system first attempts to find the model using the original model_name provided in the request.
    2. Normalization Fallback: If (and only if) the original name is not found, the system applies a regex transformation to replace underscores between digits (\d_\d) with dots (\d.\d).
    3. Secondary Search: The system then attempts to find the model using this new normalized name. If the normalized name exists in the model registry, it is returned.
    4. Final Fallback: If the normalized name also fails, the original name is returned.

    Constraints and Behavior

    • Regex Precision: The transformation only targets underscores between digits (e.g., model_1_2 becomes model_1.2). Underscores in other contexts (e.g., my_custom_model) are ignored.
    • No Data Mutation: The normalization happens only at the parsing layer; the original request body remains unchanged.
    • Caching: The system uses JetCache (L1/L2). When a fallback occurs, the result (the normalized name) is cached under the original requested key, ensuring subsequent identical requests do not trigger the fallback logic again.
    // Conceptual logic flow in ModelService.java
    public String fetchTerminalModelName(String modelName) {
        // 1. Try original name
        List<String> path = getPath(modelName);
        if (!CollectionUtils.isEmpty(path)) {
            return path.get(path.size() - 1);
        }
    
        // 2. Fallback to normalized name
        String normalized = tryNormalizeModelName(modelName);
        if (!normalized.equals(modelName)) {
            path = getPath(normalized);
            if (!CollectionUtils.isEmpty(path)) {
                return path.get(path.size() - 1);
            }
        }
        return modelName;
    }
  7. Understand ApiKey Manager Delegation

    develop

    The ApiKey Manager system allows an API Key (AK) owner to delegate daily management permissions to a specific user (manager). This allows the manager to perform operations on the AK and its child AKs without owning them.

    Key Concepts:

    • Delegation Scope: Supports both personal and organizational AKs.
    • Manager Permissions: Managers have near-owner permissions (RESET, RENAME, UPDATE_MANAGER, CREATE_CHILD, etc.) but cannot perform the TRANSFER operation (transferring ownership).
    • Inheritance: Management permissions automatically flow from a parent AK to all its child AKs. Even if a child AK's manager is changed, the parent AK's manager retains control over that child via permission tracing.
  8. How channel capacity is calculated

    develop

    The CapacityCalculator determines the remaining processing capability of a channel using a multi-layered strategy. It combines historical 429 error data, real-time concurrency, and completed RPM (Requests Per Minute).

    Capacity Calculation Formula

    Remaining Capacity = 1.0 - (Current Load / Total Capacity)

    Where:

    • Current Load = Current Concurrent Requests + Current Completed RPM
    • Total Capacity is determined by a three-level fallback strategy:
      1. Level 1 (Most Accurate): EMA (Exponential Moving Average) fit based on historical 429 data.
      2. Level 2 (Conservative): 70% of the historical maximum RPM.
      3. Level 3 (Fallback): Fully open (100% capacity) used during cold starts or data anomalies.

    Data Sources and Storage

    • Historical 429 Metrics: Stored in Redis at bella-openapi-channel-metrics:{channelCode}:rpm_429_history. Format: {timestamp, rpm, avg_response_time}.
    • Real-time Completed RPM: bella-openapi-channel-metrics:{channelCode}:total.completed.
    • Current Concurrency: bella-openapi-channel-concurrent:{entityCode}.
    • Concurrency Tracking: Managed via a modified concurrent.lua script tracking both API Key (bella-openapi-limiter-concurrent:{entityCode}:{akCode}) and Channel (bella-openapi-channel-concurrent:{entityCode}) dimensions.
  9. Override speech_rate using extra_body

    develop

    If you need to provide a specific speech_rate that differs from the calculated value of the speed parameter, you can use the extra_body field. Parameters provided within extra_body.audio_params have higher priority and will override the value calculated from the top-level speed parameter.

    To ensure the override works correctly, pass the speech_rate inside the audio_params object within extra_body.

    {
      "model": "Doubao_Seed_TTS_2",
      "input": "test text",
      "voice": "xxx",
      "speed": 1.5,
      "extra_body": {
        "audio_params": {
          "speech_rate": 80
        }
      }
    }
  10. Authentication and Security in Bella OpenAPI

    develop

    Bella OpenAPI provides a unified login service with multiple authentication methods to secure API access:

    • OAuth 2.0
    • CAS Single Sign-On (SSO)
    • API Key Authentication

    Security features include fine-grained user permission control (Role-based access control) and complete audit logs for tracking and troubleshooting.

  11. Requirements and Constraints for RocksDB Log Storage

    develop

    To use or implement the RocksDB full log storage, the following technical constraints must be met:

    • Runtime: Must be compatible with Java 8 (use rocksdbjni 7.x series; 8.x requires Java 11+).
    • Storage: The PVC must be block storage (e.g., Ceph RBD, EBS, or local SSD). NFS is not supported.
    • Performance: Reading logs via shardPath is designed for < 50ms latency using bloom filters, ReadOnly mode, and Caffeine caching.
    • Data Lifecycle: Logs are subject to an automatic 7-day cleanup policy.
    • Operational Behavior: The feature can be toggled via configuration. When disabled, the system behaves as it did previously (truncated logs only).
  12. Understand OpenAPI error source and upstream HTTP codes

    develop

    The OpenAPI error response includes two new metadata fields to help callers distinguish between errors originating from the gateway itself versus errors from downstream channels:

    • source: A String indicating the origin of the error. Possible values are:
      • gateway: Errors originating from the Bella OpenAPI gateway (e.g., rate limiting, authorization, safety checks, or routing failures).
      • channel: Errors originating from the selected downstream channel.
      • network: Network-related errors.
      • timeout: Timeout-related errors.
    • upstreamHttpCode: An Integer representing the original HTTP status code returned by the downstream channel. This field is omitted (null) if the error is a gateway error or if no upstream code is available.

    Note: These fields are only present in error responses. Successful responses will not include these fields in the JSON output.