GPT-Load

GPT-Load is a high-performance, enterprise-grade AI API transparent proxy service built with Go. It is designed for developers and enterprises needing to integrate multiple AI services. It features intelligent key management (key pools with auto-rotation and failover), load balancing across upstream endpoints, and comprehensive monitoring.

Key capabilities include:

• Transparent Proxying: Maintains native API formats for OpenAI, Google Gemini, and Anthropic Claude.
• Intelligent Key Management: Group-based management with automatic rotation and blacklisting of failed keys.
• Dynamic Configuration: Supports hot-reloading of system and group settings without restarts.
• High Performance: Utilizes zero-copy streaming and connection pool reuse.
• Dual Authentication: Separates management access from proxy access, supporting both global and group-level keys.

GPT-Load uses a dual-layer configuration system:

1. Static Configuration (Environment Variables): Used for infrastructure settings like database connections, server ports, and authentication keys. These are read at startup and require an application restart to change.
2. Dynamic Configuration (Hot-Reload): Used for application behavior. These settings are stored in the database and support hot-reloading, meaning changes take effect immediately without a restart.

Configuration Priority: Group Configuration > System Settings > Environment Configuration.

GPT-Load uses a two-tier configuration architecture to manage settings:

1. Static Configuration (Environment Variables): Loaded at application startup. These are immutable during runtime and require an application restart to take effect. Use these for infrastructure settings like database connections, server ports, and authentication keys.
2. Dynamic Configuration (Hot Reload): Stored in the database and supports hot reloading. Changes take effect immediately without a restart.
   • System Settings: Provide a unified standard for the entire application.
   • Group Settings: Allow customization for specific groups and can override System Settings.

Configuration Precedence: Group Settings > System Settings > Environment Variables.

GPT-Load supports encrypted storage for API keys. You can enable encryption, disable it, or rotate keys using the migrate-keys command.

Important Safety Rules:

• Backup your database before any migration.
• Stop the service before performing migrations to avoid data inconsistency.
• Never lose your ENCRYPTION_KEY; if lost, encrypted data cannot be recovered.
• Use a random string of at least 32 characters for security.

Scenarios:

• Enable encryption: Use --to <new_key>
• Disable encryption: Use --from <current_key>
• Rotate keys: Use --from <current_key> --to <new_key>

For a fast, single-container deployment, use the following Docker command. Ensure you replace your-secure-key-here with a strong password to secure your management interface.

docker run -d --name gpt-load \
    -p 3001:3001 \
    -e AUTH_KEY=your-secure-key-here \
    -v "$(pwd)/data":/app/data \
    ghcr.io/tbphp/gpt-load:latest

The management console is used to manage AI service provider groups, API keys, request logs, and system settings. By default, it is accessible at http://localhost:3001.

Features:

• Dashboard: Real-time statistics and system status.
• Key Management: Create/configure provider groups and monitor API keys.
• Request Logs: Detailed request history and debugging.
• System Settings: Global configuration and hot-reload.

For high availability, use a leader-follower architecture.

Requirements:

• All nodes must connect to the same MySQL (or PostgreSQL) and Redis instances.
• Redis is mandatory for cluster mode.
• All nodes must have identical AUTH_KEY, DATABASE_DSN, and REDIS_DSN configurations.
• Follower nodes must set the environment variable IS_SLAVE=true.

To build from source, you need Go 1.24+ and a local database (SQLite, MySQL, or PostgreSQL). Redis is optional for caching and distributed coordination.

1. Clone the repository and tidy modules.
2. Create a .env file from the example.
3. Configure AUTH_KEY, DATABASE_DSN, and optionally REDIS_DSN in .env.
4. Run using make run.

# Clone and build
git clone https://github.com/tbphp/gpt-load.git
cd gpt-load
go mod tidy

# Setup configuration
cp .env.example .env

# [Edit .env: AUTH_KEY, DATABASE_DSN, REDIS_DSN]

# Run
make run

GPT-Load routes requests to different AI services via group names. To use the proxy, replace the original service base URL with the GPT-Load proxy endpoint and use the Proxy Key configured in the management interface.

Proxy Endpoint Format: http://localhost:3001/proxy/{group_name}/{original_api_path}

• {group_name}: The name of the group created in the management interface.
• {original_api_path}: The original API path (must remain identical).

Authentication:

• Use the Proxy Key (set in the Web UI) instead of the original provider's API key.
• Global Proxy Key: Set in system settings, works for all groups.
• Group Proxy Key: Set per group, only works for that specific group.
• Multiple keys can be provided, separated by commas.

To run GPT-Load quickly using a single Docker container, use the following command. Ensure you replace your-secure-key-here with a strong password to secure your management interface.

After running, access the management interface at http://localhost:3001.

docker run -d --name gpt-load \
    -p 3001:3001 \
    -e AUTH_KEY=your-secure-key-here \
    -v "$(pwd)/data":/app/data \
    ghcr.io/tbphp/gpt-load:latest