DOCSEARCHbyAlgolia
Submit Docs

Gemini Balance

repository·main·Indexed 26 days ago

https://github.com/snailyp/gemini-balance

A Python FastAPI-based proxy and load balancer for the Google Gemini API. It features key rotation for multiple API keys, OpenAI API compatibility for chat and embeddings, and support for image generation, web search, and native Gemini Text-to-Speech (TTS) with single and multi-speaker configurations.

Tokens
4.9K
Snippets
11
Records
22
Agent score
91%

What's inside gemini-balance

  1. Overview of Gemini Balance features

    main

    Gemini Balance is a FastAPI-based proxy and load balancer for Google Gemini API keys.

    Key capabilities include:

    • Multi-Key Load Balancing: Automatically rotates through configured API_KEYS to improve availability and concurrency.
    • Dual Protocol Support: Proxies both Gemini and OpenAI API formats.
    • Specialized Model Features: Supports image generation/editing (via IMAGE_MODELS) and web search (via SEARCH_MODELS) by appending -image or -search to the configured model names.
    • Resilience: Implements automatic retries (MAX_RETRIES) and automatically disables keys that exceed a failure threshold (MAX_FAILURES).
    • Observability: Provides a /keys_status dashboard for monitoring key health and detailed error logging.

  2. Overview of Gemini Balance

    main
    Gemini Balance is a proxy and load balancer for the Google Gemini API built with Python FastAPI. It enables managing multiple Gemini API keys through sequential polling (key rotation), authentication, model filtering, and real-time status monitoring. It also provides compatibility with the OpenAI API format for both chat and embeddings, and supports image generation and web search features.

  3. Run Gemini Balance locally for development

    main

    To run the application directly on your host machine for development purposes:

    1. Clone the repository and install dependencies:
      git clone https://github.com/snailyp/gemini-balance.git
cd gemini-balance
pip install -r requirements.txt
    2. Create a `.env` file from `.env.example`.
3. Start the application using Uvicorn:

```bash
uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload

  4. Deploy Gemini Balance using Docker CLI

    main

    You can run Gemini Balance as a standalone container using Docker.

    1. Pull the latest image:
      docker pull ghcr.io/snailyp/gemini-balance:latest
    2. Prepare your `.env` file from `.env.example`.
3. Run the container with the following command to map ports, persist data, and load environment variables:

```bash
docker run -d -p 8000:8000 --name gemini-balance \
-v ./data:/app/data \
--env-file .env \
ghcr.io/snailyp/gemini-balance:latest

  5. Deploy Gemini Balance using Docker Compose

    main

    The recommended way to deploy Gemini Balance is using Docker Compose, which starts both the application and a MySQL database.

    1. Obtain the docker-compose.yml file from the repository.
    2. Create a .env file by copying .env.example and renaming it to .env.
    3. In the .env file, ensure DATABASE_TYPE is set to mysql and provide the necessary MYSQL_* configuration.
    4. Run the following command to start the services in the background:
    docker-compose up -d

  6. Use OpenAI-Compatible TTS

    main

    For OpenAI-compatible TTS requests, use the /v1/audio/speech endpoint. This service is separate from the native Gemini TTS enhancement and follows standard OpenAI patterns: it uses the Authorization: Bearer header and returns an audio file directly instead of a JSON response.

    curl -X POST "https://your-domain.com/v1/audio/speech" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer your-token" \
  -d '{
    "model": "tts-1",
    "input": "这是一个OpenAI兼容格式的TTS测试。",
    "voice": "alloy"
  }' \
  --output openai_tts_test.wav

  7. Use Native Gemini TTS (Single and Multi-speaker)

    main

    The project supports native Gemini Text-to-Speech (TTS) via the /v1beta/models/{model}:generateContent endpoint. This feature is automatically enabled when a request includes responseModalities: ["AUDIO"] and a speechConfig. It supports both single-speaker and multi-speaker configurations without requiring any manual setup or environment variables.

    ### 1. Native Gemini Single-Speaker TTS

```bash
curl -X POST "https://your-domain.com/v1beta/models/gemini-2.5-flash-preview-tts:generateContent" \
  -H "Content-Type: application/json" \
  -H "x-goog-api-key: your-token" \
  -d '{
    "contents": [{
      "parts": [{
        "text": "Hello, this is a single speaker test."
      }]
    }],
    "generationConfig": {
      "responseModalities": ["AUDIO"],
      "speechConfig": {
        "voiceConfig": {
          "prebuiltVoiceConfig": {
            "voiceName": "Kore"
          }
        }
      }
    }
  }'

    2. Native Gemini Multi-Speaker TTS

    curl -X POST "https://your-domain.com/v1beta/models/gemini-2.5-flash-preview-tts:generateContent" \
  -H "Content-Type: application/json" \
  -H "x-goog-api-key: your-token" \
  -d '{
    "contents": [{
      "parts": [{
        "text": "Alice: Hello everyone, welcome to our show today.\nBob: Hi Alice, and hello to all our listeners! Today we are talking about AI development."
      }]
    }],
    "generationConfig": {
      "responseModalities": ["AUDIO"],
      "speechConfig": {
        "multiSpeakerVoiceConfig": {
          "speakerVoiceConfigs": [
            {
              "speaker": "Alice",
              "voiceConfig": {
                "prebuiltVoiceConfig": {
                  "voiceName": "Puck"
                }
              }
            },
            {
              "speaker": "Bob",
              "voiceConfig": {
                "prebuiltVoiceConfig": {
                  "voiceName": "Kore"
                }
              }
            }
          ]
        }
      }
    }
  }'

  8. Install Gemini Balance using Docker Command

    main

    You can run Gemini Balance as a standalone container using the following steps:

    1. Pull the latest image:
    docker pull ghcr.io/snailyp/gemini-balance:latest
    1. Prepare your .env file from .env.example.
    2. Run the container with port mapping and volume persistence:
    docker run -d -p 8000:8000 --name gemini-balance \
-v ./data:/app/data \
--env-file .env \
ghcr.io/snailyp/gemini-balance:latest

  9. Install Gemini Balance for Local Development

    main

    To run the application locally for development:

    1. Clone the repository and install dependencies:
    git clone https://github.com/snailyp/gemini-balance.git
cd gemini-balance
pip install -r requirements.txt
    1. Configure your environment by copying .env.example to .env.
    2. Start the Uvicorn server:
    ```bash
uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload

    Access the application at http://localhost:8000.

  10. Install Gemini Balance using Docker Compose

    main

    The recommended way to deploy Gemini Balance is using Docker Compose.

    1. Download the docker-compose.yml file from the repository.
    2. Create a .env file by copying .env.example and configure your settings. Note that DATABASE_TYPE must be set to mysql and you must provide valid MYSQL_* credentials.
    3. Run the following command in the directory containing the files:
    docker-compose up -d

  11. Configure Stream and Fake Stream features

    main

    Control the behavior of streaming outputs and fake streaming (for compatibility) using these variables:

    Stream Optimizer

    | Variable | Description | Default | | --- | --- | STREAM_OPTIMIZER_ENABLED | Enable stream output optimization | false | | STREAM_MIN_DELAY | Minimum stream output delay | 0.016 | | STREAM_MAX_DELAY | Maximum stream output delay | 0.024 | | STREAM_SHORT_TEXT_THRESHOLD| Short text threshold | 10 | | STREAM_LONG_TEXT_THRESHOLD | Long text threshold | 50 | | STREAM_CHUNK_SIZE | Stream output chunk size | 5 |

    Fake Stream

    | Variable | Description | Default | | --- | --- | FAKE_STREAM_ENABLED | Enable fake streaming | false | | FAKE_STREAM_EMPTY_DATA_INTERVAL_SECONDS | Heartbeat interval for fake streaming (seconds) | 5 |

  12. Configure Gemini Balance environment variables

    main

    Gemini Balance is configured via environment variables. Below are the key configuration groups.

    Required API Configuration

    VariableDescription
    API_KEYSRequired, list of Gemini API keys
    ALLOWED_TOKENSRequired, list of access tokens

    Database Configuration

    | Variable | Description | Default | | --- | --- | DATABASE_TYPE | mysql or sqlite | mysql | | SQLITE_DATABASE | Path for SQLite database file | default_db | | MYSQL_HOST | MySQL host address | localhost | | MYSQL_PORT | MySQL port | 3306 | | MYSQL_USER | MySQL username | your_db_user | | MYSQL_PASSWORD | MySQL password | your_db_password | | MYSQL_DATABASE | MySQL database name | defaultdb |

    Advanced Model & Feature Configuration

    | Variable | Description | Default | | --- | --- | TEST_MODEL | Model for testing key validity | gemini-2.5-flash-lite | | IMAGE_MODELS | Models supporting image generation | ["gemini-2.0-flash-exp", "gemini-2.5-flash-image-preview"] | | SEARCH_MODELS | Models supporting web search | ["gemini-2.5-flash","gemini-2.5-pro"] | | TOOLS_CODE_EXECUTION_ENABLED | Enable code execution tool | false | | SHOW_SEARCH_LINK | Display search result links in response | true | | SHOW_THINKING_PROCESS | Display model's thinking process | true | | THINKING_MODELS | Models supporting thinking process | [] | | THINKING_BUDGET_MAP | Budget map for thinking function (model:budget) | {} | | URL_NORMALIZATION_ENABLED | Enable smart URL routing | false | | URL_CONTEXT_ENABLED | Enable URL context understanding | false | | URL_CONTEXT_MODELS | Models supporting URL context | [] |

    Image Generation & Upload Providers

    | Variable | Description | Default | | --- | --- | CREATE_IMAGE_MODEL | Image generation model | imagen-3.0-generate-002 | | UPLOAD_PROVIDER | Image upload provider: smms, picgo, cloudflare_imgbed, aliyun_oss | smms | | OSS_ACCESS_KEY | Aliyun AccessKey ID | LTAI5txxxxxxxxxxxxxxxx | | OSS_ACCESS_KEY_SECRET | Aliyun AccessKey Secret | yXxxxxxxxxxxxxxxxxxxxxx | | OSS_BUCKET_NAME | Aliyun OSS bucket name | your-bucket-name | | SMMS_SECRET_TOKEN | SM.MS API Token | your-smms-token | | PICGO_API_KEY | PicoGo API Key | your-picogo-apikey | | CLOUDFLARE_IMGBED_AUTH_CODE| CloudFlare ImgBed auth key | your-cloudflare-imgber-auth-code |