qwen2api

qwen2API is a self-hosted gateway that converts Qwen Web capabilities into standard API protocols. It provides a local WebUI for managing upstream accounts, downstream API keys, runtime configurations, and testing models (chat, image, and video).

Supported API Protocols

• OpenAI Compatible: /v1/chat/completions, /v1/responses, /v1/models, /v1/files, /v1/images/generations, /v1/videos/generations
• Anthropic Compatible: /v1/messages, /anthropic/v1/messages, /v1/messages/count_tokens
• Gemini Compatible: /v1beta/models/{model}:generateContent, /v1beta/models/{model}:streamGenerateContent

Core Features

• Account Pool: Supports multi-account polling, single-account concurrency control, and cooldown records for chat/image/video tasks.
• WebUI: Interface for account management, API key management, and various testing modes.
• Maintenance: Includes health checks (/healthz), readiness checks (/readyz), and keepalive probes (/keepalive).

The recommended way to deploy is using Docker Hub images. This method ensures that your accounts, keys, and logs persist across updates by mounting host directories to the container.

1. Prepare the directory structure:

mkdir qwen2api
cd qwen2api
mkdir -p data logs

2. Create a .env file with your configuration:

HOST_PORT=7860
HOST_DATA_DIR=./data
HOST_LOGS_DIR=./logs
ADMIN_KEY=replace-with-your-own-strong-random-key

3. Create a docker-compose.yml file:

services:
  qwen2api:
    image: ${QWEN2API_IMAGE:-yujunzhixue/qwen2api:latest}
    container_name: qwen2api
    restart: unless-stopped
    init: true
    env_file:
      - .env
    ports:
      - "${HOST_PORT:-7860}:${PORT:-7860}"
    volumes:
      - ${HOST_DATA_DIR:-./data}:/app/data
      - ${HOST_LOGS_DIR:-./logs}:/app/logs
    shm_size: "512m"
    healthcheck:
      test: ["CMD-SHELL", "curl -fsS http://127.0.0.1:${PORT:-7860}/healthz || exit 1"]
      interval: 30s
      timeout: 10s
      start_period: 120s
      retries: 3

4. Pull and start the service:

docker compose pull
docker compose up -d
docker compose logs -f qwen2api

Access URLs:

• WebUI: http://127.0.0.1:7860
• Health Check: http://127.0.0.1:7860/healthz
• Keepalive Probe: http://127.0.0.1:7860/keepalive

The following environment variables can be used to configure the runtime behavior. Note that some variables are for runtime-only injection and are not persisted to JSON stores.

To prevent prompt length overflow, the system uses a planContextOffload strategy. This determines how to handle long conversation histories based on the ContextInlineMaxChars and ContextForceFileMaxChars settings.

Offloading Modes:

1. inline: Used when the estimated prompt length is within limits. The entire message history is sent directly in the payload.
2. hybrid: Used when the history is long but manageable. The system summarizes older messages and attaches them as a generated context file (e.g., qwen2api_context_history.txt), while keeping the most recent user message inline.
3. file: Used when the estimated prompt length exceeds ContextForceFileMaxChars. The history is heavily offloaded into generated files to minimize the immediate prompt size.

Generated Files: When offloading, the system creates LocalContextFile objects which include the text content and a SHA256 hash for integrity.

The App struct provides several recovery methods to handle common failures in the tool-calling lifecycle. These methods use a retry mechanism by injecting specialized 'guard' prompts into a new chat session to guide the model toward correct behavior.

Recovery Strategies

• recoverBlockedToolNameOutput: Used when the model's output contains a tool name that is blocked by the client-side safety filters. It attempts to retry the request with a prompt explaining that the tool is a client-side action and is available.
• recoverUnparsedToolMarkup: Handles cases where tool markup (QNML/XML/JSON) is malformed, truncated, or unclosed. It uses continuation prompts to help the model complete the tool call.
• recoverMissingToolContinuation: Triggered when a model provides a tool result but fails to follow up with the next required tool call. It uses an ordered workflow hint to guide the model to the next planned step.
• recoverMissingInitialToolCall: Used when a task clearly requires a tool, but the model's first response contains only prose. It prompts the model to review the goal and emit a tool call.
• recoverInvalidToolCallArgs: Handles cases where the model emits a tool call with incorrect or incomplete parameters (e.g., a patch tool without a diff). It provides specific guidance on required fields for the attempted tool.
• recoverRepeatedToolCall: Detects and breaks infinite loops where the model repeatedly calls the same tool with the same arguments. It prompts the model to try a different approach.

The AccountPool manages a collection of Account objects used to interact with upstream services. It handles concurrency, rate limiting, and account health.

Key Operations

• Load(): Loads accounts from the persistent JSONStore and merges them with accounts provided via environment variables.
• Acquire(ctx, preferredEmail): Retrieves an available account. It will wait up to 60 seconds for an account to become available if the pool is currently at capacity.
• Release(acc): Returns an account to the pool after use, decrementing the Inflight count.
• MarkSuccess(acc): Marks an account as valid and clears its rate limits.
• MarkInvalid(acc, status, message): Marks an account as invalid with a specific error status.
• MarkRateLimited(acc, cooldown, message): Sets a cooldown period for an account when a rate limit is encountered.
• Status(): Returns a summary of the pool, including total, valid, and available accounts for different usage types (chat, image, video).

// Example: Acquiring and releasing an account
acc, err := pool.Acquire(ctx, "")
if err != nil {
    return err
}
defer pool.Release(acc)

// ... use account ...

pool.MarkSuccess(acc)