Cloudpods Documentation

Cloudpods is an open-source, cloud-native multi-cloud/hybrid-cloud management platform implemented in Golang. It acts as a "cloud on top of clouds," providing a unified API to manage heterogeneous infrastructure resources. It abstracts the differences in data models and APIs across various cloud providers, allowing users to manage local virtual machines, physical machines, and multiple public/private cloud accounts through a single interface.

Key capabilities include:

• Multi-Cloud Management: Unified access to public clouds (AWS, Azure, GCP, etc.) and private clouds (OpenStack, VMware, etc.).
• AI Cloud: A dedicated platform for managing LLM inference services and AI container applications, featuring GPU scheduling and model management.
• Resource Abstraction: Standardized management of servers, load balancers, object storage, NAS, RDS, and VPCs.

Use approve.sh to merge a specific Pull Request after performing automated safety checks.

Safety Checks performed:

1. Verifies the PR status is open.
2. Verifies the PR mergeable status is true (ensures no merge conflicts).
3. Verifies all CI checks have passed.
4. If the check_reviewers argument is provided, verifies that all requested reviewers have provided an lgtm status.

Warning: This script is a convenience tool for merging and does not replace manual code review. Ensure you have manually reviewed the code before execution.

./scripts/approve.sh <PRN> [check_reviewers]

Deleting an llm_deployment automatically triggers the cleanup of associated aiproxy resources, including ai_provider, ai_model, and ai_routing.

• Providers are identified via llm_deployment_id or llm_id.
• Routing is identified via llm_deployment.aiproxy_routing_id.

This cleanup occurs regardless of whether auto_register_aiproxy is set to true.

To set up a custom gateway as a provider, you must define the base_url, a top-level secret, and the api_mode (openai or anthropic). Note that creating a provider does not automatically inject catalog models; you must manually create ai_model resources and configure routing.

// OpenAI mode example
{
  "generate_name": "my-gateway",
  "provider_key": "custom",
  "secret": "sk-xxx",
  "config": {
    "base_url": "https://llm.example.com/v1",
    "api_mode": "openai"
  }
}

// Anthropic mode example
{
  "generate_name": "my-anthropic-gateway",
  "provider_key": "custom",
  "secret": "sk-ant-xxx",
  "config": {
    "base_url": "https://llm.example.com/anthropic",
    "api_mode": "anthropic"
  }
}

To install Cloudpods, you must select the appropriate installation scenario based on your specific use case. Please refer to the official documentation for step-by-step instructions.

Official Getting Started Guide: https://www.cloudpods.org/en/docs/getting-started/

When using the Anthropic SDK or Claude Code with aiproxy, you must use the aiproxy virtual_key for authentication, NOT the upstream provider's API key.

Crucial Configuration Rules:

1. Base URL: Set base_url to ${AIPROXY_URL}/ai/anthropic. Do not append /v1 to the URL, as the SDK will append it automatically (resulting in /v1/v1/messages if you do).
2. API Key: Use the virtual_key generated by aiproxy.

Python Example

import anthropic
client = anthropic.Anthropic(
    base_url=f"{AIPROXY_URL}/ai/anthropic",  # Correct: SDK appends /v1/messages
    api_key=VIRTUAL_KEY,                     # aiproxy virtual_key
)
client.messages.create(model="claude-sonnet-4-5", max_tokens=128, messages=[...])

Environment Variables

export ANTHROPIC_BASE_URL="${AIPROXY_URL}/ai/anthropic"
export ANTHROPIC_API_KEY="${VIRTUAL_KEY}"

Use climc ai-codex-config to generate configuration files required for the Codex CLI to work with the Cloudpods AI Gateway. This command generates config.toml, models_catalog.json, and aiproxy.env based on an existing ai_virtual_key.

Key Details:

• Base URL: Points to {gateway_root_url}/ai/openai/v1. If --routing is specified, it uses the access_address of the bound ai_proxy_node.
• Authentication: Codex uses the OPENAI_API_KEY environment variable to carry the virtual_key. You must source aiproxy.env before starting Codex.
• Model Catalog: When --routing is used, models_catalog.json contains the intersection of models available to the virtual key and models defined in the route's catalog.

Use approve_all.sh to merge a set of related Pull Requests, typically consisting of a main PR on the master branch and its corresponding cherry-pick PRs backported to other branches.

Check Logic:

• For the Master PR: Performs all 4 checks (Open status, Mergeable status, CI pass, and Reviewer check if requested).
• For Backport PRs: Performs only the first 3 checks (Open status, Mergeable status, and CI pass).

./scripts/approve_all.sh <PRN>