PrivateGPT Documentation

PrivateGPT implements the full Claude API, making it compatible with Claude-native tools and any client that requires a Claude-compatible interface.

Claude-Native Integrations

PrivateGPT can power first-party Anthropic applications that typically require specific API capabilities (like structured output, tool use, and tokenizer endpoints) which standard local inference providers (e.g., Ollama) often lack. Documented integrations include:

• Claude Code: Anthropic's official CLI. You can route requests through your PrivateGPT server instead of api.anthropic.com.
• Claude Desktop: Connect Claude Desktop to your PrivateGPT server using third-party inference.
• Claude for Microsoft 365: Connect Claude add-ins for Word, Excel, PowerPoint, and Outlook to your PrivateGPT server.
• OpenCode: An open-source AI coding agent that can be pointed at a PrivateGPT server via configuration.

Other Compatible Tools

Because PrivateGPT exposes a standard Claude-compatible interface, it is also compatible with tools designed for local OpenAI-compatible providers, such as:

• OpenClaw
• Hermes Agent
• n8n
• VS Code
• Cline

PrivateGPT is an open-source, Claude API-compatible orchestration layer designed to turn local or self-hosted model servers into production-ready AI application backends. It does not run models itself; instead, it connects to an external OpenAI-compatible inference server (e.g., Ollama, vLLM, llama.cpp) via the OPENAI_API_BASE environment variable.

Key Capabilities

• Messaging: Standard messages API with streaming, batch/async processing, and token counting.
• Knowledge & RAG: File/artifact ingestion (including PDFs), retrieval with citations, and embeddings.
• Tool Use: Built-in tools (web search, extraction, database querying, CSV/tabular analysis), custom tools support, and MCP (Model Context Protocol) connectors.
• Agentic Features: Code execution, skills, and extended reasoning support.

Architecture Model

Your app / agent / workflow / UI
              |
        PrivateGPT API
              |
    Self-hosted LLM Server

To choose the right tool for your stack, understand how PrivateGPT differs from inference providers and application interfaces:

PrivateGPT vs. Inference Providers (Ollama, LM Studio, vLLM, llama.cpp)

• Inference Providers answer how to run a model.
• PrivateGPT answers how to build a useful AI application on top of that model.
• Usage: Run your model with an inference server, then use PrivateGPT as the Claude-style backend for your application.

PrivateGPT vs. AI Applications (Onyx, Open WebUI)

• Onyx/Open WebUI are self-hosted, app-first experiences focused on chat and enterprise search.
• PrivateGPT is an API-first layer designed for developers to build their own applications.

PrivateGPT vs. Zylon

• PrivateGPT is the open-source application API layer.
• Zylon is a complete enterprise production platform that includes PrivateGPT plus integrated inference, Kubernetes deployment, API gateways, and enterprise connectors (SharePoint, LDAP, etc.).

Choose a provider based on your requirements for throughput, features, and local vs. cloud deployment:

† Model-dependent: The server must support the protocol and the loaded model must support the capability.

Key Considerations

• Structured Output: Only vLLM supports the structured output (JSON schema enforcement) endpoint. Other providers use prompt-based JSON extraction, which is less reliable for complex schemas.
• Missing Tokenizer Endpoint: If your provider (like Ollama) does not expose a /tokenize endpoint, PrivateGPT uses a character-based estimate (4 chars = 1 token). This can lead to inaccurate context management. To mitigate this, explicitly set a conservative context_window in your model profile.

The PrivateGPT Workbench UI is a single-file runtime implementation located at ../index.html. It follows the Fern-generated OpenAPI schema found at ../../fern/openapi/openapi.json.

Key UI architectural details:

• Global Collection: The active document collection is managed in Settings via state.context.documents.defaultCollection and is used for all document and chat operations.
• Onboarding: The first-run experience is controlled by state.onboarding. The overlay is displayed as long as state.onboarding.completed !== true.
• Appearance: UI themes and visibility are driven by state.uiAppearance and applied via applyAppearance(). Theme generation is performed by sending a brief to the chat API (POST /v1/messages), which returns JSON to update the appearance fields.
• Navigation: The UI uses hash-based navigation (syncHash() / restoreFromHash()) to manage views. Supported formats include #context/{tab}, #chat/{id}, #settings, and #apiDebugger.