DOCSEARCHbyAlgolia
Submit Docs

huggingface/chat-ui

repository·main·Indexed 27 days ago

https://github.com/huggingface/chat-ui

A SvelteKit-based chat interface for LLMs that powers the official HuggingChat app. It supports any OpenAI-compatible API provider, including Hugging Face Inference Providers, llama.cpp, Ollama, and OpenRouter. Features include an LLM Router (Omni) for smart model selection based on input signals, Model Context Protocol (MCP) server integration for tool calling, and customizable app theming. Version 0.20.0.

Tokens
11.3K
Snippets
45
Records
76
Agent score
89%

What's inside chat-ui

  1. Understand Chat UI architectural components

    main

    Chat UI is built with SvelteKit, Tailwind, and MongoDB. Its architecture is centered around several key modules:

    • routes: Contains the majority of backend and frontend logic. Routes are rendered with SSR via SvelteKit. Shared modules are located in lib (client) and lib/server (server).
    • textGeneration: The standard interface for chat features. It handles model output, tool calls, and streaming, emitting MessageUpdates for status updates like new tokens or tool results.
    • endpoints: Provides the streaming interface for OpenAI-compatible endpoints. It fetches and caches models from ${OPENAI_BASE_URL}/models.
    • mcp: Implements the Model Context Protocol (MCP) client functionality for tool discovery and execution.
    • llmRouter: Contains routing logic to select models based on request heuristics (e.g., routing to a multimodal model if image input is present, or an agentic route if MCP tools are active).
    • migrations: Manages MongoDB migrations to ensure backwards compatibility during schema changes.

  2. Quickstart Chat UI with OpenAI-compatible APIs

    main

    Chat UI requires an OpenAI-compatible API. The fastest way to start is using the Hugging Face Inference Providers router.

    1. Create a .env.local file in the root directory.
    2. Configure OPENAI_BASE_URL and OPENAI_API_KEY based on your provider.
    3. Install dependencies and launch the development server.

    Common provider configurations:

    • Hugging Face Inference Providers: OPENAI_BASE_URL=https://router.huggingface.co/v1 and OPENAI_API_KEY=hf_xxx
    • llama.cpp server: OPENAI_BASE_URL=http://127.0.0.1:8080/v1 and OPENAI_API_KEY=any-string
    • Ollama: OPENAI_BASE_URL=http://127.0.0.1:11434/v1 and OPENAI_API_KEY=ollama
    • OpenRouter: OPENAI_BASE_URL=https://openrouter.ai/api/v1 and OPENAI_API_KEY=sk-or-v1-...
    OPENAI_BASE_URL=https://router.huggingface.co/v1
OPENAI_API_KEY=hf_************************
    git clone https://github.com/huggingface/chat-ui
cd chat-ui
npm install
npm run dev -- --open

  3. Configure OpenID Connect authentication

    main

    To authenticate users using OpenID Connect instead of browser sessions, configure the following environment variables. Ensure your provider's redirect URI is set to https://your-domain.com/login/callback.

    OPENID_CLIENT_ID=your_client_id
OPENID_CLIENT_SECRET=your_client_secret
OPENID_SCOPES="openid profile"
OPENID_PROVIDER_URL=https://your-provider.com

  5. Configure MongoDB for Chat UI

    main

    Chat UI requires a MongoDB instance for data persistence.

    Development Mode

    In development, MongoDB is optional. If MONGODB_URL is not set, Chat UI uses an embedded MongoDB server that persists data to the ./db folder.

    Production Mode

    For production, you must provide a MONGODB_URL pointing to a dedicated MongoDB instance.

    Option 1: Local MongoDB via Docker

    Run a local MongoDB container using the following command:

    docker run -d -p 27017:27017 -v mongo-chat-ui:/data --name mongo-chat-ui mongo:latest

    Then, set the following in your environment: MONGODB_URL=mongodb://localhost:27017

    Option 2: MongoDB Atlas (Managed)

    Use a connection string from a managed service like MongoDB Atlas and set it as MONGODB_URL.

  6. Launch Chat UI in Development and Production

    main

    Use the following commands to manage the application lifecycle:

    • Development: Run npm run dev. The server listens on http://localhost:5173 by default.
    • Production Build: Run npm run build to create a production version.
    • Production Preview: Run npm run preview to test the production build locally.
    npm install
npm run dev

  7. Run Chat UI with an external MongoDB

    main

    If you prefer to use an existing MongoDB instance (such as a local installation or MongoDB Atlas), use the ghcr.io/huggingface/chat-ui image. You must provide the MONGODB_URL environment variable. To connect to a MongoDB instance running on your host machine from within the container, you can use mongodb://host.docker.internal:27017.

    docker run -p 3000:3000 \
  -e OPENAI_BASE_URL=https://router.huggingface.co/v1 \
  -e OPENAI_API_KEY=hf_*** \
  -e MONGODB_URL=mongodb://host.docker.internal:27017 \
  ghcr.io/huggingface/chat-ui

  8. Use Multimodal and Tools shortcuts

    main

    You can bypass the JSON policy file by using shortcut environment variables for image inputs and MCP tools. When enabled, these shortcuts route directly to a specific model regardless of the policy definitions.

    Multimodal Shortcut:

    • Set LLM_ROUTER_ENABLE_MULTIMODAL=true to bypass the policy and use LLM_ROUTER_MULTIMODAL_MODEL when an image is attached.

    Tools Shortcut:

    • Set LLM_ROUTER_ENABLE_TOOLS=true to bypass the policy and use LLM_ROUTER_TOOLS_MODEL when at least one MCP server is enabled.
    # Multimodal shortcut
LLM_ROUTER_ENABLE_MULTIMODAL=true
LLM_ROUTER_MULTIMODAL_MODEL=moonshotai/Kimi-K2.6

# Tools shortcut
LLM_ROUTER_ENABLE_TOOLS=true
LLM_ROUTER_TOOLS_MODEL=moonshotai/Kimi-K2.6

  9. Configure OpenAI-compatible API endpoints

    main

    Set the OPENAI_BASE_URL and OPENAI_API_KEY environment variables in a .env.local file to connect to different providers.

    Common configurations include:

    ProviderOPENAI_BASE_URLOPENAI_API_KEY
    Hugging Facehttps://router.huggingface.co/v1hf_xxx
    Ollamahttp://127.0.0.1:11434/v1ollama
    llama.cpphttp://127.0.0.1:8080/v1sk-local
    OpenRouterhttps://openrouter.ai/api/v1sk-or-v1-xxx
    OPENAI_BASE_URL=https://router.huggingface.co/v1
OPENAI_API_KEY=hf_************************

  10. Add User MCP Servers via UI

    main

    Users can add their own custom MCP servers directly through the Chat UI. These servers are stored in the browser and work alongside the administrator-configured base servers.

    To add a server:

    1. Open the chat input and click the + button (or navigate to Settings).
    2. Select MCP Servers.
    3. Click Add Server.
    4. Enter the server name and URL.
    5. Run the Health Check to verify the connection.

  11. Quick Start Chat UI with bundled MongoDB

    main

    To run Chat UI quickly with a built-in MongoDB instance, use the ghcr.io/huggingface/chat-ui-db image. This image includes MongoDB and is recommended for rapid setup. You must provide an OPENAI_API_KEY and an OPENAI_BASE_URL. Use a Docker volume (e.g., chat-ui-data:/data) to persist data.

    docker run -p 3000:3000 \
  -e OPENAI_BASE_URL=https://router.huggingface.co/v1 \
  -e OPENAI_API_KEY=hf_*** \
  -v chat-ui-data:/data \
  ghcr.io/huggingface/chat-ui-db

  12. Configure OpenAI-compatible API providers

    main

    Chat UI connects to any OpenAI-compatible API. You can switch providers by setting the OPENAI_BASE_URL environment variable.

    Common provider configurations:

    • Hugging Face: https://router.huggingface.co/v1
    • Ollama: http://127.0.0.1:11434/v1
    • llama.cpp: http://127.0.0.1:8080/v1
    • OpenRouter: https://openrouter.ai/api/v1