DOCSEARCHbyAlgolia
Submit Docs

Lemonade SDK Documentation

repository·main·Indexed 26 days ago

https://github.com/lemonade-sdk/lemonade

A local AI server and embeddable binary providing private, high-performance multi-modal AI capabilities (chat, image, speech, transcription), optimized for AMD Ryzen AI and Radeon systems. It features an HTTP API with compatibility layers for OpenAI, Ollama, Anthropic, and llama.cpp, as well as an MCP Gateway. The SDK includes 'Omni Models' managed via OmniRouter and a portable 'Embeddable Lemonade' version for integration into third-party applications.

Tokens
96.2K
Snippets
255
Records
516
Agent score
89%

What's inside Lemonade

  1. Overview of Lemonade HTTP API endpoints

    main

    The Lemonade HTTP service provides a variety of standards-compliant and custom endpoints designed to work out-of-the-box with popular local AI applications. The API is organized into several compatibility layers:

    • OpenAI-Compatible API: The primary API surface used by most standard SDKs and clients.
    • Ollama-Compatible API: For clients expecting Ollama-style routes and behavior.
    • Anthropic-Compatible API: For clients built around Anthropic's specific message formats.
    • MCP Gateway: Exposes Lemonade as a Model Context Protocol server via the POST /mcp endpoint.
    • llama.cpp-Specific API: Provides compatibility with llama.cpp conventions.
    • Lemonade-Specific API: A local-first API used for managing the service lifecycle, configuration, and backends.

  2. Overview of available Lemonade recipes and backends

    main

    Lemonade supports various AI recipes, each with specific selectable backends and hardware requirements. Some recipes allow manual selection of the backend via CLI flags, while others use specific hardware (like NPU) automatically.

    Key information for selecting a recipe:

    • Selectable backend: Indicates if you can manually choose the backend (e.g., cuda, vulkan, cpu) via CLI flags.
    • Uses ctx_size: Indicates if the recipe supports the --ctx-size flag to manage model context.
    RecipeNameSelectable backendUses ctx_sizeBackends
    acestepACE-Stepyesnocuda, rocm, vulkan
    flmFastFlowLM NPUnoyesnpu
    kokoroKokorononocpu, metal
    llamacppLlama.cpp GPUyesyescpu, cuda, metal, rocm, system, vulkan
    moonshineMoonshinenonocpu
    onnxruntimeONNX Runtimenonocpu
    openmossOpenMOSS TTSyesnocuda, rocm, vulkan
    ryzenai-llmRyzen AI LLMnoyesnpu
    sd-cppStableDiffusion.cppyesnocpu, cuda, metal, rocm, vulkan
    thinksoundThinkSoundyesnocuda, rocm, vulkan
    trellisTRELLIS.2yesnocuda, rocm, vulkan
    vllmvLLM ROCm (experimental)yesyesrocm
    whispercppWhisper.cppyesnocpu, metal, npu, rocm, vulkan

  3. Overview of Lemonade's OpenAI-Compatible API

    main

    Lemonade provides an API implementation compatible with the OpenAI specification. It supports various modalities including chat completions, text completions, embeddings, audio transcription, text-to-speech, image generation, and more. This allows developers to use existing OpenAI SDKs by pointing them to the Lemonade local server.

    | Method | Endpoint | Description | Modality |
|--------|----------|-------------|----------|
| `POST` | [`/v1/chat/completions`](#post-v1chatcompletions) | Chat Completions | messages -> completion |
| `POST` | [`/v1/completions`](#post-v1completions) | Text Completions | prompt -> completion |
| `POST` | [`/v1/embeddings`](#post-v1embeddings) | Embeddings | text -> vector representations |
| `POST` | [`/v1/responses`](#post-v1responses) | Responses API | prompt/messages -> event |
| `POST` | [`/v1/audio/transcriptions`](#post-v1audiotranscriptions) | Audio Transcription | audio file -> text |
| `POST` | [`/v1/audio/speech`](#post-v1audiospeech) | Text to speech | text -> audio |
| `WS` | [`/realtime`](#ws-realtime) | Realtime Audio Transcription, OpenAI SDK compatible | streaming audio -> text |
| `POST` | [`/v1/images/generations`](#post-v1imagesgenerations) | Image Generation | prompt -> image |
| `POST` | [`/v1/images/edits`](#post-v1imagesedits) | Image Editing | image + prompt -> edited image |
| `POST` | [`/v1/images/variations`](#post-v1imagesvariations) | Image Variations | image -> varied image |
| `POST` | [`/v1/images/upscale`](#post-v1imagesupscale) | Image Upscaling | image + ESRGAN model -> upscaled image |
| `GET` | [`/v1/models`](#get-v1models) | List models available locally | n/a |
| `GET` | [`/v1/models/{model_id}`](#get-v1modelsmodel_id) | Retrieve a specific model by ID | n/a |

  4. Overview of Lemonade local AI

    main

    Lemonade is a local AI server that provides capabilities similar to cloud APIs (chat, coding, speech, and image generation) while remaining 100% free and private. It runs on your own NPU and GPU and is optimized for AMD hardware (Ryzen AI, Radeon, and Strix Halo).

    It is available in two forms:

    • Lemonade Server: A service you can connect to via standard OpenAI, Anthropic, and Ollama APIs.
    • Embeddable Lemonade: A portable binary for packaging multi-modal local AI into your own applications.

  5. Overview of Lemonade Components

    main

    Lemonade consists of several core executables depending on your use case:

    • lemond: The core HTTP server responsible for request handling and LLM backend orchestration.
    • lemonade: The CLI client used for terminal operations (e.g., list, pull, delete, run, status, logs, launch, backends, scan).
    • LemonadeServer.exe (Windows only): A GUI application that embeds the server and provides a system tray icon.
    • lemonade-tray (macOS/Linux): A lightweight tray client that connects to a running lemond instance.

  6. Understand Lemonade Router Schema Types

    main

    The Lemonade Router uses frozen JSON schemas to manage generic routing. The engine performs pure model selection based on boolean rules over classifiers using a first-match-wins strategy. If no rules match, it fails-open to a default_model.

    There are three primary schema types:

    1. route_policy.schema.json: Defines the routing block within a collection.router collection JSON. You invoke this by pointing the OpenAI model field to the collection name.
    2. request.schema.json: Defines extension fields for the OpenAI chat body. It supports metadata (string-valued routing inputs) and an optional route_trace. This schema allows additional properties but validates these specific fields.
    3. decision.schema.json: Defines the x_lemonade_route decision object that is attached to the chat response.

  7. Understand the Auto-Tune feature

    main
    Auto-Tune is a performance optimization mechanism in Lemonade designed to automatically select the best backend and tuning parameters (such as batch size, GPU layer offload, thread count, and context size) based on your machine's hardware profile. It aims to eliminate the need for manual benchmarking or guessing backend flags by detecting hardware archetypes (GPU architecture, VRAM capacity, memory bandwidth, etc.) and applying community-validated performance parameters.

  8. Understand the Lemonade C++ Project Structure

    main

    The Lemonade C++ core is organized into several functional directories. Developers building or extending the core should note the following key areas:

    • src/cpp/server/: Contains the main HTTP server implementation, routing, model management, and CLI parsing.
    • src/cpp/server/backends/: Contains specific implementations for different AI models (e.g., llamacpp_server.cpp for LLMs, sd_server.cpp for Stable Diffusion, and whisper_server.cpp for transcription).
    • src/cpp/include/lemon/: The public header directory. This is where the primary interfaces for the server, backends, and utilities are defined.
    • src/cpp/tray/: Contains the implementation for the system tray application, including platform-specific code for Windows, macOS, and Linux.
    • src/cpp/resources/: Holds static assets like the Web UI and model registry JSON files.
    • src/cpp/installer/: Contains WiX installer definitions for Windows.

  9. Get started with Lemonade

    main

    Lemonade is designed to simplify local AI for both users and developers.

    • Users: Start with the User Guide to learn how to use the Command Line Interface (CLI) and explore customization options.
    • Builders: If you want to integrate Lemonade into your own applications, you can serve models using the API or embed the Lemonade portable binary directly into your application.

  11. Understand the Job Recipe Expression Language

    main

    Job recipes (sent via POST /v1/jobs) use a tiny expression language for two purposes:

    1. References/Interpolation: Resolving values in a step's params using the job's shared context.
    2. Boolean Conditions: Evaluating when clauses in steps or branch[].when clauses to control execution flow.

    The language is not a general scripting language; it lacks variables, assignments, function calls, and loops. It only supports references, literals, comparisons, boolean logic, and basic arithmetic.

  12. Understand Lemonade Architecture

    main

    Lemonade uses a client-server architecture consisting of three main components:

    1. lemond (Server Component): A pure HTTP server that provides OpenAI-compatible REST API endpoints (/api/v0 and /api/v1). It manages model loading/unloading, routes requests to backends (llamacpp, fastflowlm, ryzenai), and supports multiple simultaneous models using an LRU (Least Recently Used) eviction policy.

      • Multi-Model Support: You can configure the number of loaded models using --max-loaded-models N (default is 1).
      • NPU Exclusivity: Only one model can use the NPU at a time.

    2. lemonade (CLI Client): A console application for terminal users. It provides a command-based interface (e.g., list, pull, run, status) and communicates with the lemond server via HTTP. It uses UDP beacon broadcasting on port 13305 to auto-discover running servers.

    3. lemonade-tray / LemonadeServer.exe (GUI Tray Application): A desktop application that exposes the server via a system tray icon.

      • Windows: LemonadeServer.exe is a GUI app with no console window. It uses a global mutex (Global//LemondMutex) to prevent multiple instances.
      • Linux: lemonade-tray requires GTK3 + AppIndicator3 and uses flock() to prevent duplicate instances.