DOCSEARCHbyAlgolia
Submit Docs

Open Genie Documentation

website·Indexed 30 days ago

https://docs.opengenie.dev

Developer resources for the Open Genie self-hosted platform, including guides for building, validating, and publishing plugins. Covers the plugin runtime model, manifest configuration, permissions, background jobs, webhooks, and the Hub Registry API contract.

Tokens
132.8K
Snippets
521
Records
915
Agent score
50%

What's inside Open Genie

  1. Overview of the Open Genie Plugin Publishing Process

    Publishing a plugin to the Open Genie Hub involves creating a self-contained .zip file that conforms to the manifest schema. The general workflow is: build the plugin locally $\rightarrow$ sign the plugin $\rightarrow$ upload via the CLI or web Submit page $\rightarrow$ admin review $\rightarrow$ marketplace publication.

  2. Overview of OpenGenie brand font architecture

    The web UI and Electron desktop app use self-hosted fonts to ensure the application renders correctly fully offline. All font files are stored in apps/web/client/public/fonts/ and served from /fonts/*.woff2. The @font-face rules and locale-switching logic are managed in apps/web/client/styles.css.

  3. Overview of Open Genie Background Tasks

    Open Genie uses a background task system to defer expensive, non-urgent work to optimal times (e.g., at night or when the machine is idle and on AC power). This system sits on top of the Scheduler: the Cron engine decides when to enqueue a job, and the Governor decides when that job actually runs. All processing occurs within the single server process without requiring external dependencies like Redis.

  4. Overview of Talk Mode voice conversations

    Talk Mode provides a hands-free, turn-taking voice conversation loop: listen → transcribe → think (chat) → speak → re-arm. It uses the same chat brain as typed chat, meaning it supports tools, actions, RAG memory, and history. All spoken exchanges are routed through POST /api/chat and persisted as normal conversation history. The system is half-duplex; the microphone is closed while Genie is speaking to prevent the reply from being transcribed as a new turn. Users can manually interrupt a reply by tapping the bubble or the Interrupt button.

  5. Overview of Open Genie WebSocket endpoints

    Open Genie provides three WebSocket endpoints on the same port, each serving different client types with specific authentication requirements:

    • /ws: For paired phones, tablets, and TVs. Requires a JWT token (via header or opengenie.bearer.<token> subprotocol). Always enabled.
    • /ws/iot: For ESP32 firmware and IoT hardware. Requires a per-device secret (Authorization: Bearer) and a non-secret clientId. Always enabled.
    • /ws/vault: For other Open Genie instances (vault sessions). Requires a vault session bearer (vault.<random>) obtained via POST /api/vault/session-token. Enabled only when GENIE_VAULT_ENABLED=1.

  6. Overview of the Planting plugin

    The Planting plugin tracks plants through growth stages (seedling → vegetative → flowering → fruiting → harvest) using a combination of species knowledge tables and LLMs. It integrates with multi-function IoT devices for soil-moisture and light sensing, uses PTZ cameras with saved presets for visual health checks, and manages watering based on sensor data and OpenWeatherMap forecasts.

  7. Overview of the Open Genie Hub architecture

    Open Genie is a self-hosted Edge AI hub implemented as a single monolithic Node.js process. It integrates several core components into one binary and database, including a chat assistant (local or cloud LLMs), persistent memory, a sandboxed plugin runtime, camera vision, a media library, and automation. The hub avoids service meshes, RPC, or service discovery in favor of a monolithic design where features interact via direct function calls.

  8. Open Genie Clipper authentication and security

    The connection flow between the browser extension and the server uses a PKCE-based handshake to ensure tokens are never exposed in URLs:

    1. The extension stores a secret verifier and sends a challenge to the consent page.
    2. Upon approval, the server returns a single-use code.
    3. The extension exchanges the code and verifier for a token via a direct request.

    Privacy and Security constraints:

    • The extension does not store data or model keys.
    • It communicates only with the configured server URL (no third-party telemetry).
    • Page content is read only when the extension is explicitly invoked via the active tab, not in the background.

  9. Understand the OpenGenie plugin submission pipeline

    Every plugin update (initial publish or new version) follows a sequential pipeline. Failures at server-side checks (Zip safety, Virus scan, Manifest, Signature) short-circuit the process and return an immediate rejection. Linting failures do not short-circuit; they are persisted as warnings for human moderators to review.

    Pipeline Flow:

    1. Upload (CLI or web)
    2. Content-length check (10 MB limit)
    3. Zip safety scan
    4. Virus scan (ClamAV, if enabled)
    5. Manifest parse and validation
    6. Signature verification (if provided)
    7. Automated linting
    8. Submission record persisted (status: pending_review)
    9. Email notification to publisher and moderators
    10. Human moderation
    11. Outcome: approved (becomes current version), changes_requested (fix required), rejected (terminal), or quarantined (security review required).

  10. Understand Open Genie Action Architecture

    Actions are the universal unit of work in Open Genie. They are named, parameterized operations that can be triggered by AI tool calls, cron jobs, webhooks, or direct API calls. The flow follows this sequence: Trigger $\rightarrow$ Action Registry $\rightarrow$ Action Handler (function) $\rightarrow$ ActionResult ({ output, err }).

  11. Understand the Open Genie Chat Pipeline flow

    The chat pipeline (lib/chat/pipeline.ts) is a provider-agnostic system that handles conversational AI. The sequence of operations for a user message is as follows:

    1. resolveRole(): Selects the provider and model based on opengenie.json roles.
    2. createConversation(): Creates a database row and persists history.
    3. buildSystemPrompt(): Constructs the prompt using stable and volatile sections (e.g., SOUL, household, time).
    4. retrieveMemory(): Performs RAG vector search over memory_chunks if GENIE_RAG_ENABLED=1 is set.
    5. inferSpeaker(): Maps the deviceId to a specific person.
    6. provider.chat(): Calls the LLM (Ollama or Anthropic) with streaming and tool support (up to 5 rounds).
    7. addMessage(assistant): Persists tokens as they stream.
    8. extractMemories(): Asynchronously distills durable facts into memory_chunks.
    9. updateConversationTitle(): Updates the title on the first turn.

    The final output is a ReadableStream<Uint8Array> delivered via HTTP SSE or WebSocket frames.

  12. Open Genie Camera Vision Architecture Overview

    Open Genie monitors RTSP, HTTP, and MJPEG camera streams using a periodic AI-powered analysis pipeline. The process follows this flow: Camera Worker (staggered capture loop) → FFmpeg Capture (frame extraction) → Ollama Vision (AI analysis via custom prompt) → Alert Rules (condition evaluation) → Event Storage/Notifications.