Omnigent Documentation

Omnigent Desktop is a thin Electron-based shell that wraps the existing Omnigent web UI. It provides the same user experience as a web browser but adds native desktop features including:

• OS-native notifications: Alerts for agent turn completions (running → idle/failed), new elicitation requests, or runner disconnections. Notifications include a preview of the agent's final message.
• Foreground attention cues: On macOS and Windows, the app bounces the dock icon or flashes the taskbar frame to alert you even when the app is in the foreground.
• Unread session badges: A dock/taskbar badge shows the number of unread sessions (sessions that finished a turn or asked for input while not being actively viewed).
• Native menus: Standard text-editing shortcuts (Cmd/Ctrl-A, C, V, etc.) and a dedicated Server submenu for managing connections.
• Native file support: Supports browser-style drag-and-drop for files like images directly into text fields.
• Voice dictation: Integrated microphone support using a server-side fallback for transcription.

Omnigent Desktop is an Electron-based application designed with a security-first approach. It uses a main process to manage settings, window creation, and permissions, while the UI runs in a sandboxed environment.

Key Components

• Main Process (src/main.js): Manages settings persistence, window lifecycle, application menus, microphone permissions, and IPC handlers (e.g., normalize_url, change_server, navigate-to-server).
• Preload Script (src/preload.js): Acts as the sole, serialization-safe bridge between the untrusted SPA and the main process using contextBridge. It exposes window.omnigentDesktop and omnigentSetup.
• Browser Pane: Managed via src/browserViewRegistry.js and src/browserIpc.js to handle per-conversation web content.
• Find Bar: A built-in find-in-page feature (Cmd/Ctrl+F) anchored to the top-right corner.

Security Model

• Isolation: Uses nodeIntegration: false and contextIsolation: true.
• Origin Pinning: Every window is pinned to the specific server origin the user connected to. Privileged IPC handlers (like notify or setBadgeCount) verify that both the calling frame and the top-level page match the pinned origin.
• Protocol Handling: Non-web schemes (e.g., vscode://, ssh://) trigger an OS protocol handler with a consent prompt showing the requesting origin and URL.

The Omnigent Desktop shell uses a native Electron WebContentsView to host an embedded browser pane. This is a real Chromium page that is not an in-page webview or iframe, allowing for full screenshots, arbitrary in-page JS, and cross-origin navigation.

Key Components

• WebContentsView: A native Chromium view positioned over a placeholder <div> in the SPA. It runs with nodeIntegration:false, contextIsolation:true, and sandbox:true for security.
• browserViewRegistry.js: Manages a per-conversation Map of up to 10 WebContentsViews. Views are detached (not destroyed) when switching conversations to keep background pages running.
• browserIpc.js: The IPC surface handling commands like open-or-navigate, set-active, resize, screenshot, execute, close, and toolbar actions (go-back, go-forward, reload).
• preload.js: Exposes the window.omnigentDesktop bridge to the renderer, providing access to browser controls and event subscriptions (e.g., onBrowserUrlChanged, onBrowserElementSelected).

Agent Interaction Flow

Agents drive the browser via messages over the session stream:

1. Agent sends browser_* tool request to Omnigent server.
2. Server sends browser.action_request (via SSE) to the Desktop Renderer.
3. Renderer claims the action using an atomic token.
4. Renderer drives the local WebContentsView via IPC.
5. Result is posted back to the server and returned to the Agent.

Omnigent uses a client-side message queue to manage user inputs before they are sent to the server. This allows users to edit, delete, reorder, or 'steer' messages while an agent is busy.

Key Concepts:

• Queue (Strip): A pre-POST draft buffer where messages are editable and visible as "⏱ Queued". Messages in the queue are not yet owned by the server.
• Bubble: Once a message is POSTed (via auto-flush or manual steer), it is promoted from the strip to a chat bubble. Once in a bubble, it can no longer be edited, deleted, or reordered.
• Steer: An action that immediately POSTs a queued message, attempting to deliver it mid-turn if the harness supports it.
• Auto-flush: When the agent's sessionStatus becomes idle, the head of the queue (FIFO) is automatically sent as the next turn.

Message Actions:

• Edit: Pulls the message back into the composer (client-side only).
• Delete: Removes the message from the queue.
• Steer: Immediately POSTs the message to jump the queue.
• Reorder: Uses a drag handle to change the order of messages within the queue.