DOCSEARCHbyAlgolia
Submit Docs

VCPChat Documentation

repository·main·Indexed 20 days ago

https://github.com/lioensky/vcpchat

A distributed, AI-native full-stack engine and desktop client (vcp-chat-desktop v1.0.0) for VCP servers. It features VCPdesktop, a rendering layer for pushing interactive HTML widgets to the Windows desktop via a sandbox API (vcpAPI, musicAPI), and Flowlock, an autonomous loop module for AI agents to execute self-sustaining reasoning and execution cycles.

Tokens
95.5K
Snippets
195
Records
416
Agent score
72%

What's inside VCPChat

  1. Overview of Promptmodules modes

    main

    Promptmodules provides three distinct modes for managing system prompts in VCPChat, catering to different levels of complexity:

    ModeStorage FieldUse CaseCore Features
    Original Rich TextoriginalSystemPromptSimple, direct editingTraditional text area with auto-height
    Modular BlocksadvancedSystemPromptComplex, composable promptsDrag-and-drop, multi-content variants, small warehouse management
    Presets & TemporarypresetSystemPromptUsing templates or quick scene switchingPreset folders, placeholder replacement

    Use prompt-manager.js to coordinate and switch between these modes.

  2. Overview of VCPHumanToolBox

    main

    VCPHumanToolBox is an independent Electron application that serves as a visual tool invocation panel for the VCP ecosystem. It allows human users to call AI tools via dynamic forms instead of writing manual VCP commands.

    Key Capabilities:

    • Tool Grid: Displays available tools as cards with search and category filtering.
    • Dynamic Forms: Automatically generates input interfaces based on tool parameter definitions, supporting 7 different parameter types.
    • Multimodal Results: Renders results including Markdown, images, and video.
    • Plugin Management: Dynamically imports plugin definitions from the backend and allows visual parameter editing.
    • Workflow Editor: A node-based orchestration interface (partially implemented).

    Important Distinctions:

    • It is not an AI chat interface (that is the VCPChat main window).
    • It is not the execution backend (execution is handled by VCPDistributedServer).
    • It is not a permission management system.

  3. What is VCPdesktop

    main

    VCPdesktop is a desktop rendering layer extension for VChat. It operates as an independent canvas window within an Electron instance, allowing AI to perform real-time rendering of streamed HTML/CSS/JS content directly onto the user's operating system desktop.

    Key capabilities include:

    • Streaming Widget Creation: AI outputs HTML in chat, which is rendered token-by-token on the desktop.
    • Shadow DOM Isolation: Each widget runs in its own Shadow DOM to prevent CSS/JS pollution.
    • Script Sandboxing: <script> tags within widgets are automatically injected with Shadow DOM proxies for secure execution.
    • AI Remote Control: Through the DesktopRemote plugin, AI can push wallpapers, query desktop state (widget IDs, status, paths), and read widget source code for debugging or modification.

  4. Understand the VCPChat Tech Stack

    main

    VCPChat is built on a multi-layered stack designed for cross-platform desktop AI applications. Developers should be aware of the following core technologies used for different functional layers:

    Desktop & Backend

    • Electron: Cross-platform desktop application framework.
    • Node.js: Backend runtime environment.
    • File & Content Processing: fs-extra (file system), marked (Markdown), pdf-parse (PDF text extraction), and mammoth (DOCX text extraction).
    • Communication: ws (WebSocket communication for VCPLog).
    • Plugin Support: glob & minimatch (pattern matching) and node-schedule (task scheduling).

    Python Audio Engine

    The audio engine relies on a Python-based web service stack:

    • Web Framework: flask, flask_cors, and flask_socketio.
    • Audio Processing: soundfile, sounddevice, numpy, and pydub.
    • Networking: gevent and gevent-websocket for asynchronous support.

    Frontend Capabilities

    • Code & Math: highlight.js (syntax highlighting) and KaTeX (LaTeX rendering).
    • Computation & 3D: Pyodide (Python in WebAssembly) and three.js (3D rendering).

  5. Use the PTYShellExecutor plugin for local shell execution

    main

    The PTYShellExecutor plugin enables VCPChat to execute local shell commands in a Linux desktop environment. Unlike SSH-based executors, this plugin uses a Pseudo-Terminal (PTY) to maintain session state (environment variables and working directories) and supports both synchronous and asynchronous execution modes.

    Core Capabilities

    • PTY Persistent Sessions: Uses node-pty to create real pseudo-terminals, ensuring session state is preserved.
    • Multi-Shell Support: Automatically detects and selects shells in the following priority: fish > zsh > bash.
    • Dual Execution Modes:
      • Synchronous: For short commands that return immediately.
      • Asynchronous: For long-running tasks managed in the background.
    • Smart Output Cleaning: Automatically filters ANSI sequences, Shell Integration markers, and OSC/CSI sequences from the output.
    • GUI Terminal: Includes an Electron-based window (ShellViewer.html) for real-time command monitoring that follows VCPChat themes.

    Limitations

    • Environment: Only supported on Linux.
    • Interactive Programs: Does not support interactive programs like vim or less. It is recommended to use pipes to bypass these requirements.
    • GUI Dependency: The terminal GUI functionality requires Electron.

  6. Understand the VCPChat technology stack

    main

    VCPChat is built on a distributed, AI-native full-stack architecture. The stack is divided into desktop, backend, audio, and frontend layers:

    Desktop & Backend

    • Electron: Cross-platform desktop application framework.
    • Node.js: Backend runtime environment.
    • Core Dependencies:
      • fs-extra: Enhanced file system operations.
      • marked: Markdown parsing and rendering.
      • ws: WebSocket communication (used for VCPLog).
      • pdf-parse: PDF text extraction.
      • mammoth: DOCX text extraction.
      • glob & minimatch: File system pattern matching (used by distributed plugins).
      • node-schedule: Task scheduling (used by distributed plugins).

    Python Audio Engine

    • Web Services: flask, flask_cors, and flask_socketio.
    • Audio Processing: soundfile, sounddevice, numpy, and pydub.
    • Networking: gevent and gevent-websocket for asynchronous support.

    Frontend Features

    • Syntax Highlighting: highlight.js.
    • Math Rendering: KaTeX.
    • Python in Browser: Pyodide (via WebAssembly/CDN).
    • 3D Rendering: three.js (via CDN).

  7. VCPChat Technology Stack Overview

    main

    VCPChat is built using a multi-layered stack designed for cross-platform desktop applications, high-performance audio processing, and AI-native capabilities. The stack is divided into the following core components:

    Desktop & Backend

    • Electron: For cross-platform desktop application construction.
    • Node.js: Serving as the backend runtime.
    • Core Dependencies:
      • fs-extra: Enhanced file system operations.
      • marked: Markdown parsing and rendering.
      • ws: WebSocket communication (used for VCPLog).
      • pdf-parse: PDF text extraction.
      • mammoth: DOCX text extraction.
      • glob & minimatch: Pattern matching for distributed plugins.
      • node-schedule: Task scheduling for distributed plugins.

    Python Audio Engine

    • Web Services: flask, flask_cors, and flask_socketio.
    • Audio Processing: soundfile, sounddevice, numpy, and pydub.
    • Networking: gevent and gevent-websocket for asynchronous support.

    Frontend Capabilities

    • Syntax Highlighting: highlight.js (via CDN or local).
    • Math Rendering: KaTeX (via CDN or local).
    • Python in Browser: Pyodide (via WebAssembly/CDN) for client-side Python execution.
    • 3D Rendering: three.js (via CDN) for interactive 3D content.

  8. What is the TopicSponsor plugin?

    main

    TopicSponsor is a synchronous distributed plugin for the VCPChat frontend server (VCPDistributedServer). It grants Agents the ability to proactively initiate conversations, manage topics, and reply to messages, rather than just responding to user prompts.

    Key Capabilities:

    • Proactive Interaction: Agents can initiate greetings, share inspirations, or send task reminders.
    • Agent-to-Agent Collaboration: One Agent can leave messages within another Agent's topic.
    • Scheduled Tasks: Can be integrated with timely_contact for periodic messaging.
    • System Integration: Can deliver content from other systems (like AgentDream) directly to the conversation list.

    Relationship with TopicMemo:

    • TopicSponsor: Has read/write permissions. Used to create topics, reply to messages, and modify topic states.
    • TopicMemo: Has read-only permissions. Used to query topic lists and content.

  9. What is LoomController and how does it work?

    main

    LoomController is a VCP Loom Agent controller that provides Agents with the ability to create, query, open, close, read source code, and edit configurations for LoomAPPs.

    It operates as a hybridservice using the direct communication protocol. It does not start extra sub-processes or use local HTTP proxies; instead, it accesses the Electron main process's initialized LoomManager directly via a service container.

    Call Chain: Agent $\rightarrow$ VCP Backend Server $\rightarrow$ VCPDistributedServer $\rightarrow$ LoomControllerService $\rightarrow$ VCPLoomManager $\rightarrow$ LoomAPP (BrowserWindow/WebContentsView/Disk Source).

  10. Overview of VCPChat Audio Engine capabilities

    main

    The professional audio engine features:

    • Core: Custom decoding kernel and signal processing algorithms (FIR-EQ, dynamic compensation).
    • High-Res Support: Supports Hi-Res audio via WASAPI Exclusive and hardware DSD decoding.
    • Performance: Uses Rust for high-performance, double-precision (f64) resampling.

  11. What is Flowlock (心流锁)?

    main

    Flowlock is the Agent autonomous loop module for VCPChat. It allows an Agent to enter a self-sustaining cycle of reasoning and execution.

    Core Rule: The autonomous loop is only triggered after a reply is fully completed and saved to disk. This ensures a clear state boundary between messages, tool calls, and autonomous heartbeats. Unlike standard tool-calling loops, Flowlock does not bind the autonomous cycle to tool requests; tool requests can execute within a single reply, but the Flowlock state machine only migrates once the final assistant message is complete.

  12. How to use LoRAs in ComfyUI workflows

    main

    The module handles LoRA injection based on the presence of specific nodes in your workflow:

    Recommended Method (Automatic Injection)

    For automatic multi-LoRA stacking, your ComfyUI workflow must include the weilin-comfyui-prompt-all-in-one node (which may appear in the UI as WeiLinComfyUIPromptToLoras). When this node is present, the LoRA list configured in the settings will be automatically combined and injected into the workflow.

    Alternative Methods (If the node is missing)

    If the required node is not in your workflow, the settings-defined LoRAs will be ignored to prevent breaking your workflow. You can instead:

    1. Add the node: Manually add the weilin-comfyui-prompt-all-in-one node and connect it to your prompt/processing chain.
    2. Manual LoRA Tokens: Manually write LoRA tokens directly into your prompts using the format: <lora:name:strength:clipStrength>. This method does not rely on the specialized node.