Open Cowork Documentation

repository·main·Indexed 23 days ago

https://github.com/opencoworkai/open-cowork

Open-source AI agent desktop application for Windows and macOS (v3.3.1) that provides a GUI for LLMs including Claude, OpenAI, Gemini, and DeepSeek. Features include GUI automation, MCP integration, a specialized 'Skills' system for document generation (PPTX, DOCX, PDF, XLSX), and sandbox isolation using WSL2 on Windows and Lima on macOS.

Tokens
30.7K
Snippets
30
Records
154
Agent score
83%

What's inside Open Cowork

  1. Overview of Open Cowork Features

    main

    Open Cowork is an open-source desktop AI assistant that provides a graphical interface for models like Claude, OpenAI, Gemini, and DeepSeek.

    Key Capabilities:

    • GUI Automation: Controls desktop applications (Gemini-3-Pro is recommended for best results).
    • Skills System: Built-in workflows for generating/processing PPTX, DOCX, PDF, and XLSX files.
    • MCP Integration: Connects to external services like Browsers and Notion via Model Context Protocol (MCP) connectors.
    • Remote Control: Integrates with platforms like Feishu (Lark) for workflow automation.
    • Sandbox Isolation: Uses WSL2 (Windows) or Lima (macOS) to run commands in a secure environment.
    • Multimodal Input: Supports dragging and dropping files and images directly into the input box.
  2. Core Features of Open Cowork

    main

    Open Cowork is an open-source AI agent desktop application with the following capabilities:

    • Built-in Skills: Workflows to generate PPTX, DOCX, XLSX, and PDF documents, or create custom skills via the skill-creator toolkit.
    • MCP Tool Integration: Connects to browsers, Notion, and desktop apps using the Model Context Protocol.
    • GUI Automation: Automates desktop applications via computer use (Gemini-3-Pro is recommended for this feature).
    • Remote Control: Integrate with Feishu (Lark) and Slack to send commands and receive results remotely.
    • Data Privacy: The application runs locally. No telemetry or data is sent to Open Cowork servers; communication only occurs with your configured AI model API.
  3. How Subagent execution works

    main

    Subagents are implemented as in-process sessions rather than subprocesses. This approach ensures:

    • No dependency on external binaries (like pi) being in the system PATH.
    • Reuse of existing authStorage and modelRegistry from the main runner.
    • Tighter integration with MCP (Model Context Protocol), sandboxing, and permission systems.
    • Lower overhead compared to spawning new OS processes.

    Child Session Lifecycle:

    1. Creation: Uses createAgentSession({ sessionManager: inMemory(), model, tools: <subset>, cwd }).
    2. Communication: Subscribes to events to stream progress back to the parent session.
    3. Control: Supports AbortSignal propagation from parent to child.
    4. Cleanup: Calls .dispose() upon completion.
    5. Observability: Token usage is aggregated and reported back to the parent's UI.
  4. Agent Configuration Awareness and Tools

    main

    The roadmap for Open Cowork includes making the agent aware of its own configuration and providing tools to interact with it.

    Configuration Access

    Agents can be granted access to read and modify their settings via custom tools:

    • config_read: A tool that allows the agent to inspect its current configuration (e.g., model, contextWindow, provider, sandboxEnabled, memoryEnabled).
    • config_write: A tool that allows the agent to modify non-sensitive configuration fields (e.g., switching models or toggling thinking mode).

    Security Constraints

    • Sensitive Data: API keys and secrets (found in profiles, configSets, or remote-config.json) are encrypted and must never be exposed in plaintext configuration files or via the config_write tool.
    • Permissions: Any use of config_write is subject to the project's permission policy (typically requiring an 'always-ask' user confirmation).
  5. Configure Sandbox Security (WSL2 and Lima)

    main

    Open Cowork provides multi-level sandbox protection to ensure system security. Commands are executed in isolated virtual machines where available.

    Windows (WSL2)

    If WSL2 is installed, Open Cowork automatically detects it. All Bash commands are routed to the isolated Linux VM, and the workspace is bi-directionally synchronized.

    macOS (Lima)

    To enable enhanced isolation on macOS, install Lima:

    brew install lima

    Open Cowork will automatically create and manage a Lima VM (internal instance name: 'claude-sandbox'). Commands will run in an Ubuntu VM with /Users mounted.

    Fallback Mode

    If no virtual machine is available, commands execute on the host machine but are protected by path restrictions (limiting operations to the selected workspace folder).

  6. Built-in Skills and Customization

    main

    Open Cowork uses a Skills system located in .claude/skills/. These skills allow the AI to perform complex file-based workflows. Users can add or delete custom skills.

    Available Built-in Skills:

    • pptx: PowerPoint generation
    • docx: Word document processing
    • pdf: PDF processing and forms
    • xlsx: Excel spreadsheet support
    • skill-creator: A toolkit for developing new skills
  7. How Skills and MCP work in Open Cowork

    main

    Open Cowork extends AI capabilities through two primary mechanisms:

    • Skills: Built-in or custom workflows for specific file-based tasks. Default skills are located in .claude/skills/ and include support for pptx, docx, pdf, and xlsx. You can develop your own using the skill-creator toolkit.
    • MCP (Model Context Protocol): Allows the AI to connect to external tools and services. Open Cowork supports MCP connectors for browsers, Notion, and other desktop applications, enabling the agent to interact with data beyond the local filesystem.
  8. Understand the SDK compaction algorithm

    main

    The SDK's compaction logic follows these steps:

    1. shouldCompact(): Checks if contextTokens > contextWindow - reserveTokens.
    2. prepareCompaction(): Identifies the cut point by walking backward through messages while respecting keepRecentTokens. It ensures tool_call/tool_result pairs are never split.
    3. generateSummary(): Uses an LLM with a structured template (Goal, Constraints, Progress, Key Decisions, Next Steps, Critical Context) to create a summary. It supports an UPDATE variant for incremental merges.
    4. Result: Returns a CompactionResult containing the summary, firstKeptEntryId, tokensBefore, and details (including readFiles and modifiedFiles).

    Default Settings:

    • reserveTokens: 16384
    • keepRecentTokens: 20000
  9. Understand the Open Cowork Architecture

    main

    Open Cowork is an Electron-based desktop application designed as a local AI workspace. The architecture is divided into three main layers:

    1. Main Process (Node.js): Handles core logic including the Agent SDK (claude/), configuration management (config/), database persistence via SQLite (db/), and security via sandbox path resolution (sandbox/). It also manages skills/ (workflow templates) and tools/ (execution logic).
    2. Preload Scripts: Bridges the gap between the Main and Renderer processes via IPC.
    3. Renderer Process (React + Tailwind): The user interface, featuring components for chat (ChatView.tsx), configuration (ConfigModal.tsx), file context (ContextPanel.tsx), and AI reasoning traces (TracePanel.tsx).

    Custom skills are stored in the .claude/skills/ directory, which includes specialized support for pptx, docx, pdf, and xlsx.

  10. How Sandbox Isolation Works

    main

    Open Cowork provides multi-level security to protect your host system:

    • Basic Protection: All platforms implement fundamental path restrictions.
    • VM-Level Isolation: On Windows and macOS, you can enable virtual machine isolation.
      • Windows: Uses WSL2.
      • macOS: Uses Lima (the internal Lima instance name is claude-sandbox).

    When VM isolation is enabled, all commands are executed within the isolated Linux environment, preventing direct access to your host machine's files and system.

  11. Security: VM Sandbox Isolation

    main

    To protect your host system, Open Cowork executes all AI-generated commands within an isolated Linux virtual machine (VM). This ensures that even if an AI agent executes a command incorrectly or maliciously, your host files remain protected.

    • Windows: Uses WSL2 (Windows Subsystem for Linux).
    • macOS: Uses Lima.
  12. Configure AI Models in Open Cowork

    main

    Open Cowork supports multiple AI model providers. You can configure:

    • Claude: Via Anthropic or OpenRouter.
    • OpenAI Compatible Interfaces: Any provider that offers an OpenAI-compatible API (e.g., DeepSeek, Zhipu GLM, MiniMax, Kimi).

    Note: Open Cowork is free under the MIT license, but you are responsible for paying the API usage costs to your chosen provider.