Qwen Code Documentation

repository·main·Indexed 31 days ago

https://github.com/qwenlm/qwen-code

Qwen Cua Driver is a cross-platform background computer-use driver for AI agents. It provides a Model Context Protocol (MCP) interface over stdio and supports ACP Streamable HTTP transport via a single /acp endpoint. Features include window management, input actions (click, drag), and specialized support for Qwen-VL models via relative-coordinate normalization.

Tokens
535.6K
Snippets
708
Records
2.4K
Agent score
97%

What's inside qwen-code

  1. Overview of Virtual Viewport implementation

    main

    The Virtual Viewport implementation (introduced in PR #4146) addresses performance issues (flickering, lag, and unbounded scrollbar growth) caused by rendering entire conversation histories in ink's append-only <Static> component.

    Key Features included in V.2:

    • Core Viewport: Efficiently renders only visible items.
    • ASCII Scrollbar: Includes auto-hide animations.
    • SGR Mouse Support: Enables mouse-wheel interaction.
    • Keyboard Navigation: Supports scroll keys.
    • ui.useTerminalBuffer Gate: Allows switching between the legacy <Static> path and the new ScrollableList path.

    Planned Features (V.3+):

    • Scrollbar dragging and click-to-position.
    • In-app / search.
    • Alternate-buffer mode (full alt-screen takeover).
  2. Overview of cua-driver test harness applications

    main

    The test-harness/apps directory contains self-contained host applications used to drive and test cua-driver. These apps share fixtures via ../shared/scenarios.json and ../shared/web/index.html to ensure that AutomationIds, AX identifiers, and DOM ids remain consistent between the applications and integration tests.

    Supported Test Applications

    PathHost OSTesting Focus
    cross-platform/electronanyElectron + CDP (exercises page tool)
    macos/appkitmacOS 13+AppKit / Cocoa (NSButton, NSScrollView, NSMenu, etc.)
    macos/swiftuimacOS 13+SwiftUI (popovers, declarative state)
    windows/wpfWindowsWPF / UIA (popups, layered windows, modal MessageBox, hwndhost)
    windows/winui3WindowsWinUI3 unpackaged (DComp popups, XAML Popup primitive)
    windows/webview2WindowsMicrosoft.Edge.WebView2 (DOM parity with Electron)
  3. Overview of the ACP SDK for Java

    main

    The acp-sdk is a Java implementation of the Agent Client Protocol (ACP), designed to facilitate standardized communication between AI agents and client applications. It enables developers to implement features such as session management, file system operations, terminal command execution, and tool call handling within Java applications.

    Key Capabilities:

    • Session Management: Create, load, and manage conversation sessions.
    • File System Operations: Read and write text files.
    • Terminal Access: Execute commands via the terminal.
    • Tool Integration: Handle tool calls and manage permissions for sensitive operations.
    • Rich Content Support: Process text, images, audio, and resources.
    • MCP Integration: Works with Model Context Protocol (MCP) servers.
  4. Overview of Qwen Code Hooks

    main

    Qwen Code hooks allow you to extend and customize application behavior by executing custom scripts or programs at specific lifecycle points (e.g., before/after tool execution, session start/end). Hooks are enabled by default. You can globally disable all hooks by setting disableAllHooks: true in your settings file.

    Common use cases include:

    • Monitoring and auditing tool usage
    • Enforcing security policies
    • Injecting context into conversations
    • Integrating with external systems
    {
      "disableAllHooks": true,
      "hooks": {
        "PreToolUse": [...]
      }
    }
  5. Overview of Computer Use capabilities

    main

    The agent uses the cua-driver to interact with your desktop. It is recommended to use element-addressed actions (targeting element_index from the accessibility tree) rather than raw pixel coordinates for better stability.

    Key Tool Categories:

    • Mouse: click, double_click, right_click, drag, move_cursor, scroll
    • Keyboard: type_text, press_key, hotkey
    • Windows / UI: list_windows, get_window_state, get_accessibility_tree, set_value, zoom
    • Apps: launch_app, list_apps, bring_to_front, kill_app (Note: bring_to_front is Windows-only)
    • Browser: page (execute JS, read text, query DOM, click elements)
    • Screenshots: get_window_state (captures PNG), page
    • Recording: start_recording, stop_recording, replay_trajectory
    • Sessions: start_session, end_session
  6. Understand Session Artifacts in Qwen-Code

    main

    In Qwen-Code, Artifacts are explicitly registered, structured outputs within a session that are reusable, clickable, previewable, or downloadable.

    Key distinctions:

    • Artifacts: Explicitly declared outputs like generated reports, HTML files, PDFs, images, or specific business resource URLs (e.g., a dashboard link from a tool).
    • Not Artifacts: Standard source code changes (these are file changes/diffs), markdown links in assistant responses, URLs found in shell stdout, or URLs from web_fetch (these are considered noise or inputs).
  7. Understand the Daemon Event Schema v1

    main

    The Qwen Daemon emits events via Server-Sent Events (SSE) on the GET /session/:id/events endpoint. Every event follows a standard envelope shape:

    { id, v, type, data, originatorClientId?, _meta? }

    • id: Unique identifier for the event.
    • v: Schema version (currently 1).
    • type: A string from the DAEMON_KNOWN_EVENT_TYPE_VALUES set.
    • data: The event-specific payload.
    • originatorClientId?: The ID of the client that originated the action.
    • _meta?: Envelope-level metadata stamped at the SSE write boundary.

    Forward Compatibility: SDK consumers can use asKnownDaemonEvent(evt) to handle events. If the daemon introduces a new event type that the current SDK version does not recognize, asKnownDaemonEvent returns undefined. The session reducer will then record these as unrecognizedKnownEventCount, allowing the client to remain functional without a lockstep upgrade.

  8. Understand LLM Request Timing Telemetry in qwen-code

    main

    The qwen-code.llm_request span provides detailed telemetry to help operators diagnose LLM latency and reliability issues. Instead of a single opaque duration_ms, the telemetry decomposes the request lifecycle into specific phases.

    Key metrics captured include:

    • TTFT (Time-to-First-Token): The latency until the first chunk containing user-visible content is received. Note that this includes 'thinking' or reasoning content.
    • Request Setup Duration (request_setup_ms): The time spent on network setup and initial request preparation.
    • Sampling Duration: The time elapsed between the start of the request and the first token, which can be calculated as duration_ms - ttft_ms - request_setup_ms.
    • Retry Visibility: Detailed telemetry for retries occurring via retryWithBackoff (e.g., rate limits or 5xx errors), providing visibility into how many attempts were made and why.

    This telemetry is designed to answer "why was this LLM call slow?" by distinguishing between model generation time, network setup, and retry overhead.

  9. Understand AutoSkill procedural memory extraction

    main

    AutoSkill is a mechanism for automatically extracting procedural memory from agent sessions. When an agent completes a task involving high tool-call density, the system evaluates the session for reusable operational workflows and saves them as project-level skills.

    Key Differences from Memory Extract

    FeatureMemory ExtractAutoSkill
    Memory TypeDeclarative (user identity, project context)Procedural (how to perform specific tasks)
    TriggerEnd of every sessionWhen tool calls reach a specific threshold within a session
    Storage Path${projectRoot}/.qwen/memory/${projectRoot}/.qwen/skills/
    ContentUser preferences, context, feedbackReusable steps, best practices
    LifecyclePeriodic integration/pruning (Dream)On-demand updates by a review agent
  10. Understand the /review skill architecture and blocker recognition

    main

    The /review skill uses an ensemble of specialized agents to perform code reviews. To prevent human-reported blockers from being ignored, the system uses a semantic recognition pattern called carriesBlockerSignal rather than relying on a literal [Critical] marker.

    Any body (inline thread or issue comment) asserting a blocking defect—using phrases like [Critical], (blocker), is a blocker, must fix, still reproducible, broken, fails, or the Chinese 阻塞项/问题/点—is automatically promoted to a "Blockers to re-check" section. This section is rendered in full to ensure agents can see the complete context, preventing blockers from being lost due to character limits or snippet truncation.

  11. Understand the DaemonWorkspaceService Architecture

    main

    The DaemonWorkspaceService is a facade layer (L2) designed to manage workspace-scoped operations. It is distinct from the AcpSessionBridge (formerly HttpAcpBridge), which handles session-scoped operations.

    Scope Separation Rule

    • Session-scoped (use AcpSessionBridge): Operations targeting a specific sessionId, such as prompt, cancel, close, model, approval, metadata, or heartbeat.
    • Workspace-scoped (use DaemonWorkspaceService): Operations targeting the entire workspace, such as file, auth, agents, memory, mcp-status, skills, env, preflight, tool-toggle, or init.

    Layered Architecture

    1. L1 Transport: REST+SSE or /acp (jsonrpc/sse).
    2. L2 Application Layer: Contains AcpSessionBridge (session lifecycle/EventBus) and DaemonWorkspaceService (workspace management via sub-services like FileService, AuthService, etc.).
    3. L3 ACP-client: The interface to the child process.
    4. L4 Agent: The end-user agent.
  12. Understand Async Memory Recall design

    main

    Async Memory Recall is a design pattern used to reduce user-visible latency during AI agent interactions. Instead of awaiting memory retrieval (which can block the main request path for up to 2.5s during cold starts), the system fires a memory recall side-query immediately upon a UserQuery and never awaits it.

    The result is consumed opportunistically at one of two points:

    1. UserQuery consume point: A synchronous check just before turn.run(). If the memory has already settled, it is prepended to systemReminders. If not, the query proceeds without waiting.
    2. ToolResult inject point: If the memory settles after the initial query but before the next turn, it is injected as a system-reminder appended to the requestToSend array. This ensures the memory context is available to the model before its next response without breaking the required functionCall/functionResponse pairing.