DOCSEARCHbyAlgolia
Submit Docs

pilotty

repository·main·Indexed 41 days ago

https://github.com/msmps/pilotty

AI-friendly terminal automation tool version 0.0.11 that enables AI agents to interact with TUI applications through PTY management. It provides VT100 emulation, screen snapshots with JSON output, input simulation (typing, key sequences, and mouse clicks), and output inspection. Features include a daemon-client architecture, change detection via content hashes, and stability flags to handle progressive rendering.

Tokens
25.1K
Snippets
88
Records
112
Agent score
48%

What's inside pilotty

  1. Understand the session lifecycle

    main

    Spawn

    When you run pilotty spawn --name myapp my-command, the daemon:

    1. Creates a PTY.
    2. Forks the child process with the specified command.
    3. Initializes a terminal emulator (default size 80x24).
    4. Returns a session ID.

    Active Use

    During an active session, the screen buffer updates based on process output, the cursor position is tracked, and the terminal size can be modified using the resize command.

    Process Exit

    When the child process exits, the session is marked for cleanup. Cleanup typically occurs within 500ms, and the session is removed from the active list.

    Manual Kill

    Use pilotty kill -s <name> to manually terminate a session. This sends a SIGTERM to the child process and then performs cleanup.

    pilotty kill -s myapp

  2. How pilotty sessions work

    main

    pilotty manages multiple isolated terminal sessions. Each session is a self-contained environment consisting of:

    • PTY: A pseudo-terminal for the application.
    • Screen buffer: The state of the terminal emulator.
    • Child process: The actual running application.

    Sessions are fully isolated from one another, meaning they have separate PTY file descriptors, independent screen buffers, and independent cursor positions. Killing one session does not affect others.

  3. How pilotty's daemon architecture works

    main

    pilotty operates using a background daemon for session management, which simplifies the user experience:

    • Auto-start: The daemon starts automatically on your first command.
    • Auto-stop: The daemon shuts down after 5 minutes of inactivity (no active sessions).
    • Shared state: Multiple CLI calls can interact with the same sessions because they share the daemon's state.
    • Final evidence: When a session exits, the daemon retains its status, final screen, and bounded output for up to 10 minutes, allowing for post-mortem inspection.

  4. Understand the pilotty daemon architecture and lifecycle

    main

    pilotty operates using a daemon-client architecture. The CLI communicates with a background daemon via a Unix Socket using JSON-line messages.

    Lifecycle

    • Auto-start: The daemon starts automatically when you run your first command (e.g., pilotty spawn vim).
    • Session Cleanup: Sessions are automatically removed within 500ms of the child process exiting.
    • Auto-stop (Idle Timeout): The daemon automatically shuts down after 5 minutes of inactivity (no active sessions).
    • Persistence: The daemon runs in the background and survives the closing of your terminal.
    • Shared State: Multiple CLI invocations can interact with the same running sessions.
    • Graceful Shutdown: Use pilotty stop to terminate all sessions and the daemon.

    Socket Location Priority

    The daemon socket is created following this priority order:

    1. $PILOTTY_SOCKET_DIR/{session}.sock (if set)
    2. $XDG_RUNTIME_DIR/pilotty/{session}.sock (Linux standard)
    3. ~/.pilotty/{session}.sock (Home directory fallback)
    4. /tmp/pilotty/{session}.sock (Last resort)

  5. Use the 'Reliable Action + Wait' pattern

    main

    Avoid using sleep to wait for terminal updates. Instead, use the content_hash to detect when a screen has actually changed. This makes automation much more robust.

    Recommended Workflow:

    1. Capture the current screen's content_hash using pilotty snapshot.
    2. Perform your action (e.g., pilotty key Enter).
    3. Use pilotty snapshot --await-change <HASH> to wait until the screen content differs from the baseline.
    # The pattern: capture hash, act, await change
HASH=$(pilotty snapshot | jq '.content_hash')
pilotty key Enter
pilotty snapshot --await-change $HASH --settle 50

  6. Wait for terminal screen changes and stability

    main

    To avoid flaky automation caused by sleep commands, use the --await-change flag. This allows you to block execution until the terminal screen actually updates.

    Workflow Pattern:

    1. Capture the current content_hash from a snapshot.
    2. Perform a terminal action (e.g., pilotty key Enter).
    3. Use pilotty snapshot --await-change <HASH> to block until the screen changes.

    Advanced Stability Flags:

    • --settle <MS>: After a change is detected, wait for the screen to remain stable for the specified number of milliseconds. This is critical for apps that render content progressively (like AI streaming or slow TUI loaders).
    • -t, --timeout <MS>: Maximum time to wait (default: 30000ms).
    • --strict: If the deadline is reached or the session exits, the command exits with a non-zero code (3 or 4) instead of returning the last available evidence.
    # 1. Capture baseline hash
HASH=$(pilotty snapshot | jq '.content_hash')

# 2. Perform action
pilotty key Enter

# 3. Wait for change + 100ms stability
pilotty snapshot --await-change $HASH --settle 100

  7. Capture and wait for screen changes

    main

    The snapshot command provides the current state of the terminal. To avoid using sleep, use the --await-change flag with a content_hash from a previous snapshot to block until the screen updates.

    Snapshot Formats

    • --format full: (Default) Full JSON with text content and hash.
    • --format compact: JSON without the text field.
    • --format text: Plain text with a cursor indicator (e.g., [_]).

    Efficient Waiting Pattern

    1. Capture the current hash: HASH=$(pilotty snapshot | jq '.content_hash')
    2. Perform an action: pilotty key Enter
    3. Wait for change: pilotty snapshot --await-change $HASH
    4. (Optional) Wait for stability: pilotty snapshot --await-change $HASH --settle 50

    Snapshot Outcomes

    OutcomeMeaningAgent action
    immediateNo wait requestedRead the current screen
    changed--await-change was satisfiedContinue with the new screen
    settledThe requested stability window was satisfiedTreat the screen as stable
    deadlineTime expired; latest evidence is includedInspect evidence, then retry with more time if useful
    exitedThe session ended; final evidence is includedDo not send more input; inspect exit metadata or output

    Use --strict to make the command exit with a non-zero status on deadline or exited outcomes.

    # Wait for screen to change (eliminates need for sleep!)
HASH=$(pilotty snapshot | jq '.content_hash')
pilotty key Enter
pilotty snapshot --await-change $HASH           # Block until screen changes
pilotty snapshot --await-change $HASH --settle 50  # Wait for 50ms stability

  8. Install pilotty via npm or from source

    main

    You can install pilotty globally using npm, which is the recommended method. Alternatively, you can build it from source using Cargo, provided you have Rust 1.70+ installed.

    Note: Windows is not supported due to dependencies on Unix domain sockets and POSIX PTY APIs. Supported platforms include macOS (Intel and Apple Silicon) and Linux (x64 and arm64).

    # npm (recommended)
npm install -g pilotty

# From Source
git clone https://github.com/msmps/pilotty
cd pilotty
cargo build --release
./target/release/pilotty --help

  9. Manage pilotty sessions

    main

    Each pilotty session is isolated with its own PTY, screen buffer, and child process. You can run multiple applications in parallel by assigning them unique names using the --name flag.

    CRITICAL: Argument Positioning The --name flag must be placed before the command. Any arguments following the command are passed directly to that command, not to pilotty.

    Common session commands:

    • pilotty spawn --name <name> <command>: Starts a new session.
    • pilotty list-sessions: Lists all active sessions.
    • pilotty kill -s <name>: Terminates a specific session.
    • pilotty snapshot -s <name>: Targets a specific session for a snapshot.
    • pilotty key -s <name> <key>: Targets a specific session for key input.

    If no --name is provided during spawn, the session is automatically named default.

    # Run multiple apps with unique names
pilotty spawn --name monitoring htop
pilotty spawn --name editor vim file.txt

# Target a specific session
pilotty key -s editor Ctrl+S

# List all sessions
pilotty list-sessions

# Kill a specific session
pilotty kill -s editor

  10. Spawn a TUI application with pilotty

    main

    Use the spawn command to start a Terminal User Interface (TUI) application within a managed PTY session. You can also specify a working directory using the --cwd flag.

    To spawn a standard application:

    pilotty spawn htop

    To spawn in a specific directory:

    pilotty spawn --cwd /path/to/project bun src/app.tsx

  11. Create and manage sessions

    main

    Default Session

    If you run pilotty spawn without a --name, a session named default is created. Subsequent commands without a -s flag will target this default session.

    Named Sessions

    To run multiple concurrent applications, use the --name flag. Crucially, --name must appear before the command.

    Naming Rules:

    • Names must be unique per daemon instance.
    • Names are sanitized to alphanumeric characters, hyphens, and underscores.
    • Path traversal attempts (e.g., ../) are rejected.

    Targeting Sessions

    Use the -s or --session flag to direct a command to a specific named session. If omitted, commands target the most recently used session or default.

    # Create a default session
pilotty spawn htop

# Create named sessions
pilotty spawn --name monitoring htop
pilotty spawn --name editor vim file.txt

# Target a specific session
pilotty snapshot -s monitoring
pilotty key -s editor Ctrl+S
pilotty kill -s monitoring

  12. Take a terminal snapshot

    main

    The snapshot command captures the current state of the terminal screen. It returns structured JSON containing the terminal size, cursor position, and the text content. This is used by agents to perceive the current UI state.

    Snapshot JSON Schema:

    {
  "outcome": "immediate",
  "size": { "cols": 80, "rows": 24 },
  "cursor": { "row": 5, "col": 10, "visible": true },
  "text": "... plain text content ..."
}

    Wait for screen changes: To avoid arbitrary sleep durations, capture the current content_hash from a snapshot and use the --await-change flag. The --settle flag specifies how long to wait for the screen to stabilize.

    # Capture hash
HASH=$(pilotty snapshot | jq '.content_hash')

# Perform action
pilotty key Enter

# Wait for the screen to update and settle
pilotty snapshot --await-change $HASH --settle 50
    pilotty snapshot