DOCSEARCHbyAlgolia
Submit Docs

ypi Documentation

repository·master·Indexed 18 days ago

https://github.com/rawwerks/ypi

ypi is a recursive coding agent built on top of the Pi coding agent (version 0.6.1). It allows LLMs to decompose complex problems by spawning child agents that can call themselves recursively using tools like bash and a specialized rlm_query function. The project includes a CLI wrapper for shell ergonomics and a pure extension (pi-recursive) for native Pi integration, along with various extensions for LSP support, semantic code search via colgrep, and repository indexing.

Tokens
11.7K
Snippets
38
Records
60
Agent score
63%

What's inside ypi

  1. Use rlm_query for recursive task decomposition

    master

    The core pattern for solving large tasks is size up → search → delegate → combine. Instead of attempting to process massive files or complex repositories in a single context window, use rlm_query to spawn sub-agents. Each sub-agent receives a fresh context window, allowing you to stay effective on long-running tasks.

    Execution Modes:

    • Synchronous (Default): Blocks the parent process until the child completes. Use this when the next step depends on the child's output.
    • Asynchronous (--async): Returns immediately, running the child in the background. This is the preferred method for parallel work. It returns a JSON object containing job_id, output (path to result), sentinel (path to a completion file), and pid.
    # Synchronous: Child inherits environment and blocks
rlm_query "Refactor the error handling in src/api.py"

# Asynchronous: Returns immediately for parallel work
rlm_query --async "Write tests for the auth module"

# Piping data as context (Synchronous)
sed -n '100,200p' bigfile.txt | rlm_query "Summarize this section"

  2. Understand the ypi and pi-recursive package relationship

    master

    The rawwerks/ypi repository ships two distinct npm packages from a single source. They are versioned in lockstep (the same version number) and must always be released together.

    1. ypi (located in the repo root): A CLI wrapper providing the following binaries:

      • ypi
      • rlm_query
      • rlm_cost
      • rlm_parse_json
      • rlm_sessions
      • rlm_cleanup

    2. pi-recursive (located in ./pi-recursive): A pure Pi extension providing the pi.extensions surface. It does not provide a CLI binary. Its contents (like extensions/, SYSTEM_PROMPT.md, and LICENSE) are gitignored build artifacts that must be staged via make build-pi-recursive before publishing.

  3. Understand the difference between pi-recursive and ypi

    master

    It is important to distinguish between the pure extension and the CLI wrapper:

    • pi-recursive (Pure Extension): Provides a native rlm_query tool directly inside Pi. It does not include shell helpers, launchers, or the jj requirement. It is intended for use within the Pi environment.
    • ypi (CLI Wrapper): A convenience layer around this extension. It provides shell-compatible rlm_query (supporting pipes and async), cost/session helpers (like rlm_cost), and CLI ergonomics. These features are opt-in via YPI_SHELL_HELPER=1.

  4. How ypi recursion works

    master

    ypi implements a recursive agent pattern where a parent Pi process can spawn child Pi processes using the rlm_query tool.

    Recursion Depth:

    • Depth 0 (root): Full Pi with native rlm_query and bash tools.
    • Depth 1 (child): Full Pi with native rlm_query and bash tools.
    • Depth 2 (leaf): Full Pi with bash tools, but rlm_query is disabled (max depth).

    File Isolation with jj: If jj (Jujutsu) is installed and RLM_JJ is not 0, recursive children use jj workspaces for safe file isolation. Without jj, children default to read-only tools in the current checkout. To allow writable children without jj, set RLM_UNSAFE_NO_JJ_WRITE=1 (use with caution).

  5. Supported languages and auto-detection

    master

    The extension automatically detects language servers based on project markers and installed binaries. If a server is not detected, ensure the corresponding binary is installed and the root marker is present in your project directory.

    | Language | Server | Root markers | Install |
|----------|--------|-------------|---------|
| Rust | `rust-analyzer` | `Cargo.toml` | `rustup component add rust-analyzer` |
| TypeScript/JS | `typescript-language-server` | `package.json`, `tsconfig.json` | `npm i -g typescript-language-server typescript` |
| Go | `gopls` | `go.mod` | `go install golang.org/x/tools/gopls@latest` |
| Python | `pylsp` | `pyproject.toml`, `setup.py` | `pip install python-lsp-server` |
| Zig | `zls` | `build.zig` | [zigtools/zls](https://github.com/zigtools/zls) |
| C/C++ | `clangd` | `compile_commands.json` | Package manager |
| Lua | `lua-language-server` | `.luarc.json` | [LuaLS](https://github.com/LuaLS/lua-language-server) |

  6. Version strategy for ypi and pi-recursive

    master

    The project follows semver for versioning. Because the packages are shipped from one source, the version field in both package.json (root) and pi-recursive/package.json must be identical.

    Semver rules applied:

    • patch (e.g., 0.1.0 → 0.1.1): Bug fixes and documentation updates.
    • minor (e.g., 0.1.x → 0.2.0): New features, new environment variables, or new guardrails.
    • major (e.g., 0.x → 1.0): Breaking changes to CLI arguments, environment variables, or the rlm_query interface.

    Consistency Requirement: You must ensure the pinned Pi version is consistent across:

    • package.json → dependencies["@earendil-works/pi-coding-agent"]
    • pi-recursive/package.json → peerDependencies["@earendil-works/pi-coding-agent"]
    • .pi-version

  7. How pi-lsp behaves with ypi / recursive agents

    master
    When used with ypi (recursive Pi), the extension optimizes resource usage by checking the RLM_DEPTH environment variable. The lsp tool is only registered at depth 0. Child agents skip the extension to prevent wasted tokens and unnecessary server spawns.

  8. Project Structure and Components of ypi

    master

    The ypi project is structured around a thin launcher and a core recursive extension. Understanding these components helps in knowing where logic resides:

    • ypi: The thin launcher that sets environment variables and loads extensions.
    • rlm_query: An optional shell-compatible command used for recursive sub-calls.
    • extensions/recursive.ts: The canonical ypi Pi extension that enables recursion.
    • extensions/ypi/: Contains native modules for the tool, environment, prompts, and status.
    • extensions/ypi.ts: A compatibility alias for recursive.ts.
    • SYSTEM_PROMPT.md: Contains the instructions that teach the LLM how to be recursive and edit code.
    • pi-mono/: A Git submodule containing the upstream Pi coding agent.

  9. Run tests for ypi

    master

    The project uses a Makefile to manage various testing suites. Use these commands to verify your changes:

    • make test-fast: Runs unit tests and guardrail tests (no LLM, fast).
    • make test-extensions: Tests compatibility between the latest Pi and the extension, including minimal mode.
    • make pre-push-checks: Runs shared local/CI gates (recommended before pushing).
    • make test-e2e: Runs real LLM calls (slow, incurs costs).
    • make test-recursion-e2e: Verifies that ypi correctly invokes rlm_query.
    • make test-extension-recursion-e2e: Verifies direct pi -e native tool recursion.
    • make test-parity-e2e: Verifies parity between the wrapper and direct extension.
    • make test: Runs all available tests.
    make test-fast
make test-e2e

  10. Install ypi

    master

    You can install the ypi CLI tool using bun, npm, a shell script, or by manual cloning. ypi provides a preconfigured recursive coding agent experience with a launcher and shell-compatible helpers.

    # bun (global)
bun install -g ypi

# or npm (global)
npm install -g ypi

# or run without installing
bunx ypi "What does this repo do?"

# or curl
curl -fsSL https://raw.githubusercontent.com/rawwerks/ypi/master/install.sh | bash

# or manual
git clone https://github.com/rawwerks/ypi.git && cd ypi
git submodule update --init --depth 1
export PATH="$PWD:$PATH"

  11. Perform recursive file edits with jj workspaces

    master

    For complex repository restructuring or multi-file refactors, using jj (Jujutsu) is strongly encouraged.

    • Isolation: In a jj workspace, child agents spawned via rlm_query operate in isolated workspaces. Edits do not affect the parent worktree until they are explicitly absorbed.
    • Detection: Check if you are in a jj workspace using jj root.
    • Safety: If jj is unavailable, sub-agents run in the current checkout. By default, they are read-only unless the environment variable RLM_UNSAFE_NO_JJ_WRITE=1 is set.
    • Action: Always write files directly (using write, sed, or redirection) rather than just describing changes.
    # Detect jj workspace
jj root 2>/dev/null && echo "jj workspace detected"

# Example: Multi-file refactor using async loop
grep -rl "old_api_call" src/ | while read f; do
    rlm_query --async "In $f, replace all old_api_call() with new_api_call(). Update the imports."
done

  12. Install the pi-lsp extension

    master

    To install the pi-lsp extension, clone the repository into Pi's extensions directory and install its dependencies using bun. Pi will automatically discover the extension in ~/.pi/agent/extensions/.

    # Copy to Pi's extensions directory
git clone https://github.com/rawwerks/pi-lsp.git ~/.pi/agent/extensions/lsp
cd ~/.pi/agent/extensions/lsp && bun install