Ponytail Prompting Framework

repository·main·Indexed 13 days ago

https://github.com/dietrichgebert/ponytail

A prompting framework designed to optimize AI agents for 'lazy senior dev mode,' prioritizing minimal, efficient, and safe code. It utilizes a hierarchical decision ladder to favor native features, standard libraries, and existing codebase assets over over-engineered solutions. Ponytail includes an MCP server (ponytail-mcp) and supports integration with Claude Code, GitHub Copilot CLI, OpenCode, Gemini CLI, and Antigravity CLI.

Tokens
29.2K
Snippets
80
Records
125
Agent score
100%

What's inside Ponytail

  1. What is Ponytail

    main

    Ponytail is a tool designed to be integrated into AI agents to prevent over-engineering and reduce code verbosity. It follows a principle of 'lazy but not negligent' development, ensuring that agents write the minimum amount of code necessary to solve a task without sacrificing security, error handling, validation, or accessibility.

    Instead of installing new libraries or creating complex wrappers for common tasks, Ponytail encourages agents to use native browser features, standard libraries, or existing dependencies. For example, instead of installing a heavy date-picker library, Ponytail directs the agent to use a native <input type="date">.

  2. Understand the Agentic Safety Benchmark findings

    main

    The 2026-06-17 agentic safety benchmark compared ponytail against several other prompting strategies (baseline, caveman, YAGNI, and YAGNI-oneliner) using real headless Claude Code sessions.

    Key findings include:

    • Code-size gap: When compared against a fair agentic baseline (rather than a conversational model), ponytail provides a small code-size advantage (~4% reduction in mean source LOC) rather than the previously claimed 80-94%.
    • Safety Floor: Instructions to strictly minimize lines (e.g., yagni-oneliner) significantly drop safety. For example, a 5-LOC Python function might pass correctness on clean data but crash on malformed rows, whereas ponytail (8 LOC) maintains a 100% safety rate by including necessary error handling.
    • Over-engineering: On well-scoped tasks, current models (including ponytail) did not exhibit over-engineering behavior, scoring near 0.00 on an auditable LLM judge scale.
  3. Understand the Agentic Benchmark concept

    main

    The Agentic benchmark evaluates coding agents by running real, headless Claude Code sessions in temporary workspaces. Unlike single-shot benchmarks that measure one prompt to one completion, this benchmark simulates actual usage: an agent is given a seeded codebase and must edit existing files to fulfill tasks.

    Key differences from single-shot benchmarks:

    • Unit: A full Claude Code session in a temp workspace vs. a single prompt/completion.
    • Baseline: The 'real agent' (Claude Code) with no specific skill applied.
    • Task: Editing existing files (seeded stubs) vs. writing new code from scratch.
    • Safety: Explicitly measured by running produced code against adversarial input.
    • Over-engineering: Measured via source LOC and source file count (excluding tests).
    • Tests: Tracked as a positive signal (wrote_tests_rate) rather than being counted as bloat.
  4. How Ponytail's decision ladder works

    main

    Ponytail operates using a hierarchical decision-making process (a "ladder") that the agent follows after understanding the problem. The agent evaluates the task against these rungs in order, stopping at the first one that applies:

    1. Does this need to exist? → If no, skip it (YAGNI principle).
    2. Already in this codebase? → Reuse existing code instead of rewriting.
    3. Stdlib does it? → Use the standard library.
    4. Native platform feature? → Use native features (e.g., <input type="date"> instead of a heavy library).
    5. Installed dependency? → Use an existing dependency.
    6. One line? → If it can be done in one line, do it.
    7. Only then: Write the minimum code required to work.

    Important: The ladder is applied after the agent has read the relevant code and traced the data flow. It is designed to be "lazy about the solution, never about reading."

  5. How Ponytail agent portability works

    main

    Ponytail uses an adapter-based architecture to distribute skills across different AI agents.

    • Core Behavior: Stored in the skills/ directory. These files contain the actual logic and instructions for the skills.
    • Adapters: Host-specific files that map the core behavior to a specific agent's capabilities (e.g., plugins, hooks, or project instructions).

    When creating or using an adapter, follow the Adapter Rule: Keep adapters thin. If a host supports skills or hooks, point it directly at the existing skills/ and hooks/ files. If the host only supports project instructions, ensure the copied rule text remains aligned with the content in AGENTS.md.

  6. How 'The Ladder' works to find the simplest solution

    main

    When Ponytail is active, it attempts to stop at the first 'rung' of the ladder that satisfies the requirement. This mental model prevents unnecessary code by checking in this specific order:

    1. Existence Check (YAGNI): Does this task actually need to exist?
    2. Codebase Reuse: Is there an existing helper, util, or pattern in the current codebase?
    3. Standard Library: Can the standard library handle it?
    4. Native Platform Features: Can CSS, HTML5, or DB constraints solve it instead of JS/App code?
    5. Existing Dependencies: Can an already-installed library solve it?
    6. Minimization: Can it be written in one line?
    7. Minimum Viable Code: Only if the above fail, write the minimum code required to work.

    Note: You must understand the full context and trace the code flow before climbing the ladder. A small diff in the wrong place is not lazy; it is a bug.

  7. Understand the correctness gate behavior and fixes

    main

    The correctness gate (implemented in correctness.js) is used to validate model outputs. Historically, it had two major bugs that caused under-reporting of correctness, especially for terse models:

    1. Fenced Code Requirement: Previously, extractBlocks() only recognized code within

    fenced blocks. If a model provided bare code (common in terse styles), it was scored as a failure. The fix allows the gate to fall back to treating the entire response as a single code block if no fences are present. 2. Deliverable Mismatch (Debounce Task): The debounce task previously expected a reusable debounce(fn, delay) utility, but prompts often asked for inline implementations (e.g., using clearTimeout). This caused correct inline answers to fail. The task has been updated to explicitly ask for the reusable function the validator expects.

    Key Takeaway: Ponytail's tendency to produce shorter, more direct code (roughly halving median LOC) does not inherently degrade correctness on instruction-following models, provided the validator can handle unfenced code.

  8. When NOT to use Ponytail laziness

    main

    While Ponytail aims for minimalism, it must never compromise on critical software engineering principles. Do not simplify away:

    • Trust Boundaries: Input validation at boundaries.
    • Data Integrity: Error handling that prevents data loss.
    • Safety/Compliance: Security measures and accessibility basics.
    • Explicit Requests: If a user insists on a full implementation, build it.
    • Comprehension: Never skip reading or tracing the code to achieve a smaller diff.
    • Physical Realities: Do not simplify away calibration for hardware/sensors that drift.
    • Verification: Non-trivial logic (branches, loops, parsers) must include at least one small, runnable check (e.g., an assert-based demo() or a single test_*.py).
  9. Understand the Ponytail v4 hardening rules

    main

    Ponytail v4 introduces three specific rules to improve code quality, safety, and maintainability without increasing code bloat:

    1. Test reflex (5.1): For any non-trivial logic, the agent must provide exactly ONE runnable check. This can be an assert-based demo() or __main__ block, or a single small test_*.py file. Frameworks should be avoided; one-liners do not require a test.
    2. Ceiling comments (5.2): When using a ponytail: shortcut that has a known limitation (a "ceiling"), the comment must explicitly name the ceiling and describe the intended upgrade path.
    3. Robust variant rule (5.3): When presented with two standard library options of similar size, the agent must choose the one that is correct for edge cases.
  10. Understand the Ponytail Gain scoreboard metrics

    main

    The scoreboard provides a comparison between no-skill (baseline) and ponytail (optimized) across three key metrics based on published benchmark medians:

    • Lines of code: Shows the percentage of code reduction (typically 80–94% savings).
    • Cost: Shows the reduction in cost (typically 47–77% savings).
    • Speed: Shows the speedup factor (typically 3–6× faster).

    Important Honesty Boundary: The scoreboard displays global benchmark medians. It will never print a per-repo savings number (e.g., "you saved X lines here"). Because the unbuilt version of your specific code was never written, there is no real baseline to subtract from in a live repository. For actual per-repo figures, use /ponytail-debt.

  11. Understand the code reuse ladder rung (#217)

    main

    To prevent 'AI slop' (duplicated code), ponytail includes a specific ladder rung (rung 2) that enforces code reuse within the existing codebase.

    Directive: "Already in this codebase? Reuse it, don't re-write it."

    This encourages the agent to discover and use existing helpers (e.g., a slugify function in a separate module) rather than implementing a new, potentially divergent version (e.g., a hand-rolled regex) of the same logic.

  12. Understand the root-cause vs symptom patching directive (#245)

    main

    To prevent agents from performing 'dangerously lazy' patches (patching only the symptom mentioned in a bug report), ponytail uses an operational directive designed to favor root-cause fixes.

    Instead of just telling the agent to 'trace the flow', the directive instructs the agent to:

    1. Grep every caller of the function being touched.
    2. Fix the shared function once at the root cause.

    This is framed as the 'lazier' (smaller diff) approach to align with the agent's instinct for minimal changes, ensuring that sibling callers (which might not be named in the bug report) are also repaired.