DOCSEARCHbyAlgolia
Submit Docs

little-coder

repository·main·Indexed 23 days ago

https://github.com/itayinbarr/little-coder

A coding agent optimized for small local language models, built as an extension-heavy layer on top of the pi agent substrate. It implements scaffold-model-fit adaptations as pi extensions and provides specialized tools, skills, and research modes for software engineering tasks. It supports local providers including llama.cpp, Ollama, and LM Studio, as well as cloud providers like Anthropic and OpenAI.

Tokens
25.5K
Snippets
47
Records
123
Agent score
82%

What's inside little-coder

  1. Assess Deep Research production-readiness

    main

    The Deep Research feature has been evaluated through two rounds of testing (R1 and R2) to measure the impact of grounding refinements. The current verdict is READY-WITH-CAVEATS.

    While the refinement successfully eliminated the primary failure mode of fabricated sources (improving from 0.0 to ~27.5 unique sources per report), users should be aware of the following residual risks:

    • Marginal Fabrication: A 35B local model may still produce plausible-looking but unverifiable specifics, such as future-dated CVE IDs or arXiv IDs. Exact identifiers, versions, and figures should always be verified.
    • Latency: If subagents hang, the watchdog mechanism will kill them, which can increase total execution time (up to ~54 minutes in serial mode, though parallel mode in production mitigates this).
    • Report Thinness: If research subagents are killed by the watchdog, the resulting report may be thinner than expected due to missing information slices.
    • Judge Unreliability: Do not rely on local 35B models to automatically gate quality via a 'judge' metric, as they can be unreliable and may penalize honest reports that flag information gaps.

  2. Overview of little-coder tool providers and subsystems

    main

    Tools in little-coder are organized into functional groups and register themselves into the tool_registry.py at import time.

    Tool Groups:

    • Core Tools (tools.py): Read, Write, Edit, Bash, Glob, Grep, WebFetch, WebSearch, NotebookEdit, GetDiagnostics, SleepTimer.
    • Memory Tools (memory/tools.py): MemorySave, MemoryDelete, MemorySearch, MemoryList.
    • Multi-Agent Tools (multi_agent/tools.py): Agent, CheckAgentResult, ListAgentTasks.
    • Skill Tools (skill/tools.py): Skill, SkillList.
    • Task Tools (task/tools.py): TaskCreate, TaskUpdate, TaskGet, TaskList.
    • Dynamic Tools: Provided via MCP (mcp/tools.py) or plugins (plugin/loader.py).

    Key Subsystems:

    • Memory: Persistent file-based memory using an index and per-entry markdown files.
    • Multi-Agent: A threaded manager for running sub-agents.
    • Skill: A markdown-based skill loader and executor (includes /commit and /review).
    • MCP: A Model Context Protocol client using user and project configurations.
    • Checkpoint: Provides file-snapshot hooks and the ability to rewind state.

  3. Understand the time economics of passing vs failing exercises

    main
    Failing exercises consume significantly more wall-clock time than passing ones. On average, failing exercises take 2.8x longer to process. This is attributed to failing exercises exhausting the full turn budget (averaging 19.0 turns), whereas passing exercises tend to converge earlier (averaging 11.6 turns).

  4. How the second-attempt retry (pass_2) mechanism works

    main
    The agent includes a built-in retry mechanism for failed attempts. If the first implementation attempt (pass_1) fails, the agent receives the test output as context and is granted a second chance (pass_2) to fix the specific failures. This mechanism is particularly effective in languages with rich test-output feedback, such as Go and JavaScript.

  5. How little-coder extensions work

    main

    The core philosophy of little-coder is that it is a minimal base. Every specialized mechanism is a pi extension that hooks into specific lifecycle events, such as:

    • before_agent_start
    • context
    • before_provider_request
    • tool_call
    • tool_result
    • turn_end
    • session_compact

    Managing Extensions

    • Discovery: The launcher automatically discovers extensions located in .pi/extensions/*/index.ts.
    • Loading: Extensions are loaded explicitly. You can run pi with --no-extensions to disable them all.
    • Adding/Removing:
      • To remove an extension, delete its directory in .pi/extensions/.
      • To add a custom extension, drop its directory next to the existing ones or use the -e <path> flag at launch.

  6. Understand the little-coder scaffold performance gap

    main

    Benchmark results comparing little-coder and vanilla Aider using the same model (ollama/qwen3.5) and test protocol show that the little-coder scaffold significantly outperforms Aider.

    Key Performance Metrics:

    • Pass Rate: little-coder achieves ~45% pass rate, while vanilla Aider achieves ~19.11%.
    • Throughput: little-coder produces ~1.72× more passes per wall-clock hour compared to Aider.
    • Scaffold Signal: The performance gap is most evident when comparing 'Phase 1' (exercises the model is capable of solving) vs 'Phase 2' (exercises the model fails). little-coder passes significantly more exercises that the model is demonstrably capable of solving, proving the advantage is due to scaffold architecture rather than model weights.

  7. How little-coder relates to pi

    main

    little-coder is built on top of pi, which provides the agent loop, TUI, and toolset.

    Key distinction: little-coder does not shadow the pi CLI. It runs pi with the --no-extensions flag and manually wires in its own bundled set of ~30 extensions and skill markdown files. This ensures a predictable, low-token context (~7k tokens) that is not affected by globally installed pi extensions.

    Loading custom extensions: To use extensions outside of the bundled set, you can:

    1. Drop them in ~/.config/little-coder/extensions/.
    2. Set the LITTLE_CODER_EXTRA_EXTENSIONS environment variable to point to specific files.
    3. Relaunch with the --with-pi-extensions flag to allow pi to discover its own registered extensions.

  8. Analyze the impact of the retry mechanism on pass rates

    main
    The benchmark analysis shows that the second-attempt retry mechanism significantly improves overall pass rates across all languages. While the total mean pass rate is 45.6%, the rate without the retry mechanism (first-attempt only) drops to 37.6%, representing an 8 percentage point gain. The effectiveness of retry varies by language, with C++ seeing the highest gain (+13.5 pp) and Java seeing the lowest (+5.3 pp).

  9. Understand the little-coder architecture (v0.0.x Python implementation)

    main

    The v0.0.x architecture is a Python-based CLI designed for LLM coding agents. It uses a multi-turn agent loop (agent.py) that manages tool use, permission gates, and quality gating. The system is modular, consisting of a core REPL, a system prompt builder (context.py), and various specialized sub-packages for memory, multi-agent coordination, and skill injection.

    Core Components:

    • Agent Loop (agent.py): Orchestrates the multi-turn interaction, manages providers (Anthropic, OpenAI, Ollama), and handles context window management via compaction.py.
    • Context Builder (context.py): Constructs the system prompt by aggregating base instructions, Git context, persistent memory, CLAUDE.md, skills, knowledge, and MCP tools.
    • Tool Registry (tool_registry.py): A central registry where all tools (core, memory, multi-agent, skill, task, MCP, and plugin-provided) are registered.
    • Local Preprocessing Pipeline (local/): A specialized pipeline for small-model optimization, including skill/knowledge augmentation, prompt compression, and response quality detection.

    Note on Versions: This architecture describes the Python implementation. Version v0.1.0+ moved the agent to the pi platform using TypeScript extensions, though the core behavioral invariants (like the Write-vs-Edit invariant and compaction) are preserved.

  10. Evaluate exercise determinism and flakiness

    main

    Exercises in the benchmark are categorized by their outcome stability across runs:

    • Deterministic (79.1%): Produce the same outcome regardless of sampling randomness.
    • Flaky (20.9%): Produce different outcomes (pass vs fail) due to temperature-0.3 variance.
    • Always pass: Both runs succeeded.
    • Always fail: Both runs failed.

    JavaScript and Go exhibit the highest flaky rates (29% and 26% respectively), indicating the agent operates near the pass/fail boundary in these languages.

  11. How bounded thinking with reasoning reuse works

    main
    The system manages the model's reasoning process using a thinking-budget cap. When the model's reasoning token stream exceeds a budget of 2,048 tokens, the generation is aborted. To prevent loss of progress, the partial reasoning generated up to that point is reinjected into the assistant context, and the request is retried with thinking disabled. This ensures the model commits to an implementation without exhausting the context window or crowding out implementation tokens with excessive reasoning.

  12. When to use Write vs Edit

    main

    To avoid tool errors and optimize token usage, follow these selection rules:

    Use Write when:

    • The file does not exist yet and you are creating it from scratch.

    Use Edit when:

    • Modifying existing files: Any change (bug fixes, refactors, formatting, adding functions, renaming variables) must use Edit.
    • Iterating after failures: If a test fails, do not re-write the whole file; use Edit to patch the specific error.
    • Replacing content: If you need to completely overwrite an existing file, use Edit by passing the entire current content as oldText and the new content as newText. (Note: You should Read the file first to ensure you have the correct oldText).