DOCSEARCHbyAlgolia
Submit Docs

Nano Agent

repository·main·Indexed 42 days ago

https://github.com/disler/nano-agent

An MCP (Model Context Protocol) server for autonomous engineering agents, designed to test and compare the agentic capabilities of various LLMs across performance, speed, and cost. It supports OpenAI, Anthropic, and local Ollama models, utilizing a nested agent architecture and a HOP/LOP (Higher Order Prompt/Lower Order Prompt) pattern for systematic model evaluation.

Tokens
33.7K
Snippets
90
Records
114
Agent score
52%

What's inside Nano Agent

  1. How the HOP/LOP pattern works for model evaluation

    main

    The Higher Order Prompt (HOP) and Lower Order Prompt (LOP) pattern is used for systematic, parallel evaluation of multiple models.

    • HOP (Higher Order Prompt): The orchestrator. It reads test files, delegates tasks to multiple agents in parallel, and grades the results.
    • LOP (Lower Order Prompt): The test definition. These files contain the specific prompt, expected outputs, and grading rubrics.

    Execution Flow:

    1. HOP reads the LOP file.
    2. HOP extracts the prompt and the list of target agents.
    3. Agents are executed in parallel via the nano-agent MCP server.
    4. Results are collected and graded based on Performance, Speed, and Cost.
    5. A comparison table is generated showing the ranked results.
    /perf:hop_evaluate_nano_agents .claude/commands/perf/lop_eval_3__file_operations_test.md

  2. Core Concepts of Model Context Protocol (MCP)

    main

    The Model Context Protocol (MCP) is an open standard for communication between AI applications and data sources. An MCP server acts as a bridge by exposing three primary abstractions:

    • Tools: Executable functions that can have side effects (e.g., performing a calculation or updating a database).
    • Resources: Read-only data endpoints that provide context to the LLM (e.g., configuration files or documentation).
    • Prompts: Reusable templates designed to guide LLM interactions.

    Communication occurs over a Transport layer, which can include stdio, SSE, or Streamable HTTP.

  3. Key differences and limitations of Anthropic OpenAI compatibility

    main

    When using this compatibility layer, be aware of the following behaviors:

    • System Messages: System messages are hoisted and concatenated at the beginning of the message list.
    • Function Calling: The strict parameter for function calling is ignored.
    • Unsupported Features: Audio input and prompt caching (via OpenAI SDK) are not supported. For prompt caching, use the native Anthropic SDK.
    • Rate Limits: Limits follow Anthropic's standard limits, not OpenAI's.
    • Production Warning: This layer is intended for testing and comparison, not for production use. For full feature support (PDFs, citations, prompt caching), use the native Anthropic SDK.

  4. How the HOP/LOP Multi-Model Evaluation System works

    main

    The nano-agent uses a hierarchical HOP/LOP (Higher Order Prompt / Lower Order Prompt) pattern to orchestrate parallel model testing and benchmarking. This system ensures a fair comparison by using the same execution environment (OpenAI Agent SDK) across different providers.

    The Hierarchy:

    1. Higher Order Prompt (HOP): Orchestrates the entire evaluation process. It accepts test case files as $ARGUMENTS, formats/grades results, and generates performance comparison tables.

      • Example file location: .claude/commands/perf/hop_evaluate_nano_agents.md

    2. Lower Order Prompt (LOP): Defines the specific test cases. It lists the agents to be tested (using @agent-nano-agent-* references), specifies expected outputs, and provides grading rubrics.

      • Example file location: .claude/commands/perf/lop_eval_*.md

    3. Claude Code Sub-Agents: Individual agent configurations that specify a model + provider combination. These are color-coded in Claude Code for visual scanning:

      • Green: GPT-5 series (nano, mini, standard)
      • Blue: GPT-OSS series (20b, 120b)
      • Purple: Claude 4 Opus models
      • Orange: Claude 4 Sonnet & Claude 3 Haiku
      • Example file location: .claude/agents/nano-agent-*.md

    4. Nano-Agent MCP Server: The execution layer. It provides the prompt_nano_agent(prompt, model, provider) function, which creates an isolated agent instance per request using the OpenAI Agent SDK to ensure consistent metrics (time, tokens, and costs) across all providers.

  5. Best practices for engineering agent implementation

    main

    When building or configuring long-running engineering agents, follow these best practices for reliability and observability:

    1. System Prompt Design

    Ensure prompts include a clear mission statement, explicit workflow steps, error recovery procedures, quality standards, and defined completion criteria.

    2. Managing Long-Running Tasks

    • Use high max_turns (e.g., 50-100).
    • Implement retry logic and session management for continuity.
    • Monitor progress and handle errors gracefully.

    3. Monitoring & Observability

    Implement comprehensive logging, execution metrics, tool usage tracking, and result persistence. Integration with OpenAI traces is recommended.

    4. Streaming & UI

    For better user experience, implement real-time progress updates, token-by-token streaming, tool execution visibility, and handoff notifications.

    5. Production Safety

    Use input validation with guardrails, safety checks, resource limits (timeouts), and robust configuration management.

  6. Core Architecture and Primitives of OpenAI Agents SDK

    main

    The SDK is built around several key primitives designed for building production-ready agentic applications:

    • Agents: LLMs equipped with specific instructions, tools, and context.
    • Handoffs: The mechanism used to delegate tasks between specialized agents.
    • Guardrails: Systems for input and output validation.
    • Sessions: Objects that manage automatic conversation history.
    • Streaming: Enables real-time monitoring of agent progress.

  7. Configure OpenAI Models

    main

    The SDK provides two primary model types for OpenAI:

    1. OpenAIResponsesModel (Recommended): The default model type. It uses the new Responses API and supports advanced features like structured outputs, multimodal input, and hosted file/web search.
    2. OpenAIChatCompletionsModel: A fallback that uses the Chat Completions API. Use this for broader compatibility with OpenAI-compatible endpoints that do not support the Responses API.

  8. Use Agent Handoffs for Multi-Agent Orchestration

    main

    Handoffs allow an orchestrator agent to delegate specific parts of a task to specialized agents. This is achieved by:

    1. Creating specialized Agent instances (e.g., DebugSpecialist, TestSpecialist).
    2. Defining a handoff_description for each specialist to tell the orchestrator when to use them.
    3. Adding these agents to the handoffs list of the orchestrator agent.

    When the orchestrator decides a specialist is needed, it transfers control to that agent. The specialist can then use its own unique set of tools and instructions.

    # Specialist Agent
debug_agent = Agent(
    name="DebugSpecialist",
    instructions="Instructions for debugging...",
    tools=[read_file, write_file],
    handoff_description="Debugging complex issues",
)

# Orchestrator Agent
orchestrator = Agent(
    name="Orchestrator",
    instructions="Coordinate work and delegate to specialists.",
    handoffs=[debug_agent],
)

  9. Understand the Nano Agent nested hierarchy

    main

    Nano Agent operates using a two-layer nested agent architecture. This design allows an outer agent (like Claude Code or any MCP client) to interact with a highly capable inner agent through a single interface.

    1. Outer Agent (MCP Client): Communicates via the MCP protocol. It only sees one tool: prompt_nano_agent. It sends natural language prompts to the Nano Agent server.
    2. Nano Agent MCP Server: Receives prompts from the outer agent and spawns an internal agent to handle the request.
    3. Inner Agent (OpenAI GPT with function calling): A fresh agent instance created for every prompt_nano_agent call. It possesses its own set of autonomous tools that are not visible to the outer agent, including:
      • read_file: Read file contents
      • list_directory: List directory contents
      • write_file: Create/overwrite files
      • get_file_info: Get file metadata

    The inner agent runs an autonomous loop (up to 20 turns) and returns the final result back to the outer agent via the MCP server.

  10. How lifecycle hooks work in OpenAI Agent SDK

    main

    The OpenAI Agent SDK provides two types of lifecycle hooks to monitor agent execution events:

    1. RunHooks: Used with Runner.run() to monitor the entire execution lifecycle, including agent start/end, tool execution, and handoffs between agents.
    2. AgentHooks: Set directly on a specific Agent instance via agent.hooks to monitor that specific agent's individual lifecycle.

    All hook methods must be defined as async def.

  12. Interact with Nano Agent via Claude Code (MCP)

    main

    Once configured in your .mcp.json, you can interact with Nano Agent within Claude Code using two patterns:

    1. Direct MCP Calls: Use the mcp nano-agent: prompt_nano_agent command to send specific instructions to the agent with a chosen model.
    2. Sub-agent Pattern: Use the @agent- prefix followed by the specific model name to delegate tasks to a nano-agent instance. This allows you to switch between different LLM providers (GPT, Claude, or local OSS models) seamlessly within your conversation.
    # Direct Call
mcp nano-agent: prompt_nano_agent "Create a hello world script in python" --model gpt-5

# Sub-agent Delegation
@agent-nano-agent-gpt-5-mini "Create a hello world script in python"
@agent-nano-agent-gpt-oss-20b "<insert agentic prompt here>"