DOCSEARCHbyAlgolia
Submit Docs

Hermes Agent Self-Evolution

repository·main·Indexed 26 days ago

https://github.com/nousresearch/hermes-agent-self-evolution

A framework for the automatic optimization of Hermes Agent's skills, prompts, tool descriptions, and code. It utilizes DSPy, GEPA (Genetic-Pareto Prompt Evolution), and a Darwinian Evolver to perform reflective evolutionary search via API calls. The system optimizes four tiers of targets: skill files (SKILL.md), tool descriptions, system prompt components, and tool implementation code, employing automated guardrails such as full pytest suites and size limits to ensure stability.

Tokens
4K
Snippets
12
Records
21
Agent score
40%

What's inside hermes-agent-self-evolution

  1. Understand the Hermes Agent Self-Evolution Architecture

    main

    Hermes Agent Self-Evolution is a standalone optimization pipeline designed to improve the hermes-agent performance by evolving skills, prompts, tool descriptions, and code. It operates as an external process that reads from the hermes-agent repository and outputs improvements via Git branches and Pull Requests (PRs).

    Optimization Engines

    • DSPy + GEPA: The primary engine used for evolving skills, prompts, instructions, and tool descriptions. It uses reflective evolution by reading execution traces to understand failure modes.
    • DSPy MIPROv2: A fallback Bayesian optimizer used for optimizing few-shot examples and instruction text.
    • Darwinian Evolver: An external CLI-based engine used specifically for evolving code files and tool implementations.

    Note: No GPU training is required. All optimizations are performed via LLM API calls to mutate text (prompts, instructions, code) rather than model weights.

  2. Install and Setup Hermes Agent Self-Evolution

    main

    To use the self-evolution pipeline, clone the repository and install it in editable mode with development dependencies. You must also point the tool to your existing hermes-agent installation using the HERMES_AGENT_REPO environment variable.

    # Clone and install
git clone https://github.com/NousResearch/hermes-agent-self-evolution.git
cd hermes-agent-self-evolution
pip install -e ".[dev]"

# Point at your hermes-agent repo
export HERMES_AGENT_REPO=~/.hermes/hermes-agent
    git clone https://github.com/NousResearch/hermes-agent-self-evolution.git
cd hermes-agent-self-evolution
pip install -e ".[dev]"

# Point at your hermes-agent repo (auto-detected from ~/.hermes/hermes-agent or env var)
export HERMES_AGENT_REPO=~/.hermes/hermes-agent

  3. Optimize tool descriptions for better selection accuracy

    main

    Phase 2 focuses on optimizing the natural language descriptions in tool schemas to improve tool selection reliability.

    Target Fields

    Tool descriptions are located in tools/*.py files and include:

    • Top-level description field (usage guidance)
    • Per-parameter description fields
    • Specific constants like TERMINAL_TOOL_DESCRIPTION

    Constraints

    • Max Length: Tool descriptions must be $\le$ 500 characters; parameter descriptions must be $\le$ 200 characters.
    • Schema Integrity: The schema structure (parameter names, types, required fields) is frozen and cannot be changed.
    • Accuracy: Descriptions must remain factually accurate.

    Evaluation Strategy

    Optimization is performed using GEPA to mutate descriptions based on a dataset of (task_description, correct_tool, correct_params) triples. Evaluation must be cross-tool (evaluating all descriptions simultaneously) to prevent one tool's description from "stealing" selection accuracy from another.

  5. Enforce Constraints and Guardrails for Evolved Variants

    main

    Every evolved variant (skill text, tool description, or code) must pass these strict constraints to be considered valid. Failure in any constraint results in immediate rejection.

    1. Functional Correctness

    All variants must pass the full test suite with zero tolerance for failure:

    python -m pytest tests/ -q

    2. Size Budgets (Character/Token Limits)

    To prevent context bloat and cost increases, the optimizer applies a length penalty. Limits are:

    • Skill files (SKILL.md): Default 15KB (configurable per skill).
    • Tool descriptions: 500 characters maximum.
    • System prompt sections: Must not exceed current section size by more than 20%.

    3. Deployment and Caching Rules

    • No Hot-Swapping: Evolved content is never injected into active conversations. Changes only take effect at the start of a new session.
    • Tool Schemas: You may evolve the description text, but the schema structure (parameter names, types) must not change.
    • Deployment: All changes must be deployed via Pull Request (PR), never via direct commit.

  6. Optimize agent skills via DSPy and GEPA

    main

    Skill evolution (Phase 1) uses DSPy and GEPA to optimize SKILL.md files. The process involves wrapping a skill as a DSPy module, generating an evaluation dataset, and running the GEPA optimization runner.

    Evaluation Dataset Sources

    • Synthetic Generation: Use a strong model (e.g., Claude Opus) to generate 15-30 (task_input, expected_behavior) pairs from the SKILL.md file. Split into 10 train / 5 val / 5-10 holdout.
    • SessionDB Mining: Extract tasks and responses from real usage where the skill was loaded. Use LLM-as-judge to score pairs; high-scoring pairs are "good" examples, low-scoring are failure cases.
    • Hand-curated Golden Sets: Manually written test cases stored in JSONL format at ~/.hermes/evolution/datasets/<skill-name>/golden.jsonl.
    • Skill-specific Auto-evaluation: Automated checks (e.g., planting a bug for systematic-debugging or searching for known papers for arxiv).

    Scoring Mechanism

    Uses an LLM-as-judge with skill-specific rubrics. Common metrics include:

    • Procedure adherence (0-1)
    • Output correctness/usefulness (0-1)
    • Conciseness/token budget adherence (0-1)

  7. Evolve tool implementation code via Darwinian Evolver

    main

    Phase 4 uses the Darwinian Evolver (an external CLI) to evolve the actual Python source code in tools/*.py files to fix bugs and improve performance.

    Safety Guardrails and Constraints

    Code evolution is high-risk and subject to strict requirements:

    • Test Pass Rate: The full pytest suite (2550+ tests) must pass with 100% success; failures result in immediate rejection.
    • Frozen API: No changes are allowed to function signatures or registry.register() calls.
    • Safety: No removal of error handling or existing safety checks.
    • Human Review: All code mutations require manual human review before merging.

    Fitness Function Components

    • pytest results (hard gate)
    • Benchmark scores (e.g., TBLite pass rate)
    • Specific failure case resolution (verifying a bug fix via reproduction scripts)
    • Code quality heuristics

  9. Understand the Benchmark Gate System

    main

    The optimization pipeline uses three specific benchmarks as regression gates to ensure that improvements in one area do not degrade overall performance. Benchmarks are used as gates, while task-specific datasets serve as the actual fitness functions.

    Optimization Flow

    1. pytest: Must pass 100% (Functional Correctness Gate).
    2. TBLite fast subset (20 tasks): Quick capability check (~20 min).
    3. Task-specific eval dataset: Determines the fitness score for the specific skill/tool/prompt.
    4. Full TBLite (100 tasks): Thorough regression check for top candidates.
    5. YC-Bench fast_test: Coherence check for multi-turn behavior.

    Benchmark Reference

    BenchmarkPurposeSpeedCost
    TBLiteCoding/sysadmin regression gate~1-2 hours~$20-50
    TerminalBench2Thorough validation (Docker sandboxes)~2-4 hours~$50-200
    YC-BenchLong-horizon strategic coherence check~3-6 hours~$50-200

  10. Evolve a skill using synthetic evaluation data

    main

    Use the evolution.skills.evolve_skill module to evolve a specific skill. When using --eval-source synthetic, the system generates its own evaluation dataset to drive the optimization process.

    python -m evolution.skills.evolve_skill \
    --skill github-code-review \
    --iterations 10 \
    --eval-source synthetic

  11. Deploy Evolved Changes via Pull Request

    main

    All evolutionary improvements must follow a specific Git workflow to ensure lineage and metric transparency. The PR body must include before/after scores on training, validation, and holdout sets, the full diff, and the optimization cost.

    Workflow Example

    # 1. Create a new branch for the evolution run
git checkout -b evolve/<target>-<timestamp>

# 2. Apply the evolved changes and commit with detailed metrics
git add <files>
git commit -m "evolve: <target> — score improved X% → Y%\n\nOptimizer: GEPA (N iterations, M candidates evaluated)\nEval dataset: <dataset name> (K examples)\nBefore: <baseline score>\nAfter: <evolved score>\nHoldout: <holdout score>"

# 3. Push and create the PR
git push -u origin evolve/<target>-<timestamp>
gh pr create --title "evolve: <target>" --body "<metrics, diff, comparison>"
    git checkout -b evolve/<target>-<timestamp>
# Apply evolved changes
git add <files>
git commit -m "evolve: <target> — score improved X% → Y%

Optimizer: GEPA (N iterations, M candidates evaluated)
Eval dataset: <dataset name> (K examples)
Before: <baseline score>
After: <evolved score>
Holdout: <holdout score>"
git push -u origin evolve/<target>-<timestamp>
gh pr create --title "evolve: <target>" --body "<metrics, diff, comparison>"

  12. Evolve a skill using real session history

    main

    To optimize a skill based on actual usage patterns, use --eval-source sessiondb. This utilizes real session history from sources like Claude Code, Copilot, and Hermes.

    python -m evolution.skills.evolve_skill \
    --skill github-code-review \
    --iterations 10 \
    --eval-source sessiondb