DOCSEARCHbyAlgolia
Submit Docs

GEPA (Genetic-Pareto) Framework

repository·main·Indexed 26 days ago

https://github.com/gepa-ai/gepa

A framework for optimizing textual system components—including AI prompts, code snippets, and agent architectures—using LLM-based reflection and Pareto-efficient evolutionary search. GEPA diagnoses execution traces (errors, logs) to evolve parameters, making it suitable for scenarios with expensive rollouts, scarce data, or API-only models. It includes specialized adapters for DSPy, LangChain, RAG, and the Model Context Protocol (MCP), and provides a dedicated `optimize_anything` API for non-prompt text artifacts.

Tokens
70K
Snippets
142
Records
333
Agent score
91%

What's inside gepa

  1. Overview of GEPA

    main

    GEPA is an optimization framework designed to automatically optimize any text-based component of an AI system, including prompts, code, agent architectures, and configurations. It utilizes LLM-based reflection and Pareto-efficient evolutionary search to achieve high performance.

    Key capabilities include:

    • Automated Prompt Optimization: Optimizing prompts for any AI system.
    • Broad Optimization Scope: Beyond prompts, it can optimize code and agent architectures.
    • Efficiency: Claims to be up to 90x cheaper and 35x faster than Reinforcement Learning (RL) methods.

  2. Understand GEPA Core Concepts

    main

    GEPA (Genetic-Pareto) is a text evolution engine designed to optimize any artifact representable as text, including prompts, code, agent architectures, configurations, and policies.

    Unlike traditional optimization methods (like RL or Bayesian optimization) that rely on numerical rewards, GEPA uses Actionable Side Information (ASI). ASI is diagnostic, domain-specific textual feedback (e.g., error messages, reasoning logs, profiling traces) that an LLM uses to reason about why a candidate failed and how to fix it.

    The Three-Stage Pipeline

    Each iteration follows this flow:

    1. Executor: Runs the candidate on a minibatch of tasks and captures full execution traces (reasoning, errors, metrics, and ASI).
    2. Reflector: A strong LLM (reflection_lm) analyzes traces to diagnose failure modes and causal patterns.
    3. Curator: Generates an improved candidate based on the reflector's diagnostic insights.

    Candidate Generation Strategies

    GEPA uses two adaptive strategies:

    • Reflective Mutation: Samples one candidate from the Pareto frontier and proposes an improved version via reflection.
    • System-Aware Merge: Samples two candidates and strategically combines their modules based on evolution history to create hybrid candidates.

  3. Identify use cases for GEPA

    main

    GEPA is optimized for scenarios where traditional Reinforcement Learning (RL) is impractical. Use GEPA when:

    • Expensive rollouts are required: For scientific simulations, complex agents with tool calls, or slow compilation processes where you only have 100–500 evaluations available (compared to 10K+ required for RL).
    • Data is scarce: When you have very few examples (as few as 3).
    • Using API-only models: When you do not have access to model weights and need to optimize models like GPT-5, Claude, or Gemini via their APIs.
    • Interpretability is needed: When you require human-readable optimization traces to understand why prompts are changing.
    • Complementing RL: You can use GEPA for rapid initial optimization before applying RL or fine-tuning for further gains.

  4. Understand Fast-Slow Training (FST) for LLMs

    main

    Fast-Slow Training (FST) is a training blueprint that interleaves Reinforcement Learning (RL) for model parameters (slow weights) with prompt optimization using GEPA (fast weights).

    In this approach:

    • Slow Loop (RL): Updates model parameters $\theta$ based on scalar rewards to capture broadly useful reasoning strategies.
    • Fast Loop (GEPA): Updates the context/prompts $\Phi$ using reflective optimization and rich text feedback (thoughts, tool calls, errors).

    By allowing the prompt to absorb task-specific information, the model weights can focus on general reasoning, leading to better data efficiency, higher performance ceilings, and maintained plasticity (the ability to learn new tasks without catastrophic forgetting).

  5. Use available GEPA adapters

    main

    GEPA provides several built-in adapters to plug into existing frameworks:

    • DSPy Adapter: Integrates GEPA into DSPy to optimize the signature instructions of any DSPy module.
    • Default Adapter: Integrates GEPA into a single-turn LLM environment. It optimizes the system prompt where the task is a user message and the answer is in the assistant response.
    • AnyMaths Adapter: Integrates GEPA with litellm and ollama specifically for solving single-turn mathematical problems.
    • LangChain Adapter: Integrates GEPA with LangChain. It supports any chat model via init_chat_model, tool-using agents (create_agent), LangGraph graphs, and RAG pipelines.

  6. Understand ConfidenceAdapter scoring mechanisms

    main

    The ConfidenceAdapter improves optimization through two primary mechanisms:

    1. Continuous Scoring via LinearBlendScoring: Instead of binary correct/incorrect signals, it provides a gradient that allows the optimizer to distinguish between "confidently correct" and "barely correct" predictions. This penalizes lucky guesses and rewards prompts that produce high-certainty predictions.

    2. Rich, Tiered Feedback: It provides detailed feedback (including logprobs and top alternatives) to the reflection LLM. This enables the generation of targeted disambiguation rules rather than generic prompt corrections, especially for high-conviction errors.

  7. Understand the gskill pipeline for automated skill learning

    main

    gskill is a fully automated pipeline designed to learn repository-specific skills for coding agents. It uses SWE-smith to generate verifiable software engineering tasks from a GitHub repository and GEPA's optimize_anything API to iteratively evolve these skills through an optimization loop.

    The gskill Workflow:

    1. Task Generation: SWE-smith converts a target GitHub repository into an active training environment by generating diverse, verifiable tasks with associated tests.
    2. Optimization Loop: Using optimize_anything, the pipeline starts with an initial (possibly empty) set of skills.
    3. Evaluation: The agent performs rollouts using the current skills, and the results are evaluated for fitness.
    4. Reflective Proposal: A more powerful LLM reflects on the evaluation results and feedback to propose updated candidate skills.
    5. Selection: The best candidates are selected into a pool, and the process repeats until convergence.

    Benefits:

    • Transferability: Skills learned on smaller, cheaper models (e.g., gpt-5-mini) can be transferred to production-grade agents like Claude Code.
    • Efficiency: Learned skills can reduce task duration and cost by helping agents navigate repositories more effectively.
    • Language Agnostic: The pipeline can generate skills for various languages (e.g., Python, Go).

  8. Understand GEPA Core Architecture

    main

    The GEPA engine consists of three primary components that interface with your system:

    • Adapter: The bridge to your system. It runs candidates against evaluation tasks, captures traces, and returns scores and Actionable Side Information (ASI). Built-in adapters exist for DSPy, standalone LLM calls, and the optimize_anything API.
    • Proposer: Implements the three-stage pipeline (Executor $\rightarrow$ Reflector $\rightarrow$ Curator) and the dual mutation/merge strategies.
    • Pareto Tracker: Maintains a frontier of candidates that excel on different subsets of tasks or objectives. Instead of a single rank, it preserves candidates that perform well on different specific objectives (Pareto-efficiency).

  9. Evaluate learned skills with Mini-SWE-agent and Claude Code

    main

    After training, you can evaluate the best_skills.txt output using the following commands:

    Mini-SWE-agent

    Evaluates both with-skills and without-skills conditions:

    python -m gepa.gskill.gskill.evaluate.mini_swe_agent \
  --config gepa_results/logs/run_xxx/config.json \
  --workers 16

    Claude Code

    Baseline (no skills):

    python -m gepa.gskill.gskill.evaluate.claude_code \
  --config gepa_results/logs/run_xxx/config.json \
  --model haiku --workers 4

    With skills (injects best_skills.txt as CLAUDE.md):

    python -m gepa.gskill.gskill.evaluate.claude_code \
  --config gepa_results/logs/run_xxx/config.json \
  --model haiku --workers 4 --use-skills

    With Claude Code Skills (uses .claude/skills/<repo>/SKILL.md):

    python -m gepa.gskill.gskill.evaluate.claude_code_skills \
  --config gepa_results/logs/run_xxx/config.json \
  --model sonnet --workers 4 --use-skills

  10. Optimize arbitrary artifacts with optimize_anything

    main

    The optimize_anything API allows you to optimize any text artifact (code, agent architectures, configurations, etc.). You must provide an evaluator function that returns a score.

    For richer optimization, the evaluator can return a (score, side_info_dict) tuple. Use gepa.optimize_anything.log() to log diagnostic information (Actionable Side Information) that the reflection model can use to improve the candidate.

    import gepa.optimize_anything as oa
from gepa.optimize_anything import optimize_anything, GEPAConfig, EngineConfig

def evaluate(candidate: str) -> tuple[float, dict]:
    """Score a candidate and return score with diagnostic side info."""
    result = run_my_system(candidate)
    # Use oa.log to provide diagnostic feedback (ASI)
    oa.log(f"Output: {result.output}")
    oa.log(f"Error: {result.error}")
    return result.score, {
        "Error": result.stderr,
        "Output": result.stdout,
    }

result = optimize_anything(
    seed_candidate="<your initial artifact>",
    evaluator=evaluate,
    objective="Describe what you want to optimize for.",
    config=GEPAConfig(engine=EngineConfig(max_metric_calls=100)),
)

print("Best candidate:", result.best_candidate)

  11. Use ConfidenceAdapter for classification optimization

    main

    The ConfidenceAdapter is designed for classification tasks where the LLM returns structured JSON with enum-constrained fields. It uses token-level log-probabilities to distinguish between genuine understanding and 'lucky guesses'.

    Prerequisites:

    • The LLM must support token-level logprobs (e.g., OpenAI gpt-4.1, gpt-4.1-mini or Google Gemini gemini-2.5-flash).
    • The response_format must use enum constraints to allow the adapter to measure confidence over the allowed categories.
    import gepa
from gepa.adapters.confidence_adapter import ConfidenceAdapter

adapter = ConfidenceAdapter(
    model="openai/gpt-4.1-mini",
    field_path="category_name",
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "classification",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {
                    "category_name": {
                        "type": "string",
                        "enum": [
                            "Bills/Electricity",
                            "Bills/Gas & Oil",
                            "Food & Drinks/Restaurants",
                            "Shopping/Electronics",
                            "Shopping/Video Games",
                        ],
                    }
                },
                "required": ["category_name"],
                "additionalProperties": False,
            },
        },
    },
)

result = gepa.optimize(
    seed_candidate={"system_prompt": "Classify the following transaction."},
    trainset=[
        {"input": "UBER EATS payment", "answer": "Food & Drinks/Restaurants", "additional_context": {}},
        {"input": "LIGHT electricity bill", "answer": "Bills/Electricity", "additional_context": {}},
        {"input": "Steam purchase", "answer": "Shopping/Video Games", "additional_context": {}},
    ],
    adapter=adapter,
    reflection_lm="openai/gpt-4.1",
    max_metric_calls=500,
)

  12. Install gskill for repository-specific skill learning

    main

    To use gskill for learning repository-specific skills for coding agents, install the gskill extra for gepa along with the required dependencies:

    pip install gepa[gskill]
pip install mini-swe-agent swebench

    Before running, ensure Docker is running and set your OPENAI_API_KEY. You must also download the SWE-smith images for your target repository:

    export OPENAI_API_KEY=<your-key>

# Verify Docker is running
docker ps

# Download SWE-smith images
python -m swesmith.build_repo.download_images
    pip install gepa[gskill]
pip install mini-swe-agent swebench