DOCSEARCHbyAlgolia
Submit Docs

MLEvolve Documentation

repository·main·Indexed 18 days ago

https://github.com/internscience/mlevolve

An agentic machine learning engineering system designed to automatically solve Kaggle-style ML competitions. MLEvolve utilizes Monte Carlo Graph Search (MCGS), multi-agent collaboration, and experience-driven memory with BM25 and FAISS retrieval to discover and evolve ML algorithms.

Tokens
946
Snippets
3
Records
4
Agent score
14%

What's inside MLEvolve

  1. How MLEvolve works: Core Concepts

    main

    MLEvolve is an agentic Machine Learning Engineering (MLE) system that uses Monte Carlo Graph Search (MCGS) and multi-agent collaboration to solve ML competitions. Its core technical pillars include:

    • Multi-Mode Planning & Code Generation: The system adaptively switches between single-shot planning and memory-enhanced (two-stage RAG) planning. It supports various code generation strategies like single-pass, stepwise multi-agent pipelines, and incremental SEARCH/REPLACE diff patching.
    • Experience-Driven Memory: A global memory layer stores plans, code, metrics, and success/failure labels. It uses a combination of BM25 + FAISS retrieval to allow agents to reinforce successful strategies and avoid past mistakes.
    • Progressive MCGS with Cross-Branch Fusion: The search process uses an extended UCT (Upper Confidence Bound applied to Trees) algorithm with piecewise exploration decay and stagnation detection. When progress stalls, the system performs cross-branch fusion, merging insights from different high-performing solution branches to propose new candidates.

  2. Run a single task with MLEvolve

    main

    You can execute a single task using the run_single_task.sh script. This script requires an experiment ID and the dataset directory. An optional server ID can also be provided.

    Usage: bash run_single_task.sh <EXP_ID> <DATASET_DIR> [SERVER_ID]

    Output: Results, including search tree logs, the best solution code, and top-K candidate submissions, are saved to ./runs/<timestamp>_<exp_id>/.

    # Example: Running a task with ID 'denoising-dirty-documents' using dataset at '/mle-bench/data' on server 1
bash run_single_task.sh denoising-dirty-documents /mle-bench/data 1

  3. Install MLEvolve dependencies

    main

    To install the necessary dependencies for MLEvolve, use pip with the --no-deps flag on the provided requirement files. This ensures you manage dependencies according to the project's specific environment needs.

    Run the following commands in order:

    pip install --no-deps -r requirements_base.txt
pip install --no-deps -r requirements_ml.txt
pip install --no-deps -r requirements_domain.txt

  4. Configure MLEvolve via config.yaml

    main

    MLEvolve requires configuration through config/config.yaml. You must provide the path to your mle-bench dataset and your LLM API credentials.

    Required Fields:

    • dataset_dir: The absolute path to your mle-bench data directory.
    • agent.code.base_url: The endpoint for your code generation LLM (supports OpenAI-compatible APIs like GPT, Qwen, DeepSeek, etc.).
    • agent.code.api_key: Your API key for the code generation LLM.
    • agent.feedback.base_url: The endpoint for the feedback LLM.
    • agent.feedback.api_key: Your API key for the feedback LLM.

    Optional/Advanced Configuration:

    • If agent.use_global_memory is set to True, you must specify agent.memory_embedding_model_path (a HuggingFace model name or local path) and agent.memory_embedding_device (e.g., cpu or cuda).
    • To disable the optional cold-start model feature, set coldstart.use_coldstart: False.
    dataset_dir: "/path/to/mle-bench/data"

agent:
  code:
    base_url: "https://your-gemini-endpoint"
    api_key: "your-api-key"
  feedback:
    base_url: "https://your-gemini-endpoint"
    api_key: "your-api-key"