DOCSEARCHbyAlgolia
Submit Docs

AIDE ML Documentation

repository·main·Indexed 23 days ago

https://github.com/wecoai/aideml

An LLM-driven agentic tree-search tool designed to autonomously write, debug, and optimize machine learning code based on user-defined goals and metrics. It provides a CLI, a Streamlit-based Web UI, and a Python API via the `aide.Experiment` class. AIDE ML is model-neutral, supporting providers like OpenAI, Anthropic, Gemini, and local LLMs via Ollama.

Tokens
2.2K
Snippets
7
Records
13
Agent score
81%

What's inside AIDE ML

  1. Run AIDE ML via Docker

    main

    You can run AIDE ML in a containerized environment. Ensure you mount volumes for logs and workspaces to persist data.

    docker build -t aide .
docker run -it --rm \
  -v "${LOGS_DIR:-$(pwd)/logs}:/app/logs" \
  -v "${WORKSPACE_BASE:-$(pwd)/workspaces}:/app/workspaces" \
  -v "$(pwd)/aide/example_tasks:/app/data" \
  -e OPENAI_API_KEY="your-actual-api-key" \
  aide data_dir=/app/data/house_prices goal="Predict price" eval="RMSE"

  2. Run AIDE ML with local LLMs (Ollama)

    main

    AIDE ML is model-neutral. You can use local LLMs by setting the OPENAI_BASE_URL environment variable to point to your local provider (e.g., Ollama).

    Note on Evaluators: By default, the agent uses gpt-4o as the evaluator. To run a fully local setup (both code generation and evaluation), you must explicitly set the agent.feedback.model flag.

    Example: Local Code Generation only

    export OPENAI_BASE_URL="http://localhost:11434/v1"
aide agent.code.model="qwen2.5" data_dir=… goal=… eval=…

    Example: Fully Local (Code + Evaluator)

    export OPENAI_BASE_URL="http://localhost:11434/v1"
aide agent.code.model="qwen2.5" agent.feedback.model="qwen2.5" data_dir=… goal=… eval=…

  3. Run an optimisation via CLI

    main

    Use the aide command to start an agentic tree search. You must provide a data_dir, a goal (in plain English), and an eval metric. Ensure your LLM API key is set in your environment variables.

    After the run, results are stored in logs/<id>/:

    • best_solution.py: The best code found.
    • tree_plot.html: An interactive HTML visualization of the solution tree.

  5. Use the AIDE ML Web UI

    main

    AIDE ML includes a Streamlit-based Web UI for prototyping ML solutions. You can upload data, set goals and metrics, and view live logs and the solution tree through a browser.

    To launch the UI:

    1. Install the package (includes streamlit).
    2. Navigate to the aide/webui directory.
    3. Run streamlit run app.py.
    pip install -U aideml
cd aide/webui
streamlit run app.py

  6. Visualize AIDE experiment results

    main

    Once an experiment completes, the WebUI provides several ways to inspect the results via tabs:

    • Tree Visualization: Renders an interactive HTML tree plot showing the decision paths taken by the agent.
    • Best Solution: Displays the Python code for the highest-performing model found during the run.
    • Config: Shows the OmegaConf YAML configuration used for the experiment.
    • Journal: Provides a JSON representation of the experiment journal, including step-by-step code, metrics, and bug status.
    • Validation Plot: Displays a Plotly line chart showing the progress of the validation score across steps.

  7. Configure API keys in the WebUI

    main

    The WebUI allows you to set LLM API keys through a sidebar interface. These keys are stored in the Streamlit session state and then applied to the system environment variables (os.environ) when an experiment starts.

    Supported keys include:

    • OPENAI_API_KEY
    • ANTHROPIC_API_KEY
    • GEMINI_API_KEY
    • OPENROUTER_API_KEY

    If a .env file is present, the WebUI will attempt to load these values automatically on initialization.

  8. Run an AIDE experiment via WebUI

    main

    To start an optimization run in the WebUI, follow these steps:

    1. Upload Data: Use the file uploader to provide .csv, .txt, .json, or .md files, or click "Load Example Experiment" to use pre-configured house price data.
    2. Define Goal: Enter a text description of the machine learning task (e.g., "Predict the sales price for each house").
    3. Set Evaluation Criteria: Define how the agent should measure success (e.g., "Use the RMSE metric...").
    4. Select Steps: Use the slider to choose the number of optimization steps (1-20).
    5. Execute: Click the "Run AIDE" button.

  9. Use the AIDE ML WebUI

    main

    The WebUI class provides a Streamlit-based graphical interface to interact with the AIDE Machine Learning Engineer Agent. It allows users to upload data files, define experiment goals and evaluation criteria, set API keys, and visualize the agent's progress and results through interactive tabs.

    To launch the interface, run the script directly using Python. The UI will handle environment variable loading (via .env) and session state management for running experiments.

    if __name__ == "__main__":
    app = WebUI()
    app.run()

  10. Example Task: Bitcoin Price Timeseries Forecasting

    main

    This example task demonstrates how to use AIDE ML to build a timeseries forecasting model for Bitcoin close prices. The objective is to predict future prices based on historical data.

    Evaluation Metric: The model's performance is measured using the Root-Mean-Squared-Error (RMSE) calculated between the logarithm of the predicted value and the logarithm of the observed price.

  11. Use AIDE ML inside Python

    main

    Integrate AIDE ML into your own Python scripts using the aide.Experiment class. This allows for programmatic control over the optimization process.

    1. Initialize aide.Experiment with data_dir, goal, and eval.
    2. Call .run(steps=N) to execute the search.
    3. Access the best_solution object to retrieve the valid_metric and the generated code.
    import aide
import logging

def main():
    logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')
    aide_logger = logging.getLogger("aide")
    aide_logger.setLevel(logging.INFO)
    print("Starting experiment...")
    exp = aide.Experiment(
        data_dir="example_tasks/bitcoin_price",  # replace this with your own directory
        goal="Build a time series forecasting model for bitcoin close price.",  # replace with your own goal description
        eval="RMSLE"  # replace with your own evaluation metric
    )

    best_solution = exp.run(steps=2)

    print(f"Best solution has validation metric: {best_solution.valid_metric}")
    print(f"Best solution code: {best_solution.code}")
    print("Experiment finished.")

if __name__ == '__main__':
    main()

  12. Configure AIDE ML via CLI flags

    main

    You can customize the agent's behavior using dot-notation flags in the CLI. Common configuration options include:

    FlagPurposeDefault
    agent.code.modelLLM used to write codegpt-4-turbo
    agent.stepsNumber of improvement iterations20
    agent.search.num_draftsNumber of drafts generated per step5

    Example: Running 50 steps with a specific model:

    aide agent.code.model="claude-4-sonnet" agent.steps=50 data_dir=... goal=... eval=...
    aide agent.code.model="claude-4-sonnet" \
     agent.steps=50 \
     data_dir=… goal=… eval=…