DOCSEARCHbyAlgolia
Submit Docs

AI Scientist-v2 Documentation

repository·main·Indexed 27 days ago

https://github.com/sakanaai/ai-scientist-v2

An autonomous scientific research system by Sakana AI that utilizes agentic tree search to generate hypotheses, conduct experiments, analyze data, and write scientific manuscripts. Designed for open-ended exploration in Machine Learning domains, it supports integration with OpenAI, Gemini, and Claude (via AWS Bedrock) models, and utilizes the Semantic Scholar API for literature search and novelty checking.

Tokens
1.8K
Snippets
4
Records
9
Agent score
42%

What's inside AI Scientist-v2

  2. Run AI Scientist-v2 Paper Generation Experiments

    main

    Launch the main pipeline using the JSON file generated during the ideation step. This uses agentic tree search to run experiments and generate a paper.

    Key Arguments:

    • --load_ideas: Path to the generated JSON idea file.
    • --load_code: Initialize experimentation with a code snippet (omit if not desired).
    • --add_dataset_ref: Adds dataset references.
    • --model_writeup: LLM for the write-up phase.
    • --model_citation: LLM for the citation phase.
    • --model_review: LLM for the review phase.
    • --model_agg_plots: LLM for aggregating plots.
    • --num_cite_rounds: Number of citation rounds.

    Configuration: Tree search parameters (like num_workers, steps, max_debug_depth, etc.) are managed in bfts_config.yaml.

    Outputs:

    • Logs & Visualization: Found in experiments/<timestamp_ideaname>/logs/0-run/unified_tree_viz.html.
    • Final Paper: A PDF file named <timestamp_ideaname>.pdf located in the experiment folder.
    python launch_scientist_bfts.py \
 --load_ideas "ai_scientist/ideas/my_research_topic.json" \
 --load_code \
 --add_dataset_ref \
 --model_writeup o1-preview-2024-09-12 \
 --model_citation gpt-4o-2024-11-20 \
 --model_review gpt-4o-2024-11-20 \
 --model_agg_plots o3-mini-2025-01-31 \
 --num_cite_rounds 20

  3. Generate Research Ideas with perform_ideation_temp_free.py

    main

    Before running experiments, use the ideation script to brainstorm and refine research ideas based on a topic description.

    1. Prepare a Topic Description: Create a Markdown file (e.g., my_research_topic.md) containing Title, Keywords, TL;DR, and Abstract. Place it in a directory like ai_scientist/ideas/.
    2. Run the Script: Execute ai_scientist/perform_ideation_temp_free.py with the following arguments:
      • --workshop-file: Path to your topic Markdown file.
      • --model: The LLM to use (e.g., gpt-4o-2024-05-13).
      • --max-num-generations: Number of distinct ideas to attempt.
      • --num-reflections: Number of refinement steps per idea.
    3. Output: A JSON file (e.g., ai_research_topic.json) containing structured hypotheses and proposed experiments.
    python ai_scientist/perform_ideation_temp_free.py \
 --workshop-file "ai_scientist/ideas/my_research_topic.md" \
 --model gpt-4o-2024-05-13 \
 --max-num-generations 20 \
 --num-reflections 5

  4. Install AI Scientist-v2

    main

    The AI Scientist-v2 is designed for Linux with NVIDIA GPUs using CUDA and PyTorch. Follow these steps to set up the environment:

    1. Create and activate a Conda environment with Python 3.11.
    2. Install PyTorch with CUDA support (ensure the pytorch-cuda version matches your system setup).
    3. Install PDF and LaTeX tools using poppler and chktex.
    4. Install Python dependencies via requirements.txt.

    Caution: This codebase executes LLM-written code. Run it within a controlled sandbox environment (e.g., a Docker container) to mitigate risks like uncontrolled web access or unintended processes.

    # Create a new conda environment
conda create -n ai_scientist python=3.11
conda activate ai_scientist

# Install PyTorch with CUDA support (adjust pytorch-cuda version for your setup)
conda install pytorch torchvision torchaudio pytorch-cuda=12.4 -c pytorch -c nvidia

# Install PDF and LaTeX tools
conda install anaconda::poppler
conda install conda-forge::chktex

# Install Python package requirements
pip install -r requirements.txt

  5. Configure API Keys and Supported Models

    main

    The system requires environment variables for the LLMs and services you intend to use:

    • OpenAI Models: Set OPENAI_API_KEY.
    • Gemini Models: Set GEMINI_API_KEY (accessed via OpenAI API).
    • Claude Models (via AWS Bedrock):
      1. Install the adapter: pip install anthropic[bedrock].
      2. Set AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_REGION_NAME.
    • Semantic Scholar (Literature Search): Set S2_API_KEY for higher throughput and better novelty checking. The system can function without it, but you may encounter rate limits.
    export OPENAI_API_KEY="YOUR_OPENAI_KEY_HERE"
export S2_API_KEY="YOUR_S2_KEY_HERE"
# Set AWS credentials if using Bedrock
# export AWS_ACCESS_KEY_ID="YOUR_AWS_ACCESS_KEY_ID"
# export AWS_SECRET_ACCESS_KEY="YOUR_AWS_SECRET_KEY"
# export AWS_REGION_NAME="your-aws-region"

  7. Run AI Scientist-v2 for custom subject fields

    main

    To run experiments on a specific topic or subject field, follow these steps:

    1. Create an Ideation Prompt: Create a new Markdown file describing your desired subject field or topic. Follow the structure of the example file: ai_scientist/ideas/i_cant_believe_its_not_better.md.
    2. Generate Ideas: Run the perform_ideation_temp_free.py script using your Markdown file as input to generate a JSON idea file.
    3. Launch Experiments: Use the launch_scientist_bfts.py script to run the paper generation experiments, passing the generated JSON file via the --load_ideas argument.

  8. Troubleshoot CUDA Out of Memory errors

    main

    If you encounter a CUDA Out of Memory error, it means the model being loaded or run exceeds your available GPU memory.

    Solution: Update your ideation prompt file (e.g., ai_scientist/ideas/my_research_topic.md) to explicitly instruct the agent to suggest using smaller models for the experiments.