Agent Laboratory

To improve research outcomes, you can provide specific instructions to agents by modifying the task_notes_LLM structure within ai_lab_repo.py. This allows you to define requirements for specific research phases, such as 'plan formulation', 'data preparation', or 'running experiments'.

You can use these notes to:

• Specify the number of experiments.
• Define preferred LLM models for sub-tasks (e.g., using gpt-4o-mini for experiments).
• Provide specific code snippets for inference.
• Inform agents about available hardware resources (e.g., GPU types, CPU cores, or using mps on MacBook).
• Set constraints on dataset sizes or design preferences for figures.

task_notes_LLM = [
    {"phases": ["plan formulation"],
     "note": f"You should come up with a plan for TWO experiments."},

    {"phases": ["plan formulation", "data preparation",  "running experiments"],
     "note": "Please use gpt-4o-mini for your experiments."},

    {"phases": ["running experiments"],
     "note": f"Use the following code to inference gpt-4o-mini: \nfrom openai import OpenAI\nos.environ[\"OPENAI_API_KEY\"] = \"{api_key}\"\nclient = OpenAI()\ncompletion = client.chat.completions.create(\nmodel=\"gpt-4o-mini-2024-07-18\", messages=messages)\nanswer = completion.choices[0].message.content\n"},

    {"phases": ["running experiments"],
     "note": f"You have access to only gpt-4o-mini using the OpenAI API, please use the following key {api_key} but do not use too many inferences. Do not use openai.ChatCompletion.create or any openai==0.28 commands. Instead use the provided inference code."},

    {"phases": ["running experiments"],
     "note": "I would recommend using a small dataset (approximately only 100 data points) to run experiments in order to save time. Do not use much more than this unless you have to or are running the final tests."},

    {"phases": ["data preparation", "running experiments"],
     "note": "You are running on a MacBook laptop. You can use 'mps' with PyTorch"},

    {"phases": ["data preparation", "running experiments"],
     "note": "Generate figures with very colorful and artistic design."},
    ]

To provide specific instructions to agents (e.g., experiment details, hardware constraints, or preferred coding styles), modify the task_notes_LLM list within ai_lab_repo.py. Each note can be scoped to specific research phases like plan formulation, data preparation, or running experiments.

Example configuration for specific phases:

task_notes_LLM = [
    {"phases": ["plan formulation"],
     "note": f"You should come up with a plan for TWO experiments."},

    {"phases": ["running experiments"],
     "note": "I would recommend using a small dataset (approximately only 100 data points) to run experiments in order to save time."},

    {"phases": ["data preparation", "running experiments"],
     "note": "You are running on a MacBook laptop. You can use 'mps' with PyTorch"},
]

task_notes_LLM = [
    {"phases": ["plan formulation"],
     "note": f"You should come up with a plan for TWO experiments."},

    {"phases": ["plan formulation", "data preparation",  "running experiments"],
     "note": "Please use gpt-4o-mini for your experiments."},

    {"phases": ["running experiments"],
     "note": f"Use the following code to inference gpt-4o-mini: \nfrom openai import OpenAI\nos.environ[\"OPENAI_API_KEY\"] = \"{api_key}\"\nclient = OpenAI()\ncompletion = client.chat.completions.create(\nmodel=\"gpt-4o-mini-2024-07-18\", messages=messages)\nanswer = completion.choices[0].message.content\n"},

    {"phases": ["running experiments"],
     "note": f"You have access to only gpt-4o-mini using the OpenAI API, please use the following key {api_key} but do not use too many inferences. Do not use openai.ChatCompletion.create or any openai==0.28 commands. Instead use the provided inference code."},

    {"phases": ["running experiments"],
     "note": "I would recommend using a small dataset (approximately only 100 data points) to run experiments in order to save time. Do not use much more than this unless you have to or are running the final tests."},

    {"phases": ["data preparation", "running experiments"],
     "note": "You are running on a MacBook laptop. You can use 'mps' with PyTorch"},

    {"phases": ["data preparation", "running experiments"],
     "note": "Generate figures with very colorful and artistic design."},
    ]

To improve research outcomes, you can provide detailed instructions to the agents by modifying the task_notes_LLM structure within ai_lab_repo.py. These notes allow you to specify:

• Project goals and stylistic preferences.
• Specific experiments to perform.
• API keys for required services.
• Hardware and compute resource availability (e.g., GPU types, CPU cores, storage limits, or using 'mps' with PyTorch on MacBook).
• Preferred models for specific phases.

Each note is associated with specific research phases (e.g., plan formulation, data preparation, running experiments).

task_notes_LLM = [
    {"phases": ["plan formulation"],
     "note": f"You should come up with a plan for TWO experiments."},

    {"phases": ["plan formulation", "data preparation",  "running experiments"],
     "note": "Please use gpt-4o-mini for your experiments."},

    {"phases": ["running experiments"],
     "note": f"Use the following code to inference gpt-4o-mini: \nfrom openai import OpenAI\nos.environ[\"OPENAI_API_KEY\"] = \"{api_key}\"\nclient = OpenAI()\ncompletion = client.chat.completions.create(\nmodel=\"gpt-4o-mini-2024-07-18\", messages=messages)\nanswer = completion.choices[0].message.content\n"},

    {"phases": ["running experiments"],
     "note": f"You have access to only gpt-4o-mini using the OpenAI API, please use the following key {api_key} but do not use too many inferences. Do not use openai.ChatCompletion.create or any openai==0.28 commands. Instead use the provided inference code."},

    {"phases": ["running experiments"],
     "note": "I would recommend using a small dataset (approximately only 100 data points) to run experiments in order to save time. Do not use much more than this unless you have to or are running the final tests."},

    {"phases": ["data preparation", "running experiments"],
     "note": "You are running on a MacBook laptop. You can use 'mps' with PyTorch"},

    {"phases": ["data preparation", "running experiments"],
     "note": "Generate figures with very colorful and artistic design."},
    ]

To improve research outcomes, you can provide the agent with specific instructions, style preferences, API keys, or hardware specifications (GPU/CPU/Memory). This is done by modifying the task_notes_LLM list within the ai_lab_repo.py file. Each note is associated with specific research phases (e.g., plan formulation, data preparation, running experiments).

task_notes_LLM = [
    {"phases": ["plan formulation"],
     "note": f"You should come up with a plan for TWO experiments."},

    {"phases": ["plan formulation", "data preparation",  "running experiments"],
     "note": "Please use gpt-4o-mini for your experiments."},

    {"phases": ["running experiments"],
     "note": f"Use the following code to inference gpt-4o-mini: \nfrom openai import OpenAI\nos.environ[\"OPENAI_API_KEY\"] = \"{api_key}\"\nclient = OpenAI()\ncompletion = client.chat.completions.create(\nmodel=\"gpt-4o-mini-2024-07-18\", messages=messages)\nanswer = completion.choices[0].message.content\n"},

    {"phases": ["running experiments"],
     "note": f"You have access to only gpt-4o-mini using the OpenAI API, please use the following key {api_key} but do not use too many inferences. Do not use openai.ChatCompletion.create or any openai==0.28 commands. Instead use the provided inference code."},

    {"phases": ["running experiments"],
     "note": "I would recommend using a small dataset (approximately only 100 data points) to run experiments in order to save time. Do not use much more than this unless you have to or are running the final tests."},

    {"phases": ["data preparation", "running experiments"],
     "note": "You are running on a MacBook laptop. You can use 'mps' with PyTorch"},

    {"phases": ["data preparation", "running experiments"],
     "note": "Generate figures with very colorful and artistic design."},
    ]

Follow these steps to set up Agent Laboratory using a Python virtual environment (recommended Python 3.12):

1. Clone the repository:

git clone git@github.com:SamuelSchmidgall/AgentLaboratory.git

2. Set up and activate the Python environment:

python -m venv venv_agent_lab
source venv_agent_lab/bin/activate

3. Install dependencies:

pip install -r requirements.txt

4. Install pdflatex (Optional): To enable agents to compile LaTeX source, install pdflatex via sudo apt install pdflatex. If you do not have sudo access, disable LaTeX compilation by passing the --compile-latex "false" flag when running the application.

# Installation sequence
git clone git@github.com:SamuelSchmidgall/AgentLaboratory.git
python -m venv venv_agent_lab
source venv_agent_lab/bin/activate
pip install -r requirements.txt
# Optional: sudo apt install pdflatex
python ai_lab_repo.py --yaml-location "experiment_configs/MATH_agentlab.yaml"