AI-Researcher

Use the Research Agent to generate novel research ideas from a list of reference papers and then conduct experiments (Level 2 tasks). This requires setting up environment variables for Docker workplace, base images, and LLM models.

Execute the command found in research_agent/run_infer_level_2.sh to run the run_infer_idea.py script.

current_dir=$(dirname "$(readlink -f "$0")")
cd $current_dir
export DOCKER_WORKPLACE_NAME=workplace_paper

export BASE_IMAGES=tjbtech1/paperagent:latest

export COMPLETION_MODEL=claude-3-5-sonnet-20241022
export CHEEP_MODEL=claude-3-5-haiku-20241022

category=vq
instance_id=one_layer_vq
export GPUS='"device=0,1"'

python run_infer_idea.py --instance_path ../benchmark/final/${category}/${instance_id}.json --container_name paper_eval --model $COMPLETION_MODEL --workplace_name workplace --cache_path cache --port 12372 --max_iter_times 0 --category ${category}

Follow these three steps to generate the innovation benchmark dataset:

1. Prepare search terms: Create a list of paper titles or keywords in the papers_to_search file.
2. Collect papers: Run python 0_crawl_paper.py to collect papers and their related metadata.
3. Generate dataset: Run python 1_create_inno_graph.py to produce the final innovation dataset.

Important Configuration Notes:

• Keyword Search: To collect papers based on keywords rather than exact titles, set exact_match to False in 0_crawl_paper.py.
• OpenAI API Key: You must set your OpenAI key globally in your environment or edit utils/openai_utils.py directly.

Output Locations:

• The final results are stored at innovation_graph/innovation_graph_final.json.
• Note that the specific dataset used in the project's evaluation is merged_papers_with_fields.json, which includes manual revisions.

Use the Research Agent to conduct extensive surveys and experiments based on a provided idea (Level 1 tasks). This requires setting up environment variables for Docker workplace, base images, and LLM models (Completion and Cheap models).

Execute the command found in research_agent/run_infer_level_1.sh to run the run_infer_plan.py script.

current_dir=$(dirname "$(readlink -f "$0")")
cd $current_dir
export DOCKER_WORKPLACE_NAME=workplace_paper

export BASE_IMAGES=tjbtech1/paperagent:latest

export COMPLETION_MODEL=claude-3-5-sonnet-20241022
export CHEEP_MODEL=claude-3-5-haiku-20241022

category=vq
instance_id=one_layer_vq
export GPUS='"device=0,1"'

python run_infer_plan.py --instance_path ../benchmark/final/${category}/${instance_id}.json --container_name paper_eval --task_level task1 --model $COMPLETION_MODEL --workplace_name workplace --cache_path cache --port 12372 --max_iter_times 0 --category ${category}

AI-Researcher requires configuration via an environment variable file (created from .env.template). The configuration is divided into container, LLM, and task settings.

Container Configuration

• DOCKER_WORKPLACE_NAME: Name of the research agent's workplace.
• BASE_IMAGES: Base image for the agent (e.g., tjbtech1/airesearcher:v1).
• COMPLETION_MODEL: Primary completion model name (compatible with LiteLLM).
• CHEEP_MODEL: Secondary/cheaper model name.
• GPUS: GPU allocation (e.g., '"device=0"', '"device=0,1"', '"all"', or None).
• CONTAINER_NAME: Name of the container.
• WORKPLACE_NAME: Name of the workplace.
• CACHE_PATH: Path for caching.
• PORT: Port for the research agent.
• PLATFORM: Platform (e.g., linux/amd64).

LLM Configuration

• GITHUB_AI_TOKEN: GitHub AI token for the agent.
• OPENROUTER_API_KEY: OpenRouter API key.
• OPENROUTER_API_BASE: OpenRouter API base URL.

Task Configuration

• CATEGORY: Research category (e.g., diffu_flow, gnn, reasoning, recommendation, vq).
• INSTANCE_ID: Instance ID for the research task.
• TASK_LEVEL: Task level (task1 or task2).
• MAX_ITER_TIMES: Maximum number of research iterations.