AgentFlow

Rollout trajectories are saved during training for analysis. These are organized by public IP and experiment name/timestamp. Note that step numbers restart from 0 whenever training is restarted.

Directory Structure:

rollout_data/
└── {PUBLIC_IP}/
    └── {EXPERIMENT_NAME}_{TIMESTAMP}/
        ├── .init.lock
        ├── .run_info
        └── {MODEL_NAME}_{TIMESTAMP}/
            ├── train/
            └── validation/
                ├── .val.lock
                └── step_0/
                    ├── idx_0/
                    │   └── rollout_{uuid}.json
                    └── ...

Rollout JSON Schema: Each rollout_{uuid}.json file contains the following fields:

• prompt: The original problem or query.
• groundtruth: The expected answer.
• answer_extracted: The model's predicted answer.
• reward: The score (0.0 for incorrect, positive value for correct).
• total_result: A full execution trace containing:
  • query_analysis: Problem breakdown.
  • memory: Tool execution history.
  • direct_output: The final response.
  • Tool prompts and responses.
• timestamp: The time of generation.

Each benchmark task's run.sh script (e.g., test/bamboogle/run.sh) can be customized by editing several key variables:

1. Set Parallelism

Adjust THREADS to control data parallelism for inference. Setting this too high may exceed API rate limits.

THREADS=20  # Number of parallel workers

2. Select Tasks

Enable or disable specific benchmarks by commenting or uncommenting them in the TASKS array:

TASKS=(
    "aime24"
    "gameof24"
    "bamboogle"
    # "gpqa"
)

3. Define Models

Specify the models and their configurations using the MODELS array. Each entry follows a specific pipe-separated format.

# Example configuration in test/bamboogle/run.sh
TASK="bamboogle"
THREADS=20
DATA_FILE_NAME="data.json"

MODELS=(
    "8000:vllm-AgentFlow/agentflow-planner-7b,AgentFlow-7B,\nBase_Generator_Tool|Python_Coder_Tool|Google_Search_Tool|Wikipedia_Search_Tool,\ngpt-4o-mini|gpt-4o-mini|Default|Default,\ntrainable|gpt-4o|gpt-4o|gpt-4o"
)

Model checkpoints are stored in a hierarchical directory structure based on your config.yaml settings. Checkpoints include both the model weights in HuggingFace format and the training state.

Directory Structure:

checkpoints/
└── {PROJECT_NAME}/
    └── {EXPERIMENT_NAME}/
        ├── global_step_2/
        │   ├── actor/
        │   │   └── huggingface/  # HuggingFace format (ready for inference)
        │   └── data.pt           # Training state
        ├── global_step_4/
        └── latest_checkpointed_iteration.txt  # Points to latest checkpoint

Configuration (train/config.yaml):

• trainer.save_freq: Interval for saving checkpoints (default: every 2 epochs).
• trainer.test_freq: Interval for validation (default: every 2 epochs).
• trainer.total_epochs: Total training epochs (default: 5).

How to use checkpoints:

• For VLLM inference: Configure the paths in scripts/serve_vllm.sh.
• For direct loading with Transformers: Use the path to the huggingface subdirectory within a specific global step.

AgentFlow allows you to customize the LLM engine for different agent modules. The configuration depends on which agent you are modifying:

Planner Agent

To change the model for the Planner, modify the llm_engine_name parameter in the corresponding shell script used to run the agent (for example, test/bamboogle/run.sh).

Other Agents (Executor, Verifier, Generator)

By default, these agents use a fixed engine (Qwen-2.5-7B-Instruct via DashScope). To use a custom model, you must modify the source code:

1. Modify the Planner's fixed engine: In agentflow/agentflow/models/planner.py, update self.llm_engine_fixed using create_llm_engine with your desired model_string.
2. Modify the Executor: In agentflow/agentflow/solver.py, update the llm_engine_name parameter during the Executor instantiation.

For a list of supported engines and the specific model_string formats required, refer to the llm_engine.md documentation.

# Example: Modifying the Planner's fixed engine in agentflow/agentflow/models/planner.py
self.llm_engine_fixed = create_llm_engine(model_string="your-engine", is_multimodal=False, temperature=temperature)

# Example: Modifying the Executor in agentflow/agentflow/solver.py
executor = Executor(
    llm_engine_name="dashscope",
    root_cache_dir=root_cache_dir,
    verbose=verbose,
    temperature=temperature
)