DOCSEARCHbyAlgolia
Submit Docs

ReTool Documentation

repository·main·Indexed 17 days ago

https://github.com/retool-rl/retool

A tool-augmented reinforcement learning framework designed to train LLMs to use external computational tools strategically during reasoning tasks. ReTool focuses on improving accuracy and convergence speed in mathematical reasoning and provides pre-trained weights for ReTool-Qwen-32B and ReTool-DeepSeek-R1-Distill-Qwen-32B.

Tokens
418
Snippets
2
Records
3
Agent score
14%

What's inside ReTool

  1. Run model inference and evaluation

    main

    ReTool uses the STILL3 inference framework for evaluating trained checkpoints. To run the quick start evaluation, navigate to the evaluation directory and execute the evaluation script.

    cd evaluation
bash scripts/eval.sh

  2. Install the ReTool environment

    main

    To set up the environment for ReTool, install the required dependencies including vllm, flash-attn, deepspeed, and symeval. Note that flash-attn should be installed with the --no-build-isolation flag.

    pip install vllm==0.7.3
pip install packaging
pip install ninja
pip install flash-attn --no-build-isolation
pip install deepspeed
pip install accelerate
pip install datasets
pip install "git+https://github.com/tongyx361/symeval.git"
pip install timeout_decorator

  3. Access ReTool model weights and datasets

    main

    ReTool provides pre-trained model weights and datasets on Hugging Face for both SFT (Supervised Fine-Tuning) and RL (Reinforcement Learning) workflows.

    Model Weights

    • ReTool-Qwen-32B: Based on Qwen2.5-32B-Instruct (achieves 67% on AIME 24).
    • ReTool-DeepSeek-R1-Distill-Qwen-32B: Based on DeepSeek-R1-Distill-Qwen-32B (achieves 72% on AIME 24).

    Datasets

    • Cold-Start SFT: JoeYing/ReTool-SFT on Hugging Face.
    • RL Training: BytedTsinghua-SIA/DAPO-Math-17k on Hugging Face.
    • RL Validation: AIME 2024 and AIME 2025 datasets, located in the evaluation/dataset directory of this repository.