R1-V Framework

To evaluate OOD counting performance on SuperCLEVR, download the required images, unzip them, and run the evaluation script. You can modify the script to test on your own datasets.

cd ./src/eval
wget https://www.cs.jhu.edu/~zhuowan/zhuowan/SuperCLEVR/to_be_released/images.zip
unzip images.zip

# change the model path in the script
python test_qwen2vl_counting_superclevr.py 

Perform Supervised Fine-Tuning (SFT) using accelerate launch. You must provide a configuration file and a specific SFT task config file.

accelerate launch --config_file src/r1-v/configs/zero2.yaml src/r1-v/src/open_r1/sft.py --config src/r1-v/configs/qwen2vl_sft_config.yaml 

Use query_r1.py to obtain predictions and reasoning traces from R1 by querying SiliconFlow.

Note on Output Formats:

• v1: Does not constrain output format, making answers difficult to parse.
• v2: Uses explicit prompting to ensure the model generates the answer using the marker **The answer is: ** for easier parsing.

<think>
Okay, let's see. The user is asking how many items are there in the described scene... [reasoning trace]
</think>

There are 7 items in the described scene. Each entry corresponds to one distinct object, listed by their properties, coordinates, and rotations.

To evaluate on the GEOQA test set (direct answer form), you must first clone the Geo170K dataset from Hugging Face and unzip the images before running the evaluation script. For faster inference using multiple GPUs, use the multi-GPU script provided in the repository.

# prepare images for testing
cd ./src/eval
git lfs install
git clone https://huggingface.co/datasets/Luckyjhg/Geo170K
cd Geo170K
unzip images.zip

# Evaluation Script
python test_qwen2vl_geoqa.py

# For multi-GPU inference:
bash src/scripts/test_grpo_geoqa_multigpu.sh

To set up the R1-V environment, create a new Conda environment with Python 3.11 and run the provided setup script. If you encounter issues, ensure your environment matches the specifications in ./src/requirements.txt.

conda create -n r1-v python=3.11 
conda activate r1-v

bash setup.sh

To generate reasoning datasets, create scene description strings by combining object metadata (location, depth, etc.) using a template. For each scene, generate counting-relevant questions and include a mandatory question: How many items are there in the described scene? to ensure all objects are counted.

Refer to generate_scene_qa_pairs.ipynb for the implementation details.

To use generated QA pairs with a Large Language Model (like DeepSeek-R1), you must format the query to include the scene description followed by the specific question. This provides the model with the necessary context to answer based on the text description rather than just the image.

def format_query(qa_dict):
    query = "Answer the question according to scene description.\n\n"
    query += qa_dict['description']
    query += f"\nQuestion:\n{qa_dict['q']}"
    return query

You can perform supervised fine-tuning for decoder language models using the src/open_r1/sft.py script via accelerate launch. This script supports vision-language models and requires specific arguments for the model path, dataset, and training hyperparameters.

Example command for a single node with 8 x H100 GPUs:

accelerate launch --config_file=configs/zero3.yaml src/open_r1/sft.py \
    --model_name_or_path Qwen/Qwen2.5-1.5B-Instruct \
    --dataset_name HuggingFaceH4/Bespoke-Stratos-17k \
    --learning_rate 2.0e-5 \
    --num_train_epochs 1 \
    --packing \
    --max_seq_length 4096 \
    --per_device_train_batch_size 4 \
    --gradient_accumulation_steps 4 \
    --gradient_checkpointing \
    --bf16 \
    --logging_steps 5 \
    --eval_strategy steps \
    --eval_steps 100 \
    --output_dir data/Qwen2.5-1.5B-Open-R1-Distill