DOCSEARCHbyAlgolia
Submit Docs

CLIP-ReID

repository·master·Indexed 19 days ago

https://github.com/syliz517/clip-reid

A framework for Image Re-Identification using Vision-Language Models like CLIP without requiring concrete text labels. It supports CNN and ViT-based architectures and incorporates advanced techniques such as Spatial Information Embedding (SIE) and Online Prompt Learning (OLP). Compatible with datasets including Market-1501, MSMT17, DukeMTMC-reID, Occluded-Duke, VehicleID, and VeRi-776.

Tokens
774
Snippets
5
Records
6
Agent score
18%

What's inside CLIP-ReID

  1. Evaluate trained CLIP-ReID models

    master

    To test a trained model (e.g., a ViT-based model for MSMT17), use the test_clipreid.py script and provide the path to your trained checkpoint using the TEST.WEIGHT flag.

    CUDA_VISIBLE_DEVICES=0 python test_clipreid.py --config_file configs/person/vit_clipreid.yml TEST.WEIGHT 'your_trained_checkpoints_path/ViT-B-16_60.pth'

  2. Prepare Datasets for CLIP-ReID

    master

    To use the project, download the supported datasets and unzip them into a directory of your choice (e.g., your_dataset_dir). Supported datasets include:

    • Market-1501
    • MSMT17
    • DukeMTMC-reID
    • Occluded-Duke
    • VehicleID
    • VeRi-776

  3. Train CNN-based CLIP-ReID baseline

    master

    To train a CNN-based baseline (e.g., for Market-1501), first modify the configuration file (e.g., configs/person/cnn_base.yml) by adding or updating the DATASETS and OUTPUT_DIR keys at the bottom of the file:

    DATASETS:
   NAMES: ('market1501')
   ROOT_DIR: ('your_dataset_dir')
OUTPUT_DIR: 'your_output_dir'

    Then, execute the training script using train.py.

    CUDA_VISIBLE_DEVICES=0 python train.py --config_file configs/person/cnn_base.yml

  4. Install CLIP-ReID via Conda and Pip

    master

    Set up the environment using Conda and install the required dependencies. This project requires Python 3.8 and specific versions of PyTorch and torchvision.

    conda create -n clipreid python=3.8
conda activate clipreid
conda install pytorch==1.8.0 torchvision==0.9.0 torchaudio==0.8.0 cudatoolkit=10.2 -c pytorch
pip install yacs
pip install timm
pip install scikit-image
pip install tqdm
pip install ftfy
pip install regex

  5. Train ViT-based CLIP-ReID

    master

    To train a ViT-based model (e.g., for MSMT17), modify the configuration file (e.g., configs/person/vit_clipreid.yml) with your dataset details:

    DATASETS:
   NAMES: ('msmt17')
   ROOT_DIR: ('your_dataset_dir')
OUTPUT_DIR: 'your_output_dir'

    Then, execute the training script using train_clipreid.py.

    CUDA_VISIBLE_DEVICES=0 python train_clipreid.py --config_file configs/person/vit_clipreid.yml

  6. Train ViT-based CLIP-ReID with SIE and OLP

    master

    To run the advanced ViT-based CLIP-ReID with SIE (Spatial Information Embedding) and OLP (Online Prompt Learning), use train_clipreid.py with additional model flags passed via the command line.

    CUDA_VISIBLE_DEVICES=0 python train_clipreid.py --config_file configs/person/vit_clipreid.yml  MODEL.SIE_CAMERA True MODEL.SIE_COE 1.0 MODEL.STRIDE_SIZE '[12, 12]'