DOCSEARCHbyAlgolia
Submit Docs

PiDiNet (Pixel Difference Networks)

repository·master·Indexed 20 days ago

https://github.com/hellozhuo/pidinet

A lightweight PyTorch framework for efficient visual edge detection. PiDiNet provides implementations for various model scales and includes tools for training, evaluating performance, and converting models to vanilla CNNs. It supports multiple datasets including BSDS, NYUD, and Multicue, and provides a utility to measure GPU throughput (FPS).

Tokens
2.5K
Snippets
10
Records
13
Agent score
20%

What's inside PiDiNet

  1. Download and prepare datasets

    master

    The project uses augmented BSDS500, PASCAL VOC, and NYUD datasets. You can download the base files using wget:

    wget http://mftp.mmcheng.net/liuyun/rcf/data/HED-BSDS.tar.gz
wget http://mftp.mmcheng.net/liuyun/rcf/data/PASCAL.tar.gz
wget http://mftp.mmcheng.net/liuyun/rcf/data/NYUD.tar.gz

    BSDS500 Setup

    1. Create a folder /path/to/BSDS500.
    2. Extract HED-BSDS.tar.gz to /path/to/BSDS500/HED-BSDS.
    3. Extract PASCAL.tar.gz to /path/to/BSDS500/PASCAL.
    4. (Optional) For the BSDS500 val set, download the images from Google Drive and extract to /path/to/BSDS500/HED-BSDS/val.
    5. Copy .lst files from data/BSDS500/HED-BSDS to /path/to/BSDS500/HED-BSDS/.
    6. Copy .lst files from data/BSDS500 to /path/to/BSDS500/.

    NYUD Setup

    1. Create a folder /path/to/NYUD.
    2. Extract NYUD.tar.gz to /path/to/NYUD.
    3. Copy .lst files from data/NYUD to /path/to/NYUD/.

  2. Set up the running environment

    master

    To use PiDiNet, ensure your environment meets the following requirements:

    For Training (PyTorch):

    • OS: Ubuntu 18.04
    • PyTorch: 1.9
    • CUDA: 10.1
    • cuDNN: 7.5

    For Evaluation (Matlab):

    • Matlab: 2019a

  3. Test GPU throughput (FPS)

    master

    Use throughput.py to measure the Frames Per Second (FPS) on your GPU using the converted model.

    python throughput.py --model pidinet_converted --config carv4 --sa --dil -j 1 --gpu 0 --datadir /path/to/BSDS500 --dataset BSDS

  4. Train the PiDiNet model

    master

    Use main.py to train the model. Checkpoints are saved in the directory specified by --savedir/save_models/ during training.

    Example for training the table5_pidinet model:

    python main.py --model pidinet --config carv4 --sa --dil --resume --iter-size 24 -j 4 --gpu 0 --epochs 20 --lr 0.005 --lr-type multistep --lr-steps 10-16 --wd 1e-4 --savedir /path/to/table5_pidinet --datadir /path/to/BSDS500 --dataset BSDS

  5. Generate edge maps using the original model

    master

    To generate edge maps using a trained checkpoint with the original PiDiNet architecture, use the --evaluate flag pointing to your .pth checkpoint.

    python main.py --model pidinet --config carv4 --sa --dil -j 4 --gpu 0 --savedir /path/to/table5_pidinet --datadir /path/to/BSDS500 --dataset BSDS --evaluate /path/to/table5_pidinet/save_models/checkpointxxx.pth

  6. Generate edge maps using the converted vanilla CNN model

    master

    PiDiNet can be converted to a vanilla CNN. This process uses the saved checkpoint to produce a model that should yield identical results to the original. Use the --model pidinet_converted and the --evaluate-converted flags.

    python main.py --model pidinet_converted --config carv4 --sa --dil -j 4 --gpu 0 --savedir /path/to/table5_pidinet --datadir /path/to/BSDS500 --dataset BSDS --evaluate /path/to/table5_pidinet/save_models/checkpointxxx.pth --evaluate-converted

  7. Evaluate models using Matlab

    master

    Evaluation is performed in Matlab.

    1. Download the evaluation code and extract it to /path/to/edge_eval_matlab.
    2. Edit the path settings in the first few lines of eval_bsds.m, eval_nyud.m, or eval_multicue.m depending on your dataset.
    3. Run Matlab in headless mode:
      matlab -nosplash -nodisplay -nodesktop
    4. Inside the Matlab prompt, run the appropriate evaluation script (e.g., eval_bsds).

    Note: You can adjust the number of parallel workers in /path/to/edge_eval_matlab/toolbox.badacost.public/matlab/fevalDistr.m at line 100 (default is 16).

  9. Train Pixel Difference Networks

    master

    Use main.py to train the model on various datasets. You can specify the dataset, model architecture, and training hyperparameters via CLI arguments. The training process saves checkpoints and logs to the directory specified by --savedir.

    python main.py --dataset BSDS --model baseline --savedir ./my_results --epochs 50 --lr 0.005

  10. Evaluate a trained checkpoint

    master

    To evaluate a specific model checkpoint, use the --evaluate flag followed by the path to the checkpoint file. If you want to evaluate the model after converting it to a vanilla CNN (removing the pixel difference convolution layers), use the --evaluate-converted flag.

    # Standard evaluation
python main.py --evaluate /path/to/checkpoint.pth

# Evaluation using a converted vanilla CNN model
python main.py --evaluate /path/to/checkpoint.pth --evaluate-converted

  11. Generate edge maps for custom images

    master

    To run inference on your own images, set --dataset Custom and point --datadir to your image folder. Use the converted model for efficiency.

    python main.py --model pidinet_converted --config carv4 --sa --dil -j 4 --gpu 0 --savedir /path/to/savedir --datadir /path/to/custom_images --dataset Custom --evaluate /path/to/table5_pidinet/save_models/checkpointxxx.pth --evaluate-converted

  12. Reference: CLI arguments for main.py

    master

    The following arguments are available for controlling the training and evaluation process in main.py:

    --savedir (str): path to save result and checkpoint (default: 'results/savedir')
--datadir (str): dir to the dataset (default: '../data')
--only-bsds: only use bsds for training
--ablation: not use bsds val set for training
--dataset (str): data settings for BSDS, Multicue and NYUD datasets (default: 'BSDS')
--model (str): model to train the dataset (default: 'baseline')
--sa: use CSAM in pidinet
--dil: use CDCM in pidinet
--config (str): model configurations, please refer to models/config.py (default: 'carv4')
--seed (int): random seed
--gpu (str): gpus available
--checkinfo: only check the informations about the model: model size, flops
--epochs (int): number of total epochs to run (default: 20)
--iter-size (int): number of samples in each iteration (default: 24)
--lr (float): initial learning rate for all weights (default: 0.005)
--lr-type (str): learning rate strategy [cosine, multistep] (default: 'multistep')
--lr-steps (str): steps for multistep learning rate (e.g., '10-20')
--opt (str): optimizer [adam, sgd] (default: 'adam')
--wd (float): weight decay for all weights (default: 1e-4)
-j, --workers (int): number of data loading workers (default: 4)
--eta (float): threshold to determine the ground truth (default: 0.3)
--lmbda (float): weight on negative pixels (default: 1.1)
--resume: use latest checkpoint if have any
--print-freq (int): print frequency (default: 10)
--save-freq (int): save frequency (default: 1)
--evaluate (str): full path to checkpoint to be evaluated
--evaluate-converted: convert the checkpoint to vanilla cnn, then evaluate