DOCSEARCHbyAlgolia
Submit Docs

perceptualsimilarity

repository·master·Indexed 26 days ago

https://github.com/richzhang/perceptualsimilarity

Implementation of the Learned Perceptual Image Patch Similarity (LPIPS) metric and the Berkeley Adobe Perceptual Patch Similarity (BAPPS) dataset. It provides tools to evaluate and optimize image similarity using deep features via the lpips Python package or command-line scripts, supporting backbone networks such as AlexNet, VGG, and SqueezeNet.

Tokens
843
Snippets
4
Records
6
Agent score
38%

What's inside perceptualsimilarity

  1. Train a perceptual metric using BAPPS

    master

    You can train a linear layer on top of an existing representation using the BAPPS dataset.

    • Standard training/testing: Run bash train_test_metric.sh. This trains for 10 epochs on the full training set and tests on all validation sets.
    • Train from scratch: Run bash train_test_metric_scratch.sh.
    • Fine-tune: Run bash train_test_metric_tune.sh.

    Training results (checkpoints) are saved in the checkpoints directory.

  3. Download the BAPPS dataset

    master

    The Berkeley Adobe Perceptual Patch Similarity (BAPPS) dataset can be downloaded using the provided shell scripts. The full dataset is approximately 6.6 GB.

    • Full dataset: bash ./scripts/download_dataset.sh
    • Validation set only (1.3 GB): bash ./scripts/download_dataset_valonly.sh
    bash ./scripts/download_dataset.sh

  4. Use the LPIPS metric in Python

    master

    You can use the lpips.LPIPS class to calculate the perceptual distance between image patches.

    Important: Input images must be RGB and normalized to the range [-1, 1].

    Configuration Options:

    • net: Specifies the backbone network.
      • 'alex': (Default) Fastest and best performing as a forward metric.
      • 'vgg': Recommended when using the metric for optimization (backpropping).
      • 'squeeze': SqueezeNet variant.
    • lpips: (Default True) Adds a linear calibration on top of intermediate features. Set to False to weight all features equally.
    • version: Set to '0.0' to use the initial release version. Current version is 0.1.

  5. Evaluate a perceptual metric on the BAPPS dataset

    master

    Use test_dataset_model.py to evaluate different perceptual models on the BAPPS dataset.

    Available Flags:

    Dataset Flags:

    • --dataset_mode: 2afc or jnd.
    • --datasets:
      • If 2afc: train/traditional, train/cnn, val/traditional, val/cnn, val/superres, val/deblur, val/color, val/frameinterp.
      • If jnd: val/traditional, val/cnn.

    Model Flags:

    • --model: lpips (learned), baseline (uncalibrated classification net), l2 (Euclidean), ssim (Structured Similarity).
    • --net: squeeze, alex, or vgg (ignored for l2 and ssim).
    • --colorspace: Lab or RGB (used for l2 and ssim, ignored for net models).

    Misc Flags:

    • --batch_size: Evaluation batch size (default 1).
    • --use_gpu: Enable GPU usage.

    Example:

    python ./test_dataset_model.py --dataset_mode 2afc --datasets val/traditional val/cnn --model lpips --net alex --use_gpu --batch_size 50
    python ./test_dataset_model.py --dataset_mode 2afc --datasets val/traditional val/cnn --model lpips --net alex --use_gpu --batch_size 50

  6. Run LPIPS via Command Line

    master

    The repository provides scripts to calculate distances between images or directories via the CLI.

    • Two specific images: python lpips_2imgs.py -p0 <img0> -p1 <img1> --use_gpu
    • Two directories (corresponding pairs): python lpips_2dirs.py -d0 <dir0> -d1 <dir1> -o <output_file> --use_gpu
    • All pairs within a single directory: python lpips_1dir_allpairs.py -d <dir> -o <output_file> --use_gpu
    python lpips_2imgs.py -p0 imgs/ex_ref.png -p1 imgs/ex_p0.png --use_gpu