DOCSEARCHbyAlgolia
Submit Docs

CycleGAN

repository·master·Indexed 12 days ago

https://github.com/junyanz/cyclegan

A Torch implementation for unpaired image-to-image translation, enabling the learning of mappings between two domains—such as horses to zebras or photos to paintings—without requiring direct input-output pairs.

Tokens
2K
Snippets
7
Records
9
Agent score
46%

What's inside CycleGAN

  1. How to structure training and test data

    master

    To train or test CycleGAN on custom datasets, organize your data into a folder structure with two subdirectories:

    • trainA and trainB: For training images from domain A and domain B.
    • testA and testB: For test images from domain A and domain B.

    Note on Dataset Selection: CycleGAN works best when the two domains share similar visual content (e.g., landscape painting <\leftrightarrow> landscape photographs). It may fail on datasets with very different content (e.g., cats <\leftrightarrow> dogs).

  2. Understand CycleGAN failure cases and limitations

    master

    CycleGAN is most effective for translation tasks involving color and texture changes. Users should be aware of the following limitations:

    • Domain Mismatch: The model performs poorly when test images differ significantly from the training distribution (e.g., training on horses without riders but testing on horses with riders).
    • Geometric Changes: The method has limited success with tasks requiring significant geometric transformations (e.g., dog<->cat transfiguration often results in minimal changes).
    • Label Permutation: In certain semantic tasks (like cityscapes photos $\leftrightarrow$ labels), the model may incorrectly permute labels (e.g., confusing trees and buildings).
    • Unpaired vs. Paired Gap: There remains a performance gap between this unpaired method and methods using paired training data, which may be difficult to close in some tasks.

  3. Apply a pre-trained CycleGAN model

    master

    You can use pre-trained models to perform image-to-image translation without training.

    1. Download test photos using ./datasets/download_dataset.sh <dataset_name>.
    2. Download a pre-trained model using ./pretrained_models/download_model.sh <model_name>.
    3. Run the test script using th test.lua with the following environment variables:
      • DATA_ROOT: Path to your images.
      • name: The name of the pre-trained model.
      • model: Set to one_direction_test for single-direction generation.
      • phase: Set to test.
      • loadSize / fineSize: Resolution settings.
      • resize_or_crop: Scaling strategy.

    Results are saved to ./results/<name>_pretrained/latest_test/index.html.

    bash ./datasets/download_dataset.sh ae_photos
bash ./pretrained_models/download_model.sh style_cezanne

DATA_ROOT=./datasets/ae_photos name=style_cezanne_pretrained model=one_direction_test phase=test loadSize=256 fineSize=256 resize_or_crop="scale_width" th test.lua

  4. Install CycleGAN

    master

    To install CycleGAN, you need a Linux or OSX environment with an NVIDIA GPU and CUDA CuDNN. For macOS users, ensure gfind and gwc are installed via brew install findutils coreutils.

    Follow these steps:

    1. Install torch and its dependencies from the official Torch distribution.
    2. Install required Torch packages using luarocks.
    3. Clone the repository.

    Note: This implementation is in Torch. For an actively developed PyTorch version, see the pytorch-CycleGAN-and-pix2pix repository.

    luarocks install nngraph
luarocks install class
luarocks install https://raw.githubusercontent.com/szym/display/master/display-scm-0.rockspec

git clone https://github.com/junyanz/CycleGAN
cd CycleGAN

  5. Test a trained CycleGAN model

    master

    To test a model you have trained:

    1. Run th test.lua with the following environment variables:
      • DATA_ROOT: Path to the test images.
      • name: The name of your trained experiment.
      • phase: Set to test.

    Options:

    • model=one_direction_test: Generates outputs in only one direction.
    • which_direction=AtoB or which_direction=BtoA: Specifies the direction if using one_direction_test.
    • results_dir: Change the output directory (default is ./results/<name>).

    Results are saved as an HTML file at ./results/<name>/latest_test/index.html.

    DATA_ROOT=./datasets/horse2zebra name=horse2zebra_model phase=test th test.lua

  6. Train a CycleGAN model

    master

    To train a model on a new dataset:

    1. Download a dataset using bash ./datasets/download_dataset.sh <dataset_name>.
    2. Run the training command using th train.lua.

    Environment Variables:

    • DATA_ROOT: Path to the dataset folder.
    • name: The experiment name (used for saving checkpoints).
    • gpu: Set to 0 to force CPU mode.
    • cudnn: Set to 0 to force CPU mode.

    Checkpoints: Models are saved to ./checkpoints/<name>. You can change this by passing checkpoint_dir=your_dir to train.lua.

    Optional Display: To view results during training, start the display server with th -ldisplay.start 8000 0.0.0.0 and open http://localhost:8000 in your browser.

    bash ./datasets/download_dataset.sh horse2zebra

# Standard training
DATA_ROOT=./datasets/horse2zebra name=horse2zebra_model th train.lua

# CPU-only training
DATA_ROOT=./datasets/horse2zebra name=horse2zebra_model gpu=0 cudnn=0 th train.lua

  7. Reference: Available Datasets

    master

    Download datasets using bash ./datasets/download_dataset.sh <dataset_name>.

    Supported Datasets:

    • facades: CMP Facades dataset.
    • cityscapes: Cityscapes training set (requires manual download due to license; see ./datasets/prepare_cityscapes_dataset.py).
    • maps: Google Maps images.
    • horse2zebra: ImageNet horse and zebra images.
    • apple2orange: ImageNet apple and orange images.
    • summer2winter_yosemite: Yosemite summer and winter images.
    • monet2photo, vangogh2photo, ukiyoe2photo, cezanne2photo: Wikiart paintings and Flickr landscapes.
    • iphone2dslr_flower: Flickr iPhone and DSLR flower photos.
    bash ./datasets/download_dataset.sh horse2zebra

  8. Reference: Pre-trained Model Zoo

    master

    Download pre-trained models using bash ./pretrained_models/download_model.sh <model_name>. Models are saved to ./checkpoints/<model_name>/latest_net_G.t7. To download CPU versions, append _cpu to the name.

    Available Models:

    • orange2apple / apple2orange (ImageNet categories)
    • horse2zebra / zebra2horse (ImageNet categories)
    • style_monet (landscape photo $\rightarrow$ Monet painting)
    • style_vangogh (landscape photo $\rightarrow$ Van Gogh painting)
    • style_ukiyoe (landscape photo $\rightarrow$ Ukiyo-e painting)
    • style_cezanne (landscape photo $\rightarrow$ Cezanne painting)
    • monet2photo (Monet paintings $\rightarrow$ real landscape)
    • cityscapes_photo2label / cityscapes_label2photo (Street scenes)
    • map2sat / sat2map (Maps $\leftrightarrow$ Aerial photos)
    • iphone2dslr_flower (iPhone flowers $\rightarrow$ DSLR flowers)
    bash ./pretrained_models/download_model.sh style_monet
# For CPU
bash ./pretrained_models/download_model.sh style_monet_cpu

  9. Cite CycleGAN in research

    master

    If using this implementation for research, please cite the original paper using the following BibTeX entry:

    @inproceedings{CycleGAN2017,
  title={Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networkss},
  author={Zhu, Jun-Yan and Park, Taesung and Isola, Phillip and Efros, Alexei A},
  booktitle={Computer Vision (ICCV), 2017 IEEE International Conference on},
  year={2017}
}