neosr Framework

repository·master·Indexed 43 days ago

https://github.com/neosr-project/neosr

An open-source framework for training super-resolution models version 1.0.0. It provides a reproducible environment for image restoration across domains such as photos, anime, medical imaging, and geospatial data. The framework supports a wide array of architectures (including Real-ESRGAN, SwinIR, and HAT), optimizers, losses, and augmentations, utilizing the neosr-train command for configuration-based training.

Tokens
2.1K
Snippets
3
Records
14
Agent score
6%

What's inside neosr

  1. Overview of neosr

    master

    neosr is an open-source framework designed for training super-resolution models. It provides a reproducible environment for image restoration across various domains, including photos, anime/cartoons, illustrations, medical imaging, forensics, and geospatial data.

    Key characteristics:

    • Accessible: Supports a wide range of networks, losses, optimizers, and augmentations via configuration.
    • Efficient: Optimized for faster training, quicker convergence, and low GPU requirements.
    • Reproducible: Provides deterministic training environments for bit-exact reproducible models on the same platform.
    • Simple: Uses readable Python code managed by uv, ruff, mypy, and torchfix.
  2. Understand the neosr license and third-party acknowledgements

    master

    The neosr project is released under the Apache 2.0 license.

    Because neosr integrates various architectures, optimizers, losses, and augmentations from the research community, it includes numerous third-party components. These components are subject to their own respective licenses, including but not limited to:

    • Apache 2.0 (e.g., BasicSR, SwinIR, OmniSR, DAT, DITN, etc.)
    • MIT (e.g., HAT, Focal Frequency Loss, DiffJPEG, Real-CUGAN, etc.)
    • BSD / BSD 3-Clause (e.g., Real-ESRGAN, patchgan)
    • CC-NC 4.0 / CC-NC-SA 4.0 (e.g., SRFormer, DISTS weights, TOPIQ)

    Users should refer to the specific license of each component if they are using specific architectures or modules within neosr for commercial or redistributed purposes.

  3. Available datasets for training and validation

    master

    neosr provides access to several curated datasets designed to distill high-quality images for super-resolution tasks. These include the BHI series, Nomos datasets, and various community-curated datasets.

    BHI Series

    The BHI series is recommended for different network scales:

    • Full version: Recommended for large networks.
    • Small version: Recommended for lightweight SISR.
    • Mini version: Lightweight option.
    • BHI100: Used for validation.

    Nomos Datasets

    • Nomos-v2: A multipurpose dataset containing 6000 images.
    • nomos_uni: A multipurpose dataset containing 2989 images, specifically intended for lightweight networks with fewer than 800k parameters.

    Community Datasets

    Various community-curated datasets are available for specialized tasks (e.g., digital art, animation, faces, or nature):

    • digital_art_v3 (Digital art)
    • ModernAnimation_v3 (Modern anime)
    • ClassicAnimation (Analog/cel anime)
    • DF2k-BHI (Curated DF2k)
    • FaceUp (Curated FFHQ)
    • SSDIR (Curated LSDIR)
    • ArtFaces (Curated MetFaces)
    • Nature Dataset (Curated iNaturalist)
  4. Quick start training with neosr-train

    master

    To begin training a super-resolution model, use the neosr-train command followed by the path to your configuration file (typically a .toml file).

    Example:

    neosr-train config.toml

    Configuration templates are available in the options/ directory of the repository. For a detailed explanation of available configuration options, refer to the Configuration Walkthrough in the project wiki.

    neosr-train config.toml
  5. Install neosr

    master

    Install neosr using the platform-specific installation scripts. Note that neosr will be installed in your terminal's current path.

    Prerequisites:

    • CUDA >= 12.4

    Windows (PowerShell): Run the following command in PowerShell:

    powershell -ExecutionPolicy ByPass -c "irm https://raw.githubusercontent.com/neosr-project/neosr/refs/heads/master/install_windows.ps1 | iex"

    Linux (Bash): Run the following command in your terminal:

    curl -LsSf https://raw.githubusercontent.com/neosr-project/neosr/refs/heads/master/install_linux.sh | sh
    powershell -ExecutionPolicy ByPass -c "irm https://raw.githubusercontent.com/neosr-project/neosr/refs/heads/master/install_windows.ps1 | iex"
    
    # OR for Linux
    curl -LsSf https://raw.githubusercontent.com/neosr-project/neosr/refs/heads/master/install_linux.sh | sh
  6. Reference supported architectures (arch)

    master

    When configuring a model, use the arch option to specify the neural network architecture. The following architectures are supported:

    ArchitectureOption(s)
    Real-ESRGANesrgan
    SRVGGNetCompactcompact
    SwinIRswinir_small, swinir_medium
    HAThat_s, hat_m, hat_l
    OmniSRomnisr
    SRFormersrformer_light, srformer_medium
    DATdat_small, dat_medium, dat_2
    DITNditn
    DCTLSAdctlsa
    SPANspan
    Real-CUGANcugan
    CRAFTcraft
    SAFMNsafmn, safmn_l
    RGTrgt, rgt_s
    ATDatd, atd_light
    PLKSRplksr, plksr_tiny
    RealPLKSRrealplksr, realplksr_s, realplksr_l
    DRCTdrct, drct_l, drct_s
    MSDANmsdan
    SPANPlusspanplus, spanplus_sts, spanplus_s, spanplus_st
    HiT-SRFhit_srf, hit_srf_medium, hit_srf_large
    HMAhma, hma_medium, hma_large
    MANman, man_tiny, man_light
    light-SAFMN++light_safmnpp
    GRFormergrformer, grformer_medium, grformer_large
    EIMNeimn, eimn_a, eimn_l
    LMLTlmlt, lmlt_tiny, lmlt_large
    DCTdct
    KRGNkrgn
    PlainUSRplainusr, plainusr_ultra, plainusr_large
    HASNhasn
    FlexNetflexnet, metaflexnet
    CFSRcfsr
    Sebicasebica, sebica_mini
    NinaSRninasr, ninasr_b0, ninasr_b2
    RCANrcan
    MoESRmoesr
    ASIDasid, asid_d8
    MoSRV2mosrv2
    ESCesc, esc_fp, esc_large
    CATANet (testing)catanet

    For arch-specific parameters, refer to the project wiki.

  7. Reference supported losses

    master

    Configure the loss function for training using these options:

    Loss TypeOption(s)
    L1 LossL1Loss, l1_loss
    L2 LossMSELoss, mse_loss
    Huber LossHuberLoss, huber_loss
    CHC (Clipped Huber with Cosine Similarity)chc_loss
    NCC (Normalized Cross-Correlation)ncc_opt, ncc_loss
    Perceptual Lossperceptual_opt, vgg_perceptual_loss
    GANgan_opt, gan_loss
    MS-SSIMmssim_opt, mssim_loss
    LDL Lossldl_opt, ldl_loss
    Focal Frequencyff_opt, ff_loss
    DISTSdists_opt, dists_loss
    Wavelet Guidedwavelet_guided
    Perceptual Patch Lossperceptual_opt, patchloss, ipk
    Consistency Loss (Oklab and CIE L*)consistency_opt, consistency_loss
    KL Divergencekl_opt, kl_loss
    MS-SWDmsswd_opt, msswd_loss
    FDL + DINOv2 backendfdl_opt, fdl_loss