FlashVSR

repository·main·Indexed 23 days ago

https://github.com/openimaginglab/flashvsr

A diffusion-based one-step streaming framework for real-time video super-resolution (VSR). It utilizes sparse attention and a distilled pipeline to achieve high efficiency, such as ~17 FPS at 768x1408 on an NVIDIA A100. The framework is optimized for 4× video super-resolution and requires the Block-Sparse Attention backend for efficient inference.

Tokens
1.1K
Snippets
4
Records
5
Agent score
33%

What's inside FlashVSR

  1. Install FlashVSR

    main

    Follow these steps to set up FlashVSR on your local machine. This project is optimized for 4× video super-resolution; using the 4× SR setting is strongly recommended for stability and results.

    1. Clone the Repository

    git clone https://github.com/OpenImagingLab/FlashVSR
    cd FlashVSR

    2. Set Up the Python Environment

    Create and activate a Conda environment using Python 3.11.13:

    conda create -n flashvsr python=3.11.13
    conda activate flashvsr

    Install the project dependencies:

    pip install -e .
    pip install -r requirements.txt
    git clone https://github.com/OpenImagingLab/FlashVSR
    cd FlashVSR
    conda create -n flashvsr python=3.11.13
    conda activate flashvsr
    pip install -e .
    pip install -r requirements.txt
  2. Download FlashVSR Model Weights

    main

    Model weights are available on Hugging Face via Git LFS. You can choose between the original v1 or the recommended v1.1 (which offers enhanced stability and fidelity).

    First, ensure Git LFS is installed on your machine. Then, navigate to the examples/WanVSR directory and clone the desired version:

    Weights Structure: After cloning, your directory should contain:

    • LQ_proj_in.ckpt
    • TCDecoder.ckpt
    • Wan2.1_VAE.pth
    • diffusion_pytorch_model_streaming_dmd.safetensors
    • README.md

    Note: Inference scripts automatically load weights from these folders.

    # From the repo root
    cd examples/WanVSR
    
    # Install Git LFS (once per machine)
    git lfs install
    
    # Clone v1 (original) or v1.1 (recommended)
    git lfs clone https://huggingface.co/JunhaoZhuang/FlashVSR          # v1
    # or
    git lfs clone https://huggingface.co/JunhaoZhuang/FlashVSR-v1.1      # v1.1
  3. Install Block-Sparse Attention (Required)

    main

    FlashVSR requires the Block-Sparse Attention backend to enable flexible and dynamic attention masking for efficient inference.

    Important Notes:

    • The build process can be memory-intensive (OOM risk) when compiling with multiple ninja jobs. Ensure sufficient memory is available during compilation.
    • Hardware Compatibility: Works with ideal acceleration on NVIDIA A100 and A800 (Ampere). Works on H200 (Hopper) but with limited acceleration. Compatibility for RTX 40/50 series or H800 is currently unknown.

    It is recommended to install this in a separate clean folder outside the FlashVSR repository.

    # ✅ Recommended: clone and install in a separate clean folder (outside the FlashVSR repo)
    git clone https://github.com/mit-han-lab/Block-Sparse-Attention
    cd Block-Sparse-Attention
    pip install packaging
    pip install ninja
    python setup.py install
  4. Quality Warning for Third-Party Implementations

    main

    Some third-party implementations (such as certain early ComfyUI versions) may not include the Locality-Constrained Sparse Attention (LCSA) module, falling back to dense attention instead.

    Using implementations without LCSA can lead to noticeable quality degradation, texture aliasing, and visual artifacts, especially at higher resolutions. For the best preservation of fine structures and details, use the official FlashVSR pipeline.

  5. Run FlashVSR Inference

    main

    Run inference using the provided scripts located in examples/WanVSR. You can choose between the original v1 models or the recommended v1.1 models, and select between full or tiny versions depending on your requirements.

    Note: All commands should be executed from the examples/WanVSR directory.

    # From the repo root
    cd examples/WanVSR
    
    # v1 (original)
    python infer_flashvsr_full.py
    # or
    python infer_flashvsr_tiny.py
    # or
    python infer_flashvsr_tiny_long_video.py
    
    # v1.1 (recommended)
    python infer_flashvsr_v1.1_full.py
    # or
    python infer_flashvsr_v1.1_tiny.py
    # or
    python infer_flashvsr_v1.1_tiny_long_video.py