DOCSEARCHbyAlgolia
Submit Docs

vs-rife: RIFE for VapourSynth

repository·master·Indexed 41 days ago

https://github.com/holywu/vs-rife

A VapourSynth plugin for Real-Time Intermediate Flow Estimation (RIFE) used for video frame interpolation to increase frame rates. Version 5.7.0 supports TensorRT acceleration, scene change detection, and various RIFE models. Requires PyTorch 2.10.0+, VapourSynth R69+, and a CUDA-capable GPU.

Tokens
1.4K
Snippets
7
Records
10
Agent score
99%

What's inside vs-rife

  1. Download RIFE models

    master

    You can download the required RIFE models using two methods:

    1. Download all models at once: Run the module directly from the command line:
      python -m vsrife
    2. On-demand download: Set auto_download=True in the rife() function call. The model will only be downloaded when first requested.
    python -m vsrife

  3. Configure dependencies for vs-rife

    master

    Ensure your environment meets the following requirements:

    • PyTorch: version 2.10.0 or later
    • VapourSynth: R69 or later

    If you are using the trt (TensorRT) backend, you must also install:

    • TensorRT: 10.14.1 or later
    • Torch-TensorRT: 2.10.0 or later

    To install the latest stable versions of PyTorch and Torch-TensorRT with CUDA 13.0 support, use the following commands:

    pip install -U packaging setuptools wheel
pip install -U torch torchvision torch_tensorrt --extra-index-url https://download.pytorch.org/whl/cu130 --extra-index-url https://pypi.nvidia.com

  4. Detect scene changes in RIFE

    master

    To prevent interpolation artifacts during scene cuts, use the sc parameter in rife(). This utilizes the sc_detect() function which:

    1. Resizes the clip to grayscale (vs.GRAY8).
    2. Calculates the difference between consecutive frames using PlaneStats.
    3. Sets a _SceneChangeNext property on frames where the difference exceeds sc_threshold.

    If sc=True is passed to rife(), the function will skip interpolation for frames marked as scene changes.

  5. Use the rife() function for video frame interpolation

    master

    The primary entry point for frame interpolation is the rife() function. Pass a VapourSynth clip to the function to receive the interpolated result.

    Note: For a full list of available parameters, refer to the vsrife/__init__.py file.

    from vsrife import rife

ret = rife(clip)

  6. Use the rife() function for video frame interpolation

    master

    The rife() function performs Real-Time Intermediate Flow Estimation to interpolate video frames using VapourSynth. It supports various models, TensorRT acceleration, and scene change detection.

    Key Requirements

    • Input Format: The input clip must be in vs.RGBH (FP16) or vs.RGBS (FP32) format.
    • Hardware: Requires a CUDA-capable GPU.
    • Models: Models must be downloaded (use auto_download=True or run python -m vsrife to download all models).

    Common Parameters

    • clip: The vs.VideoNode to process.
    • model: The model version string (e.g., `

  7. Use download_model() to fetch specific models

    master

    The download_model function allows for programmatic downloading of model files from a provided URL. It automatically determines the filename from the URL and saves the file into a models subdirectory relative to the module's location.

    from vsrife.__main__ import download_model

download_model("https://github.com/HolyWu/vs-rife/releases/download/model/flownet_v4.26.pkl")

  8. Configure RIFE interpolation parameters

    master

    The rife() function provides several parameters to control the interpolation behavior and performance:

    ParameterDescription
    factor_num / factor_denNumerator and denominator to multiply the frame rate (e.g., 5, 2 for 2.5x).
    fps_num / fps_denTarget frame rate numerator/denominator. Overrides factor settings.
    scaleResolution scale for optical flow. Must be 0.25, 0.5, 1.0, 2.0, or 4.0. Use 0.5 for 4K.
    ensembleEnables smoothing in uncertain estimation areas.
    scEnables scene change detection to avoid interpolating across cuts.
    sc_thresholdThreshold for scene change detection (0.0 to 1.0).
    trtEnables TensorRT for high-performance inference (not supported in models 4.0 and 4.1).
    trt_static_shapeIf True, builds with static shapes. If False, uses dynamic shapes defined by trt_min_shape, trt_opt_shape, and trt_max_shape.
    # Example: 2x interpolation using model 4.25 with TensorRT
clip = rife(
    clip=my_clip,
    model="4.25",
    factor_num=2,
    factor_den=1,
    trt=True,
    trt_static_shape=True
)

  9. Configure TensorRT settings in RIFE

    master

    When trt=True is used, you can fine-tune the TensorRT engine construction. Note that engines are cached in trt_cache_dir and are specific to the GPU, model, and settings used.

    Dynamic Shape Parameters

    If trt_static_shape=False, you must provide:

    • trt_min_shape: Minimum dimensions [height, width].
    • trt_opt_shape: Optimal dimensions [height, width].
    • trt_max_shape: Maximum dimensions [height, width].

    Optimization Parameters

    • trt_workspace_size: Size constraints of workspace memory pool.
    • trt_max_aux_streams: Max auxiliary streams for parallel kernel execution (set to 0 for optimal memory usage).
    • trt_optimization_level: Builder optimization level (integers 0 to 5, default is 3).
    • trt_cache_dir: Directory where TensorRT engine files are stored.

  10. Download RIFE models via CLI

    master

    You can run the vsrife module directly from the command line to download all available RIFE model files (.pkl) into the package's models directory. This is useful for ensuring all necessary model weights are present for the library to function.

    python -m vsrife