ffsubsync

ffsubsync treats subtitle synchronization as a signal-alignment problem using a three-step process:

1. Discretize: The reference (audio stream or existing subtitle timings) and the input subtitles are divided into 10 ms windows.
2. Label speech: Each 10 ms window is labeled as 'speech' (1) or 'not speech' (0). For subtitles, a window is 'speech' if any subtitle is on screen. For audio, a voice-activity detector (VAD) is used.
3. Align: The algorithm compares the two resulting binary strings and searches for the temporal shift that maximizes the alignment score.

If framerate correction is enabled, the algorithm repeats this search across various candidate framerate ratios to find the best overall result.

When using the web version of ffsubsync, all syncing happens locally in your browser tab. Your subtitle and video/audio files are read directly from your disk and are never uploaded.

Network traffic includes only:

• A one-time download of the WebAssembly runtime and support libraries.
• Anonymous, aggregate Google Analytics usage events (e.g., whether a sync started/completed, the reference type used, and which options like --gss, --no-fix-framerate, or --split-penalty were toggled).

Filenames, file contents, and file sizes are never sent over the network.

The reference is the ground truth used for timing. ffsubsync selects a strategy based on the reference file's extension or provided flags to create a speech signal for alignment. Choosing the right reference type helps balance speed and accuracy:

• Media files (Video/Audio): Uses Voice-Activity Detection (VAD) on extracted audio. This is the most general but most expensive method.
• Subtitle files (.srt, .ass, .ssa, .sub): Uses existing timings. This is the fastest path (typically < 1s).
• PGS Image Subtitles: Uses subtitle display timings from image-based tracks (common in Blu-rays).
• Whisper Transcription: Uses a speech-recognition model for high-accuracy timing when no subtitles exist.
• Serialized Speech (.npy, .npz): Reuses a pre-computed speech signal to avoid re-decoding audio.

To install the latest development version directly from the source:

pip install git+https://github.com/smacke/ffsubsync@latest

You can pull the latest image from the GitHub Container Registry:

Pull image: docker pull ghcr.io/smacke/ffsubsync:latest

Run container: Mount your local directory to /video inside the container: docker run --rm -v "$PWD":/video ghcr.io/smacke/ffsubsync:latest video.mp4 -i unsynchronized.srt -o synchronized.srt

Build from source: docker build -t ffsubsync .

Build with specific PyPI version: docker build -t ffsubsync --build-arg FFSUBSYNC_VERSION=0.4.31 .

docker run --rm -v "$PWD":/video ghcr.io/smacke/ffsubsync:latest \
  video.mp4 -i unsynchronized.srt -o synchronized.srt

The web version is a static bundle built from the web/ directory of the repository. Developers can use make commands to manage the site locally.

Commands:

• Build the site: make site
• Serve the site locally: make serve
• Run tests: make test

The ffsubsync command-line tool can be invoked using any of the following three command names. They all provide access to the same set of command-line options:

• ffs
• subsync
• ffsubsync

You can use ffsubsync entirely in your web browser without installing Python or ffmpeg. The tool runs client-side via WebAssembly, meaning your files are processed locally and never uploaded to a server.

Steps to sync subtitles:

1. Open https://smacke.github.io/ffsubsync.
2. Select a reference type: either a correctly-synced subtitle file or a video/audio file.
3. Choose your reference (the source of truth) and the subtitles to sync (the file you want to fix).
4. Click Sync subtitles.
5. Download the corrected file once the process completes. The interface will display the detected time offset and any framerate corrections applied.

When no subtitles (text or PGS) are available, you can use Whisper transcription via ffmpeg (requires ffmpeg >= 8.0 with --enable-whisper). This uses a speech-recognition model to derive timings, which is often sharper than energy-based VAD.

• Use --whisper-weights to point to a whisper.cpp ggml model file.
• Use --language to override auto-detection (e.g., --language es).
• Use --whisper-args to pass extra key=value options to the underlying ffmpeg filter (e.g., --whisper-args queue=12).
• To enable a VAD model during transcription, reuse the --vad flag with a path to a ggml VAD model (e.g., --vad ~/path/to/model.bin).

$ ffs video.mp4 -i in.srt -o out.srt \
    --whisper-weights ~/whisper.cpp/models/ggml-base.en.bin

To avoid re-decoding audio when syncing multiple subtitle files against the same video, you can use a serialized speech signal (.npy or .npz).

1. Generate the signal once using the --serialize-speech flag.
2. Use the resulting file as your reference in subsequent runs.