WhisperJAV Documentation

WhisperJAV is a subtitle generator specifically optimized for the acoustic and linguistic characteristics of Japanese Adult Videos (JAV). It addresses common ASR issues like non-verbal vocalizations, extreme audio dynamics, and hallucination loops through acoustic filtering, linguistic adaptation, and defensive decoding.

Users can choose between a Graphical User Interface (GUI) for ease of use or a Command Line Interface (CLI) for automation and batch processing.

### GUI (Recommended for most users)

```bash
whisperjav-gui

A window opens. Add your files, pick a mode, click Start.

Before installing, ensure your system meets the following requirements:

Hardware

• CPU: Minimum 4 cores (x86_64); 8+ cores recommended.
• RAM: Minimum 8 GB; 16 GB recommended (32 GB for Qwen3-ASR).
• GPU VRAM: Minimum 4 GB; 8 GB+ recommended (16+ GB for Qwen3-ASR).
• Disk Space: 15 GB minimum; 50 GB+ recommended for models and temp files.

Software

• Linux Kernel: 4.15+ (5.4+ recommended).
• Python: 3.10, 3.11, or 3.12 (3.9 and 3.13+ are NOT supported).
• NVIDIA Driver: 450+ (for CUDA 11.8) or 570+ (for CUDA 12.8).
• FFmpeg: 4.0+ (required for audio/video processing).
• Git: 2.0+.
• GCC / build-essential: Required for compiled extensions.

This document evaluates potential replacements for the current llama-cpp-python backend used by WhisperJAV for subtitle translation. The evaluation considers constraints such as non-technical Windows users, NVIDIA GPU primary support, and the need for OpenAI-compatible APIs.

Key candidates evaluated include:

• Ollama: High-potential candidate with excellent GPU auto-detection and low user complexity.
• vLLM: Rejected due to lack of native Windows/macOS support and high resource requirements.
• llama.cpp server: A direct C/C++ alternative.
• LocalAI, MLX, LM Studio: Other evaluated alternatives with varying platform and complexity profiles.

The Qwen Pipeline is a dedicated 9-phase subtitle generation pipeline specifically built around Qwen3-ASR. It operates on a "redundancy over reuse" principle, meaning it does not switch backends or fall back to Whisper.

Since the Phase 4 (Strangulation) update, the pipeline exclusively uses the assembly mode code path. While legacy mode names (like context_aware or vad_slicing) are still accepted via CLI or API, they are internally mapped to assembly mode with appropriate framer overrides and will trigger deprecation warnings.

The architecture is designed to support several upcoming generator and framer backends through a decoupled pipeline:

• WhisperSegmentFramer: Uses faster-whisper to extract segment boundaries as TemporalFrames. This enables 'Whisper-Guided Qwen' (Whisper for timing + Qwen for Japanese text).
• TransformersTextGenerator: A generic adapter for any HuggingFace ASR model. Supports load(), generate(), and unload() (to manage VRAM).
• VLLMTextGenerator: An HTTP client for models served via vLLM. load() and unload() are no-ops as the server manages VRAM; generate() sends requests to the vLLM server.

Ollama is a client-server LLM tool written in Go that abstracts the complexity of running models. It uses llama.cpp under the hood but manages model loading, unloading, and GPU acceleration automatically.

Key Characteristics

• Architecture: Runs an HTTP server (ollama serve) that clients communicate with via REST.
• GPU Detection: Features a sophisticated two-phase bootstrap system that detects CUDA, ROCm, Metal, and Vulkan, with automatic fallback to CPU if GPU fails.
• Model Management: Uses a registry system (similar to Docker) where models can be pulled via ollama pull or imported from GGUF files using a Modelfile.
• API: Provides an OpenAI-compatible API at /v1/ and a native Ollama API for management tasks.

While the standard CLI and GUI are limited to the default assembly configuration (full-scene framer, qwen3 generator, and qwen3 aligner), the underlying API supports several advanced workflows. These must be implemented programmatically as they are not yet exposed via CLI flags or GUI options.

Supported Workflows

• SRT Re-Transcription: Uses SrtSourceFramer(srt_path, keep_text=False) combined with Qwen3TextGenerator and either NoneAligner or None.
• Two-Pass Refinement: A two-step process where Pass 1 generates an SRT, which is then used as input for Pass 2 via SrtSourceFramer(pass1.srt).
• Aligner-Free Fast Mode: Uses VadGroupedFramer + Qwen3TextGenerator with aligner=None for faster processing.
• Cross-Model Benchmarking: Structurally supported, though currently only the qwen3 generator is implemented.

The EnsembleOrchestrator (Ensemble Mode) provides a high-level way to compose a custom pipeline by passing a configuration object. This allows you to explicitly override the scene detector, speech enhancer, and speech segmenter for a single execution flow. This is the most flexible way to use the project's modular components.

Example configuration structure for pass_config:

• pipeline: The base mode to use (e.g., 'qwen').
• scene_detector: The chosen scene detection method.
• speech_enhancer: The chosen enhancement backend.
• speech_segmenter: The chosen VAD segmenter.

pass1_config = {
    'pipeline': 'qwen',              # Use Qwen mode
    'scene_detector': 'semantic',    # Full scene detector options
    'speech_enhancer': 'clearvoice', # Full enhancer options
    'speech_segmenter': 'silero',    # Full segmenter options
}

To avoid the complexities of local C++ compilation and GPU driver management, WhisperJAV recommends a 'sidecar' architecture. Instead of embedding LLM inference directly into the application process, the application should act as a client that communicates with a separate, dedicated LLM server via an OpenAI-compatible HTTP API.

The Workflow

1. Detect: Probe known local server ports (e.g., Ollama at localhost:11434 or LM Studio at localhost:1234).
2. Delegate: If a server is found, send translation requests to its /v1/ endpoints.
3. Fallback: If no local server is detected, provide the user with clear options: install a local provider (like Ollama), use a cloud API (like OpenAI or DeepSeek), or configure a custom server URL.

As of version 1.8.14+, assembly is the default input mode for the Qwen pipeline. It is designed to provide higher quality results by using a decoupled architecture that allows for mid-pipeline text cleaning.

Note on Legacy Modes: If you explicitly select legacy coupled modes such as vad_slicing or context_aware, the system will function but will log a deprecation warning: input_mode='...' uses legacy coupled architecture. For best quality, use input_mode='assembly' with --qwen-framer. Coupled modes will be removed in a future release.