TurboDiffusion

The core of LTX-2 is an asymmetric dual-stream diffusion transformer with 48 layers. It processes video and audio tokens simultaneously, allocating 14B parameters to the video stream and 5B parameters to the audio stream.

Each dual-stream block performs four sequential operations:

1. Self-Attention: Within-modality attention.
2. Text Cross-Attention: Textual prompt conditioning.
3. Audio-Visual Cross-Attention: Bidirectional inter-modal exchange using 1D temporal RoPE.
4. Feed-Forward Network (FFN): Feature refinement.

For actual model loading and initialization, use the ltx-pipelines package.

You can install ltx-core either from the repository root using uv or by installing it as an editable package via pip.

# From the repository root
uv sync --frozen

# Or install as a package
pip install -e packages/ltx-core

Prerequisites

• python>=3.9
• torch>=2.7.0 (Note: torch==2.8.0 is recommended to avoid OOM issues).

Installation via pip

conda create -n turbodiffusion python=3.12
conda activate turbodiffusion

pip install turbodiffusion --no-build-isolation

Installation from source

git clone https://github.com/thu-ml/TurboDiffusion.git
cd TurboDiffusion
git submodule update --init --recursive
pip install -e . --no-build-isolation

To install the required dependencies for the recommended SageSLA + FastNorm + TileLang W8A8 inference path, navigate to the TurboDiffusion/TurboT2AV/LTX-2 directory and run the following commands. This installs local LTX packages, CUDA 12.8 PyTorch, SageAttention, SpargeAttn, and TileLang.

cd TurboDiffusion/TurboT2AV/LTX-2
pixi install
pixi run install-acceleration

If you are a developer, you can run the unit test suite in the development environment. This task installs CUDA 12.8 PyTorch and the local LTX packages before running the tests. Inference-only users do not need to perform this step.

pixi run -e dev test

The complete generation process follows these conceptual steps:

1. Text Encoding: Text prompt $\rightarrow$ Gemma encoder $\rightarrow$ separate video/audio embeddings.
2. Latent Initialization: Initialize noise latents in spatial format [B, C, F, H, W].
3. Patchification: Convert spatial latents to sequence format [B, seq_len, dim].
4. Sigma Schedule: Generate noise schedule adapted to token count.
5. Denoising Loop: Iteratively denoise using transformer predictions (includes CFG, STG guidance, and Euler updates).
6. Unpatchification: Convert sequence back to spatial format.
7. VAE Decoding: Decode latents to pixel space.

The TurboDiffusion TUI Server provides an interactive text-based interface for video generation. It loads models once and keeps them GPU-resident for multiple generations. You can launch it using three different methods:

1. As a Python module (recommended for direct control).
2. As an installed CLI (if you have run pip install -e .).
3. Via existing inference scripts by passing the --serve flag to specific Wan2.1 or Wan2.2 scripts.

# 1. Python module
PYTHONPATH=turbodiffusion python -m turbodiffusion.serve [args]

# 2. Installed CLI
PYTHONPATH=turbodiffusion turbodiffusion-serve [args]

# 3. Via existing inference scripts
PYTHONPATH=turbodiffusion python turbodiffusion/inference/wan2.1_t2v_infer.py --serve [args]
PYTHONPATH=turbodiffusion python turbodiffusion/inference/wan2.2_i2v_infer.py --serve [args]

Choose your checkpoint and command flags based on your GPU memory:

• High Memory GPUs (e.g., H100, >40GB VRAM):

  • Use unquantized checkpoints (without -quant in the filename).
  • Remove the --quant_linear flag from your command.

• Consumer GPUs (e.g., RTX 5090, RTX 4090):

  • Use quantized checkpoints (with -quant in the filename).
  • Add the --quant_linear flag to your command.

Follow these steps during an active TUI session to generate video:

1. Enter prompt: Type your text prompt. Use \ at the end of a line to continue the prompt on the next line.
2. Image path (I2V only): Enter the path to your input image.
3. Output path: Enter the desired output path or press Enter to use the default.
4. Generation: The server will generate the video and save it to the specified path.

> A cat sitting on a windowsill \
... watching the rain fall outside
output [output/generated_video.mp4]:
Generating video...
Done: output/generated_video.mp4

Weights for TurboT2AV, LTX-2, and Gemma-3 can be downloaded using pixi run hf download.

Note: Gemma is a gated Hugging Face model. You must accept the access terms on its Hugging Face page and export your token first:

export HF_TOKEN=your_huggingface_token

Base Model Weights

pixi run hf download Lightricks/LTX-2 ltx-2-19b-dev.safetensors --local-dir /path/to/checkpoints/LTX-2
pixi run hf download google/gemma-3-12b-it-qat-q4_0-unquantized --local-dir /path/to/checkpoints/gemma-3-12b-it-qat-q4_0-unquantized

TurboT2AV Main Checkpoint

pixi run hf download luyu1021/TurboT2AV \
  --include "checkpoints/turbot2av_main/*" \
  --local-dir /path/to/turbo-t2av-weights