repository·main·Indexed 23 days ago
https://github.com/modeltc/lightx2vA lightweight inference framework for high-performance image and video generation (T2V, I2V, T2I, I2I). It focuses on efficiency through quantization (FP8, NVFP4), distillation, and optimized parallelism. The framework supports models such as BAGEL-7B-MoT, Qwen-Image (2511/2512), and Wan2.1/2.2, providing tools for deployment via Gradio and a Python API through the LightX2VPipeline class.
lightx2v_platform is a functional platform designed to be independent of the core lightx2v library. Its primary purpose is to align inference interfaces for non-NVIDIA chip backends. Developers looking to support a new chip backend should focus their implementation efforts exclusively within lightx2v_platform.The Model Conversion Tool is a utility for managing model weights. It provides capabilities for:
.pth) and SafeTensors (.safetensors) formats.LightX2V and Diffusers architectures.wan_dit, hunyuan_dit, qwen_image_dit, wan_t5, and wan_clip.torchrun.LightX2V is a lightweight image and video generation inference framework designed for high-performance synthesis. It provides a unified platform for various generation tasks, where 'X2V' denotes the transformation of input modalities (X, such as text or images) into vision output (Vision).
Supported tasks include:
Users can access models via the LightX2V HuggingFace repository or try the framework online via LightX2V Studio.
python -m lightx2v.infer --model_cls bagel --task t2ipython -m lightx2v.infer --model_cls bagel --task i2iseed controlaspect_ratio presetstarget_shape overridesreturn_result_tensor=TrueTo run LightX2V locally on Windows, ensure your hardware meets the following minimum recommendations:
LightX2V supports a wide range of models including:
LightX2V supports three primary model formats for video generation inference:
lightx2v_platform is a functional platform independent of the core lightx2v library. Its purpose is to align inference interfaces for non-NVIDIA chip backends. Developers looking to support new chip backends should focus on the implementations within lightx2v_platform.Autoregressive distillation is a research-oriented feature in LightX2V designed to accelerate inference. It reduces the required inference steps from the standard 40-50 steps down to 8 steps. This technique enables infinite-length video generation by utilizing KV Cache technology.
Note: Currently, LightX2V only supports autoregressive models for Text-to-Video (T2V). Users should be aware that current performance and acceleration improvements may be mediocre, as it is primarily intended as a long-term research direction.
Disaggregation Mode splits a large generative model's inference pipeline (e.g., Wan, Qwen Image) into three distinct stages: Encoder, Transformer, and Decoder. This is designed to solve Out-of-Memory (OOM) issues on memory-constrained GPUs by ensuring each node only loads its specific subset of the model.
disagg_mode="encoder"): Loads Text Encoder, Image Encoder, and VAE Encoder. It sends encoded data via Phase1 (using Mooncake transport) to the Transformer.disagg_mode="transformer"): Loads only the DiT (Diffusion Transformer). It receives Phase1 data, runs denoising, and if a decoder_engine_rank is configured, sends latents via Phase2 to the Decoder.disagg_mode="decode"): Loads only the VAE Decoder. It receives Phase2 data, runs the VAE decode, and is responsible for task completion and storing the final result path.