repository·main·Indexed 24 days ago
https://github.com/siliconflow/onediffAn acceleration library designed to speed up diffusion models such as SDXL, SVD, and DiT using optimized GPU kernels and PyTorch compilation tools. It integrates with HF diffusers and ComfyUI, providing specialized nodes for ControlNet, LoRA with DeepCache, and Stable Diffusion 3 (SD3) acceleration.
onediffx extensions for onediff.onediff is an acceleration library for diffusion models with the following key characteristics:
For businesses requiring enterprise-level support, the OneDiff Enterprise Solution provides:
Contact: contact@siliconflow.com or visit https://siliconflow.cn/pricing.
If you are switching between models that share the same structure, you can re-use a previously compiled graph instead of re-compiling. This significantly reduces model switching time. To do this, load the new model's state dict into the _torch_module member of the object returned by oneflow_compile.
Note: This feature is not supported for quantized models.
base = StableDiffusionPipeline(...)
compiled_unet = oneflow_compile(base.unet)
base.unet = compiled_unet
# This step needs some time to compile the UNet
base(prompt)
new_base = StableDiffusionPipeline(...)
# Re-use the compiled graph by loading the new state dict into the `_torch_module` member of the object returned by `oneflow_compile`
compiled_unet._torch_module.load_state_dict(new_base.unet.state_dict())
# After loading the new state dict into the `compiled_unet._torch_module`, the weights of the compiled_unet are updated too
new_base.unet = compiled_unet
# This step doesn't need additional time to compile the UNet again because
# new_base.unet is already compiled
new_base(prompt)The Nexfort backend supports out-of-the-box dynamic shape inference. To use it, set "dynamic": true in your options dictionary.
Best Practice: To avoid over-specialization and frequent re-compilation, perform an initial call to your model with a non-typical shape (e.g., a height and width that are not equal, such as 512x768).
options = '{"mode": "max-autotune", "dynamic": true}'python3 ./onediff_diffusers_extensions/examples/text_to_image_sdxl.py \
--height 512 \
--width 768 \
--compiler nexfort \
--compiler-config '{"mode": "max-optimize:max-autotune:low-precision", "memory_format": "channels_last", "dynamic": true}' \
--run_multiple_resolutions 1 \
--run_rare_resolutions 1Quantization features (which reduce memory usage and increase speed while maintaining quality) are exclusively available to OneDiff Enterprise users.
If you have a license key, refer to the Online Quantization for ComfyUI documentation for specific instructions on using quantized models.
The first time you perform quantization, the system requires additional computation time to analyze data dependencies and identify necessary parameters (such as maximum and minimum values).
Once these parameters are established, they are stored in a cache. Subsequent quantization processes will use these cached parameters, significantly accelerating processing speed.
*.pt is cached.cache_dir/quantization_stats.json.OneDiff supports full LoRA functionality. You can use LoRAs exactly as you do with native Stable Diffusion WebUI.
OneDiffX provides an optimized implementation for loading and fusing LoRA weights into Hugging Face Diffusers pipelines. It is designed to be faster than standard Diffusers or PEFT methods by directly fusing weights into the model, bypassing the overhead of module replacement (e.g., LoRACompatible or BaseTunerLayer).
If you need to load multiple LoRAs before inference, do not repeatedly call load_and_fuse_lora as it may cause precision issues. Instead:
load_lora_and_optionally_fuse with fuse=False for each LoRA.set_and_fuse_adapters to apply them.OneDiffX provides DeepCache speedup for Stable Diffusion models. To use it, import the specialized pipeline classes from onediffx.deep_cache instead of the standard diffusers classes.
Import StableDiffusionXLPipeline from onediffx.deep_cache.
Import StableDiffusionPipeline from onediffx.deep_cache.
When calling the pipeline, use the following parameters to control DeepCache:
cache_interval: The interval between cache updates.cache_layer_id: The layer ID to use for caching.cache_block_id: The block ID to use for caching.Note: It is recommended to perform a 'warmup' run (running the pipeline once) before the actual inference to ensure the cache is properly initialized.
import torch
from onediffx import compile_pipe
from onediffx.deep_cache import StableDiffusionXLPipeline
pipe = StableDiffusionXLPipeline.from_pretrained(
"stabilityai/stable-diffusion-xl-base-1.0",
torch_dtype=torch.float16,
variant="fp16",
use_safetensors=True
)
pipe.to("cuda")
pipe = compile_pipe(pipe)
# Warmup
for i in range(1):
deepcache_output = pipe(
"A photo of a cat.",
cache_interval=3, cache_layer_id=0, cache_block_id=0,
output_type='pil'
).images[0]
# Actual inference
deepcache_output = pipe(
"A photo of a cat.",
cache_interval=3, cache_layer_id=0, cache_block_id=0,
output_type='pil'
).images[0]