HunyuanCustom Documentation

For environments with limited GPU resources, use python hymm_sp/sample_gpu_poor.py.

Key configurations for single-GPU usage:

• Set DISABLE_SP=1 to disable Spatiotemporal Parallelism.
• Use --use-fp8 to enable FP8 precision, which reduces VRAM usage.
• For extremely low VRAM environments, set CPU_OFFLOAD=1 and use the --cpu-offload flag to offload components to the CPU.

### Single-gpu Inference
```bash
cd HunyuanCustom

export MODEL_BASE="./models"
export DISABLE_SP=1
export PYTHONPATH=./
python hymm_sp/sample_gpu_poor.py \
    --ref-image './assets/images/seg_woman_01.png' \
    --pos-prompt "Realistic, High-quality. A woman is drinking coffee at a café." \
    --neg-prompt "Aerial view, aerial view, overexposed, low quality, deformation, a poor composition, bad hands, bad teeth, bad eyes, bad limbs, distortion, blurring, text, subtitles, static, picture, black border." \
    --ckpt ${MODEL_BASE}"/hunyuancustom_720P/mp_rank_00_model_states_fp8.pt" \
    --video-size 512 896 \
    --seed 1024 \
    --sample-n-frames 129 \
    --infer-steps 30 \
    --flow-shift-eval-video 13.0 \
    --save-path './results/1gpu_540p' \
    --use-fp8

Run with very low VRAM

cd HunyuanCustom

export MODEL_BASE="./models"
export CPU_OFFLOAD=1
export PYTHONPATH=./
python hymm_sp/sample_gpu_poor.py \
    --ref-image './assets/images/seg_woman_01.png' \
    --pos-prompt "Realistic, High-quality. A woman is drinking coffee at a café." \
    --neg-prompt "Aerial view, aerial view, overexposed, low quality, deformation, a poor composition, bad hands, bad teeth, bad eyes, bad limbs, distortion, blurring, text, subtitles, static, picture, black border." \
    --ckpt ${MODEL_BASE}"/hunyuancustom_720P/mp_rank_00_model_states_fp8.pt" \
    --video-size 720 1280 \
    --seed 1024 \
    --sample-n-frames 129 \
    --infer-steps 30 \
    --flow-shift-eval-video 13.0 \
    --save-path './results/cpu_720p' \
    --use-fp8 \
    --cpu-offload 

To perform high-performance video generation using multiple GPUs, use torchrun to execute hymm_sp/sample_batch.py. This method supports three primary customization modes:

1. Single-Subject Video Customization: Generates video based on a reference image and text prompts.
2. Video-Driven Video Customization (Video Editing): Uses an input video and a mask video to edit content, utilizing the --video-condition flag.
3. Audio-Driven Video Customization: Generates video driven by an audio file, utilizing the --audio-condition flag and --audio-strength to control influence.

Ensure MODEL_BASE and PYTHONPATH are set correctly before running.

### Run Single-Subject Video Customization
```bash
cd HunyuanCustom

export MODEL_BASE="./models"
export PYTHONPATH=./
torchrun --nnodes=1 --nproc_per_node=8 --master_port 29605 hymm_sp/sample_batch.py \
    --ref-image './assets/images/seg_woman_01.png' \
    --pos-prompt "Realistic, High-quality. A woman is drinking coffee at a café." \
    --neg-prompt "Aerial view, aerial view, overexposed, low quality, deformation, a poor composition, bad hands, bad teeth, bad eyes, bad limbs, distortion, blurring, text, subtitles, static, picture, black border." \
    --ckpt ${MODEL_BASE}"/hunyuancustom_720P/mp_rank_00_model_states.pt" \
    --video-size 720 1280 \
    --seed 1024 \
    --sample-n-frames 129 \
    --infer-steps 30 \
    --flow-shift-eval-video 13.0 \
    --save-path './results/sp_720p'

Run Video-Driven Video Customization (Video Editing)

cd HunyuanCustom

export MODEL_BASE="./models"
export PYTHONPATH=./
torchrun --nnodes=1 --nproc_per_node=8 --master_port 29605 hymm_sp/sample_batch.py \
    --ref-image './assets/images/sed_red_panda.png' \
    --input-video './assets/input_videos/001_bg.mp4' \
    --mask-video './assets/input_videos/001_mask.mp4' \
    --expand-scale 5 \
    --video-condition \
    --pos-prompt "Realistic, High-quality. A red panda is walking on a stone road." \
    --neg-prompt "Aerial view, aerial view, overexposed, low quality, deformation, a poor composition, bad hands, bad teeth, bad eyes, bad limbs, distortion, blurring, text, subtitles, static, picture, black border." \
    --ckpt ${MODEL_BASE}"/hunyuancustom_editing_720P/mp_rank_00_model_states.pt" \
    --seed 1024 \
    --infer-steps 50 \
    --flow-shift-eval-video 5.0 \
    --save-path './results/sp_editing_720p'
    # --pose-enhance # Enable for human videos to improve pose generation quality.

Run Audio-Driven Video Customization

cd HunyuanCustom

export MODEL_BASE="./models"
export PYTHONPATH=./
torchrun --nnodes=1 --nproc_per_node=8 --master_port 29605 hymm_sp/sample_batch.py \
    --ref-image './assets/images/seg_man_01.png' \
    --input-audio './assets/audios/milk_man.mp3' \
    --audio-strength 0.8 \
    --audio-condition \
    --pos-prompt "Realistic, High-quality. In the study, a man sits at a table featuring a bottle of milk while delivering a product presentation." \
    --neg-prompt "Two people, two persons, aerial view, overexposed, low quality, deformation, a poor composition, bad hands, bad teeth, bad eyes, bad limbs, distortion, blurring, text, subtitles, static, picture, black border." \
    --ckpt ${MODEL_BASE}"/hunyuancustom_audio_720P/mp_rank_00_model_states.pt" \
    --seed 1026 \
    --video-size 720 1280 \
    --sample-n-frames 129 \
    --cfg-scale 7.5 \
    --infer-steps 30 \
    --use-deepcache 1 \
    --flow-shift-eval-video 13.0 \
    --save-path './results/sp_audio_720p'

You can use the official HunyuanVideo Docker images to avoid manual dependency management. Pull and run the image corresponding to your CUDA version, then install the necessary Python packages for Gradio and inference.

# For CUDA 12.4
docker pull hunyuanvideo/hunyuanvideo:cuda_12
docker run -itd --gpus all --init --net=host --uts=host --ipc=host --name hunyuanvideo --security-opt=seccomp=unconfined --ulimit=stack=67108864 --ulimit=memlock=-1 --privileged hunyuanvideo/hunyuanvideo:cuda_12
pip install gradio==3.39.0 diffusers==0.33.0 transformers==4.41.2

# For CUDA 11.8
docker pull hunyuanvideo/hunyuanvideo:cuda_11
docker run -itd --gpus all --init --net=host --uts=host --ipc=host --name hunyuanvideo --security-opt=seccomp=unconfined --ulimit=stack=67108864 --ulimit=memlock=-1 --privileged hunyuanvideo/hunyuanvideo:cuda_11
pip install gradio==3.39.0 diffusers==0.33.0 transformers==4.41.2

Follow these steps to set up a manual installation on Linux. We recommend using CUDA versions 12.4 or 11.8.

1. Clone the repository and enter the directory.
2. Create and activate a Conda environment with Python 3.10.9.
3. Install PyTorch and dependencies based on your CUDA version.
4. Install pip requirements.
5. Install flash-attention v2 for acceleration (requires CUDA 11.8 or above).

# Clone the repository
git clone https://github.com/Tencent/HunyuanCustom.git
cd HunyuanCustom

# 1. Create conda environment
conda create -n HunyuanCustom python==3.10.9

# 2. Activate the environment
conda activate HunyuanCustom

# 3. Install PyTorch and other dependencies using conda
# For CUDA 11.8
conda install pytorch==2.4.0 torchvision==0.19.0 torchaudio==2.4.0 pytorch-cuda=11.8 -c pytorch -c nvidia
# For CUDA 12.4
conda install pytorch==2.4.0 torchvision==0.19.0 torchaudio==2.4.0 pytorch-cuda=12.4 -c pytorch -c nvidia

# 4. Install pip dependencies
python -m pip install -r requirements.txt

# 5. Install flash attention v2 for acceleration (requires CUDA 11.8 or above)
python -m pip install ninja
python -m pip install git+https://github.com/Dao-AILab/flash-attention.git@v2.6.3

If you encounter float point exceptions or core dumps on specific GPU types, use one of the following solutions:

Option 1: Ensure CUDA 12.4 compatibility Install nvidia-cublas-cu12 and set the LD_LIBRARY_PATH to point to the cublas library.

Option 2: Force CUDA 11.8 compiled PyTorch Uninstall all packages and reinstall PyTorch explicitly using the CUDA 11.8 wheel index, then reinstall the remaining dependencies.

# Option 1: Making sure you have installed CUDA 12.4, CUBLAS>=12.4.5.8, and CUDNN>=9.00
pip install nvidia-cublas-cu12==12.4.5.8
export LD_LIBRARY_PATH=/opt/conda/lib/python3.8/site-packages/nvidia/cublas/lib/

# Option 2: Forcing to explicitly use the CUDA 11.8 compiled version of Pytorch
pip uninstall -r requirements.txt
pip install torch==2.4.0 --index-url https://download.pytorch.org/whl/cu118
pip install -r requirements.txt
pip install ninja
pip install git+https://github.com/Dao-AILab/flash-attention.git@v2.6.3