VideoCrafter Documentation

To set up the VideoCrafter environment, it is recommended to use Anaconda. Create a new environment with Python 3.8.5 and install the required dependencies from requirements.txt.

conda create -n videocrafter python=3.8.5
conda activate videocrafter
pip install -r requirements.txt

To generate video from an image:

1. Download the pretrained I2V model from Hugging Face.
2. Place the model.ckpt file in the directory: checkpoints/i2v_512_v1/model.ckpt.
3. Execute the inference script via shell.

sh scripts/run_image2video.sh

To generate video from text:

1. Download the pretrained T2V model from Hugging Face.
2. Place the model.ckpt file in the directory: checkpoints/base_512_v2/model.ckpt.
3. Execute the inference script via shell.

sh scripts/run_text2video.sh

To use the interactive Gradio web interface, ensure you have downloaded the pretrained T2V and I2V models and placed them in their corresponding directories as specified in the inference guides, then run the Gradio application.

python gradio_app.py

The following models are available for download via Hugging Face:

Text-to-Video (T2V) Models:

• VideoCrafter2 (320x512)
• VideoCrafter1 (576x1024)
• VideoCrafter1 (320x512)

Image-to-Video (I2V) Models:

• VideoCrafter1 (640x1024) - Note: This is the DynamiCrafter model
• VideoCrafter1 (320x512)