DOCSEARCHbyAlgolia
Submit Docs

ToonCrafter Documentation

repository·main·Indexed 27 days ago

https://github.com/doubiiu/tooncrafter

A generative cartoon interpolation tool using pre-trained image-to-video diffusion priors to interpolate between two cartoon images. It supports sparse sketch guidance, reference-based sketch colorization, and provides a local Gradio demo. The ToonCrafter_512 model operates at 512x320 resolution and can generate videos up to 16 frames.

Tokens
359
Snippets
3
Records
4
Agent score
41%

What's inside ToonCrafter

  1. Install ToonCrafter via Anaconda

    main

    To set up the ToonCrafter environment, it is recommended to use Anaconda. Create a new environment with Python 3.8.5 and install the required dependencies from requirements.txt.

    conda create -n tooncrafter python=3.8.5
conda activate tooncrafter
pip install -r requirements.txt

  2. Run ToonCrafter via Command Line

    main

    To perform inference via the command line, first download the pretrained ToonCrafter_512 model. Place the model.ckpt file in the following directory structure:

    checkpoints/tooncrafter_512_interp_v1/model.ckpt

    Then, execute the provided shell script to run the inference.

    sh scripts/run.sh

  3. Run ToonCrafter Local Gradio Demo

    main

    To launch a local web interface using Gradio, ensure the pretrained model is placed in the correct directory as specified in the command line instructions, then run the gradio_app.py script.

    python gradio_app.py

  4. Model Specifications and Requirements

    main

    The primary model available is ToonCrafter_512.

    • Resolution: 512x320
    • Capabilities: Supports generating videos up to 16 frames.
    • GPU Memory: Approximately 24GB (though community implementations may lower this to ~10GB).
    • Inference Time: ~24s on an A100 (using 50 DDIM steps with perframe_ae=True).

    Checkpoint Download: Hugging Face