DOCSEARCHbyAlgolia
Submit Docs

MagicAnimate Documentation

repository·main·Indexed 27 days ago

https://github.com/magic-research/magic-animate

A diffusion-based model for temporally consistent human image animation that allows users to animate static human images using motion sequences. Includes instructions for installation, pretrained model organization, inference execution, and launching a local Gradio demo.

Tokens
690
Snippets
4
Records
4
Agent score
45%

What's inside MagicAnimate

  1. Download and organize pretrained models

    main

    To run MagicAnimate, you must download several pretrained models and place them in a specific directory structure under pretrained_models/.

    Required Models:

    1. StableDiffusion V1.5
    2. MSE-finetuned VAE
    3. MagicAnimate checkpoints

    Directory Structure:

    magic-animate
|----pretrained_models
  |----MagicAnimate
    |----appearance_encoder
      |----diffusion_pytorch_model.safetensors
      |----config.json
    |----densepose_controlnet
      |----diffusion_pytorch_model.safetensors
      |----config.json
    |----temporal_attention
      |----temporal_attention.ckpt
  |----sd-vae-ft-mse
    |----config.json
    |----diffusion_pytorch_model.safetensors
  |----stable-diffusion-v1-5
    |----scheduler
       |----scheduler_config.json
    |----text_encoder
       |----config.json
       |----pytorch_model.bin
    |----tokenizer (all)
    |----unet
       |----diffusion_pytorch_model.bin
       |----config.json
    |----v1-5-pruned-emaonly.safetensors
    magic-animate
|----pretrained_models
  |----MagicAnimate
    |----appearance_encoder
      |----diffusion_pytorch_model.safetensors
      |----config.json
    |----densepose_controlnet
      |----diffusion_pytorch_model.safetensors
      |----config.json
    |----temporal_attention
      |----temporal_attention.ckpt
  |----sd-vae-ft-mse
    |----config.json
    |----diffusion_pytorch_model.safetensors
  |----stable-diffusion-v1-5
    |----scheduler
       |----scheduler_config.json
    |----text_encoder
       |----config.json
       |----pytorch_model.bin
    |----tokenizer (all)
    |----unet
       |----diffusion_pytorch_model.bin
       |----config.json
    |----v1-5-pruned-emaonly.safetensors

  2. Install MagicAnimate

    main

    MagicAnimate requires python>=3.8, CUDA>=11.3, and ffmpeg. You can install the environment using either conda or pip.

    Using Conda:

    conda env create -f environment.yaml
conda activate manimate

    Using Pip:

    pip3 install -r requirements.txt
    conda env create -f environment.yaml
conda activate manimate

  3. Launch local Gradio demo

    main

    To interact with MagicAnimate via a web interface, launch the Gradio demo locally.

    Single GPU:

    python3 -m demo.gradio_animate

    Multiple GPUs:

    python3 -m demo.gradio_animate_dist

    After running the command, open the provided URL in your local browser.

    python3 -m demo.gradio_animate