DOCSEARCHbyAlgolia
Submit Docs

AnyTop: Character Animation with Diffusion Models

repository·main·Indexed 18 days ago

https://github.com/anytop2025/anytop

A PyTorch implementation for character animation using diffusion models that supports arbitrary skeleton topologies. AnyTop enables motion synthesis for both known skeletons from the TrueBones dataset and unseen custom skeletons via BVH files. The library includes tools for preprocessing skeletons, training specialized models for different animal subsets (Bipeds, Quadrupeds, Millipeds/Snakes, Flying), performing in-between and upper-body motion editing, and computing joint and temporal correspondences.

Tokens
3.2K
Snippets
13
Records
15
Agent score
13%

What's inside AnyTop

  1. Train AnyTop models

    main

    You can train specialized models for different animal subsets or a unified model.

    Training Commands

    • Unified Model: python -m train.train_anytop --model_prefix all --objects_subset all --lambda_geo 1.0 --overwrite --balanced
    • Bipeds: python -m train.train_anytop --model_prefix bipeds --objects_subset bipeds --lambda_geo 1.0 --overwrite --balanced
    • Quadrupeds: python -m train.train_anytop --model_prefix quadropeds --objects_subset quadropeds --lambda_geo 1.0 --overwrite --balanced
    • Millipeds/Snakes: python -m train.train_anytop --model_prefix millipeds_snakes --objects_subset millipeds_snakes --lambda_geo 1.0 --overwrite --balanced
    • Flying: python -m train.train_anytop --model_prefix flying --objects_subset flying --lambda_geo 1.0 --overwrite --gen_during_training --balanced

    Key Training Flags

    • --balanced: Activates the balancing sampler for fair skeleton sampling.
    • --overwrite: Resumes training from previous checkpoints.
    • --gen_during_training: Generates motions for each saved checkpoint (improves monitoring but slows training).
    • --use_ema: Uses Exponential Moving Average to improve performance.
    • --diffusion_steps <int>: Set to 50 to train a faster model.
    • --train_platform_type {WandBPlatform, TensorboardPlatform}: Tracks results via WandB or Tensorboard.

  2. Setup environment for TrueBones evaluation

    main

    To prepare the environment for evaluating the TrueBones dataset, install the required dependencies using conda and pip. You will need eigen from conda-forge, the ganimator-eval-kernel from GitHub, and pytorch3d.

    conda install -c conda-forge eigen
pip install git+https://github.com/PeizhuoLi/ganimator-eval-kernel.git
pip install pytorch3d

  3. Run the TrueBones evaluation

    main

    Execute the evaluation module using python -m. This command compares generated motion files against ground truth files based on a specified benchmark list.

    To reproduce the results reported in Tables 3 and 8 of the paper:

    1. Generate 20 samples (120 frames each) for every benchmark skeleton.
    2. Place all generated .npy files in a single directory.
    3. Run the command with --seed 10.

    Note: Released models may differ slightly from those in the paper but achieve comparable results.

    python -m --seed <seed> eval.eval_truebones --eval_gt_dir <gt_motion_dir> --eval_gen_dir <generated_motion_dir> --benchmark_path <benchmark_path> --unique_str <output_fname_suffix>

  4. Preprocess the Truebones dataset

    main

    If you have the Truebones dataset, you must preprocess it before training or inference.

    1. Download the dataset from the official Truebones website.
    2. Place the Truebone_Z-OO directory into ./dataset/truebones/zoo/ within the repository.
    3. Run the utils.create_dataset module to begin preprocessing.

    Note: This process may take several hours due to inverse kinematics calculations and MP4 generation.

    python -m utils.create_dataset

  5. Visualize Motions in Blender

    main

    Use the visualization/bvh2skeleton.py script to convert .bvh files into animated skeletons within Blender. The script generates .blend files for each input file.

    Prerequisites: You must install the following dependencies in Blender's Python environment:

    • git+https://github.com/inbar-2344/Motion.git
    • tqdm
    • scipy

    Arguments:

    • --bvh_path: Path to a single .bvh file or a directory of files.
    • --save_dir: Directory to save the generated .blend files.
    • --subset: Specifies a subset (e.g., bipeds).
    • --sphere_radius (Optional): Joint sphere radius.
    • --cylinder_radius (Optional): Bone cylinder radius.
    • --scale (Optional): Global scale factor.
    blender -b -P visualization/bvh2skeleton.py -- --bvh_path assets/Truebones_Chicken --save_dir save/blend_files --subset bipeds

  6. Preprocess a new skeleton (In-the-wild)

    main

    You can adapt AnyTop to custom skeletons (e.g., Mixamo BVH files) by running the utils.process_new_skeleton script. This creates the required cond.npy file for motion synthesis.

    Arguments

    • --object_name: A unique name for the character (e.g., Chicken). Ensure it doesn't conflict with names in Truebones_skeletons.txt.
    • --bvh_dir: Directory containing the BVH files for the skeleton.
    • --face_joints_names: Four joint names used to align the skeleton to the Z+ and XZ plane (e.g., [right hip, left hip, right shoulder, left shoulder]).
    • --save_dir: The output directory where processed data will be stored.
    • --tpos_bvh: (Optional) A BVH file representing the character's natural rest pose. If omitted, the script attempts to find a suitable pose from the provided BVH files.
    python -m utils.process_new_skeleton --object_name Chicken --bvh_dir assets/Truebones_Chicken --save_dir dataset/truebones/zoo/Chicken --face_joints_names Bip01_R_Thigh Bip01_L_Thigh BN_Finger_R_01 BN_Finger_L_01 --tpos_bvh assets/Truebones_Chicken/Chicken_TPOSE.bvh

  8. Setup the AnyTop environment

    main

    To use AnyTop, you need Ubuntu 18.04.5 LTS (or similar), Python 3.8, conda/miniconda, and a CUDA-capable GPU. Follow these steps to initialize the environment:

    1. Create and activate the conda environment using the provided environment.yaml.
    2. Install the Motion library directly from GitHub via pip.
    conda env create -f environment.yaml
conda activate anytop
pip install git+https://github.com/inbar-2344/Motion.git

  9. Generate motion for unseen skeletons (Custom BVH)

    main

    To synthesize motion for a custom skeleton not in the Truebones dataset, you must first run the process_new_skeleton pipeline to generate a cond.npy file. Then, run the sample.generate command providing the path to that file.

    Command

    python -m sample.generate --model_path <model_path> --object_type <skeleton_name> --num_repetitions 3 --cond_path <path_to_cond_npy_file>

    Additional Options

    • --device: Specify GPU ID.
    • --seed: Set a seed for reproducible sampling.
    • --motion_length: (Text-to-motion only) Duration in seconds (max 9.8s).

    Outputs

    • <object_type>_rep_<rep_id>_#<sample_id>.npy: XYZ positions of the animation.
    • <object_type>_rep_<rep_id>_#<sample_id>.mp4: Stick figure animation.
    • <object_type>_rep_<rep_id>_#<sample_id>.bvh: The generated motion BVH file.

  10. Generate motion for Truebones skeletons

    main

    Synthesize motion for skeletons belonging to the Truebones subsets (Bipeds, Quadrupeds, Millipeds, Snakes, or Flying Creatures) using a pretrained model checkpoint.

    Example (Flying subset): To generate motion for a Parrot2 or Bat using a flying model:

    python -m sample.generate --model_path save/flying_model_dataset_truebones_bs_16_latentdim_128/model000229999.pt --object_type Parrot2 Bat --num_repetitions 3

    You can also use a subset model to generate motion for an unseen skeleton (e.g., Tyranno) by passing its name to --object_type.

  11. Perform In-between Motion Editing

    main

    The in_between edit mode generates motion for the middle portion of a sample while keeping the beginning and end fixed.

    Arguments:

    • --edit_mode in_between: Sets the mode.
    • --model_path: Path to the trained model.
    • --object_type: The type of object/character.
    • --samples: Path to the input .npy motion sample.
    • --num_repetitions: Number of times to repeat the generation.
    • --prefix_end (Optional): Fraction of frames at the start to remain fixed (default: 0.25).
    • --suffix_start (Optional): Fraction of frames at the end to remain fixed (default: 0.75).
    python -m sample.edit --edit_mode in_between --model_path save/bipeds_model_dataset_truebones_bs_16_latentdim_128/model000329999.pt --object_type Ostrich --samples 'assets/Ostrich___Attack_581.npy' --num_repetitions 3

  12. Perform Upper-body Motion Editing

    main

    The upper_body edit mode generates motion specifically for the upper body of a character.

    Arguments:

    • --edit_mode upper_body: Sets the mode.
    • --model_path: Path to the trained model.
    • --object_type: The type of object/character.
    • --samples: Path to the input .npy motion sample.
    • --num_repetitions: Number of times to repeat the generation.
    • --upper_body_root: A list of joint indices defining the roots of the upper body sub-tree. If set to 0 (default), the entire skeleton is generated.
    python -m sample.edit --edit_mode upper_body --model_path save/bipeds_model_dataset_truebones_bs_16_latentdim_128/model000329999.pt --object_type Ostrich --samples 'assets/Ostrich___Attack_581.npy' --upper_body_root 26 --num_repetitions 3