AI4AnimationPy Documentation

AI4AnimationPy is a Python framework for AI-driven character animation using neural networks. It is designed to unify model research (PyTorch/NumPy) with visualization and animation engineering, removing the need for external game engines like Unity.

Key capabilities include:

• Training neural networks on motion capture data.
• Real-time visualization of training and inference.
• Modular architecture using an Entity-Component-System (ECS).
• Support for various execution modes: Standalone (with rendering), Headless (for server-side training), and Manual (for custom update loop control).

The AdamW implementation decouples weight decay from batch gradient calculations as described in the paper "Decoupled Weight Decay Regularization".

When used with the CyclicLRWithRestarts scheduler, the learning rate is adjusted on every batch update (rather than every epoch) to follow a cosine annealing schedule. This allows the model to converge to different local minima upon each restart. The scheduler also normalizes the weight decay hyperparameter according to the length of the restart period.

optimizer = AdamW(model.parameters(), lr=1e-3, weight_decay=1e-5)
scheduler = CyclicLRWithRestarts(optimizer, batch_size, epoch_size, restart_period=5, t_mult=1.2, policy="cosine")

The framework supports three primary execution modes depending on your needs:

1. Standalone: Uses the built-in rendering pipeline for full visualization.
2. Headless: Runs without a GUI, ideal for server-side training.
3. Manual: Provides manual control over the update loop, allowing you to specify exactly when and at what intervals the update loop is invoked. This is useful for running code locally or remotely on a server with custom timing requirements.

To use the CyclicLRWithRestarts scheduler correctly, you must call scheduler.step() at the start of each epoch and scheduler.batch_step() after every optimizer step (batch update), as the scheduler operates on a per-batch basis.

    batch_size = 32
    epoch_size = 1024
    model = resnet()
    optimizer = AdamW(model.parameters(), lr=1e-3, weight_decay=1e-5)
    scheduler = CyclicLRWithRestarts(optimizer, batch_size, epoch_size, restart_period=5, t_mult=1.2, policy="cosine")
    for epoch in range(100):
        scheduler.step()
        train_for_every_batch(...)
            ...
            optimizer.zero_grad()
            loss.backward()
            optimizer.step()
            scheduler.batch_step()
        validate(...)

AI4AnimationPy supports importing mesh, skin, and animation data from GLB, FBX, and BVH files. The framework's internal motion format is .npz, which stores 3D positions and 4D quaternions for each skeleton joint per frame.

You can load these files using the Motion class and save them to the internal .npz format for optimized use within the framework.

from ai4animation import Motion

motion = Motion.LoadFromGLB("character.glb")
motion = Motion.LoadFromFBX("character.fbx")
motion = Motion.LoadFromBVH("character.bvh", scale=0.01)
motion.SaveToNPZ("character")

Use the built-in convert command-line interface to batch convert entire directories of motion files to the internal .npz format.

convert --input_dir path/to/motions --output_dir path/to/output