Animated Drawings

The Retarget Config File maps 3D skeletal motion (from a BVH actor) onto a 2D Animated Drawing character rig. It defines how 3D joint positions are projected onto 2D planes, how character body parts are rendered in depth order, and how skeletal joint orientations translate to character bone rotations.

Key configuration areas include:

• Projection: Mapping BVH joints to 2D planes (frontal, sagittal, or pca).
• Depth/Rendering: Determining the order in which character body parts (e.g., arms vs. torso) are rendered to handle overlaps.
• Root Offset: Scaling and projecting the skeleton's 3D translation to the character's 2D movement.
• Joint Mapping: Defining how character bones rotate based on the orientation of pairs of BVH joints.
• Runtime Checks: Conditional removal of joint mappings based on the character's pose (e.g., preventing a 'neck' from flipping on certain character types).

The MVC (Model-View-Controller) configuration file is the top-level configuration passed into render.start() to generate animations. It follows a Model-View-Controller design pattern where the 'Model' is referred to as scene.

Important: Do not modify the base configuration file animated_drawings/mvc_base_cfg.yaml. Instead, create a new config file containing only the parameters you wish to override. The rendering script will merge your overrides into the base configuration.

The file is organized into three main subgroups:

1. scene (Model): Parameters for the animation environment.
2. view (View): Parameters for rendering, camera, and visual output.
3. controller (Controller): Parameters for execution mode (interactive vs. video rendering).

# Example structure of an MVC config file
scene: 
  ADD_FLOOR: True
  ANIMATED_CHARACTERS: 
    - character_cfg: "path/to/char.yaml"
      motion_cfg: "path/to/motion.yaml"
      retarget_cfg: "path/to/retarget.yaml"
view:
  WINDOW_DIMENSIONS: [1920, 1080]
controller:
  MODE: "video_render"
  OUTPUT_VIDEO_PATH: "output.mp4"

To automatically generate annotations (mask, texture, and joints) from a raw image, you can use a TorchServe container to run machine learning models.

1. Build and run the Docker container from the torchserve directory:

   cd torchserve
   docker build -t docker_torchserve .
   docker run -d --name docker_torchserve -p 8080:8080 -p 8081:8081 docker_torchserve

2. Verify the server is healthy:

   curl http://localhost:8080/ping

3. Run the animation pipeline from the examples directory:

   cd ../examples
   python image_to_animation.py <path_to_image> <output_folder>

   Example: python image_to_animation.py drawings/garlic.png garlic_out

# Build and run
cd torchserve
docker build -t docker_torchserve .
docker run -d --name docker_torchserve -p 8080:8080 -p 8081:8081 docker_torchserve

# Run animation
cd ../examples
python image_to_animation.py drawings/garlic.png garlic_out

The Quadruped Extension allows you to animate drawings as four-legged animals. The workflow involves three main steps: generating annotations from your drawing, converting the human-based configuration to an animal configuration, and finally rendering the animation using a scene configuration file.

1. Generate Annotations

Run image_to_animation.py on your drawing to create the initial character configuration.

2. Convert to Animal Configuration

Use human_to_animal.py to transform the generated configuration into an animal-compatible format. Note: human_to_animal.py must be located in the same directory as the newly generated files or you must provide its full path.

3. Configure and Render the Scene

Edit a scene configuration file (e.g., four_example.yaml) to point to your new animal configuration. You must update the character_cfg field under ANIMATED_CHARACTERS.

# Step 1: Generate annotations
python image_to_animation.py path/to/your/file 

# Step 2: Convert to animal config
python human_to_animal path/to/your/yaml

The Motion Configuration file defines how a BVH (BioVision Hierarchy) file drives the character animation.

Key parameters:

• filepath (str): Path to the BVH file.
• start_frame_idx / end_frame_idx (int): Range of frames to use from the BVH.
• frame_time (float): Override the BVH's internal frame time.
• groundplane_joint (str): A joint name used to adjust the worldspace y-offset so the joint sits at y=0 at the start frame.
• forward_perp_joint_vectors (list[List[str, str]]): Pairs of joint names used to calculate the skeleton's 'forward' vector during retargeting.
• scale (float): Uniform scale for the BVH skeleton.
• up (str): The 'up' direction in the BVH. Supported values: +y or +z.

filepath: "motions/dance.bvh"
start_frame_idx: 0
end_frame_idx: 100
frame_time: 0.033
groundplane_joint: "pelvis"
forward_perp_joint_vectors:
  - ["spine", "head"]
scale: 1.0
up: "+y"