FlyGym (NeuroMechFly)

FlyGym is a Python library providing a digital twin of the adult fruit fly Drosophila melanogaster. It is designed for simulating embodied sensorimotor control.

Key features include:

• Biomechanical Model: Based on micro-CT scans of a real adult female fly with adjusted body segments (e.g., antennae).
• Vision: Simulated compound eyes with ommatidia arranged on a hexagonal lattice.
• Olfaction: Simulated odor/chemical intensity at receptors in the antennae and maxillary palps.
• Hierarchical Control: Supports a two-part CNS model consisting of a Brain (sensory integration and decision making) and a Ventral Nerve Cord (VNC) (motor control), connected via descending and ascending representations.
• Leg Adhesion: Simulates specialized adhesive structures at leg tips, which can be toggled on/off during leg stance/swing to model locomotion on vertical or overhanging surfaces.
• Mechanosensory Feedback: Provides access to joint angles, actuator forces, contact forces, and user-defined anatomical joint-site positions.

FlyGym is a framework for simulating embodied sensorimotor control using NeuroMechFly v2. To begin using the library, you can follow these primary paths:

1. Installation: Set up the environment and dependencies.
2. API Migration: If you are moving from FlyGym v1 to v2, refer to the migration guide.
3. Tutorials: Follow structured learning paths including:
   • Model Composition: Learning how to compose models and scenes (Basic and Advanced).
   • Experimental Replay: Replaying experimental recordings or using the FlyBody model.
   • Simulation: Running GPU-accelerated simulations.
   • Locomotion Controllers: Implementing CPG, rule-based, hybrid, or turning controllers.
   • Machine Learning: Implementing muscle-based imitation learning.
   • Optimization: Performance profiling.
4. API Reference: Deep dive into specific modules like anatomy, compose (fly, physics, pose, world), rendering, simulation, and warp.

The game demonstrates three distinct levels of abstraction for controlling animal behavior, ranging from high-level decisions to low-level limb management:

1. Level 1: High-level control (Central Pattern Generators - CPG)

   • Concept: Uses CPGs, which are neural oscillators that produce rhythmic movement patterns (like walking) without needing rhythmic input.
   • Control: You decide on direction (forward, backward, left, right) using a joystick or keyboard. The CPG circuits handle the complex motor coordination automatically.

2. Level 2: Medium-level control (Fixed gait pattern)

   • Concept: Focuses on specific movement patterns, such as the "tripod gait" used by insects. In a tripod gait, three legs move together to form a stable base while the other three move.
   • Control: You control groups of three legs to move forward or backward. This requires more precision than Level 1.

3. Level 3: Low-level control (Individual legs)

   • Concept: Mimics the conscious management of limb placement (similar to a climber on a cliff). Instead of controlling muscles directly, you control individual legs.
   • Control: You move one leg at a time using a joystick or keyboard. This is the most challenging level of control.

FlyGym has transitioned from a Gymnasium-compliant interface (v1.x.x) to a custom, high-performance interface (v2.x.x).

FlyGym 1.x.x (via flygym-gymnasium)

• Interface: Strictly follows the Gymnasium API.
• Use Case: Best for users who need immediate compatibility with existing RL libraries and tutorials built for the Gymnasium standard.

FlyGym 2.x.x (via flygym)

• Interface: Custom API designed for direct and flexible interaction with the simulation.
• Performance: Achieves approximately a 10× speed-up on CPU-based simulations compared to v1.x.x.
• Key Feature: Supports lazy-loading of simulated states rather than exposing all pre-configured states as a single "observation" block.
• Mathematical Model: Despite the interface change, it remains fully compatible with Partially Observable Markov Decision Process (POMDP) principles.

Note for v2 users: If you need Gymnasium compatibility while using the faster v2 engine, it is recommended to write a lightweight wrapper around the v2 API rather than using an official compatibility layer.

The game architecture consists of the following components:

• Physics Engine: MuJoCo (WebAssembly) running at dt=1e-4. It runs at a fixed playback speed (approx. 0.1x real-time) controlled by PLAYBACK_SPEED in game.js.
• Controller (game.js): A JavaScript port of the Python control logic. It uses a Controller class to manage CPG networks (Level 1) and per-leg/tripod state machines (Levels 2-3). It uses baked tables from assets/model_meta.json to avoid needing SciPy in the browser.
• Rendering: Three.js for 3D visualization.
• File Structure:
  • game.html: Contains the stage, HUD (level, timer, controls), and overlay screens.
  • game.js: The core loop containing MuJoCo-WASM, ported controllers, and Three.js rendering.
  • assets/model/fly.xml: The flattened slalom-arena MJCF model.
  • assets/model_meta.json: Contains timestep, neutral keyframes, actuator/adhesion data, CPG parameters, and baked trajectory tables.