RoboJuDo Documentation

A Policy is the core component responsible for robot control. It acts as the bridge between the environment and the robot's actuators by:

1. Receiving env_data from the environment.
2. Receiving ctrl_data from the controller.
3. Organizing observations.
4. Inferring the final actions for the robot.

Policy serves as the base class for all specific policy implementations (e.g., UnitreePolicy, AMOPolicy, AsapPolicy).

In RoboJuDo, a Controller is the component responsible for providing ctrl_data to the robot. Controller serves as the base class for all specific controller implementations, defining a common interface.

Available controllers include:

• JoystickCtrl: For joystick-based control.
• UnitreeCtrl: For controlling Unitree robots via UnitreeG1.
• KeyboardCtrl: For keyboard-based control.
• MotionCtrl: For providing reference motion (typically used in mimic tasks).
• BeyondMimicCtrl: Specifically designed for use with BeyondMimicPolicy.

ProtoMotionsTrackerPolicy deploys a ProtoMotions tracker exported as a unified ONNX pipeline.

Prerequisite: You must clone the ProtoMotions repository locally as a sibling directory named protomotions. RoboJuDo relies on importing deployment.motion_utils and deployment.state_utils from that specific local checkout at runtime.

For detailed deployment workflows, refer to the official ProtoMotions G1 deployment documentation.

RoboJuDo provides built-in access to rich kinematic data, including link poses, joint positions, and end-effector states. This allows for advanced control and monitoring without extra computation.

Tip: Use the debug_viz option for real-time sim2real monitoring. Configuration details can be found in ForwardKinematicCfg within robojudo/tools/tool_cfgs.py.

RoboJuDo supports odometry to track a robot's global position and velocity, which is essential for reliable deployment and motion alignment. There are two supported options:

• ZED: Uses a ZED camera for odometry. This requires the zed_proxy submodule.
• UNITREE: Uses the built-in sport state service provided by Unitree robots.

RoboJuDo uses DoFConfig (defined in robojudo/tools/tool_cfgs.py) to manage joint properties like joint_names, stiffness, and default_pos. It supports several advanced deployment scenarios:

1. Subset extension & cropping: Use the _subset parameter to handle locked Degrees of Freedom (DoFs). For example, a G1_12DoF configuration can be derived from a larger set.
2. Dynamic policy override: You can run a policy with a different number of DoFs than the environment. For example, a 12-DoF policy can be executed on a 29-DoF environment.
3. Strict validation: All configurations are validated within DoFConfig to ensure safety during execution.

RoboJuDo is a decoupled framework designed to bridge the gap between policy research and robot deployment. It relies on three core abstractions that are freely composable:

• Controller: Collects external inputs (e.g., joystick, keyboard, motion sequences) and forwards them as ctrl_data to the pipeline.
• Environment: The execution target (e.g., Mujoco simulation or a real robot). It processes actions from the policy and sends real-time sensor data as env_data to the pipeline.
• Policy: The trained control model. It generates actions by consuming data from both the Environment and the Controller.

This design allows users to swap policies, environments, or controllers with minimal code changes.

To set up the RoboJuDo environment, clone the repository, create a new Conda environment with Python 3.11, and install the package in editable mode. You can optionally install the CPU version of PyTorch first to speed up the installation process.

git clone https://github.com/HansZ8/RoboJuDo.git
cd RoboJuDo/
# Example using conda
conda create -n robojudo python=3.11 -y
conda activate robojudo

# Optional, install cpu version for speed up
pip install torch --index-url https://download.pytorch.org/whl/cpu
pip install -e .

Use UnitreeCppEnv for UnitreeG1 robots. This environment is based on unitree_cpp and is much faster and more stable, making it suitable for deployment on the G1's onboard pc2 computer.

1. Install the official Unitree C++ SDK unitree_sdk2. It is recommended to use the default installation path.
2. Install the unitree_cpp package using the RoboJuDo submodule installer:

   python submodule_install.py unitree_cpp

   Note: unitree_sdk2 must be installed before running this command.
3. Verify the installation by running:

   python -c "from robojudo.environment import UnitreeCppEnv"

   If an error occurs, check the installation of both unitree_sdk2 and unitree_cpp.

python submodule_install.py unitree_cpp
# Then verify
python -c "from robojudo.environment import UnitreeCppEnv"

RoboJuDo uses a plug-and-play module system. To install specific modules (like robot SDKs or specific environments):

1. Configure: Edit submodule_cfg.yaml and set install: true for the desired modules.
2. Install: Run the submodule_install.py script.

Note: If you are controlling a real robot, you must first install the corresponding Robot SDK (e.g., Unitree SDK) as described in docs/unitree_setup.md.

# Install all required modules
python submodule_install.py

# Or specify modules to install with args
python submodule_install.py unitree_cpp

To run a policy directly on the robot's onboard computer:

1. Clone the RoboJuDo repository and set up the environment on the robot's onboard computer (refer to the Basic Setup guide).
2. Install the appropriate Unitree SDK (UnitreeEnv or UnitreeCppEnv) as described in the SDK Installation section.
3. Note for G1 users: Because the G1 has limited computing resources, it is highly recommended to use UnitreeCppEnv.
4. Ensure the network interface is correctly identified. For G1, this is typically eth0.