repository·main·Indexed 25 days ago
https://github.com/leggedrobotics/rsl_rlA lightweight, GPU-accelerated reinforcement learning library implemented in PyTorch and optimized for robotics research. It features high-throughput Multi-GPU training and supports algorithms such as PPO and Distillation. RSL-RL is compatible with learning environments including Isaac Lab, Legged Gym, mjlab, and MuJoCo Playground. The library provides various model architectures (MLP, RNN, CNN), stochastic distributions, and extensions like Random Network Distillation (RND) and Symmetry augmentation.
To use RSL-RL with a custom environment, your environment must implement the :class:~rsl_rl.env.vec_env.VecEnv`` abstract interface. Specifically, it must implement the following methods:
:func:~rsl_rl.env.vec_env.VecEnv.step``:func:~rsl_rl.env.vec_env.VecEnv.get_observations``To use RSL-RL as a library in your project, install it via pip. Ensure you have Python 3.9+ installed and that you have activated your desired virtual environment (e.g., venv or conda) before running the installation command.
pip install rsl-rl-libIf you intend to contribute to RSL-RL or need to modify its source code, install it in editable mode. Clone the repository and use pip install -e . to ensure changes to the source are reflected in your environment.
git clone https://github.com/leggedrobotics/rsl_rl
cd rsl_rl
pip install -e .:func:~rsl_rl.utils.utils.resolve_callable`` utility to resolve these classes from the configuration.RSL-RL uses TensorDict to handle observations returned by the environment's VecEnv.step method. These observations are organized into observation groups.
You configure how these groups are assigned to different model components (like the actor or critic) using the obs_groups dictionary in your runner configuration. The obs_groups dictionary maps an observation set (a key like actor or critic) to a list of observation groups (the keys present in the TensorDict).
For example, if your environment returns a policy tensor (for deployment) and a privileged tensor (only for training), you can configure the actor to only see the policy group while the critic sees both.
Distillation algorithm configuration is passed via the algorithm key in the runner configuration. It is used for training a student model from a teacher.OnPolicyRunner is used for standard on-policy reinforcement learning. It requires a nested configuration dictionary containing settings for observations, environment steps, saving intervals, and sub-dictionaries for the algorithm, actor, and critic models.GaussianDistribution for stochastic model outputs with a standard deviation.Symmetry augmentation to add symmetric trajectories and loss terms to the training process.MLPModel for standard multi-layer perceptron architectures. It is the base model for other types like RNNModel and CNNModel.