DOCSEARCHbyAlgolia
Submit Docs

Gym

repository·master·Indexed 13 days ago

https://github.com/openai/gym

An open-source Python library providing a standardized API for reinforcement learning (RL) algorithms to communicate with various environments. It serves as a benchmark for developing and comparing RL algorithms, featuring a system of wrappers for modular environment modification and strict versioning for reproducibility. Note: Future development has moved to Gymnasium, which serves as a drop-in replacement.

Tokens
1.1K
Snippets
3
Records
7
Agent score
49%

What's inside Gym

  1. Quick tips for writing your own wrapper

    master

    When implementing a custom wrapper, follow these implementation guidelines to ensure it integrates correctly with the Gym ecosystem:

    • Initialization: If you override the __init__ function, you must call super(class_name, self).__init__(env) to properly initialize the wrapper.
    • Accessing Layers:
      • Use self.env to access the immediate previous layer in the wrapper chain.
      • Use self.unwrapped to access the base, original environment (bypassing all wrappers).
    • Attribute Inheritance: The following attributes are automatically copied from the previous layer to self:
      • metadata
      • action_space
      • observation_space
      • reward_range
      • spec
    • Method Overriding: To have an effect, your wrapper should override at least one of the following methods: __init__(self, env), step, reset, render, close, or seed.
    • Data Flow: Your overridden methods should typically take inputs from the previous layer (self.env) or the inner environment (self.unwrapped).

  2. Understand Gym environment versioning

    master
    Gym uses strict versioning for environments to ensure reproducibility. Every environment ID ends with a version suffix (e.g., _v0). If changes are made to an environment that could impact learning results, the version number is incremented to prevent confusion and ensure existing code continues to behave as expected.

  3. Install Gym and environment dependencies

    master

    To install the base Gym library, use pip install gym.

    Because many environment families have large or platform-specific dependencies, you can install specific sets using extras:

    • Atari environments: pip install gym[atari]
    • All environments: pip install gym[all]
    • Latest MuJoCo environments: pip install gym[mujoco]
    • Legacy MuJoCo environments (mujoco-py): pip install gym[mujoco_py]

    Supported Platforms:

    • Linux and macOS (Python 3.7, 3.8, 3.9, and 3.10).
    • Windows is not officially supported, though PRs related to it are accepted.
    pip install gym
pip install gym[atari]
pip install gym[all]
pip install gym[mujoco]

  4. Migrate from Gym to Gymnasium

    master

    Important Notice

    The Gym project has moved all future development to Gymnasium. Gymnasium is a drop-in replacement for Gym. To use it, you should change your imports from import gym to import gymnasium as gym.

    Gym will not receive any further updates. It is highly recommended to switch to Gymnasium as soon as possible.

  5. Transform environments using Wrappers

    master

    Wrappers allow you to modify or extend an environment's behavior in a modular way by wrapping an existing environment instance. You can chain multiple wrappers together to apply several transformations sequentially.

    When importing wrappers, always use the top-level gym.wrappers package to ensure compatibility even if the internal file structure changes.

    import gym
from gym.wrappers import MyWrapper

env = gym.make('Pong-v0')
env = MyWrapper(env)

  6. Interact with Gym environments

    master

    Gym models environments as Python classes. You can create an environment instance using gym.make() and interact with it using reset() and step().

    In the standard API, reset() returns an observation and an info dictionary. The step(action) method returns a tuple containing observation, reward, terminated, truncated, and info.

    import gym

env = gym.make("CartPole-v1")
observation, info = env.reset(seed=42)

for _ in range(1000):
    action = env.action_space.sample()
    observation, reward, terminated, truncated, info = env.step(action)

    if terminated or truncated:
        observation, info = env.reset()

env.close()

  7. MuJoCo environment dependencies

    master

    The MuJoCo environment implementation has transitioned away from mujoco-py:

    • Newer versions (_v4 and later): Depend on the mujoco package. Install via pip install gym[mujoco].
    • Legacy versions: Depend on mujoco-py. These are kept for compatibility but are unmaintained. Install via pip install gym[mujoco_py].