DOCSEARCHbyAlgolia
Submit Docs

D4RL Documentation

repository·master·Indexed 23 days ago

https://github.com/farama-foundation/d4rl

An open-source benchmark for offline reinforcement learning providing standardized environments and datasets. D4RL uses the OpenAI Gym API for environment creation and provides tools for dataset retrieval, score normalization, and support for platforms like the Adroit manipulation platform. Note: D4RL is being phased out in favor of Gymnasium and Minari.

Tokens
927
Snippets
2
Records
6
Agent score
33%

What's inside D4RL

  2. Install D4RL

    master

    You can install D4RL by cloning the repository or via pip directly from GitHub.

    Option 1: Clone and install in editable mode

    git clone https://github.com/Farama-Foundation/d4rl.git
cd d4rl
pip install -e .

    Option 2: Install via pip

    pip install git+https://github.com/Farama-Foundation/d4rl@master#egg=d4rl

    Dependencies:

    • MuJoCo: Required for control environments. You may need to follow mujoco_py setup instructions and obtain a license.
    • CARLA: Requires additional setup instructions found in the project wiki.
    • Flow: Requires the SUMO simulator and adding the flow repository to your PYTHONPATH.
    git clone https://github.com/Farama-Foundation/d4rl.git
cd d4rl
pip install -e .

# OR

pip install git+https://github.com/Farama-Foundation/d4rl@master#egg=d4rl

  4. Configure D4RL dataset storage directory

    master

    By default, datasets are downloaded to ~/.d4rl/datasets when get_dataset() is called.

    You can change this behavior in two ways:

    1. Set the $D4RL_DATASET_DIR environment variable to your preferred path.
    2. Pass the dataset filepath directly into the get_dataset() method.

  5. Create environments and access datasets in D4RL

    master

    D4RL uses the OpenAI Gym API. Environments are created using gym.make(). Note that you must import d4rl to register the environments.

    Each task is associated with a fixed offline dataset. You can retrieve this dataset using env.get_dataset().

    Dataset Dictionary Keys:

    • observations: N x observation_dim array.
    • actions: N x action_dim array.
    • rewards: N dimensional array.
    • terminals: N dimensional array (true on termination conditions like falling over).
    • timeouts: N dimensional array (true on reaching max episode length).
    • infos: Task-specific debugging information.

    For Q-learning algorithms, use d4rl.qlearning_dataset(env) to get a version that includes a next_observations key.

    import gym
import d4rl # Required to register environments

# Create the environment
env = gym.make('maze2d-umaze-v1')

# Standard Gym interface
env.reset()
env.step(env.action_space.sample())

# Get standard dataset
dataset = env.get_dataset()
print(dataset['observations'])

# Get dataset formatted for Q-learning (includes next_observations)
dataset = d4rl.qlearning_dataset(env)

  6. Normalize scores using env.get_normalized_score()

    master

    To compute a normalized score for an episode, use env.get_normalized_score(returns). The returns argument should be the undiscounted total sum of rewards accumulated during an episode.

    Reference values for min and max scores are located in d4rl/infos.py.