DreamerV3 Documentation

Execute the training script using dreamerv3/main.py. You must specify a --logdir for outputs and a --configs block to define the task. To reproduce specific results, use the corresponding config and task flags (e.g., --configs atari --task atari_pong).

python dreamerv3/main.py \
  --logdir ~/logdir/dreamer/{timestamp} \
  --configs crafter \
  --run.train_ratio 32

To visualize training results, install the scope package and run the scope.viewer module, pointing it to your base log directory.

pip install -U scope
python -m scope.viewer --basedir ~/logdir --port 8000

To install DreamerV3, ensure you are on Linux or Mac and using Python 3.11+. First, install JAX, then install the remaining dependencies using the provided requirements file.

pip install -U -r requirements.txt

DreamerV3 uses a configuration system defined in dreamerv3/configs.yaml. You can override any configuration option using command line flags.

Key patterns:

• Multiple Configs: You can stack multiple config blocks to override defaults in the order specified (e.g., --configs crafter size50m).
• Hardware Selection: Use --jax.platform cpu to switch from the default GPU to CPU or TPU.
• Debugging: Use the debug config block to reduce network size, batch size, and log frequency for faster testing.
• Resuming Training: To continue a stopped run, execute the same command and ensure --logdir points to the existing directory.

Common issues and solutions:

• Too many leaves for PyTreeDef: This occurs when reloading a checkpoint incompatible with the current config (e.g., reusing an old logdir with different settings).
• CUDA errors: Often caused by earlier errors like Out of Memory (OOM) or JAX/CUDA version mismatches. To test if OOM is the cause, try setting --batch_size 1.
• Environment dependencies: Some environments require additional packages. Refer to the Dockerfile for a complete list of required system/python dependencies.

When training with low-precision dtypes (like bfloat16), DreamerV3 supports prioritized and recency-based replay sampling. If config.replay.fracs.uniform is less than 1, the system uses a Mixture of selectors:

• uniform: Uniform sampling.
• priority: Prioritized Experience Replay (configured via config.replay.prio).
• recency: Sampling based on how recent the transition is (configured via config.replay.recexp).

DreamerV3 uses a hierarchical configuration system. The script key determines the execution logic in main.py. The task key determines the environment suite and specific task (e.g., atari_breakout or dmc_crafter).

Supported environment suites include:

• atari, atari100k
• dmc (DeepMind Control Suite)
• crafter
• dmlab
• gym
• loconav
• minecraft
• procgen
• bsuite
• memmaze
• pinpad
• langroom
• dm (DeepMind)

Configuration is typically passed as command-line flags that override the defaults in configs.yaml.

The make_logger function defines how metrics and scores are recorded. You can specify different output types in your configuration under logger.outputs. Supported types include:

• jsonl: Writes metrics.jsonl and scores.jsonl to the log directory.
• tensorboard: Enables TensorBoard logging with a specified fps.
• wandb: Enables Weights & Biases logging.
• expa: Specialized output for specific experiment tracking.
• scope: Logs to a specific scope within the directory.
• terminal: (Default) Prints to the terminal using elements.print.

The main.py script serves as the entrypoint for DreamerV3. It uses a configuration system based on configs.yaml and elements.Flags. You can control the execution mode using the script configuration key.

Available script modes:

• train: Standard training loop.
• train_eval: Training loop with evaluation steps.
• eval_only: Evaluation of a trained agent.
• parallel: Combined parallel execution.
• parallel_env: Parallel environment execution.
• parallel_envs: Parallel environment execution (variant).
• parallel_replay: Parallel replay buffer execution.

Configuration is loaded from configs.yaml and can be overridden via command-line arguments. The logdir is automatically updated with a timestamp unless specified otherwise.