ScenarioNet

ScenarioNet is an open-source platform designed for large-scale traffic scenario modeling and simulation. It provides a unified scenario description format that includes HD maps and detailed object annotations.

Key capabilities include:

• Database Management: Tools to build and manage databases from real-world datasets (Waymo, nuScenes, Lyft L5, nuPlan) and synthetic datasets (procedural or safety-critical).
• Multi-View Replay: Scenarios can be replayed in digital twins using various views, from Bird-Eye-View (BEV) layouts to realistic 3D rendering.
• Broad Application Support: Enables large-scale scenario generation, Autonomous Driving (AD) testing, imitation learning, and reinforcement learning in both single-agent and multi-agent environments.

ScenarioNet is an open-source platform for large-scale traffic scenario simulation and modeling. It is powered by the MetaDrive Simulator and provides the following capabilities:

• Scenario Loading: Load scenarios from real-world datasets (Waymo, nuPlan, nuScenes, l5) or synthetic datasets (procedurally generated or adversarial/safety-critical scenarios).
• Dataset Management: Tools for building training and test sets for Machine Learning applications.
• Reconstruction: Reconstruct scenarios for various applications including Autonomous Driving (AD) stack testing, Reinforcement Learning (RL), Imitation Learning, and scenario generation.

The Lyft dataset contains logs of movement for cars, cyclists, pedestrians, and other traffic agents, processed from raw lidar, camera, and radar data. It is suitable for training motion prediction models.

Dataset Details:

• 1000+ hours of traffic agent movement
• 16k miles of data from 23 vehicles
• 15k semantic map annotations

External Resources:

• Website: https://woven.toyota/en/prediction-dataset
• Download: https://woven-planet.github.io/l5kit/dataset.html
• Paper: https://proceedings.mlr.press/v155/houston21a/houston21a.pdf

Support for Lyft datasets in ScenarioNet depends on the version of the data being used:

1. Old Lyft Dataset: Can be read using nuscenes-toolkit and shares the nuScenes converter.
2. New Lyft Dataset: Maintained by Woven Planet. Support for this version via L5Kit is currently under development.

ScenarioNet provides interactive Colab notebooks for quick experimentation without local setup:

• Running a simulation: Use the Simulation Tutorial to learn how to execute simulations.
• Reading datasets: Use the Dataset Tutorial to learn how to load and inspect established ScenarioNet datasets.

To ensure nuplan-devkit can locate your data, add the following environment variables to your ~/.bashrc file:

export NUPLAN_DATA_ROOT="$HOME/nuplan/dataset"
export NUPLAN_MAPS_ROOT="$HOME/nuplan/dataset/maps"
export NUPLAN_EXP_ROOT="$HOME/nuplan/exp"

After setting these, you can verify the installation by running the tutorial located at nuplan-devkit/tutorials/nuplan_scenario_visualization.ipynb.

To use ScenarioNet with nuPlan, you must first install the nuplan-devkit. It is recommended to install from GitHub to access examples and source code for verifying your installation.

Option 1: Install from GitHub (Recommended)

# Clone the repository
git clone git@github.com:nutonomy/nuplan-devkit.git
cd nuplan-devkit

# Install dependencies and the package in editable mode
pip install -r requirements.txt
pip install -e .

# Install additional required package
pip install pytorch-lightning

Option 2: Install from PyPI

pip install nuplan-devkit

To build the ScenarioNet documentation from source, install the package with documentation dependencies in editable mode and use make html within the documentation directory.

cd scenarionet
pip install -e .[doc]
cd documentation
make html

1. Access Data: Register at the official VoD website to receive a download link. Download and unzip view_of_delft_prediction_PUBLIC.zip.

2. Directory Structure: Organize the unzipped files into the following structure:

/vod/data/path/
├── maps/
|   └── expansion/
├── v1.0-trainval/
|   ├── attribute.json
|   ├── calibrated_sensor.json
|   ├── map.json
|   ├── log.json
|   ├── ego_pose.json
|   └── ...
└── v1.0-test/

3. Data Location: By default, vod-devkit expects the data to be at /data/sets/vod. If you store the data elsewhere, you must either:
   • Create a symbolic link from your data folder to /data/sets/vod.
   • Explicitly specify the --dataroot flag when calling vod-devkit APIs or ScenarioNet converters.

The PG scenarios are generated by running simulations in the MetaDrive simulator and recording the episodes. To build the PG database, you must have MetaDrive installed. You can use the scenarionet.convert_pg module to generate a specified number of scenarios and save them to a directory.

Use the following command to generate 1000 scenarios:

python -m scenarionet.convert_pg -d /path/to/pg_database --num_scenarios 1000

To convert Waymo motion data into ScenarioNet's internal Scenario Description, you must install tensorflow and protobuf.

Note: If pip install protobuf==3.20 fails, use conda instead.

pip install tensorflow==2.11.0
conda install protobuf==3.20