py123d Documentation

The nuPlan dataset provides several modalities used by py123d.

(✓) indicates availability depends on whether the user opted-in during download (e.g., via include_cameras=true).

Conversion configurations are stored in src/py123d/script/config/conversion/dataset/ as <name>.yaml files. You should provide two distinct configuration files for each dataset:

1. Local Config (<name>.yaml): Used when data is pre-downloaded. Roots point to local paths and sensors are typically stored as file paths.
2. Stream Config (<name>-stream.yaml): Used for 'streaming during conversion'. Roots are set to null (auto-detected in a temp dir), a downloader block is included, and sensors are stored as binary data in temporary files that vanish after conversion.

Each config contains four main sections:

• parser: Instantiates your BaseDatasetParser via _target_ and defines data roots/splits.
• log_writer_config: Configures storage options (e.g., camera_store_option, lidar_store_option, lidar_codec) and flags like async_conversion.
• log_writer: Configures the ArrowLogWriter, including sync_config for timeline snapping.
• map_writer: Configures the ArrowMapWriter.

The 123D library converts driving datasets into a unified format consisting of Logs, Maps, and Sensors.

Logs

Each log is a directory containing:

• Per-modality .arrow files (e.g., ego_state_se3.arrow, box_detections_se3.arrow, traffic_light_detections.arrow, camera.pcam_f0.arrow, lidar.lidar_merged.arrow).
• A sync.arrow synchronization table.
• Optional .mp4 files if MP4 camera compression is used.

Maps

Maps are stored as .arrow files and provide a unified HD-Map API. They can be defined:

• Per-log: (e.g., in AV2 or Waymo datasets).
• Globally: For specific locations (e.g., nuPlan, nuScenes, CARLA).

Sensors

Sensor data (Camera and Lidar) is handled in one of three ways:

1. Relative Paths: Read from the original dataset via paths stored in the .arrow files.
2. Embedded Binary: Data is embedded directly within the .arrow files.
3. MP4 Compression: Camera data is stored as .mp4 files alongside the .arrow files in the log directory.

Quaternions

123D uses unit quaternions (normalized to length 1) with a scalar-first convention. The identity rotation is (1, 0, 0, 0).

Euler Angles

123D follows the Tait-Bryan ZYX intrinsic convention (yaw $\rightarrow$ pitch $\rightarrow$ roll). Angles are always in radians and normalized to $[-\pi, \pi]$.

The combined rotation matrix is computed as: $R = R_z(\text{yaw}) \cdot R_y(\text{pitch}) \cdot R_x(\text{roll})$.

The conversion pipeline is organized around three base classes defined in py123d.parser.base_dataset_parser.BaseDatasetParser.

1. Dataset Parser: A top-level orchestrator that produces lightweight, picklable handles for logs and maps. It is instantiated once on the main process and distributed to parallel workers.
2. Log Parser: A handle to a single continuous driving sequence (a 'log'). Workers use these to perform heavy I/O and yield synchronized frames.
3. Map Parser: A handle to a specific map region.

Data flows from these parsers into ArrowLogWriter and ArrowMapWriter, which store the results in Arrow format for access via the unified API.

Important: Parsers must be lightweight. Do not store open file handles, database connections, or decoded sensor data in the parser classes. Store only paths and parameters; perform expensive I/O inside the iterators which run on the worker processes.

BaseDatasetParser
  ├── get_log_parsers()  ──▶  [ LogParser_1, ..., LogParser_N ]  ──▶  ArrowLogWriter
  └── get_map_parsers()  ──▶  [ MapParser_1, ..., MapParser_M ]  ──▶  ArrowMapWriter
                                                                            │
                                                                            ▼
                                                                  Arrow storage ──▶ unified API