cityscapesscripts

The dataset follows a specific hierarchical structure:

{root}/{type}{video}/{split}/{city}/{city}_{seq:0>6}_{frame:0>6}_{type}{ext}

Key Components:

• root: The base directory (can be set via CITYSCAPES_DATASET env var).
• type: The data modality (e.g., gtFine, leftImg8bit).
• split: The dataset split (e.g., train, val, test, train_extra, demoVideo).
• city: The recording city.
• seq: 6-digit sequence number.
• frame: 6-digit frame number.
• ext: File extension (e.g., .png, _polygons.json).

Install the core library using pip:

python -m pip install cityscapesscripts

If you need the graphical tools (the image viewer and the labeling tool), which are based on Qt5, install the GUI extra:

python -m pip install cityscapesscripts[gui]

To test your method on the official test set, run your approach on the provided test images and submit your results to the Submission Page.

Result formats are defined within the specific evaluation scripts:

• cityscapesscripts/evaluation/evalPixelLevelSemanticLabeling.py
• cityscapesscripts/evaluation/evalInstanceLevelSemanticLabeling.py
• cityscapesscripts/evaluation/evalPanopticSemanticLabeling.py
• cityscapesscripts/evaluation/evalObjectDetection3d.py

To enable the Cython plugin for faster evaluation (tested on Ubuntu), run the following command in the package directory:

CYTHONIZE_EVAL= python setup.py build_ext --inplace

The Cityscapes 3D annotations use four distinct coordinate systems for 3D boxes:

1. Vehicle Coordinate System (V): Follows ISO 8855. Origin is on the ground below the rear axis center. $x$ points in the driving direction, $y$ points left, and $z$ points up. All Ground Truth (GT) annotations are provided in this system.
2. Camera Coordinate System (C): Origin is at the camera's optical center, with the same orientation as $V$.
3. Image Coordinate System (I): Origin is at the top-left image pixel. $u$ points right and $v$ points down.
4. Coordinate System (S): Shares the same origin as $C$, but uses the orientation of $I$ ($x$ points right, $y$ points down, and $z$ points in the driving direction).

When working with Box3dImageTransform, note that the object's size is always provided as [L, W, H] regardless of the coordinate system used.

The Box3dImageTransform class is the primary tool for converting 3D box annotations between coordinate systems and projecting them into 2D image space.

Workflow:

1. Create a Box3dImageTransform instance using a Camera object.
2. Load the 3D annotation into a CsBbox3d object.
3. Initialize the transform object using initialize_box_from_annotation specifying the source coordinate system (usually CRS_V).
4. Use get_vertices or get_parameters to retrieve data in target coordinate systems (CRS_V, CRS_C, or CRS_S).
5. Use get_vertices_2d() to project the box into the image coordinate system ($I$).

The following split values define the data subsets:

You can retrieve the geometric properties of a 3D box in any supported coordinate system using get_vertices(coordinate_system=...) and get_parameters(coordinate_system=...).

Supported Coordinate Systems:

• CRS_V: Vehicle coordinate system.
• CRS_C: Camera coordinate system.
• CRS_S: Coordinate system with camera origin but image orientation.

Vertex Location Encoding: Vertices are returned in a dictionary where keys are 3-character codes representing the corner location:

• 0: Back (B) or Front (F)
• 1: Left (L) or Right (R)
• 2: Bottom (B) or Top (T) Example: BLT refers to the Back-Left-Top corner.

To find the 2D projection of a 3D box (useful for generating amodal 2D bounding boxes), use the get_vertices_2d() method. This returns a dictionary of vertices in the image coordinate system ($I$), where keys are the 3-character location codes (e.g., BLT) and values are [u, v] pixel coordinates.

# Returns vertices in image coordinate system (u, v)
box_vertices_I = box3d_annotation.get_vertices_2d()

# Example: calculating an amodal 2D bounding box from projected vertices
xmin = int(min([p[0] for p in box_vertices_I.values()]))
ymin = int(min([p[1] for p in box_vertices_I.values()]))
xmax = int(max([p[0] for p in box_vertices_I.values()]))
ymax = int(max([p[1] for p in box_vertices_I.values()]))

bbox_amodal = [xmin, ymin, xmax, ymax]