OpenStereo

The StereoCarla dataset and its subsets are subject to the following restrictions:

• Academic Use Only: The dataset is strictly for academic research.
• No Commercial Use: Commercial use of the dataset or its subsets is prohibited.
• No Distribution: You may not forward, publish, or distribute the dataset or its subsets to any organization or individual without explicit permission.
• Requests: All requests for copies or sharing must be directed to the official contact email.

If the dataset is used in your research, please cite the following paper:

@article{guo2025stereocarla,
      title={StereoCarla: A High-Fidelity Driving Dataset for Generalizable Stereo}, 
      author={Xianda Guo and Chenming Zhang and Ruilin Wang and Youmin Zhang and Wenzhao Zheng and Matteo Poggi and Hao Zhao and Qin Zou and Long Chen},
      year={2025},
      journal={arXiv preprint arXiv:2509.12683}
}

To implement custom training logic, you must create a class that inherits from openstereo.stereo.modeling.trainer_template.TrainerTemplate. You can then override specific methods or define new ones required for your training workflow.

Once defined, you must 'mount' your custom trainer to your model. This is typically done within the trainer's __init__ method by instantiating the model using the configuration and passing it to the super().__init__ call.

from stereo.modeling.trainer_template import TrainerTemplate
from .psmnet import PSMNet

__all__ = {
    'PSMNet': PSMNet,
}

class Trainer(TrainerTemplate):
    def __init__(self, args, cfgs, local_rank, global_rank, logger, tb_writer):
        # Instantiate the model using the name provided in the config
        model = __all__[cfgs.MODEL.NAME](cfgs.MODEL)
        # Pass the model to the base TrainerTemplate
        super().__init__(args, cfgs, local_rank, global_rank, logger, tb_writer, model)

StereoCarla is a synthetic stereo dataset for autonomous driving built on the CARLA simulator. It includes various camera configurations (baselines, viewpoints, sensor placements) and environmental conditions (lighting, weather, road geometry).

1. Download: Obtain the dataset from https://xiandaguo.net/StereoCarla/.
2. Directory Structure: Ensure your dataset follows this specific hierarchy to be compatible with the project:

StereoCarla
└───normal
     └───town01
          ├───left
          ├───baseline_010
          │    ├───rgb
          │    ├───depth
          │    └───pose
          └───town10
     ├───pitch00
     ├───pitch30
     ├───roll05
     ├───roll15
     ├───roll30
     └───weather

Note: You can optionally provide your own .txt file to specify which parts of the dataset to use.

StereoCarla
└───normal
    └───town01
         └───left
         └───baseline_010
             └───rgb
             └───depth
             └───pose
     └───town10
└───pitch00
└───pitch30
└───roll05
└───roll15
└───roll30
└───weather

The C++ deployment example requires specific system dependencies and NVIDIA libraries.

1. Install System Dependencies

apt-get update
apt-get install libyaml-cpp-dev libopencv-dev python3-opencv

2. Prerequisites

Ensure you have installed:

• CUDA Toolkit
• TensorRT

3. Build Process

Navigate to deploy/cpp and create a build directory. You can build using the system's TensorRT or a specific TensorRT installation from a tar package.

Build with system TensorRT:

cmake .. && make

Build with a specific TensorRT path:

cmake -DTENSORRT_ROOT=<path_to_tensorrt> .. && make

Build with specific TensorRT path and CUDA architecture:

cmake -DTENSORRT_ROOT=<path_to_tensorrt> -DCMAKE_CUDA_ARCHITECTURES=<your_cuda_architecture> .. && make

4. Running Inference

Once built, run the ./main executable with the following arguments: ./main <cfg_path> <engine_path> <left_image_path> <right_image_path> <options>

Note: If using a TensorRT tar package, ensure LD_LIBRARY_PATH is correctly set. You can verify the links using ldd main | grep libnv*.

cd deploy/cpp
mkdir build && cd build
cmake -DTENSORRT_ROOT=/path/to/tensorrt -DCMAKE_CUDA_ARCHITECTURES=86 .. && make
./main config.yaml model.engine left.png right.png

To design a new stereo model, create a class that inherits from nn.Module. You must implement at least the forward() and get_loss() methods:

1. forward(self, input_data): Receives a dictionary input_data containing 'left' and 'right' images. It must return a dictionary containing the prediction, typically with the key 'disp_pred'.
2. get_loss(self, model_preds, input_data): Receives model_preds (the dictionary returned by forward) and input_data. Use input_data['disp'] to access the ground truth disparity for loss calculation.

class NewModel(nn.Module):
    def __init__(self, *args, **kwargs):
        super().__init__()

    def forward(self, input_data):
        left_img = input_data['left']
        right_img = input_data['right']
        
        # ... model logic ...
        
        return {'disp_pred': disp_pred}
    
    def get_loss(self, model_preds, input_data):
        disp_gt = input_data["disp"] 
        disp_pred = model_preds['disp_pred']
        # ... loss calculation ...