monodepth2

To use the KITTI dataset, download the archives and unzip them into kitti_data/.

Note: The dataset is approximately 175GB.

By default, the code expects images to be converted from .png to .jpg with specific chroma subsampling. This conversion command also deletes the original .png files:

find kitti_data/ -name '*.png' | parallel 'convert -quality 92 -sampling-factor 2x2,1x1,1x1 {.}.png {.}.jpg && rm {}'

Alternatively, you can train directly from raw .png files by adding the --png flag to your training command, though this will result in slower load times.

wget -i splits/kitti_archives_to_download.txt -P kitti_data/
cd kitti_data
unzip "*.zip"
cd ..

# Convert PNG to JPG (recommended):
find kitti_data/ -name '*.png' | parallel 'convert -quality 92 -sampling-factor 2x2,1x1,1x1 {.}.png {.}.jpg && rm {}'

To set up a fresh environment using Anaconda, install the required dependencies. It is recommended to use Python 3.6.6 to avoid OpenCV installation issues on newer Python versions.

To create a recommended environment:

conda create -n monodepth2 python=3.6.6 anaconda

Then install the specific versions:

conda install pytorch=0.4.1 torchvision=0.2.1 -c pytorch
pip install tensorboardX==1.4
conda install opencv=3.3.1

Note: The code is compatible with Python 2.7 and has been successfully trained with PyTorch 1.0.

Use evaluate_depth.py to evaluate trained depth models.

For monocular models, use the --eval_mono flag. For stereo models, use the --eval_stereo flag.

Note on Stereo Evaluation: Using --eval_stereo automatically disables median scaling and applies a scaling factor of 5.4 (to account for the difference between the model's effective baseline of 0.1 and the actual KITTI stereo rig baseline of 0.54m).

Before evaluating depth, you must prepare the ground truth depth maps using export_gt_depth.py. This assumes the KITTI dataset is located in ./kitti_data/.

python export_gt_depth.py --data_path kitti_data --split eigen
python export_gt_depth.py --data_path kitti_data --split eigen_benchmark

To evaluate poses predicted by models trained with --split odom, you must download the KITTI odometry dataset (color, 65GB) and ground truth poses. Ensure PNGs are converted to JPGs. If the data is unzipped to kitti_odom/, use the following pattern:

python evaluate_pose.py --eval_split odom_9 --load_weights_folder ./odom_split.M/models/weights_29 --data_path kitti_odom/
python evaluate_pose.py --eval_split odom_10 --load_weights_folder ./odom_split.M/models/weights_29 --data_path kitti_odom/

Models and Tensorboard event files are saved to ~/tmp/<model_name> by default, but you can specify a custom directory using --log_dir.

Use the following commands for different training modalities:

• Monocular: python train.py --model_name <name>
• Stereo: Requires specifying --use_stereo and a split (e.g., --split eigen_full).
• Monocular + Stereo: Requires --use_stereo and specifying frame IDs (e.g., --frame_ids 0 -1 1).

To use a specific GPU, set the CUDA_VISIBLE_DEVICES environment variable.

To finetune an existing model, use the --load_weights_folder flag to point to the directory containing the weights you wish to load.

python train.py --model_name finetuned_mono --load_weights_folder ~/tmp/mono_model/models/weights_19

You can use test_simple.py to predict scaled disparity for a single image. If you are using a stereo-trained model, you can add the --pred_metric_depth flag to estimate metric depth.

On the first run, the script will automatically download the requested pretrained model into the models/ folder.

python test_simple.py --image_path assets/test_image.jpg --model_name mono+stereo_640x192

# For metric depth with stereo models:
python test_simple.py --image_path assets/test_image.jpg --model_name mono+stereo_640x192 --pred_metric_depth

You can evaluate raw disparities or inverse depth produced by other methods by passing the path to a .npy file using the --ext_disp_to_eval flag.

python evaluate_depth.py --ext_disp_to_eval ~/other_method_disp.npy

The --eval_split parameter determines which test set is used for evaluation.