DOCSEARCHbyAlgolia
Submit Docs

GeoCalib

repository·main·Indexed 21 days ago

https://github.com/cvg/geocalib

GeoCalib is a single-image calibration tool that estimates camera intrinsics and gravity direction by combining geometric optimization with deep learning. It supports multiple camera models including pinhole, simple_radial, radial, and simple_divisional. The package provides functionality for batch processing of images, rigid multi-camera rig calibration, and an interactive webcam demo. It also includes the siclib library for training and evaluating models on datasets such as LaMAR, MegaDepth, TartanAir, and Stanford2D3D.

Tokens
18.3K
Snippets
61
Records
75
Agent score
74%

What's inside geocalib

  1. Run the interactive webcam demo

    main

    You can run an interactive demo that shows the camera feed and real-time calibration results using the geocalib.interactive_demo module.

    python -m geocalib.interactive_demo --camera_id 0

    If --camera_id is not provided, the demo will prompt for a droidcam IP address.

    Controls:

    • h: Show estimated horizon line
    • u: Show estimated up-vectors
    • l: Show estimated latitude heatmap
    • c: Show confidence heatmap (up-vectors and latitudes)
    • d: Show undistorted image
    • g: Show virtual grid of points
    • b: Show virtual box object
    • 1: Switch to pinhole model
    • 2: Switch to simple_radial model
    • 3: Switch to simple_divisional model
    • q: Quit
    python -m geocalib.interactive_demo --camera_id 0

  2. Evaluate models on the MegaDepth dataset

    main

    To evaluate models on the MegaDepth dataset, use the siclib.eval.megadepth2k (or megadepth2k_radial for radial distortion) module. Data downloads range from 1.47 GB to 2.1 GB to data/megadepth2k or data/megadepth2k-radial.

    # Evaluate GeoCalib (Pinhole)
python -m siclib.eval.megadepth2k --conf geocalib-pinhole --tag geocalib --overwrite

# Evaluate GeoCalib (Radial)
python -m siclib.eval.megadepth2k_radial --conf geocalib-pinhole --tag geocalib --overwrite model.camera_model=simple_radial

# Evaluate DeepCalib
python -m siclib.eval.megadepth2k --conf deepcalib --tag deepcalib --overwrite

# Evaluate Perspective Fields
python -m siclib.eval.megadepth2k --conf perspective-cities data.preprocessing.resize_backend="PIL" --overwrite
python -m siclib.eval.megadepth2k --conf perspective-openpano --overwrite

# Evaluate UVP
python -m siclib.eval.megadepth2k --conf uvp --tag uvp --overwrite data.preprocessing.edge_divisible_by=null

# Evaluate your own trained model
python -m siclib.eval.megadepth2k --checkpoint <experiment name> --tag <eval name> --overwrite

  3. Train DeepCalib

    main

    To train DeepCalib on the OpenPano dataset, use the deepcalib-openpano experiment.

    Note: Ensure you have generated the OpenPano dataset with radial distortion, or explicitly add the data=openpano flag to train on pinhole images.

    # Training on radial distortion dataset
python -m siclib.train deepcalib-openpano --conf deepcalib --distributed

# Training on pinhole images
python -m siclib.train deepcalib-openpano --conf deepcalib --distributed data=openpano

  4. Evaluate models on the Stanford2D3D dataset

    main

    To evaluate models on the Stanford2D3D dataset, use the siclib.eval.stanford2d3d module.

    Note: You must agree to the Stanford2D3D terms of use before running. This will download approximately 885 MB of data to data/stanford2d3d.

    # Evaluate GeoCalib
python -m siclib.eval.stanford2d3d --conf geocalib-pinhole --tag geocalib --overwrite

# Evaluate DeepCalib
python -m siclib.eval.stanford2d3d --conf deepcalib --tag deepcalib --overwrite

# Evaluate Perspective Fields
python -m siclib.eval.stanford2d3d --conf perspective-cities data.preprocessing.resize_backend="PIL" --overwrite
python -m siclib.eval.stanford2d3d --conf perspective-openpano --overwrite

# Evaluate UVP
python -m siclib.eval.stanford2d3d --conf uvp --tag uvp --overwrite data.preprocessing.edge_divisible_by=null

# Evaluate your own trained model
python -m siclib.eval.stanford2d3d --checkpoint <experiment name> --tag <eval name> --overwrite

  5. Install GeoCalib

    main

    You can install the geocalib package either by cloning the repository or directly via pip from GitHub. Requires Python >= 3.9.

    Option 1: Clone and install locally

    git clone https://github.com/cvg/GeoCalib.git && cd GeoCalib
python -m pip install -e .

    Option 2: Install via pip

    python -m pip install -e "git+https://github.com/cvg/GeoCalib#egg=geocalib"
    git clone https://github.com/cvg/GeoCalib.git && cd GeoCalib
python -m pip install -e .

  6. Prepare the OpenPano Dataset

    main

    The OpenPano dataset is used for single-image calibration training. Preparation involves downloading the Laval dataset, tonemapping HDR images, and downloading PolyHaven/HDRMAPS panoramas.

    1. Tonemap Laval HDR images: Place Laval panoramas in data/indoorDatasetCalibrated and run the tonemapping utility.
    2. Download panoramas: Use the provided script to fetch PolyHaven and HDRMAPS panoramas.
    3. Create training set: Run the dataset creation script. You can speed this up using n_workers and device=cuda.
    4. Radial Distortion: To create a version with radial distortion, use the openpano_radial configuration.
    # 1. Tonemap Laval HDR images
python -m siclib.datasets.utils.tonemapping --hdr_dir data/indoorDatasetCalibrated --out_dir data/laval-tonemap

# 2. Download PolyHaven and HDRMAPS
python -m siclib.datasets.utils.download_openpano --name openpano --laval_dir data/laval-tonemap

# 3. Create training set (standard)
python -m siclib.datasets.create_dataset_from_pano --config-name openpano

# 3. Create training set (accelerated)
python -m siclib.datasets.create_dataset_from_pano --config-name openpano n_workers=10 device=cuda

# 4. Create training set (with radial distortion)
python -m siclib.datasets.create_dataset_from_pano --config-name openpano_radial

  7. Visually inspect evaluation results

    main

    To visually inspect the results of an evaluation, use the siclib.eval.inspect module. It requires the benchmark name and one or more tags used during evaluation.

    # Syntax: python -m siclib.eval.inspect <benchmark> <one or multiple tags>

# Example: Inspect GeoCalib results on the LaMAR dataset
python -m siclib.eval.inspect lamar2k geocalib

  8. Evaluate models on the TartanAir dataset

    main

    To evaluate models on the TartanAir dataset, use the siclib.eval.tartanair module. This will download approximately 1.85 GB of data to data/tartanair.

    # Evaluate GeoCalib
python -m siclib.eval.tartanair --conf geocalib-pinhole --tag geocalib --overwrite

# Evaluate DeepCalib
python -m siclib.eval.tartanair --conf deepcalib --tag deepcalib --overwrite

# Evaluate Perspective Fields
python -m siclib.eval.tartanair --conf perspective-cities data.preprocessing.resize_backend="PIL" --overwrite
python -m siclib.eval.tartanair --conf perspective-openpano --overwrite

# Evaluate UVP
python -m siclib.eval.tartanair --conf uvp --tag uvp --overwrite data.preprocessing.edge_divisible_by=null

# Evaluate your own trained model
python -m siclib.eval.tartanair --checkpoint <experiment name> --tag <eval name> --overwrite

  9. Evaluate models on the LaMAR dataset

    main

    To evaluate models on the LaMAR dataset, use the siclib.eval.lamar2k module. Running these commands will download approximately 400 MB of data to data/lamar2k. Results are written to outputs/results/.

    # Evaluate GeoCalib
python -m siclib.eval.lamar2k --conf geocalib-pinhole --tag geocalib --overwrite

# Evaluate DeepCalib
python -m siclib.eval.lamar2k --conf deepcalib --tag deepcalib --overwrite

# Evaluate Perspective Fields (requires ParamNet-siclib setup)
python -m siclib.eval.lamar2k --conf perspective-cities data.preprocessing.resize_backend="PIL" --overwrite
python -m siclib.eval.lamar2k --conf perspective-openpano --overwrite

# Evaluate UVP (requires VP-Estimation-with-Prior-Gravity in third_party/)
python -m siclib.eval.lamar2k --conf uvp --tag uvp --overwrite data.preprocessing.edge_divisible_by=null

# Evaluate your own trained model
python -m siclib.eval.lamar2k --checkpoint <experiment name> --tag <eval name> --overwrite

  10. Train Perspective Fields

    main

    Training Perspective Fields is a two-step process:

    1. Train the perspective network: Use the perspective-net configuration.
    2. Train the full model: Place the trained weights in weights/persnet-pano-old.tar and run the param-net training command.
    # 1. Train perspective network
python -m siclib.train perspective-net --conf perspective-net --distributed

# 2. Train full model (requires weights/persnet-pano-old.tar)
python -m siclib.train param-net --conf param-net --distributed

  11. Perform single-image calibration with GeoCalib

    main

    To perform single-image calibration, initialize the GeoCalib model, load an image as a tensor in the range [0, 1] with shape [C, H, W], and call .calibrate(). The result contains the estimated camera (intrinsics) and gravity (direction).

    import torch
from geocalib import GeoCalib

device = "cuda" if torch.cuda.is_available() else "cpu"
model = GeoCalib().to(device)

# load image as tensor in range [0, 1] with shape [C, H, W]
image = model.load_image("path/to/image.jpg").to(device)
result = model.calibrate(image)

print("camera:", result["camera"])
print("gravity:", result["gravity"])