SphereSfM

Understanding the core concepts used in COLMAP:

• Camera: Represents a physical object with a specific zoom factor and lens. It defines the intrinsic projection model. A single camera can be used for multiple images if they share the same resolution, intrinsic parameters, and distortion characteristics.
• Image: A bitmap file (e.g., JPEG, PNG) on disk.
• Keypoints: Specific points detected within an image.
• Descriptors: Numerical descriptions of the appearance of keypoints.
• Matches: Pure appearance-based correspondences between keypoints/descriptors.
• Inlier Matches: Matches that have been geometrically verified and are used for the actual reconstruction.

Sparse models are stored in directories containing three files: cameras, images, and points3D.

COLMAP supports two formats:

• Binary (.bin): Machine-readable and fast. This is the default. If a directory contains both binary and text files, the binary format is preferred.
• Text (.txt): Human-readable but slower.

Conversion Methods:

• GUI: Use File > Export model (binary) or File > Export model as text (text). To convert, import the model first, then export in the desired format.
• CLI: Use the model_converter executable.
• Other Formats: Use File > Export as... in the GUI to export to VisualSfM's NVM, Bundler, PLY, VRML, etc.

Programmatic Access:

• Python: Use scripts/python/read_write_model.py (supports both binary and text).
• Matlab: Use scripts/matlab/read_model.m (supports text).

The consistency graph defines which source images a pixel is consistent with. It is stored as a mixed text and binary file.

• Text part: Follows the same format as depth/normal maps (with&height&channels& header + float32 binary data).
• Binary part: A continuous list of int32 values in the format: <row><col><N><image_idx1>...<image_idxN>.

Note that image_idx values are zero-based indices relative to the ordering in the images.txt file.

The relationship between cameras and images is 1-to-N. One camera (containing intrinsic parameters like focal length and distortion) can be shared by multiple images. Each image has its own unique extrinsic parameters (orientation and location).

Key Requirements:

• Identifiers: When manually inserting data, ensure image_id > 0 and camera_id > 0.
• Camera Parameters: Intrinsic parameters are stored as contiguous float64 binary blobs. The order of parameters depends on the camera model defined in src/base/camera_models.h.
• Image Paths: The name column in the images table must contain the unique relative path within the image folder. This allows you to move the database and image folder simultaneously as long as the relative structure remains intact.

SphereSfM uses a single SQLite database file to store all extracted information. The database consists of the following tables:

• cameras: Stores intrinsic parameters.
• images: Stores image metadata and extrinsic parameters.
• keypoints: Stores detected feature locations and geometry.
• descriptors: Stores feature appearance descriptors.
• matches: Stores feature matches between image pairs.
• two_view_geometries: Stores geometric verification results used for reconstruction.

You can interact with this database using the COLMAP GUI, the C++ database API (src/base/database.h), or the Python scripting interface (scripts/python/database.py).

Multi-View Stereo (MVS) is the second major phase of reconstruction. It takes the sparse output from the SfM process and computes depth and/or normal information for every pixel in the input images.

Key steps:

1. Depth/Normal Computation: Per-pixel information is derived from the SfM camera poses and sparse points.
2. Fusion: Depth and normal maps from multiple images are fused in 3D to produce a dense point cloud.
3. Surface Reconstruction: Using the fused point cloud, algorithms like (screened) Poisson surface reconstruction can be used to recover the final 3D surface geometry (mesh).

Structure-from-Motion (SfM) reconstructs 3D structure and camera parameters (intrinsic and extrinsic) from a series of overlapping images. COLMAP implements an incremental SfM pipeline divided into three main stages:

1. Feature detection and extraction: Identifying keypoints in images.
2. Feature matching and geometric verification: Finding correspondences between keypoints and verifying them geometrically.
3. Structure and motion reconstruction: Recovering the 3D points and camera poses.

SfM output serves as the input for Multi-View Stereo (MVS), which computes depth/normal information to produce a dense representation (point clouds and meshes).

The matches and two_view_geometries tables use a pair_id to identify matches between two unique images. This pair_id is a row-major, linear index in the upper-triangular match matrix. This allows for efficient querying even with millions of entries.

Pair ID Logic

To generate a pair_id from two image IDs:

def image_ids_to_pair_id(image_id1, image_id2):
    if image_id1 > image_id2:
        return 2147483647 * image_id2 + image_id1
    else:
        return 2147483647 * image_id1 + image_id2

To retrieve image IDs from a pair_id:

def pair_id_to_image_ids(pair_id):
    image_id2 = pair_id % 2147483647
    image_id1 = (pair_id - image_id2) / 2147483647
    return image_id1, image_id2

Constraints

• Max Images: Because pair_id relies on signed 32-bit integer math, the maximum image_id allowed is 2147483647.