DOCSEARCHbyAlgolia
Submit Docs

LoMa (Local Feature Matching Revisited)

repository·main·Indexed 18 days ago

https://github.com/davnords/loma

A family of fast and accurate local feature matchers designed as drop-in replacements for SfM and Visual Localization pipelines. LoMa provides multiple model configurations including LoMa-B, LoMa-B128, LoMa-L, LoMa-G, and a rotation-invariant LoMa-R for aerial imagery. The library includes the DeDoDeDescriptor for local feature extraction, DinoVisionTransformer helpers, and geometry utilities for coordinate conversion, warp consistency, and relative pose estimation using RANSAC.

Tokens
7.7K
Snippets
30
Records
37
Agent score
62%

What's inside LoMa

  2. Use LoMa for local feature matching

    main

    To perform local feature matching, import LoMa and a model configuration class (like LoMaB) from loma. Initialize the LoMa object with the desired model, then call .match(img_A_path, img_B_path) to get matching keypoints in image coordinates.

    Available model configurations include:

    • LoMaB (standard size, similar to LightGlue)
    • LoMaB128
    • LoMaL (Large)
    • LoMaG (Great/Heavy - most accurate)
    • LoMaR (Rotation invariant, ideal for aerial imagery)
    import cv2
from loma import LoMa, LoMaB

# load pretrained model
model = LoMa(LoMaB())

# Define image paths
img_A_path, img_B_path = "assets/0015_A.jpg", "assets/0015_B.jpg"

# Extract matching keypoints in image coordinates
kptsA, kptsB = model.match(img_A_path, img_B_path)

# Example: Find a fundamental matrix using the matches
F, mask = cv2.findFundamentalMat(
    kptsA, kptsB, ransacReprojThreshold=0.2, method=cv2.USAC_MAGSAC, confidence=0.999999, maxIters=10000
)

  3. Evaluate LoMa on benchmarks

    main

    To evaluate LoMa on benchmarks like MegaDepth, ScanNet, WxBS, or RUBIK:

    1. Prepare Data: Download necessary datasets (MegaDepth1500 and ScanNet1500) using:
      source scripts/eval_prep.sh
    2. Install Eval Dependencies: Install optional dependencies for evaluation:
      uv sync --extra eval
    3. Run Benchmark: Execute the evaluation script specifying the matcher and the benchmark name.

    Use uv run eval.py --help to see all available options.

    uv run eval.py matcher:loma-b --benchmark wxbs

  5. Configure the LoMa model via `LoMa.Cfg`

    main

    The LoMa.Cfg dataclass allows fine-grained control over the model architecture and behavior.

    Key Configuration Options:

    • input_dim: Dimension of input descriptors.
    • embed_dim: Dimension of the transformer embeddings.
    • n_layers: Number of transformer layers.
    • num_heads: Number of attention heads.
    • filter_threshold: Threshold for matching.
    • mp: Boolean to enable mixed precision (AMP).
    • compile: Boolean to enable torch.compile (Linux only).
    • descriptor: Choice of descriptor architecture ("dedode_b" or "dedode_g").
    • num_keypoints: Default number of keypoints to detect.
    • posenc_type: Type of positional encoding ("learnable" or "fixed").
    • posenc_gamma: Wavelength/scale for positional encoding.
    • weights_url: URL to download pretrained weights.

  6. Configure DeDoDeDescriptor.Cfg options

    main

    The DeDoDeDescriptor.Cfg dataclass controls the initialization of the descriptor model. Use these keys to tune the model architecture and performance:

    KeyTypeDefaultDescription
    archLiteral["dedode_b", "dedode_g"]"dedode_b"The model architecture to use.
    compileboolTrueWhether to apply torch.compile() to the model.
    descriptor_dimint256The dimensionality of the output descriptors.
    hidden_blocksint5The number of hidden blocks in the ConvRefiner layers.

  7. Configure FineFeatures via Cfg

    main

    The FineFeatures.Cfg dataclass defines the configuration for feature extraction models.

    KeyTypeDefaultDescription
    typeFineFeaturesType"vgg19bn"The type of feature extractor to use.
    patch_sizeint4Determines the depth of the VGG layers used. Valid keys for mapping to layer indices are {1, 2, 4, 8, 16}.
    from loma.features import FineFeatures

# Example: Using a different patch size
cfg = FineFeatures.Cfg(type="vgg19bn", patch_size=8)
model = FineFeatures(cfg)

  8. Configure DinoVisionTransformer parameters

    main

    The DinoVisionTransformer class can be customized via its constructor. Key parameters include:

    • img_size / patch_size: Dimensions for input and patch segmentation.
    • embed_dim: The embedding dimension of the transformer.
    • depth: Number of transformer blocks.
    • num_heads: Number of attention heads.
    • mlp_ratio: Ratio of MLP hidden dimension to embedding dimension.
    • ffn_layer: Type of Feed-Forward Network. Options: "mlp" (default), "identity" (returns nn.Identity), or other custom implementations.
    • block_chunks: (int) Splits the block sequence into block_chunks units. This is used for FSDP (Fully Sharded Data Parallel) wrapping.
    • drop_path_rate: Stochastic depth rate.
    • drop_path_uniform: If True, applies the same drop_path_rate to all blocks; otherwise, applies a decay rule.
    from loma.descriptor.transformer.dinov2 import DinoVisionTransformer

model = DinoVisionTransformer(
    img_size=224,
    patch_size=16,
    embed_dim=768,
    depth=12,
    num_heads=12,
    block_chunks=2  # Split blocks into 2 chunks for FSDP
)

  9. Reference: Available LoMa Models

    main

    LoMa provides several model variants with different trade-offs between speed and accuracy:

    ModelDescription
    LoMa-BStandard size, similar to LightGlue. Good for most use cases.
    LoMa-B128Variant of the B model.
    LoMa-LLarge model.
    LoMa-GHeavy model. Most accurate, surpasses RoMa-family on difficult benchmarks.
    LoMa-RRotation invariant model. Optimized for aerial imagery.

  11. Detect keypoints using `.detect()`

    main

    Access the frozen detector directly to get keypoints from a batch of images.

    Arguments:

    • batch: A dictionary containing img_A and/or img_B (and optionally img).
    • num_keypoints (optional): Number of keypoints to detect.

    Returns:

    • dict: A dictionary containing the detected keypoints.
    # batch should follow the Batch type definition
results = model.detect(batch, num_keypoints=1024)
keypoints = results["keypoints"]

  12. Estimate relative pose using RANSAC

    main

    Provides multiple ways to estimate the relative rotation ($R$) and translation ($t$) between two camera views based on point correspondences.

    • estimate_pose_cv2_ransac: Uses OpenCV's findEssentialMat and recoverPose. Requires normalized keypoints and camera intrinsics.
    • estimate_pose_essential: Uses the poselib library for robust relative pose estimation.
    • estimate_pose_fundamental: Estimates the Fundamental matrix using poselib and then recovers $R$ and $t$ via OpenCV.
    import numpy as np
from loma.geometry import estimate_pose_cv2_ransac

# kpts0, kpts1: (N, 2) arrays
# K0, K1: (3, 3) intrinsic matrices
# norm_thresh: RANSAC threshold
result = estimate_pose_cv2_ransac(kpts0, kpts1, K0, K1, norm_thresh=1.0)
if result:
    R, t, mask = result