DOCSEARCHbyAlgolia
Submit Docs

LightGlue Documentation

repository·main·Indexed 26 days ago

https://github.com/cvg/lightglue

LightGlue is a lightweight, high-speed deep neural network for matching sparse local features across image pairs. It utilizes adaptive pruning in network width and depth to achieve high accuracy with fast inference. The library supports various feature extraction backends including SuperPoint, DISK, ALIKED, SIFT, and DoGHardNet, and provides a match_pair convenience method for streamlined workflows.

Tokens
4.1K
Snippets
5
Records
30
Agent score
89%

What's inside LightGlue

  1. Optimize LightGlue for accuracy or speed

    main

    Maximize Accuracy

    To use all keypoints and disable adaptive mechanisms:

    extractor = SuperPoint(max_num_keypoints=None)
matcher = LightGlue(features='superpoint', depth_confidence=-1, width_confidence=-1)

    Maximize Speed

    To decrease keypoints and lower adaptive thresholds:

    extractor = SuperPoint(max_num_keypoints=1024)
matcher = LightGlue(features='superpoint', depth_confidence=0.9, width_confidence=0.95)

    Use PyTorch Compilation

    For maximum speed (requires torch >= 2.0):

    matcher = matcher.eval().cuda()
matcher.compile(mode='reduce-overhead')

    Note: For inputs with < 1536 keypoints, compilation disables point pruning due to overhead. For larger inputs, it falls back to eager mode with point pruning.

  2. Install LightGlue

    main

    Install the LightGlue repository using pip by cloning the repository and installing it in editable mode.

    git clone https://github.com/cvg/LightGlue.git && cd LightGlue
python -m pip install -e .

  3. Configure LightGlue parameters

    main

    LightGlue supports several configuration parameters to balance speed and accuracy:

    • n_layers: Number of stacked self+cross attention layers. Default: 9. Lowering this increases speed but decreases accuracy.
    • flash: Enable FlashAttention. Default: True (auto-detects availability). Significantly increases speed and reduces memory.
    • mp: Enable mixed precision inference. Default: False.
    • depth_confidence: Controls early stopping. Lower values stop more often at earlier layers. Default: 0.95. Disable with -1.
    • width_confidence: Controls iterative point pruning. Lower values prune points earlier. Default: 0.99. Disable with -1.
    • filter_threshold: Match confidence. Increase for fewer but stronger matches. Default: 0.1.

  4. Use match_pair convenience method

    main

    For a simpler workflow, use the match_pair function which handles extraction and matching in one call.

    from lightglue import match_pair
feats0, feats1, matches01 = match_pair(extractor, matcher, image0, image1)

  5. Match image pairs using LightGlue

    main

    LightGlue can be used to match local features extracted by various backends including SuperPoint, DISK, ALIKED, or SIFT. You can either use the manual extraction and matching workflow or the match_pair convenience method.

    Images should be loaded as torch.Tensor on GPU with shape (3, H, W) and normalized in the range [0, 1].

    from lightglue import LightGlue, SuperPoint, DISK, SIFT, ALIKED, DoGHardNet
from lightglue.utils import load_image, rbd

# SuperPoint+LightGlue
extractor = SuperPoint(max_num_keypoints=2048).eval().cuda()
matcher = LightGlue(features='superpoint').eval().cuda()

# load each image as a torch.Tensor on GPU with shape (3,H,W), normalized in [0,1]
image0 = load_image('path/to/image_0.jpg').cuda()
image1 = load_image('path/to/image_1.jpg').cuda()

# extract local features
feats0 = extractor.extract(image0)
feats1 = extractor.extract(image1)

# match the features
matches01 = matcher({'image0': feats0, 'image1': feats1})
feats0, feats1, matches01 = [rbd(x) for x in [feats0, feats1, matches01]]  # remove batch dimension
matches = matches01['matches']  # indices with shape (K,2)
points0 = feats0['keypoints'][matches[..., 0]]  # coordinates in image #0, shape (K,2)
points1 = feats1['keypoints'][matches[..., 1]]  # coordinates in image #1, shape (K,2)

  6. Configure SuperPoint extractor options

    main

    When initializing SuperPoint, you can pass several configuration parameters via **conf. These parameters control the feature extraction behavior:

    KeyDefaultDescription
    descriptor_dim256Dimensionality of the output descriptors
    nms_radius4Radius for Non-maximum suppression to remove nearby points
    max_num_keypointsNoneMaximum number of keypoints to keep (must be positive or None)
    detection_threshold0.0005Threshold for keypoint detection
    remove_borders4Number of pixels to discard from the image borders

    Additionally, the extractor uses a preprocess_conf which includes:

    • resize: 1024 (used for internal resizing)

  7. Configure the SIFT extractor

    main

    The SIFT extractor accepts several configuration parameters to control feature detection and processing:

    KeyTypeDefaultDescription
    rootsiftboolTrueWhether to apply RootSIFT normalization to descriptors.
    nms_radiusint/None0Non-Maximum Suppression radius. Set to None to disable filtering.
    max_num_keypointsint4096Maximum number of keypoints to extract.
    backendstr'opencv'Backend to use: 'opencv', 'pycolmap', 'pycolmap_cpu', or 'pycolmap_cuda'.
    detection_thresholdfloat0.0066667Contrast/detection threshold (from COLMAP).
    edge_thresholdint10Edge threshold for detection.
    num_octavesint4Number of octaves for the SIFT implementation.
    first_octaveint-1Only used by pycolmap (default of COLMAP).

  8. Preprocess images with ImagePreprocessor

    main

    The ImagePreprocessor class resizes and preprocesses a torch.Tensor image. It returns the processed image and the scale factor used for resizing, which is useful for mapping coordinates back to the original image size.

    Configuration options:

    • resize: Target edge length (int). If None, no resizing is performed.
    • side: `

  9. Extract features using the DISK class

    main

    The DISK class is a feature extractor that uses the Kornia implementation of DISK. It computes keypoints, scores, and descriptors for a given image.

    Input Requirements:

    • The input data dictionary must contain an `

  10. Use DoGHardNet for feature extraction

    main

    The DoGHardNet class is a feature extractor that inherits from SIFT. It extracts keypoints and 128-dimensional descriptors using a combination of Difference of Gaussian (DoG) detection and HardNet descriptors.

    Input Data Requirements:

    • The input data dictionary must contain an `

  11. Run LightGlue benchmarks

    main

    You can benchmark LightGlue on your hardware using the provided benchmark.py script.

    python benchmark.py [--device cuda] [--add_superglue] [--num_keypoints 512 1024 2048 4096] [--compile]

  12. Extract descriptors for specific keypoints with ALIKED

    main

    If you already have a set of keypoints and want to extract their corresponding descriptors using the ALIKED model, use the describe method.

    This method performs the following steps:

    1. Preprocesses the image (resizing).
    2. Normalizes the provided keypoints to the range [-1, 1].
    3. Extracts dense feature maps.
    4. Uses the desc_head to compute descriptors for the normalized keypoints.

    Parameters:

    • keypoints: torch.Tensor of shape (N, 2) representing keypoint coordinates.
    • img: torch.Tensor of shape (C, H, W) or (B, C, H, W).
    • **conf: Additional configuration passed to the ImagePreprocessor (e.g., resize).