SuperPoint

Train MagicPoint using the labels exported from MS-COCO.

Configuration: In your training config file (e.g., magic-point_coco_train.yaml), you must set the data/labels entry to point to your exported labels directory (e.g., outputs/magic-point_coco-export1).

python experiment.py train configs/magic-point_coco_train.yaml magic-point_coco

Export pseudo-ground truth interest point labels to $EXPER_DIR/outputs/<export_name>/.

Options:

• --pred_only: Use only predictions.
• --batch_size: Set the batch size.
• --export_name: Name of the export.

Configuration:

• You can enable/disable Homographic Adaptation in the config file.
• To train with resized images, you must export detections on those resized images directly using the data->preprocessing->resize parameter in the config file.

python export_detections.py configs/magic-point_coco_export.yaml magic-point_synth --pred_only --batch_size=5 --export_name=magic-point_coco-export1

Train the MagicPoint model using the generated Synthetic Shapes dataset. The training can be interrupted with Ctrl+C, and weights/Tensorboard summaries will be saved in $EXPER_DIR/<experiment_name>/.

Note: MagicPoint and SuperPoint require images with dimensions divisible by 8. You are responsible for resizing images to valid dimensions.

python experiment.py train configs/magic-point_shapes.yaml magic-point_synth

Evaluate descriptor performance using homography estimation on HPatches.

Configuration: Set data/alteration in the config file to choose between viewpoint or illumination changes.

Evaluation: Predictions are saved in $EXPER_PATH/outputs/<export_name>/. To complete the evaluation, use the notebook: notebooks/descriptors_evaluation_on_hpatches.ipynb.

python export_descriptors.py configs/superpoint_hpatches.yaml superpoint_coco --export_name=superpoint_hpatches-v

Reuse pretrained weights (MagicPoint or SuperPoint) to fine-tune the model on your own data.

Steps:

1. Download and unzip a pretrained model into your $EXPER_PATH folder.
2. Launch training using the --pretrained_model flag pointing to the model name (e.g., sp_v6).

Example (Fine-tuning SuperPoint):

python superpoint/experiment.py train superpoint/configs/superpoint_coco.yaml superpoint_finetuned --pretrained_model sp_v6

Install the Python requirements and set up the necessary paths using make install.

Requirements:

• Python 3.6.1

Setup Steps:

1. Run make install.
2. You will be prompted to provide absolute paths for:
   • An experiment directory ($EXPER_DIR): contains training and prediction outputs.
   • A dataset directory ($DATA_DIR): contains datasets like MS-COCO and HPatches.

Dataset Structure: Ensure your $DATA_DIR follows this structure:

$DATA_DIR
|-- COCO
|   |-- train2014
|   |   |-- file1.jpg
|   |   `-- ...
|   `-- val2014
|       |-- file1.jpg
|       `-- ...
`-- HPatches
|   |-- i_ajuntament
|   `-- ...
`-- synthetic_shapes  # will be automatically created

make install  # install the Python requirements and setup the paths

Evaluate the detector repeatability on the HPatches dataset.

Configuration: Set data/alteration in the config file to choose between viewpoint or illumination changes.

Evaluation: Predictions are saved in $EXPER_DIR/outputs/<export_name>/. To complete the evaluation, use the notebook: notebooks/detector_repeatability_hpatches.ipynb.

python export_detections_repeatability.py configs/magic-point_repeatability.yaml magic-point_coco --export_name=magic-point_hpatches-repeatability-v

Train the SuperPoint model using detections from a previously trained MagicPoint model.

Configuration: Set the data/labels entry in superpoint_coco.yaml to point to the MagicPoint detections.

python experiment.py train configs/superpoint_coco.yaml superpoint_coco

Compare SuperPoint and SIFT matches across two images using the provided sp_v6 weights.

Setup:

1. Extract pretrained weights to your experiments directory:

tar -xzvf pretrained_models/sp_v6.tgz $EXPER_PATH/saved_models/sp_v6

2. Run the demo.

Arguments:

• --H: Specify height for resizing.
• --W: Specify width for resizing.
• --k_best: Maximum number of keypoints to keep.

python match_features_demo.py sp_v6 $DATA_PATH/HPatches/i_pool/1.ppm $DATA_PATH/i_pool/6.ppm

The classical_detector_descriptor function allows you to run traditional computer vision algorithms like SIFT or ORB on an image. It returns both the detected keypoint locations and their corresponding descriptors.

To use it, pass the image and specify the algorithm via the method keyword argument.

The PatchesDataset class is used to load datasets like HPatches for testing or evaluation. You can configure it using a dictionary containing the dataset name, the specific subset (e.g., 'hpatches'), alterations, and preprocessing steps like resizing.

Once initialized, you can retrieve the test set using .get_test_set().

from superpoint.datasets.patches_dataset import PatchesDataset

config = {
    'name': 'patches_dataset', 
    'dataset': 'hpatches', 
    'alteration': 'v', 
    'preprocessing': {'resize': [240, 360]}
}

dataset = PatchesDataset(**config)
data = dataset.get_test_set()