DOCSEARCHbyAlgolia
Submit Docs

MinkLoc3D

repository·master·Indexed 40 days ago

https://github.com/jac99/minkloc3d

A method for large-scale place recognition using 3D point clouds. MinkLoc3D utilizes sparse voxelization and sparse 3D convolutions to generate discriminative descriptors, outperforming methods such as PointNetVLAD. It requires Python 3.8, PyTorch 1.9.1, and MinkowskiEngine 0.5.4.

Tokens
1.2K
Snippets
4
Records
5
Agent score
50%

What's inside MinkLoc3D

  1. MinkLoc3D Overview and Concept

    master

    MinkLoc3D is a learning-based method for computing discriminative 3D point cloud descriptors for large-scale place recognition.

    Core Concept

    Unlike methods like PointNetVLAD which use unordered point cloud representations and vanilla PointNet (which struggles with local geometric structures), MinkLoc3D utilizes:

    • Sparse Voxelized Point Cloud Representation: Converts point clouds into a sparse voxel grid.
    • Sparse 3D Convolutions: Uses sparse convolutions to efficiently process the voxelized data and capture local contextual information.

    This approach provides a simple and efficient architecture that outperforms several state-of-the-art methods in place recognition tasks.

  2. Evaluate pretrained MinkLoc3D models

    master

    You can evaluate the provided pretrained models using the evaluate.py script in the eval/ directory.

    Pretrained Weights

    • weights/minkloc3d_baseline.pth: Trained on the Baseline Dataset.
    • weights/minkloc3d_refined.pth: Trained on the Refined Dataset.

    Evaluation Commands

    Evaluate Baseline Model:

    cd eval
python evaluate.py --config ../config/config_baseline.txt --model_config ../models/minkloc3d.txt --weights ../weights/minkloc3d_baseline.pth

    Evaluate Refined Model:

    cd eval
python evaluate.py --config ../config/config_refined.txt --model_config ../models/minkloc3d.txt --weights ../weights/minkloc3d_refined.pth
    cd eval
python evaluate.py --config ../config/config_baseline.txt --model_config ../models/minkloc3d.txt --weights ../weights/minkloc3d_baseline.pth

  3. Generate dataset pickles for training and evaluation

    master

    Before training or evaluating, you must generate pickle files containing positive and negative point cloud tuples for each anchor.

    Important: If you are upgrading from a previous version of MinkLoc3D, you must delete existing pickles and re-create them, as the format has changed.

    Navigate to the generating_queries/ directory and run the appropriate script using the --dataset_root flag pointing to your dataset folder (e.g., /data/pointnetvlad/benchmark_datasets/). Ensure you have write permissions to the dataset root.

    Available Scripts

    • Baseline Dataset: generate_training_tuples_baseline.py
    • Refined Dataset: generate_training_tuples_refine.py
    • Evaluation/Test Sets: generate_test_sets.py
    cd generating_queries/ 

# Generate training tuples for the Baseline Dataset
python generate_training_tuples_baseline.py --dataset_root <dataset_root_path>

# Generate training tuples for the Refined Dataset
python generate_training_tuples_refine.py --dataset_root <dataset_root_path>

# Generate evaluation tuples
python generate_test_sets.py --dataset_root <dataset_root_path>

  4. Train the MinkLoc3D network

    master

    To train the model, follow these steps:

    1. Prepare Data: Download the datasets and generate training pickles using the scripts in generating_queries/.
    2. Configure: Edit the configuration files located in the config/ directory:
      • Use config_baseline.txt for the Baseline Dataset.
      • Use config_refined.txt for the Refined Dataset.
      • Key Parameters:
        • dataset_folder: Set this to your dataset root folder.
        • batch_size_limit: Adjust this based on your GPU memory. The default value of 256 requires at least 11GB of VRAM.
    3. Execute Training: Run the train.py script from the training/ directory.

    Training Commands

    For Baseline Dataset:

    cd training
python train.py --config ../config/config_baseline.txt --model_config ../models/minkloc3d.txt

    For Refined Dataset:

    cd training
python train.py --config ../config/config_refined.txt --model_config ../models/minkloc3d.txt
    cd training
python train.py --config ../config/config_baseline.txt --model_config ../models/minkloc3d.txt

  5. Set up the MinkLoc3D environment

    master

    MinkLoc3D requires a specific environment setup to ensure compatibility with MinkowskiEngine. It was tested on Ubuntu 20.04 with CUDA 10.2.

    Note: CUDA 11.1 is not recommended due to known issues with MinkowskiEngine 0.5.4.

    Required Dependencies

    • Python 3.8
    • PyTorch 1.9.1
    • MinkowskiEngine 0.5.4
    • pytorch_metric_learning >= 1.0
    • tensorboard
    • pandas

    Environment Configuration

    You must add the project root folder to your PYTHONPATH to ensure modules can be imported correctly:

    export PYTHONPATH=$PYTHONPATH:/path/to/MinkLoc3D
    export PYTHONPATH=$PYTHONPATH:/home/.../MinkLoc3D