DOCSEARCHbyAlgolia
Submit Docs

OpenVPRLab Framework

repository·main·Indexed 41 days ago

https://github.com/amaralibey/openvprlab

An open-source framework for Visual Place Recognition (VPR) research designed to simplify the training, fine-tuning, and iterative development of deep learning models. It features a modular architecture consisting of a Backbone, Aggregator, and Loss pipeline, and includes support for datasets such as GSV-Cities, MSLS, and Pitts30k.

Tokens
9.1K
Snippets
24
Records
33
Agent score
48%

What's inside OpenVPRLab

  1. How the modular VPR architecture works

    main

    OpenVPRLab uses a highly modular design that follows a specific pipeline: [Data] -> [Backbone] -> [Aggregation] -> [Loss].

    Users can implement and integrate their own components by following these steps:

    1. Backbone: The feature extractor (e.g., ResNet).
    2. Aggregator: The component that processes backbone features (e.g., MixVPR, BoQ).
    3. Loss: The optimization objective (leveraging the deep_metric_learning library).

    This modularity allows researchers to focus on a single component (like a new aggregator) without modifying the rest of the pipeline.

  2. Train a VPR model

    main

    Training is best managed via YAML configuration files located in the config/ directory. You can run a training session using a config file, or override specific hyperparameters (like --batch_size or --lr) directly via the command line. Hyperparameters for every experiment are automatically saved as hparams.yaml in the corresponding logs/ subdirectory.

    # Train using a configuration file
python run.py --config ./config/resnet50_mixvpr.yaml

# Train using a config file with command-line overrides
python run.py --config ./config/resnet50_mixvpr.yaml --batch_size 40 --lr 0.0001

  4. Download datasets for OpenVPRLab

    main

    OpenVPRLab includes four datasets: gsv-cities and gsv-cities-light for training, and msls-val and pitts30k-val for validation. Use the datasets_downloader.py script to download and unzip them into the data/ folder.

    python scripts/datasets_downloader.py

  5. Develop a custom VPR aggregator

    main

    To add a new aggregation technique to the framework, follow these steps:

    1. Create the class: Define a new torch.nn.Module in a new file (e.g., src/models/aggregators/my_agg.py).
    2. Register the aggregator: Import your class into src/models/aggregators/__init__.py so it can be referenced by name in configuration files.
    3. Configure and Run: Define the aggregator's parameters in a YAML config file under the aggregator key, specifying the module, class, and params.

    Example Aggregator Implementation:

    import torch

class ConvAP(torch.nn.Module):
    def __init__(self, in_channels=1024, out_channels=512, s1=2, s2=2):
        super().__init__()
        self.channel_pool = torch.nn.Conv2d(in_channels=in_channels, out_channels=out_channels, kernel_size=1)
        self.AAP = torch.nn.AdaptiveAvgPool2d((s1, s2))

    def forward(self, x):
        x = self.channel_pool(x)
        x = self.AAP(x)
        x = torch.nn.functional.normalize(x.flatten(1), p=2, dim=1)
        return x

    Example Configuration for Custom Aggregator:

    aggregator:
  module: src.models.aggregators
  class: ConvAP
  params:
    in_channels: 
    out_channels: 256
    s1: 3
    s2: 3

  6. Install OpenVPRLab

    main

    To install OpenVPRLab, clone the repository, create a Conda environment using the provided environment.yml, and activate the openvpr environment.

    # Clone the repository
git clone https://github.com/amaralibey/OpenVPRLab.git

# Navigate to the project directory
cd OpenVPRLab

# Create environment and install the required dependencies
conda env create -f environment.yml

# activate the openvpr environment
conda activate openvpr

  7. Understand the VPRFramework forward pass and data flow

    main

    The forward method implements a sequential pipeline: first passing the input through the backbone, then passing the resulting features through the aggregator.

    During the training_step, the framework expects input batches in the shape (P, K, c, h, w), where:

    • P: Number of places
    • K: Number of views per place
    • c: Channels
    • h: Height
    • w: Width

    The framework automatically reshapes these into a standard batch format (P * K, c, h, w) for processing.

  8. Configure the GSV-Cities dataset path in config.yaml

    main

    After downloading the GSV-Cities dataset, you must update the project's configuration file (../config/data/config.yaml) with the absolute path to the dataset so the training pipeline can locate it.

    This is typically done by loading the YAML config, setting the config["train"]["gsv_cities"] key to the resolved absolute path, and saving the file back to disk.

    import yaml
from pathlib import Path

# Define where you saved the data
save_path = "../data/train/gsv-cities"

# Load existing config
with open("../config/data/config.yaml", "r") as f:
    config = yaml.safe_load(f)

# Resolve to absolute path
absolute_save_path = Path(save_path).resolve().as_posix()

# Update the training configuration
config["train"]["gsv_cities"] = absolute_save_path

# Write the updated config back
with open('../config/data/config.yaml', 'w') as f:
    yaml.dump(config, f)

  9. Download the GSV-Cities dataset

    main

    The GSV-Cities dataset (approx. 23GB) can be downloaded from Kaggle using the kaggle CLI without requiring an API key or login.

    Prerequisites:

    • Ensure you have at least 23GB of free disk space.
    • Install the kaggle package via pip.

    Steps:

    1. Install the Kaggle CLI: pip install kaggle.
    2. Download and unzip the dataset to your desired directory using: kaggle datasets download "amaralibey/gsv-cities" -p {save_path} --unzip.
    # Install kaggle
pip install kaggle

# Download and unzip to a specific path
# Replace {save_path} with your target directory
kaggle datasets download "amaralibey/gsv-cities" -p ../data/train/gsv-cities --unzip

  10. Run a development test iteration

    main

    To verify the framework installation and ensure all components are working, run the framework in --dev mode. This requires at least the gsv-cities-light and msls-val datasets to be present. It executes one training iteration of ResNet50-MixVPR and one validation iteration.

    python run.py --dev

  12. Use the PittsburghDataset class

    main

    The PittsburghDataset class is a PyTorch Dataset used to load images and metadata for the Pittsburgh datasets (pitts30k-val, pitts30k-test, or pitts250k-test).

    Initialization

    • dataset_path (Optional[str]): The directory containing the dataset. If not provided, it defaults to the path defined in config/data/config.yaml for pitts30k-val.
    • input_transform (Optional[Callable]): An optional transformation function applied to each image (e.g., torchvision transforms).

    Data Access

    • __getitem__(index): Returns a tuple containing the loaded PIL image (or transformed tensor) and its integer index.
    • __len__(): Returns the total number of images (reference images + query images).
    from src.dataloaders.valid.pittsburgh import PittsburghDataset
from torchvision import transforms

transform = transforms.Compose([transforms.Resize((224, 224)), transforms.ToTensor()])

# Initialize with a specific path
dataset = PittsburghDataset(dataset_path='/path/to/pitts30k-val', input_transform=transform)

# Access an item
image, index = dataset[0]