NetVLAD

To train a NetVLAD network using weakly supervised learning:

1. Prepare Data: Load your training and validation datasets (e.g., using dbTokyoTimeMachine('train')).
2. Execute Training: Call trainWeakly with your dataset objects and desired hyperparameters.
3. Select Best Model: Training periodically saves checkpoints. Use pickBestNet(sessionID) to retrieve the network that performed best on the validation set.
4. Dimensionality Reduction: For optimal performance, apply PCA and whitening to the best network using addPCA.

Important trainWeakly Parameters

• netID: Name of the network architecture (caffe for AlexNet, vd16 for VGG-16).
• layerName: The last convolutional layer to crop (e.g., conv5_3 for VGG-16).
• backPropToLayer: The layer down to which learning is performed.
• method: Aggregation method (e.g., vlad_preL2_intra, max, avg).
• learningRate: SGD learning rate.
• useGPU: Boolean to enable/disable GPU acceleration.

% 1. Setup
setup;

% 2. Load Data
dbTrain= dbTokyoTimeMachine('train');
dbVal= dbTokyoTimeMachine('val');

% 3. Train
sessionID= trainWeakly(dbTrain, dbVal, ...
    'netID', 'vd16', 'layerName', 'conv5_3', 'backPropToLayer', 'conv5_1', ...
    'method', 'vlad_preL2_intra', ...
    'learningRate', 0.0001, ...
    'doDraw', true);

% 4. Pick best and add PCA
[~, bestNet]= pickBestNet(sessionID);
finalNet= addPCA(bestNet, dbTrain, 'doWhite', true, 'pcaDim', 4096);

To use a pre-trained network for place recognition, follow these steps:

1. Initialize Paths: Run setup; to set MATLAB paths.
2. Load Network: Load the .mat file using localPaths() to resolve the directory. If using an older model (v1.01 or below), wrap the network with relja_simplenn_tidy(net) to upgrade it.
3. Single Image Inference: Use computeRepresentation(net, im) on a normalized image.
4. Batch Inference: For large sets of images, use serialAllFeats. This is significantly faster as it uses batches and minimizes GPU transfers.
   • Note on Batch Size: If images have different resolutions (common in place recognition datasets), set 'batchSize', 1 to avoid errors.
5. Evaluation: To measure performance (e.g., Recall@N) on a dataset, use testFromFn with the database and query feature files.

setup;

% Load network
netID= 'vd16_tokyoTM_conv5_3_vlad_preL2_intra_white';
paths= localPaths();
load( sprintf('%s%s.mat', paths.ourCNNs, netID), 'net' );
net= relja_simplenn_tidy(net);

% Single image
im= vl_imreadjpeg({which('football.jpg')}); im= im{1};
feats= computeRepresentation(net, im);

% Batch processing
serialAllFeats(net, imPath, imageFns, outputFn, 'batchSize', 10);

% Testing Recall@N
[recall, ~, ~, opts]= testFromFn(dbTest, dbFeatFn, qFeatFn);
plot(opts.recallNs, recall, 'ro-');

The trainWeakly function accepts several categories of parameters to control the weakly supervised training process.

Main Parameters

• netID: Network name (caffe for AlexNet, vd16 for VGG-16).
• layerName: The layer to crop the initial network at (e.g., conv5 for caffe, conv5_3 for vd16).
• method: Aggregation method. Defaults to vlad_preL2_intra. Other options: max (max pooling), avg (average pooling), or vlad_preL2 (disables intra-normalization).
• margin: Loss margin parameter.
• useGPU: Boolean to enable/disable GPU.
• sessionID: Unique string for the run (avoid slashes/non-alphanumeric characters).
• computeBatchSize: Batch size for computing image representations (affects speed, not training behavior).
• compFeatsFrequency: Frequency (in training tuples) to recompute cached image representations.

SGD Parameters

• learningRate, batchSize, momentum, weightDecay, nEpoch: Standard SGD parameters. Note that batchSize refers to the number of training tuples (each tuple contains a query, a positive, and negatives).
• lrDownFactor, lrDownFreq: Learning rate decay schedule.
• backPropToLayer: Specifies the layer down to which learning is performed (name or number).
• fixLayers: Cell array of layers to keep frozen (use backPropToLayer instead for most cases).

Other Parameters

• doDraw: Plot performance curves during training.
• saveFrequency: Frequency of saving the network and data.
• recallNs: Array of N values for recall measurement.
• test0: If true, tests the network before training (off-the-shelf mode).
• nTestRankSample: Number of tuples used for loss computation during testing.
• nTestSample: Upper limit on queries used for testing (set to $\infty$ to use all).
• nNegChoice: Number of negatives sampled per tuple per epoch.
• nNegCache: Number of hardest negatives to remember for the next epoch.
• nNegCap: Total number of hardest negatives kept in the training tuple.
• excludeVeryHard: If true, ignores negatives closer than the closest potential positive.
• startEpoch: Restart training from a specific epoch (requires matching sessionID).