DOCSEARCHbyAlgolia
Submit Docs

MMFewShot Documentation

repository·main·Indexed 20 days ago

https://github.com/open-mmlab/mmfewshot

MMFewShot is an open-source few-shot learning toolbox based on PyTorch and part of the OpenMMLab project. It provides a modular unified framework for implementing and evaluating few-shot classification and detection tasks, including implementations of Baseline, Baseline++, MAML, and MatchingNet.

Tokens
80.2K
Snippets
152
Records
217
Agent score
72%

What's inside MMFewShot

  1. Overview of MMFewShot

    main

    MMFewShot is a PyTorch-based few-shot learning toolbox and benchmark, part of the OpenMMLab project. It provides a unified implementation and evaluation framework for both few-shot classification and detection tasks.

    Key features include:

    • Multi-task Support: Unified framework for few-shot classification and detection.
    • Modular Design: Decoupled task components that allow users to easily build custom few-shot algorithms by combining different modules.
    • SOTA Algorithms: Includes state-of-the-art algorithms and strong baseline models for benchmarking.

  3. Explore mmfewshot detection APIs

    main

    The mmfewshot.detection module provides tools for few-shot object detection. The API structure includes:

    • detection.apis: High-level entry points for running detection tasks.
    • detection.core: Core logic, including detection.core.evaluation for metrics and detection.core.utils.
    • detection.datasets: Data loading and management for few-shot detection.
    • detection.models: Model components, subdivided into:
      • backbones: Feature extraction networks.
      • dense_heads: Heads for dense prediction tasks.
      • detectors: The main detector architectures.
      • losses: Loss functions for detection.
      • roi_heads: Region of Interest heads.
      • utils: Model-specific helper functions.
    • detection.utils: General utility functions for the detection module.

  4. Explore mmfewshot classification APIs

    main

    The mmfewshot.classification module provides a complete suite of tools for few-shot learning in image classification tasks. The API is organized into several functional sub-modules:

    • classification.apis: High-level entry points for running classification tasks.
    • classification.core: Core logic and engine components, including classification.core.evaluation for performance metrics.
    • classification.datasets: Data loading and preprocessing utilities specifically for few-shot classification.
    • classification.models: Model architectures, which are further subdivided into:
      • backbones: Feature extraction networks.
      • classifier: Classification logic.
      • heads: Task-specific prediction heads.
      • losses: Loss functions designed for few-shot scenarios.
      • utils: Model-specific helper functions.
    • classification.utils: General utility functions for the classification module.

  5. What is MMFewShot and its core components

    main

    MMFewShot is a unified framework for implementing and evaluating few-shot classification and detection methods. The architecture is organized into four main functional parts:

    • datasets: Handles data loading and augmentation. It includes pipelines for image pre-processing transforms and datasetswrappers for flexible data sampling.
    • models: Contains the model architectures and loss functions.
    • core: Provides evaluation tools and customized hooks for model training and evaluation.
    • apis: Provides high-level APIs for model training, testing, and inference.

  6. Understand Few-Shot Learning terminologies

    main

    To use MMFewShot effectively, you should understand the following standard few-shot learning terms:

    • Training set: A large-scale dataset where every class has many samples, used to pre-train the model.
    • Support set: A small set of labeled images/instances. In classification, these are images; in detection, these are instances. The classes in the support set typically do not exist in the training set.
    • Query set: Unlabeled images/instances used for prediction, sharing the same classes as the support set.
    • N-way K-shot: Defines the support set configuration. N is the number of classes, and K is the number of samples per class.
      • Classification: A support set contains $N \times K$ images.
      • Detection: A support set contains $N \times K$ instances (the number of images may be less than $N \times K$).

  7. Customize training workflow

    main

    The workflow field is a list of (phase, epochs) tuples that defines the execution order.

    • Default: workflow = [('train', 1)] (runs 1 epoch of training).
    • With validation: workflow = [('train', 1), ('val', 1)] (runs 1 epoch of training followed by 1 epoch of validation iteratively).

    Important Notes:

    1. Model parameters are not updated during the val phase.
    2. total_epochs in the config only controls the number of training epochs, not the validation frequency.
    3. Using [('train', 1), ('val', 1)] allows the runner to calculate losses on the validation set after each training epoch.
    workflow = [('train', 1), ('val', 1)]

  8. Customize training and validation workflow

    main

    The workflow field is a list of (phase, epochs) tuples that defines the execution order.

    • Default: workflow = [('train', 1)] (runs 1 training epoch).
    • Validation inclusion: workflow = [('train', 1), ('val', 1)] (runs 1 training epoch followed by 1 validation epoch iteratively).

    Important Notes:

    • Model parameters are not updated during the val phase.
    • total_epochs in the config only controls the number of training epochs, not the total number of phases in the workflow.
    • Adding a val phase to the workflow allows the runner to calculate losses on the validation set after each training epoch.
    # Run 1 epoch of training and 1 epoch of validation iteratively
workflow = [('train', 1), ('val', 1)]

  9. Choose a training data flow in MMFewShot detection

    main

    MMFewShot supports four distinct data flows for training, depending on your few-shot strategy:

    • fine-tune based: Operates identically to regular detection training.
    • query aware: Returns both query data and support data from the same dataset.
    • n way k shot: Samples query data (regular) and support data (N-way K-shot) from separate datasets, then encapsulates them using a dataloader wrapper.
    • two branch: Samples main data (regular) and auxiliary data (regular) from separate datasets, then encapsulates them using a dataloader wrapper.

  10. Understand the TFA Config Structure

    main

    The TFA (Two-stage Fine-tuning Approach) configuration is a comprehensive dictionary that defines the entire computer vision pipeline, including data loading, augmentation, model architecture, optimization, and evaluation.

    Key components include:

    • train_pipeline & test_pipeline: Lists of data transformation dictionaries (e.g., LoadImageFromFile, Resize, Normalize).
    • data: Defines train, val, and test datasets, specifying types like FewShotCocoDefaultDataset, image prefixes, and annotation files.
    • model: The core architecture, containing backbone, neck, rpn_head, and roi_head.
    • optimizer & lr_config: Settings for training dynamics.
    • evaluation: Defines metrics (e.g., bbox) and intervals for validation.

    This structure allows for modular swapping of any component (e.g., changing a backbone from ResNet to another type) by simply updating the corresponding dictionary.

    # Example of the high-level structure of a TFA config
train_pipeline = [dict(type='LoadImageFromFile'), ...]
test_pipeline = [dict(type='LoadImageFromFile'), ...]
data = dict(train=dict(type='FewShotCocoDefaultDataset', ...), val=dict(...))
model = dict(type='TFA', backbone=dict(type='ResNet', ...), ...)
optimizer = dict(type='SGD', lr=0.001, ...)

  11. Understand the TFA Configuration Structure

    main

    The TFA (Two-stage Fine-tuning Approach) configuration is a comprehensive dictionary-based structure used to define a complete few-shot detection system. It is organized into several key top-level components:

    • train_pipeline / test_pipeline: Lists of data augmentation and preprocessing steps (e.g., LoadImageFromFile, Resize, Normalize, RandomFlip).
    • data: Defines dataset settings for train, val, and test, including dataset type, ann_cfg (annotation configuration), img_prefix, and pipeline.
    • model: The core architecture definition, including backbone (e.g., ResNet), neck (e.g., FPN), rpn_head, and roi_head.
    • optimizer & lr_config: Settings for the optimization algorithm (e.g., SGD) and learning rate scheduling (e.g., step or cosine).
    • runner: Defines the training loop type (e.g., IterBasedRunner) and total iterations.
    • evaluation: Specifies how the model is evaluated (e.g., metric='bbox') and which class splits to use (class_splits=['BASE_CLASSES', 'NOVEL_CLASSES']).
    # Example of the top-level structure
train_pipeline = [dict(type='LoadImageFromFile'), ...]
test_pipeline = [dict(type='LoadImageFromFile'), ...]

data = dict(
    train=dict(type='FewShotCocoDefaultDataset', pipeline=train_pipeline, ...),
    val=dict(type='FewShotCocoDataset', pipeline=test_pipeline, ...),
    test=dict(type='FewShotCocoDataset', pipeline=test_pipeline, ...)
)

model = dict(
    type='TFA',
    backbone=dict(type='ResNet', depth=101, ...),
    neck=dict(type='FPN', ...),
    rpn_head=dict(type='RPNHead', ...),
    roi_head=dict(type='StandardRoIHead', ...)
)

optimizer = dict(type='SGD', lr=0.001, ...)
lr_config = dict(policy='step', ...)
runner = dict(type='IterBasedRunner', max_iters=160000)

  12. How Few-Shot Classification is evaluated

    main

    Few-shot classification evaluation (also called meta test) follows this process:

    1. The dataset classes are split into three disjoint groups: train, test, and val.
    2. For each task, the system randomly samples an N-way K-shot labeled support set and a set of unlabeled query images from the test set.
    3. The model predicts the classes of the query images.
    4. This process is repeated across numerous sampled tasks to calculate the mean and standard deviation of the prediction accuracy.