Anomalib

PaDiM is a patch-based segmentation algorithm for anomaly detection and localization.

How it works:

1. Feature Extraction: Uses a pre-trained CNN to extract embeddings from different layers of an image.
2. Embedding Construction: Concatenates activation vectors from multiple layers to capture both fine-grained and global context.
3. Dimensionality Reduction: Uses random selection to reduce redundant information in the embedding vectors.
4. Distribution Modeling: Generates a multivariate Gaussian distribution for each patch embedding based on the training batch.
5. Inference: Calculates the Mahalanobis distance for each patch in a test image using the inverse of the covariance matrix learned during training. The resulting distances form an anomaly map where higher scores indicate anomalous regions.

FRE is a segmentation-type model designed for fast anomaly classification. It operates in two stages:

1. Feature Extraction: Uses a ResNet50 backbone (pre-trained on ImageNet) to extract semantic feature vectors. By default, it uses the output of layer3, resulting in a fixed-length vector of 65536.
2. Anomaly Detection: Employs a shallow linear autoencoder trained on the extracted features. The anomaly score is calculated as the reconstruction error (the norm of the error between the reconstructed output and the original high-dimensional feature). Anomaly maps are generated by reshaping and resizing the error tensor to match the input image dimensions.

Dinomaly is a segmentation-type anomaly detection model based on an encoder-decoder architecture for feature reconstruction. It leverages pre-trained DINOv2 Vision Transformer features.

Key architectural components include:

• DINOv2 Encoder: A pre-trained ViT that extracts multi-scale feature maps.
• Bottleneck MLP: A feed-forward network that collects features from the encoder's middle layers. It uses Dropout to create a "noisy bottleneck," forcing the decoder to learn only normal features.
• Vision Transformer Decoder: Uses Linear Attention to reconstruct compressed middle-level features by maximizing cosine similarity with the encoder's features.
• Loose Reconstruction: Employs a loose constraint (grouping encoder layers) and a loose loss (hard-mining global cosine loss) to prevent the model from becoming too proficient at reconstructing anomalous patterns.

Anomalib Studio is a low/no-code web application for training and deploying anomaly detection models. It supports USB/IP cameras or image folders as inputs and can output results to industrial pipelines via ROS messages, MQTT, etc.

Note: Anomalib Studio is currently in pre-release and may be unstable.

It is distributed in two ways:

1. As a Docker container.
2. As a standalone application.

PatchCore is a segmentation-type model that identifies anomalies by analyzing image patches.

Key Concepts:

• Patch Extraction: The input image is tiled into patches, which are processed by a pre-trained neural network to extract "mid" level features (from the feature extraction layer).
• Memory Bank: During training, extracted features are stored in a memory bank of neighborhood-aware patch-level features.
• Coreset Subsampling: During inference, the memory bank is subsampled using a coreset method to approximate the full set structure while reducing the computational cost of nearest neighbor searches.
• Anomaly Scoring: The anomaly score is determined by the maximum distance between a test patch and its nearest neighbor in the subsampled memory bank.

The anomalib.models.image sub-package provides classes and functions for defining image anomaly models. It supports tasks including:

• Anomaly classification
• Anomaly detection
• Anomaly segmentation

Note: These models can also process video datasets by treating them as frame-based image datasets.

CS-Flow (Fully Convolutional Cross-Scale-Flows) is a segmentation-type model designed for image-based defect detection. It handles fine-grained representations by incorporating both global and local image context through multiple scales.

Key technical aspects:

• Multi-scale feature extraction: Uses multiple scales to capture context.
• Cross-scale coupling blocks: Employs a fully-convolutional normalizing flow to process scales jointly using cross-scale convolution layers.
• Anomaly Scoring: The anomaly score for a local position $(i,j)$ is computed by aggregating values along the channel dimension using the $L_2$ norm of the latent variable $z$ ($||z^s_{i,j}||^2_2$). This allows for precise anomaly localization.

Anomalib provides type-safe data containers designed for anomaly detection tasks. These dataclasses include automatic validation and support multiple backends to allow for flexible data handling within your pipelines.

There are three primary categories of data classes:

1. Generic Classes: Base structures that define common data patterns, including Item/Batch structures, Input/Output fields, and Validation Mixins.
2. PyTorch Classes: Tensor-based implementations optimized for deep learning workflows. These include support for Image, Video, and Depth items, as well as batch processing and type-safe validation.
3. NumPy Classes: Array-based implementations designed for efficient data processing and array-based containers, including utilities for data conversion.

STFPM is an unsupervised anomaly detection algorithm that uses a pre-trained teacher network and a student network with identical architectures.

How it works:

• Training: The student network learns the distribution of anomaly-free images by matching its features with the corresponding features in the teacher network.
• Multi-scale Matching: It uses hierarchical feature matching (feature pyramids) to allow the student to learn multi-level knowledge, making it robust to anomalies of various sizes.
• Inference: During inference, the feature pyramids of both networks are compared. A larger difference between the teacher and student features indicates a higher probability of an anomaly.

Model Type: Segmentation