DOCSEARCHbyAlgolia
Submit Docs

MONAI (Medical Open Network for AI)

repository·dev·Indexed 27 days ago

https://github.com/project-monai/monai

A PyTorch-based, open-source framework for deep learning in healthcare imaging. MONAI provides specialized tools for medical imaging preprocessing, domain-specific networks, losses, and evaluation metrics. It includes modules for AutoML (auto3dseg), federated learning, interactive segmentation (DeepGrow, DeepEdit, NuClick), pathology analysis, and object detection. The framework also supports MONAI Bundles for distributing portable, self-descriptive model bundles and integrates with nnUNet.

Tokens
33.4K
Snippets
22
Records
272
Agent score
44%

What's inside MONAI

  1. Overview of MONAI core modules

    dev

    MONAI is a framework for medical AI that provides specialized components for deep learning workflows. Key modules include:

    • apps: High-level medical domain-specific deep learning applications.
    • auto3dseg: AutoML components for volumetric image analysis.
    • bundle: Components for building portable, self-descriptive model bundles.
    • config: System configuration and diagnostic output.
    • data: Datasets, readers, writers, and synthetic data generation.
    • engines: Classes for extending Ignite behavior.
    • fl: Federated learning components for integration with federated learning frameworks.
    • handlers: Functionality implementations for various training process stages.
    • inferers: Model inference methods.
    • losses: Loss functions following the torch.nn.modules.loss pattern.
    • metrics: Metric tracking types.
    • networks: Network definitions, component definitions, and PyTorch utilities.
    • optimizers: Optimizers following the torch.optim pattern.
    • transforms: Data transforms for preprocessing and postprocessing.
    • utils: Pure Python/NumPy utilities (e.g., namespace aliasing, auto module loading).
    • visualize: Data visualization utilities.

  2. Use Registration Losses in MONAI

    dev

    For medical image registration tasks, MONAI offers several loss functions to optimize alignment between moving and fixed images. These include:

    • BendingEnergyLoss: Penalizes the bending energy of the deformation field.
    • DiffusionLoss: Based on the diffusion of the deformation field.
    • LocalNormalizedCrossCorrelationLoss: Uses local NCC to measure similarity.
    • GlobalMutualInformationLoss: Uses mutual information to measure statistical dependence between images.

  3. Use Reconstruction Losses in MONAI

    dev

    For tasks involving image reconstruction (e.g., super-resolution, denoising, or generative modeling), MONAI provides the following loss functions:

    • SSIMLoss: Structural Similarity Index Measure loss.
    • PatchAdversarialLoss: Adversarial loss applied at the patch level.
    • PerceptualLoss: Uses features from a pre-trained network to measure perceptual similarity.
    • JukeboxLoss: Specialized reconstruction loss.
    • SURELoss: Stein's Unbiased Risk Estimator loss.

  4. Explore MONAI transform categories

    dev

    MONAI transforms are organized into several functional categories based on the type of data they operate on (Array vs. Dictionary) and the nature of the operation. Key categories include:

    • Crop and pad: Operations for resizing or centering data (croppad.array, croppad.dictionary, croppad.batch).
    • Spatial: Geometric transformations (spatial.array, spatial.dictionary).
    • Intensity: Operations modifying pixel/voxel values (intensity.array, intensity.dictionary).
    • IO: Input/Output operations for loading and saving data (io.array, io.dictionary).
    • Lazy: Transforms for deferred execution (lazy.array, lazy.dictionary, lazy.utils).
    • Post-processing: Operations applied after model inference (post.array, post.dictionary).
    • Regularization: Transforms for data augmentation/regularization (regularization.array, regularization.dictionary).
    • MetaTensor utilities: Dictionary-based utilities for handling MetaTensor metadata (meta_utility.dictionary).
    • Smooth field: Operations for smoothing fields (smooth_field.array, smooth_field.dictionary).
    • Signal: Signal processing transforms (signal.array).
    • Utility: General purpose transforms (utility.array, utility.dictionary).

  5. Use Loss Wrappers in MONAI

    dev

    MONAI provides loss wrappers to modify or extend the behavior of standard loss functions for specific training strategies:

    • MultiScaleLoss: Applies loss across multiple scales (often used in multi-scale architectures).
    • MaskedLoss: Applies a loss only to specific regions defined by a mask.
    • DeepSupervisionLoss: Facilitates deep supervision by applying losses to intermediate feature maps.

  6. Understand the MONAI Bundle (MB) format

    dev
    A MONAI Bundle (MB) is a portable package for deep learning models. It contains the model weights (typically a pickled state dictionary) and optional Torchscript or ONNX objects. It also includes JSON metadata files that define how the model is used, its input/output tensors, and the sequences for training, inference, and post-processing. Bundles can be distributed as a directory or a compressed .zip file.

  7. Access integrated Generative AI components in MONAI Core

    dev

    Starting with version 1.3, key components from MONAI GenerativeModels have been ported into the MONAI core codebase. This allows for consistent maintenance and release of generative AI tools.

    Users can now access integrated loss functions and metrics specifically designed for generative AI tasks directly within the core MONAI library.

  8. Use integrated MONAI Generative modules

    dev

    Modules previously located in the GenerativeModels repository (including utilities, networks, diffusion schedulers, inferers, and engines) are now integrated directly into the core MONAI codebase.

    Compatibility Note: Weights from models trained using the standalone GenerativeModels repository can be loaded into the integrated core modules.

  9. Understand MONAI Bundles

    dev

    MONAI Bundles are a standardized specification and file structure used to distribute trained MONAI models. A bundle includes the model itself along with its associated metadata, code, documentation, and other necessary resources.

    Bundles are designed to facilitate:

    • Distributing models with clear explanations of their purpose.
    • Providing instructions on how to use the model.
    • Enabling the reproduction of scientific results.
    • Allowing use in other applications like MONAI Label or deployment via MAPs.

    Bundles are intended to be used directly for training, inference, and reproducing results. They are more lightweight than MONAI Application Packages (MAPs) and typically include training scripts, whereas MAPs are focused on deployment and inference-only tasks.

  10. Use MONAI Network Blocks

    dev

    MONAI provides a wide variety of reusable building blocks for constructing neural network architectures. These blocks are located in monai.networks.blocks and include components for different tasks such as segmentation, registration, and transformer-based models.

    Key categories of blocks include:

    • U-Net Components: UnetResBlock, UnetBasicBlock, UnetUpBlock, UnetOutBlock.
    • Transformer & Attention: TransformerBlock, HyenaMixer, HyenaTransformerBlock, PatchEmbeddingBlock, SEBlock (Squeeze-and-Excitation).
    • Feature Pyramids: FeaturePyramidNetwork, ExtraFPNBlock, BackboneWithFPN.
    • Registration Blocks: RegistrationResidualConvBlock, RegistrationDownSampleBlock, RegistrationExtractionBlock, Warp.
    • Convolutional & Pooling: Convolution, DenseBlock, ResBlock (SegResnet), MaxAvgPool, UpSample, DownSample.

  11. Use Segmentation Losses in MONAI

    dev

    MONAI provides a variety of loss functions specifically designed for medical image segmentation tasks. These include Dice-based losses, Focal losses, and boundary-aware losses. Common segmentation losses available in monai.losses include:

    • Dice-based: DiceLoss, Dice, dice, MaskedDiceLoss, GeneralizedDiceLoss, generalized_dice, GeneralizedWassersteinDiceLoss, generalized_wasserstein_dice, SoftclDiceLoss, SoftDiceclDiceLoss.
    • Hybrid/Combined: DiceCELoss (Dice + Cross Entropy), DiceFocalLoss, GeneralizedDiceFocalLoss.
    • Focal/Weighting: FocalLoss, TverskyLoss.
    • Boundary/Distance: BoundaryLoss, HausdorffDTLoss.
    • Contrastive/Representation: ContrastiveLoss, BarlowTwinsLoss, NACLLoss, MCCLoss.

  12. Automate 3D medical image segmentation with Auto3DSeg

    dev

    Auto3DSeg is a solution for large-scale 3D medical image segmentation that automates the pipeline from data analysis to ensemble prediction.

    Workflow:

    1. Data Analysis: Analyzes global dataset information (intensity, dimensionality, resolution).
    2. Algorithm Generation: Generates MONAI bundle-format algorithms based on data statistics and algorithm templates.
    3. Training: Initiates model training to obtain checkpoints with optimal validation performance.
    4. Ensemble: Uses an ensemble module to rank checkpoints and create ensemble predictions.

    It is designed for both beginners and advanced researchers and has been validated on various 3D medical imaging modalities.