TorchIO Documentation

repository·main·Indexed 25 days ago

https://github.com/torchio-project/torchio

A Python package for medical image preprocessing, augmentation, and patch-based training in PyTorch-based deep learning workflows. TorchIO provides specialized tools for the efficient reading, sampling, and writing of 3D medical images, featuring a data model based on Image, Points, BoundingBoxes, Subject, and AffineMatrix. It supports lazy I/O to defer reading data from disk and offers a wide range of intensity and spatial transforms, including domain-specific medical imaging artifacts.

Tokens
41.7K
Snippets
110
Records
236
Agent score
81%

What's inside TorchIO

  1. Overview of TorchIO

    main

    TorchIO is a Python package designed for deep learning applications using PyTorch. It provides a suite of tools to efficiently handle 3D medical images, specifically for:

    • Reading and Writing: Efficient I/O for 3D medical image formats.
    • Preprocessing and Sampling: Tools to prepare and sample medical data.
    • Augmentation: A wide range of intensity and spatial transforms. These include standard computer vision operations (e.g., random affine transformations) and domain-specific medical imaging artifacts (e.g., MRI magnetic field inhomogeneity or k-space motion artifacts).
    • Patch-based Training: Support for efficient patch-based training workflows (e.g., using a Queue).
  2. Use TorchIO built-in datasets for testing and tutorials

    main

    TorchIO includes several built-in datasets designed for testing, tutorials, and demonstrations. When you call a dataset function for the first time, the data is automatically downloaded and cached locally for subsequent uses.

    Available dataset categories include:

    • Synthetic: Generated data like ZonePlate.
    • MNI: Standard brain templates (e.g., Colin27, Pediatric, Sheep).
    • IXI: Large-scale brain imaging datasets (ixi, ixi_tiny).
    • ITK-SNAP: Specific medical imaging tasks (BrainTumor, T1T2, AorticValve).
    • 3D Slicer: Data from the 3D Slicer platform.
    • FPG: FPG dataset.
    • MedMNIST: 3D datasets from MedMNIST v2.
  3. How TorchIO transforms are designed

    main

    TorchIO transforms are subclasses of torch.nn.Module. They are designed with a unified batch architecture: regardless of the input type (Subject, Image, Tensor, NumPy array, etc.), the transform internally converts the input into a SubjectsBatch before processing. This allows a single implementation to handle both individual samples and batches of data.

    Key behaviors:

    • Input/Output Consistency: Transforms return the same type they receive (e.g., a Subject input returns a Subject).
    • Batching: A single Image is treated as a batch of size 1. An image tensor of shape (C, I, J, K) reaches the internal application method as (B, C, I, J, K).
    • Per-instance Augmentation: When passing a SubjectsBatch (e.g., from a DataLoader), transforms sample independent parameters for each element by default. To share the same sampled parameters across all elements in a batch, pass per_instance=False to the transform.
    from typing import Any
    import torch
    import torchio as tio
    
    class AddValue(tio.Transform):
        """Add a fixed value to every image in a batch."""
    
        def __init__(self, value: float) -> None: 
            super().__init__()
            self.value = value
    
        def make_params(self, batch: tio.SubjectsBatch) -> dict[str, Any]:
            """Return the value to add."""
            return {"value": self.value}
    
        def apply_transform(
            self, 
            batch: tio.SubjectsBatch, 
            params: dict[str, Any]
        ) -> tio.SubjectsBatch:
            """Add the value to each 5D image tensor."""
            for image_batch in batch.images.values():
                image_batch.data = image_batch.data + params["value"]
            return batch
    
    subject = tio.Subject(
        image=tio.ScalarImage(torch.zeros(1, 2, 3, 4)),
        site="A",
    )
    batch = tio.SubjectsBatch.from_subjects([subject])
    assert subject.image.data.shape == (1, 2, 3, 4)
    assert batch.image.data.shape == (1, 1, 2, 3, 4)
    
    transformed = AddValue(2)(subject)
    assert isinstance(transformed, tio.Subject)
    assert transformed.image.data.shape == (1, 2, 3, 4)
    assert torch.all(transformed.image.data == 2)
  4. Use random parameters in TorchIO transforms

    main

    Many TorchIO transforms allow you to specify randomizable parameters (such as degrees, scales, or std) using different types of inputs. Instead of providing a single fixed value, you can provide a specification that the transform will sample from during the .apply() phase.

    Supported parameter specifications:

    • Scalar: Provides a deterministic value (e.g., degrees=10).
    • 2-tuple (a, b): Samples a value uniformly from the range $[a, b]$ (e.g., degrees=(-10, 10)).
    • 3-tuple or 6-tuple: Used for spatial parameters to set per-axis values (3-tuple) or per-axis ranges (6-tuple).
    • torchio.Choice: Samples from a discrete set of provided values.
    • torch.distributions.Distribution: Samples from the specified PyTorch distribution.
  5. How the TorchIO data model works

    main

    TorchIO's data model is built around five core classes that represent different aspects of medical imaging data: Image, Points, BoundingBoxes, Subject, and AffineMatrix.

    • Image: Represents 3D or multi-channel 3D medical images. It contains a 4D tensor (C, I, J, K) and an AffineMatrix.
    • Points: Stores $(N, 3)$ 3D coordinates (e.g., landmarks) with an AffineMatrix for coordinate conversion.
    • BoundingBoxes: Stores $(N, 6)$ 3D bounding boxes with a specified format and an AffineMatrix.
    • Subject (or Study): A container that groups Images, Points, BoundingBoxes, and metadata belonging to a single session.
    • AffineMatrix: A $4 imes 4$ matrix mapping voxel indices to world coordinates (mm).

    Typical Workflow:

    1. Create Image objects from file paths (loading is lazy).
    2. Create Points or BoundingBoxes from annotations.
    3. Group them into a Subject.
    4. Apply transforms to the Subject. This triggers data loading and produces a new transformed Subject.
    5. Access .data tensors for training.
    # Typical workflow sketch
    subject = tio.Subject(
        t1=tio.ScalarImage("t1.nii.gz"),
        seg=tio.LabelMap("seg.nii.gz"),
        landmarks=tio.Points(torch.randn(5, 3)),
        tumors=tio.BoundingBoxes(
            torch.tensor([[10, 20, 30, 50, 60, 70]]),
            format=tio.BoundingBoxFormat.IJKIJK,
        ),
        age=45,
    )
  6. Use composition strategies: Compose, OneOf, and SomeOf

    main

    TorchIO provides several ways to group transforms:

    • tio.Compose([...]): Applies all transforms in the list sequentially. Use the + operator as shorthand.
    • tio.OneOf({...}): Picks exactly one transform from a dictionary of {transform: probability}. Use the | operator as shorthand.
    • tio.SomeOf([...], num_transforms=(min, max)): Picks a random number of transforms from the list (between min and max inclusive) and applies them.
    # Compose: apply all in sequence
    pipeline = tio.Compose([
        tio.Affine(degrees=10),
        tio.Noise(std=0.05),
        tio.Gamma(log_gamma=0.3),
    ])
    
    # OneOf: pick one at random
    artifact = tio.OneOf({
        tio.Ghosting(intensity=0.5): 0.4,
        tio.Spike(intensity=1.0): 0.3,
        tio.Motion(degrees=5): 0.3,
    })
    
    # SomeOf: pick N at random
    augment = tio.SomeOf(
        [
            tio.Flip(axes=(0, 1, 2)),
            tio.Blur(std=1.0),
            tio.Noise(std=0.05),
            tio.Gamma(log_gamma=0.3),
        ],
        num_transforms=(1, 3),
    )
    
    # Operator sugar
    pipeline = tio.Flip(axes=(0,)) + tio.Noise(std=0.05) + tio.Gamma(log_gamma=(-0.3, 0.3))
    artifact = ( 
        tio.Ghosting(intensity=(0.5, 1)) | tio.Spike(intensity=(1, 3)) | tio.Motion()
    )
  7. Access batched data with SubjectsBatch and ImagesBatch

    main

    After collation, your data will be contained in one of two batch containers:

    • torchio.SubjectsBatch: A container for a batch of torchio.Subject objects. It allows you to access the aggregated data for all subjects in the batch.
    • torchio.ImagesBatch: A container for a batch of torchio.Image objects. It allows you to access the aggregated data for all images in the batch.
  8. Load and use NIfTI-Zarr images lazily

    main

    NIfTI-Zarr supports lazy partial reads. When you load a .nii.zarr file using tio.ScalarImage, accessing properties like .shape or .spacing only reads the metadata. Slicing the image (e.g., loaded[:, 50:100, 50:100, 50:100]) only reads the specific chunks required for that slice, which is efficient for large volumes.

    import torchio as tio
    
    loaded = tio.ScalarImage("output.nii.zarr")
    print(loaded.shape)    # reads only metadata
    print(loaded.spacing)  # from the stored affine
    
    # Lazy slice: reads only the needed chunks
    patch = loaded[:, 50:100, 50:100, 50:100]
    print(patch.data.mean())
  9. How metadata is handled in a batch

    main

    When multiple Subject objects are combined into a SubjectsBatch, the metadata is transformed from a single dictionary into a dictionary of lists. Each list contains the metadata value for the corresponding batch element.

    Example: If Subject A has age=30 and Subject B has age=40, the SubjectsBatch.metadata will be {'age': [30, 40]}.

    Custom transforms should preserve this shared schema and ensure metadata lists remain aligned with the batch dimension.

    import torch
    import torchio as tio
    
    subjects = [
        tio.Subject(
            image=tio.ScalarImage(torch.zeros(1, 2, 3, 4)),
            site="A",
            age=30,
        ),
        tio.Subject(
            image=tio.ScalarImage(torch.ones(1, 2, 3, 4)),
            site="B",
            age=40,
        ),
    ]
    batch = tio.SubjectsBatch.from_subjects(subjects)
    assert batch.metadata == {"site": ["A", "B"], "age": [30, 40]}
  10. Apply transforms to Subjects and Batches

    main

    Transforms in torchio are highly flexible. They accept Subject objects, Images, Tensors, NumPy arrays, SimpleITK/NiBabel images, or MONAI-style dictionaries.

    Key features:

    • Composition: Use tio.Compose to create pipelines.
    • Randomness: Transforms like tio.Flip or tio.Noise can be stochastic (using p for probability or distributions for parameters).
    • Batch Support: When applied to a batch from a SubjectsLoader, transforms are applied in a vectorized manner.
    • Type Preservation: Transforms generally return the same type they received.
    import torchio as tio
    from torch.distributions import LogNormal
    
    # Single deterministic transform
    flipped = tio.Flip(axes=(0,))(subject)
    
    # Random augmentation pipeline
    augment = tio.Compose([
        tio.Flip(axes=(0, 1, 2), p=0.5),
        tio.Noise(std=(0.01, 0.1)),   # random std each call
    ])
    augmented = augment(subject)
    
    # Custom distribution for parameters
    noisy = tio.Noise(std=LogNormal(loc=-2, scale=0.5))(subject)
    
    # Works directly on tensors too
    noisy_tensor = tio.Noise(std=0.05)(tensor)
    
    # Works with MONAI-style dicts
    data = {"image": tensor, "label": label_tensor}
    augmented = tio.Noise(std=0.1)(data)  # returns dict
    
    # Works on batches from SubjectsLoader (same params, vectorised)
    batch = next(iter(loader))  # SubjectsBatch
    augmented_batch = augment(batch)