DOCSEARCHbyAlgolia
Submit Docs

TIAToolbox

repository·develop·Indexed 19 days ago

https://github.com/tissueimageanalytics/tiatoolbox

A computational pathology toolbox built on PyTorch providing an end-to-end API for pathology image analysis. It supports data loading, pre-processing, model inference, post-processing, and visualization. Key features include WSI reading, stain normalization (Reinhard, Ruifork, Macenko, Vahadane), tissue mask and patch extraction, semantic segmentation, and nucleus instance segmentation via HoVer-Net, HoVer-Net+, and KongNet. It also includes pipelines for predicting molecular pathways (IDaRS) and HER2 status (SlideGraph+), as well as image registration tools.

Tokens
29.7K
Snippets
72
Records
116
Agent score
68%

What's inside tiatoolbox

  1. Overview of TIAToolbox example notebooks

    develop

    The examples/ directory provides Jupyter Notebooks demonstrating various TIAToolbox functionalities. The directory is organized into two main categories:

    1. General Functionalities: Notebooks demonstrating core modules (e.g., WSI reading, stain normalization, patch extraction).
    2. Pipelines:
      • full-pipelines/: Examples for training neural networks.
      • inference-pipelines/: Examples for WSI inference for high-level applications like patient survival or MSI status prediction.

    Core Functionality Examples available:

    • Reading Whole Slide Images
    • Stain Normalization
    • Tissue Mask Extraction
    • Patch Extraction
    • Patch Prediction
    • Semantic Segmentation

  2. Understanding the IDaRS algorithm

    develop

    IDaRS (Iterative Draw and Rank Sampling) is a weakly supervised deep learning algorithm used to infer high-resolution tile-level labels from low-resolution slide-level (WSI) labels.

    It is designed for scenarios where a feature $F$ (e.g., a genetic abnormality) is known to exist at the WSI level, and the goal is to estimate the probability $q(T) ext{ } orall T ext{ } orall ext{ tiles } T$ that a specific tile contains that feature.

    Key Concepts:

    • Input: WSIs provided with binary labels ($True=1$ or $False=0$).
    • Goal: Estimate $q(T) ext{ } orall T ext{ } orall ext{ tiles } T$, the probability that tile $T$ has feature $F$.
    • Mechanism: It uses an iterative process of sampling tiles to build a training set (nts). In each epoch, it selects the $k$ tiles with the highest predicted probability from the current training set and supplements them with $r$ randomly chosen tiles to update the model via stochastic gradient descent.

  4. Predict HER2 Status in Breast Cancer using SlideGraph+

    develop

    TIAToolbox can reproduce the SlideGraph+ method to predict HER2 status from H&E stained WSIs. The process involves:

    1. Generating a graph that represents the WSI.
    2. Feeding the graph into a convolutional graph network (SlideGraph).

    There are separate notebooks for training the SlideGraph model and performing WSI inference.

  5. Extract Features Using Foundation Models

    develop

    You can extract features from WSIs using pre-trained models from the timm library. The workflow involves:

    • Selecting appropriate model architectures.
    • Extracting features.
    • Visualizing features using UMAP feature embedding to verify if different tissue types are correctly separated in the feature map.

  6. Run models on selected image regions

    develop

    You can interactively run models on specific regions of a slide:

    1. Select a region using box select or points.
    2. Select the desired model from the dropdown menu (e.g., hovernet).
    3. Click go.

    To save the resulting annotations, use the save button. Currently, annotations are saved as a SQLiteStore .db file in the format {slide_name}_saved_anns.db within your overlays folder.

  7. Register and Align Image Pairs

    develop

    TIAToolbox provides tools for registering image pairs using:

    1. Deep Feature Based Registration (DFBR): A pre-alignment step followed by DFBR.
    2. Non-rigid alignment: Using SimpleITK to handle non-rigid deformations between tiles after an initial affine transformation is computed (often using thumbnails).

  8. Organize files for the Visualization Interface

    develop

    To use the TIAToolbox visualization interface, organize your files into two main folders:

    1. slides folder: Contains all the Whole Slide Images (WSIs) you want to view.
    2. overlays folder: Contains the graphs, segmentations, heatmaps, etc., that you want to overlay on the slides.

    Matching Overlays to Slides: When a slide is selected in the interface, the system automatically looks for overlay files in the overlays folder that have the exact same name (excluding the file extension) as the selected slide. For example, if your slide is sample_01.svs, the interface will look for sample_01.db, sample_01.json, or sample_01.png in the overlays folder.

  9. Visualize image registration and transformations

    develop

    You can visually inspect the accuracy of precomputed registration (e.g., H&E to IHC) by providing a registration matrix (e.g., .mha or .npy file).

    Dual Window Mode (Side-by-Side)

    1. Open Dual Window Mode.
    2. In Window 1, load the source image (e.g., H&E).
    3. In Window 2, load the target image (e.g., IHC).
    4. Load the registration file as an overlay on the source image.

    Overlay Mode (Direct Overlay)

    1. Open the source image.
    2. Overlay the target image on the source image.
    3. Load the registration file as an overlay on the source image.

    CRITICAL: Always load the target image first when using overlays to ensure the system correctly identifies the transformation direction. Ensure the loading order matches how your registration matrix was computed.

  10. Perform Stain Normalization on Histology Images

    develop

    Stain normalization is used to reduce color variation caused by different scanners or staining protocols. TIAToolbox provides several built-in algorithms to make models more robust against these variations. You can use any of the following algorithms:

    • Reinhard stain normalization
    • Ruifork
    • Macenko
    • Vahadane

    Additionally, you can provide your own custom stain matrix to perform normalization.

  11. Prepare segmentation overlays using AnnotationStore

    develop

    While the visualization interface can load GeoJSON (.geojson) and HoVerNet-style (.dat) files, it is highly recommended to convert them to the AnnotationStore format (typically a .db file) before using the interface. This avoids a conversion delay during loading and provides an optimized experience.

    Conversion Methods:

    • From HoVerNet .dat files: Use tiatoolbox.utils.misc.store_from_dat (also handles Cerberus .dat outputs).
    • From GeoJSON: Use AnnotationStore.from_geojson().
    • From PatchPredictor output: Use tiatoolbox.utils.misc.dict_to_store.

    Creating an AnnotationStore from scratch:

    If your data is in a custom format (like raw centroids or contours), you can build an AnnotationStore using a small script. Note that setting a property named type allows the UI to toggle specific annotation types on/off.

    # Example: Converting GeoJSON to AnnotationStore
from pathlib import Path
from tiatoolbox.annotation.storage import SQLiteStore

geojson_path = Path("path/to/annotations.geojson")
db1 = SQLiteStore.from_geojson(geojson_path)
db1.dump("path/to/annotations.db")

  12. Run TIAToolbox example notebooks on Google Colab

    develop

    You can run TIAToolbox examples remotely using Google Colab without installing any local software. Each notebook in the examples/ directory contains links to open the notebook directly in Colab.

    Important Steps for Colab:

    1. Installation & Restart: After installing tiatoolbox in a Colab session, you must restart the runtime via Runtime → Restart runtime to ensure prerequisite packages are loaded correctly.
    2. Hardware Acceleration: To use GPU acceleration, go to Runtime → Change runtime type → Hardware accelerator and select GPU.
    3. Device Configuration: If you are not using a GPU, you must manually change the device variable in the notebook code to 'cpu' to avoid errors.