HistoQC 2.0

HistoQC operates as a sequential pipeline of modules applied to an image. The core mechanism is the refinement of a pixel mask called img_mask_use.

1. Initial State: When an image is loaded, it is assigned a "True mask" (img_mask_use) where every pixel is considered artifact-free and "useful" for analysis.
2. Sequential Refinement: As modules run, they update img_mask_use to exclude pixels that fail certain criteria (e.g., background, artifacts, or low-intensity areas).
3. Module Interaction: Many modules include a limit_to_mask option. When enabled, the module only performs operations on pixels currently marked as true in the img_mask_use mask.

Important: Because modules refine the same mask, the order of modules in your pipeline is critical. A module that identifies and masks out background (like LightDarkModule.getIntensityThresholdPercent:tissue) should be placed before modules that calculate statistics (like HistogramModule.getHistogram) to ensure the statistics are only calculated on valid tissue areas.

Classification modules in HistoQC use a model-based approach rather than simple statistics. To set this up, you must provide a set of exemplar images and their corresponding binary masks using the examples parameter.

Each line in the examples parameter must specify one exemplar set using the format: path/to/original_image.png:path/to/binary_mask.png.

Requirements for exemplars:

• Mask Format: The mask must be a binary image containing only values {0, 1}. A value of 1 identifies pixels for the positive class, and 0 identifies pixels for the negative class.
• Image Size: For optimal performance, exemplar images should be of the same magnification specified by the image_work_size parameter.
• Pathing: You can use either relative or absolute paths.

examples: ./pen/1k_version/pen_green.png:./pen/1k_version/pen_green_mask.png
          #./pen/1k_version/pen_red.png:./pen/1k_version/pen_red_mask.png

The HistoQC pipeline is defined using a configuration file following Python's configparser syntax. The core of the pipeline is the [pipeline] section, which defines the sequence of modules to be executed on each image.

Module Naming and Reusability

You can use the same module multiple times in a single pipeline by assigning it a unique instance name using the syntax ModuleName.method:instance_name.

When you create an instance (e.g., LightDarkModule.getIntensityThresholdPercent:tissue), you must define a corresponding section in the config file named exactly after that instance (e.g., [LightDarkModule.getIntensityThresholdPercent:tissue]). This section must contain a name: parameter, which determines the output name of the image and the column name in the resulting TSV file.

Example Pipeline Configuration

[pipeline]
steps= BasicModule.getBasicStats
    BasicModule.getMag
    ClassificationModule.byExampleWithFeatures:pen_markings
    ClassificationModule.byExampleWithFeatures:coverslip_edge
    LightDarkModule.getIntensityThresholdPercent:tissue
    LightDarkModule.getIntensityThresholdPercent:darktissue
    MorphologyModule.removeSmallObjects
    MorphologyModule.fillSmallHoles
    BlurDetectionModule.identifyBlurryRegions
    BasicModule.finalProcessingSpur
    BasicModule.finalProcessingArea
    HistogramModule.compareToTemplates
    HistogramModule.getHistogram
    BrightContrastModule.getContrast
    BrightContrastModule.getBrightness
    DeconvolutionModule.seperateStains
    SaveModule.saveFinalMask
    SaveModule.saveThumbnail
    BasicModule.finalComputations

The recommended workflow for processing histology images with HistoQC involves a two-step approach to ensure data homogeneity and prevent computational errors (like out-of-memory issues).

Step 1: Discovery and Cohort Splitting

Run HistoQC using a minimal pipeline (e.g., config_first.ini). This step is used to:

• Identify images scanned at different magnifications (e.g., 20x vs 40x).
• Perform basic tissue detection (white threshold).
• Compute histograms and differences to target templates.

Goal: Split your dataset into sub-cohorts that share the same magnification, the same number of internal storage levels, and similar appearance properties (lightness, stain intensity, etc.).

Using the Web UI:

• Use the parallel coordinates graph to create filters by clicking and dragging on axes.
• Use "Save Filtered" to save a specific subset of images based on your filters.
• Use "Delete" (after multi-selecting) to remove unsuitable images from the table.
• Click "Save" to generate an output file containing only the selected subset.

Step 2: Full Pipeline Execution

Once sub-cohorts are established, rerun the pipeline using an expanded set of models (e.g., using config.ini designed for 40x H&E images). This step allows you to:

• Identify images with artifacts unsuitable for computation.
• Verify if suggested masks are appropriate for downstream tasks by using the "compare" dropdown and selecting _fuse.png to view the original image alongside fused images.