DOCSEARCHbyAlgolia
Submit Docs

Cellpose Documentation

repository·main·Indexed 25 days ago

https://github.com/mouseland/cellpose

A general-purpose algorithm for cell segmentation in biological 2D and 3D images. It features a GUI for interactive segmentation, a CLI for batch processing and training, and specialized models like Cellpose-SAM and DINOv3-based models (cpdino, cpdino-vitb). The tool supports GPU acceleration via CUDA and provides the CellposeModel class for programmatic inference and the cellpose.train module for custom model training.

Tokens
23.6K
Snippets
62
Records
121
Agent score
81%

What's inside Cellpose

  1. Overview of Cellpose-SAM

    main

    Cellpose-SAM is a cellular and nucleus segmentation tool designed for superhuman generalization. It is optimized for various data conditions, including:

    • 3D applications
    • Images with shot noise
    • Anisotropic blur
    • Undersampling
    • Contrast inversions
    • Variable channel orders and object sizes

    For batch processing without local installation, you can use the Cellpose-SAM website via Hugging Face.

  2. Use `preprocessing_steps` for custom processing and multi-channel segmentation

    main

    Distributed Cellpose can execute custom preprocessing functions on each block before segmentation.

    Rules for Preprocessing Functions:

    • The first parameter must be the image.
    • The last parameter must be the crop (which contains the slices for the current block).
    • You can include any number of other parameters in between.
    • Example signature: def my_step(image, param1, crop): ...

    Use Cases:

    1. Smoothing: Applying Gaussian filters.
    2. Multi-channel segmentation: Using a preprocessing step to stack a second channel (from another Zarr array) onto the current block's image using the crop to ensure spatial alignment.
    3. Background subtraction: Subtracting a background channel using the crop to index into a background Zarr array.
    from scipy.ndimage import gaussian_filter
from cellpose.contrib.distributed_segmentation import distributed_eval

# Example: Preprocessing with Gaussian smoothing and channel stacking
def pp_step_one(image, sigma, crop):
    return gaussian_filter(image, sigma)

def stack_channels(image, crop):
    # second_channel_zarr must be a Zarr array
    return np.stack((image, second_channel_zarr[crop]), axis=1)

preprocessing_steps = [
    (pp_step_one, {'sigma': 2.0}), 
    (stack_channels, {})
]

# Pass to distributed_eval
segments, boxes = distributed_eval(
    input_zarr=large_zarr_array,
    blocksize=(256, 256, 256),
    write_path='/path/to/output.zarr',
    preprocessing_steps=preprocessing_steps,
    model_kwargs=model_kwargs,
    eval_kwargs={'channels': [2, 1], ...},
    cluster_kwargs=cluster_kwargs,
)

  3. Optimizing Cellpose for large objects and diameters

    main

    Cellpose models are trained on images with diameters ranging from 7.5 to 120 pixels (mean size 30 pixels). If your objects differ significantly, use these parameters:

    • Large Diameters: Specify the diameter parameter. For example, diameter=60 downsamples the image by a factor of 2 relative to the 30-pixel mean.
    • Dynamics: For cells with very large diameters, you may need to increase the niter parameter to allow the mask creation dynamics step to run longer.
    • DINO Tile Size: For DINO-based models, increase the bsize parameter if you have very large or long objects.

  4. How image restoration models work in Cellpose

    main

    The denoise module provides image restoration via four types of operations:

    • denoising
    • deblurring
    • upsampling
    • one-click (trained on all degradation types)

    Models are categorized by their training dataset: cyto3, cyto2, or nuclei.

    Available model names follow the pattern {type}_{dataset} (e.g., 'denoise_cyto3', 'upsample_nuclei'). Models are automatically downloaded upon first use. Some models also include reconstruction loss variants, such as 'denoise_rec_cyto2'.

  5. Supported image formats and channel ordering

    main

    Cellpose supports TIFF, PNG, and JPEG formats. Images are loaded using tifffile or cv2.

    For single-plane images, you can provide data in either (nY, nX, channels) or (channels, nY, nX) format. The channels setting in Cellpose will handle the necessary reshaping for the neural network.

    Note on Normalization: By default, the model normalizes each channel such that the 0 value corresponds to the 1st percentile of image values and the 1 value corresponds to the 99th percentile.

  6. Adjust the diameter parameter

    main
    Cellpose-SAM is trained on ROI diameters from 7.5 to 120 pixels (mean: 30). While the model is size-invariant and diameter is optional, you can use it to control processing speed. If you provide a large diameter (e.g., 90), the image will be downsampled (e.g., by a factor of 3), increasing run speed for very large cells.

  7. Handle multiclass segmentation by training multiple models

    main
    Cellpose does not natively support multiclass segmentation (recognizing different cell types in a single image). To achieve this, train individual models for each specific cell type and run Cellpose multiple times on the same image. You can then combine the resulting outputs during post-processing to identify different cell types.

  8. Handle normalization when re-training models

    main

    When re-training a model using image crops, normalization happens per crop, which may differ from the normalization of full images. To approximate per-crop normalization on full images during evaluation, you can adjust the normalize parameter in model.eval.

    Options for model.eval(normalize=...):

    • Tile Normalization: Use {"tile_norm_blocksize": 128} to handle local normalization.
    • Percentile Scaling: Adjust the overall scaling using {"percentile": [lower, upper]}. For example, {"percentile": [3, 98]}.

    To visualize how normalization looks in a notebook, you can use cellpose.transforms.normalize99(img, lower=3, upper=98) with matplotlib.

  9. Understand Human-in-the-Loop (HITL) training

    main

    HITL training in Cellpose is an iterative process designed to produce a model that is both fine-tuned to your specific data and general enough to segment new images.

    The Workflow:

    1. Start with a pretrained model: Always start from a generalist pretrained model rather than a previously fine-tuned model to prevent the network from memorizing data and losing generalization capabilities.
    2. Predict: Use the current model to predict segmentations on new images.
    3. Annotate: Accept correct masks and edit/add incorrect ones using the Cellpose GUI.
    4. Train: Incorporate the newly annotated images into the training set. Each iteration increases the amount of training data.
    5. Repeat: Continue the cycle until the model performs accurately on your target data without further training.

  10. Use Distributed Cellpose for larger-than-memory data

    main

    The cellpose.contrib.distributed_cellpose module allows running Cellpose on 3D datasets that exceed system memory. It works by dividing the dataset into overlapping blocks, segmenting each block, and stitching the results back together into a seamless segmentation.

    Key Concepts:

    • Input Format: Data must be a zarr array.
    • Compute Resources: Supports workstations and LSF clusters. Blocks can be run in parallel, in series, or both.
    • Foreground Masking: You can provide a foreground mask to avoid processing empty background areas. The mask does not need the same resolution as the input, but must have the same field of view (physical length of axes).
    • Preprocessing: You can pass a list of preprocessing_steps (functions and their arguments) to be distributed along with the segmentation. This is useful for tasks like multi-channel segmentation or smoothing.

  11. Understand the outputs of `models.CellposeModel.eval`

    main

    When running models.CellposeModel(gpu=True).eval(img) in a notebook, the function returns three variables: masks, flows, and styles.

    • masks: A numpy array of size (Lz x) Ly x Lx where 0 represents no ROI and integers 1, 2, ... represent unique ROI labels.
    • flows: Contains the network's predicted flows. The Y and X flows are used to simulate a dynamical system on pixels where cellprob > cellprob_threshold. The cellprob values typically range from -10 to +10.
    • styles: The sum over pixels of the output of the last downsampling layer of the network.
    from cellpose import io, models
img = io.imread("img.tif")
masks, flows, styles = models.CellposeModel(gpu=True).eval(img)

  12. Run Cellpose from the command line

    main

    You can run Cellpose using the python -m cellpose command. You must specify parameters such as the input directory or image path, and desired output formats.

    Note: It is recommended to use absolute paths when providing the --dir argument.