DOCSEARCHbyAlgolia
Submit Docs

OmniCloudMask

repository·main·Indexed 41 days ago

https://github.com/dpird-dma/omnicloudmask

A sensor-agnostic deep learning Python library for segmenting clouds and cloud shadows in high to moderate resolution satellite imagery. It requires Red, Green, and NIR bands and provides classification for clear sky, thick clouds, thin clouds, and cloud shadows. The library includes tools for inference via `predict_from_array` and `predict_from_load_func`, as well as Docker support for running Jupyter Lab workflows with Sentinel-2, HLS, PlanetScope, and Maxar data.

Tokens
32.7K
Snippets
104
Records
138
Agent score
52%

What's inside OmniCloudMask

  1. Overview of OmniCloudMask

    main

    OmniCloudMask is a Python library designed for state-of-the-art cloud and cloud shadow segmentation in high to moderate resolution satellite imagery. It is compatible with sensors providing Red, Green, and NIR bands, such as Sentinel-2, Landsat, PlanetScope, and Maxar.

    Key Capabilities

    • Resolution Support: Works with imagery from 10 m to 50 m resolution (and down to 5 m with recent model versions).
    • Data Compatibility: Supports various processing levels including L1C, L2A, TOA, and surface reflectance.
    • Hardware Acceleration: Supports inference via CUDA, MPS (Apple Silicon), and CPU.
    • Processing: Features fast, multi-threaded patch-based processing and optional confidence map export.

  2. OmniCloudMask Model Architecture

    main
    The OCM architecture consists of an ensemble of two U-Net models. Predictions from these models are combined using soft voting. This ensemble approach is used to improve both detection accuracy and generalization capabilities compared to using a single U-Net model.

  3. Interpret confidence map output

    main

    When export_confidence=True is enabled, the output array has a shape of (4, height, width). Each channel represents the probability of a specific class (values range from ~0.001 to ~0.999):

    • Channel 0: Clear probability
    • Channel 1: Thick cloud probability
    • Channel 2: Thin cloud probability
    • Channel 3: Cloud shadow probability

  4. How Mixed Resolution Training works

    main

    To handle varying sensor resolutions (from 3m PlanetScope to 30m Landsat), OCM utilizes mixed resolution training.

    Training Mechanism:

    • During training, data is randomly resampled to resolutions between 9m and 50m.
    • This acts as a data augmentation technique that improves accuracy at the native training resolution while teaching the model to recognize features at different scales.

    Inference Behavior:

    • Higher resolution imagery (e.g., 3m PlanetScope) is resampled to 10m for inference.
    • For model versions V3 and later, the model supports super-resolved Sentinel-2 imagery (2×), extending the supported range down to 5m. Note that 5m resolution is not more accurate than 10m; it simply increases the range of compatible imagery.

  5. How Dynamic Z-score Normalization works

    main

    Unlike traditional deep learning that uses global dataset statistics for normalization, OCM uses dynamic Z-score normalization. This process is applied per-patch and per-channel during both training and inference.

    The Process:

    1. Calculate the mean and standard deviation for each individual patch.
    2. Normalize each channel to zero mean and unit standard deviation.

    Key Benefits for Users:

    • Sensor Agnosticism: Imagery from any sensor is mapped to a common representation regardless of its original value range (e.g., L1C vs L2A, TOA vs surface reflectance).
    • Input Flexibility: You can provide raw pixel values or pre-normalized imagery; OCM will re-normalize the patches independently, ensuring correct operation.
    # Conceptually, for each patch and channel:
normalized = (patch - patch.mean()) / patch.std()

  7. How OmniCloudMask achieves sensor-agnostic detection

    main

    OmniCloudMask (OCM) is designed to detect clouds and cloud shadows across different satellite sensors (e.g., Sentinel-2, Landsat, PlanetScope, Maxar) without requiring sensor-specific training. It achieves this through two core mechanisms:

    1. Dynamic Z-score Normalization: Decouples the model from sensor-specific pixel value distributions by normalizing each input patch independently.
    2. Mixed Resolution Training: Decouples the model from a fixed spatial resolution by training on data randomly resampled between 9m and 50m, allowing it to generalize across different spatial scales.

    Because of these techniques, the model focuses on image texture and spatial relationships rather than absolute pixel values.

  10. Access Jupyter Lab in OmniCloudMask

    main

    Once the container is running, access the Jupyter Lab interface via your web browser at:

    http://localhost:8888

    Note: No password or token is required by default. If port 8888 is already in use on your host, map a different host port (e.g., 9999) using -p 127.0.0.1:9999:8888 and access it at http://localhost:9999.

    # Example of running on a different port if 8888 is busy
docker run -p 127.0.0.1:9999:8888 omnicloudmask:local

  12. Optimize accuracy using spatial context in OmniCloudMask

    main

    The accuracy of OmniCloudMask predictions is highly dependent on the size of the input patch provided to predict_from_array or predict_from_load_func. When processing small areas of interest, ensure you provide enough surrounding context to the model to avoid sharp performance drops.

    Key Recommendations

    • Minimum Patch Size: Use at least 96×96 pixels centered on your area of interest. While accuracy gains occur between 32×32 and 96×96, performance drops sharply below 96×96 due to insufficient context.
    • Prefer Larger Patches: Accuracy improves beyond 96×96, though at a slower rate. The default patch_size=1000 is recommended for reliable results and more efficient tiling of large scenes.
    • Class-Specific Sensitivity: If your application requires accurate detection of thin cloud or cloud shadow, you must use larger patches. These specific classes suffer the most from small patch sizes, whereas 'Clear' and 'Thick Cloud' are more resilient to minimal context.

    When to apply these guidelines

    Apply these recommendations if:

    1. You have manually set a small patch_size in your prediction functions.
    2. Your input scene is small enough that OmniCloudMask automatically reduces the patch size to match the scene dimensions.
    3. You are deciding whether to downsample imagery for speed versus processing at native resolution.