decoupler

repository·main·Indexed 42 days ago

https://github.com/scverse/decoupler

A Python package providing an ensemble of statistical methods to infer biological enrichment scores (activities) from omics data within a unified framework. Version 2.2.1 includes modules for enrichment scoring (mt), OmniPath resource integration (op), visualization (pl), preprocessing (pp), and benchmarking (bm).

Tokens
20.9K
Snippets
75
Records
105
Agent score
53%

What's inside decoupler

  1. Overview of decoupler submodules

    main

    The library is organized into namespaced submodules. The standard way to use the library is to import it as dc.

    • dc.mt: Methods. Contains enrichment/activity algorithms like ulm, mlm, ora, gsea, gsva, aucell, viper, zscore, waggr, mdt, and udt. Also includes dc.mt.decouple (multi-method) and dc.mt.consensus.
    • dc.op: Prior knowledge (OmniPath). Functions for fetching/handling collectri, dorothea, progeny, hallmark, etc.
    • dc.pp: Preprocessing. Tools for pseudobulk, filtering (filter_by_expr, filter_by_prop), and data manipulation (get_obsm, extract, swap_layer, read_gmt).
    • dc.ds: Datasets. Bundled example datasets like toy, pbmc3k, and covid5k. Note: Check if the dataset is raw or normalized before use.
    • dc.tl: Tools. Functions for ranking scores: rankby_group, rankby_obsm, rankby_order.
    • dc.pl: Plotting. Visualization tools: barplot, dotplot, volcano, network, obsm.
    • dc.bm: Benchmarking. Tools for evaluating methods: benchmark, metric, pl.
    import decoupler as dc
  2. Access datasets in decoupler.ds

    main
    The decoupler.ds module provides access to various datasets categorized by data type: Bulk, Single-cell, Spatial, and Toy datasets. These are useful for testing workflows, benchmarking, or as reference data for enrichment score inference.
  3. Handle different `data` input types and return values

    main

    The return type of a dc.mt method depends on the format of the data argument provided:

    1. AnnData: The method returns None. Results are written in-place to adata.obsm (e.g., adata.obsm['score_method'] and adata.obsm['padj_method']).
    2. pandas.DataFrame: Returns a tuple (es, pv) where es is the enrichment scores DataFrame and pv is the p-values DataFrame.
    3. list: Returns a tuple (es, pv) where the input list must be in the format [matrix, obs_names, var_names] (e.g., a numpy/sparse matrix and its corresponding index arrays).
  4. Understand the `net` (Prior Knowledge) format

    main

    Every enrichment method in decoupler requires a net: a long-format pandas.DataFrame that describes the relationship between biological programs and features (genes).

    Net Structure

    columnmeaningrequired
    sourcethe program (TF, pathway, gene set)yes
    targetthe feature (gene)yes
    weightsigned importance of the edgeonly for weighted methods

    Usage Rules

    • Weighted methods (ulm, mlm, viper, zscore, waggr, mdt, udt) use the weight column. The sign of the weight encodes activation or repression.
    • Gene-set methods (ora, gsea, aucell, gsva) ignore the weight column.
    • One row per edge: Each row represents a single connection between a source and a target.
    import decoupler as dc
    # Example of a net with source, target, and weight columns
    net = dc.op.collectri(organism="human")
    net.head()      # columns: source, target, weight
  5. How return types work based on input type

    main

    The return type of every dc.mt (method) depends on whether you pass an AnnData object, a DataFrame, or a list. This is a critical distinction to avoid losing data.

    • AnnData input: The method writes results in place into the object and returns None.
    • DataFrame or list input: The method returns an (es, pv) tuple containing the results.

    Warning: Do not use the assignment pattern adata = dc.mt.method(adata, ...) when passing an AnnData object, as this will overwrite your adata variable with None and cause you to lose your data.

    # CORRECT for AnnData
    dc.mt.ulm(adata, net, tmin=3)
    # Results are now in adata.obsm
    
    # INCORRECT for AnnData (will overwrite adata with None)
    adata = dc.mt.ulm(adata, net, tmin=3)
  6. Choosing a method for enrichment score inference

    main

    Decoupler provides several methods to infer enrichment scores, which can be categorized based on the type of network (prior) you are using:

    1. Weighted Networks

    Use these if your network contains signed weights (e.g., CollecTRI, DoRothEA, or PROGENy). Recommended methods include:

    • ulm (Univariate Linear Model): The recommended default for TF/pathway activity. It is fast and robust.
    • mlm (Multivariate Linear Model): Accounts for correlated regulons. Note that it can be unstable if the network is rank-deficient (highly collinear regulons).
    • viper (VIPER regulon analysis): Corrects for pleiotropy and target overlap.
    • waggr (Weighted Aggregate): Uses weighted mean, sum, or median with permutation p-values.
    • zscore: A simple standardized weighted mean.

    2. Unweighted Gene Sets

    Use these if you are using gene sets without weights (e.g., MSigDB hallmarks or .gmt files). Recommended methods include:

    • ora (Over-Representation Analysis): Fisher test on top/bottom features.
    • gsea (Gene Set Enrichment Analysis): Ranked enrichment of gene sets.
    • aucell (AUCell): Rank-based gene-set activity per cell.
    • gsva (Gene Set Variation Analysis): Sample-relative gene-set variation.

    3. Decision Trees

    • mdt (Multivariate Decision Tree): Uses random-forest feature importance.
    • udt (Univariate Decision Tree): Uses single-tree importance.

    Tip: If you are unsure, you can run several methods and take a consensus using the multi-method approach.

  7. Where enrichment scores are stored and how to retrieve them

    main

    The location of your results depends entirely on the type of data you pass to the method. This is a critical distinction for retrieving scores.

    1. Using AnnData (In-place modification)

    If you pass an AnnData object, the method returns None and writes the results directly into the object's .obsm attribute.

    • Scores: Stored in .obsm["score_<method>"].
    • P-values: If the method performs statistical testing, p-values are stored in .obsm["padj_<method>"].
    • Retrieval: Use dc.pp.get_obsm(adata, "score_<method>") to safely access these scores.

    2. Using pandas.DataFrame or a list (Return values)

    If you pass a DataFrame or a list [matrix, obs, var], the method returns an (es, pv) tuple.

    • es: The enrichment score matrix (observations × sources).
    • pv: The p-values (this will be None for methods that do not perform statistical testing).
    # If using AnnData
    dc.mt.ulm(adata, net)
    # Scores are NOT returned; they are inside adata
    scores = dc.pp.get_obsm(adata, "score_ulm")
    
    # If using DataFrame
    es, pv = dc.mt.ulm(df, net)
  8. Best practices and limitations for multi-method analysis

    main

    When using dc.mt.decouple and dc.mt.consensus, keep the following in mind:

    Best Practices

    • Robustness: Using consensus across multiple methods is a recommended way to infer TF activity when you want to avoid the bias of a single method.
    • Reproducibility: For stochastic methods like gsea or waggr, ensure you set a seed via the args parameter: args={"gsea": {"seed": 0}}.
    • Consistency: While consensus standardizes scores (making it safe to mix methods with different ranges), you must use the same network (net) across all methods so the sources remain aligned.

    Limitations (Footguns)

    • Minimum Methods: Consensus requires at least two method scores to be present. Running consensus on a single method is mathematically meaningless.
  9. How decoupler methods are called and what inputs they accept

    main

    All enrichment/activity methods in decoupler follow a unified calling convention: dc.mt.<method>(data, net, ...).

    Accepted Input Types for data:

    • AnnData object
    • pandas.DataFrame
    • A list containing [matrix, obs, var]

    Required Input Formats:

    • Data Matrix: Must be normalized (e.g., log1p transformed), not raw counts. Use the layer argument to specify which layer to use if using AnnData.
    • Prior-knowledge Network (net): Must be a long-format pandas.DataFrame containing the columns source (the biological program/TF), target (the feature/gene), and an optional signed weight.

    Shared Arguments:

    • tmin (default: 5): The minimum number of targets a source must have in the data to be included. Sources with fewer than tmin targets are silently dropped. For small or toy networks, you may need to set tmin=3.
    • raw: Boolean indicating if the input is raw (use with caution as normalization is expected).
    • layer: String specifying the layer in AnnData to use.
    • empty: Behavior for handling empty intersections.
    • bsize: Batch size for processing.
    import decoupler as dc
    # Example of the unified call pattern
    dc.mt.ulm(data=adata, net=net, tmin=3, layer='log1p_counts')
  10. Install decoupler

    main

    You can install decoupler using pip or conda. Ensure you have Python 3.11 or newer installed.

    PyPI Installation

    • Minimal dependencies: pip install decoupler
    • Full release (with extra dependencies): pip install decoupler[full]
    • Development version: pip install git+https://github.com/scverse/decoupler.git@main

    Conda Installation

    Use mamba or conda to install the version from conda-forge. Note the -py suffix:

    mamba create -n=dcp conda-forge::decoupler-py
    pip install decoupler
    # or for full dependencies
    pip install decoupler[full]
    # or via conda
    mamba create -n=dcp conda-forge::decoupler-py
  11. Rank data using decoupler.tl tools

    main

    The decoupler.tl module provides utility functions for ranking data within an AnnData object. These tools allow you to rank enrichment scores or other metrics based on specific groupings, observational metadata, or a predefined order.

    Available ranking functions:

    • tl.rankby_group: Ranks data based on group membership.
    • tl.rankby_obsm: Ranks data based on values stored in the .obsm attribute of the AnnData object.
    • tl.rankby_order: Ranks data based on a specific provided order.
  12. Visualize enrichment scores with decoupler.pl

    main

    The decoupler.pl module provides a suite of plotting functions for visualizing enrichment scores and their relationships within single-cell datasets. Available visualization methods include:

    • Bar plots: pl.barplot for displaying scores.
    • Dot plots: pl.dotplot for visualizing scores across different groups or samples.
    • Network plots: pl.network to visualize connections between targets and sources.
    • Volcano plots: pl.volcano for visualizing statistical significance and effect size.
    • Obs/Obsm plots: pl.obsbar and pl.obsm for visualizing scores relative to sample metadata or low-dimensional embeddings.
    • Leading edge analysis: pl.leading_edge to identify contributing genes.

    Other utility functions in this module include filtering tools (pl.filter_by_expr, pl.filter_by_prop, pl.filter_samples) and ordering functions (pl.order, pl.order_targets, pl.source_targets).