DOCSEARCHbyAlgolia
Submit Docs

tangermeme

repository·main·Indexed 43 days ago

https://github.com/jmschrei/tangermeme

A library for studying sequence-to-function (S2F) machine learning models in genomics. It provides tools for sequence manipulation, batch predictions, feature attribution using DeepLIFT and SHAP, marginalization, ablation, saturation mutagenesis, and sequence design via greedy substitution.

Tokens
58.8K
Snippets
166
Records
230
Agent score
50%

What's inside tangermeme

  1. Overview of tangermeme capabilities

    main

    tangermeme is a Python package designed for sophisticated genomic analyses using machine learning models. It provides the infrastructure for genomic workflows while allowing you to provide your own predictive models.

    Core capabilities include:

    • I/O and Data Loading: Handling genomic sequences and model inputs.
    • Sequence Manipulation: Altering sequences (e.g., inserting or scrambling motifs).
    • Region Identification: Identifying matched region sets and annotating regions.
    • Experimental Workflows: Running marginalization experiments, ablation studies, and saturation mutagenesis.
    • Attribution-aware Analysis: Extending classic MEME-suite-style analyses to incorporate model attribution values (e.g., highlighting important nucleotides).

  2. Overview of core tangermeme modules

    main

    While many features have dedicated reference files, the following modules provide single-call operations:

    tangermeme.predict.predict

    Performs batched, memory-efficient inference.

    • Returns: The model's parameter dtype. You can override this with dtype=.
    • Behavior: Upcasts each batch from X. For example, int8 sequences are automatically upcast. Multi-output models return a list.
    • Contract: Satisfies the func= contract.

    tangermeme.ersatz

    Provides atomic sequence operations:

    • insert
    • substitute
    • multisubstitute
    • delete
    • shuffle
    • dinucleotide_shuffle (Note: most motif-add ops use substitute to preserve length; use start or end to confine shuffles to a specific region).

    tangermeme.pisa.pisa

    Per-position (PISA) attribution using DLS hooks.

    • Warning: Does not upcast X. You must pass float tensors. Passing int8 will fail.
    • Warning: Some paths return tensors on the input device (e.g., CUDA) rather than the CPU.

    tangermeme.utils

    Utility functions including:

    • one_hot_encode (returns int8)
    • characters
    • random_one_hot
    • reverse_complement (for single sequences)
    • pwm_consensus
    • set_seed
    • gc_content

    tangermeme.kmers

    K-mer operations including k-mer counts, batched k-mers, and gapped k-mers.

    Note: Motif scanning (FIMO/TOMTOM) is handled by the external memelite package, though annotate_seqlets wraps TOMTOM internally.

  3. Score variant effects with `tangermeme.variant_effect`

    main

    The tangermeme.variant_effect module scores how sequence edits (substitutions, deletions, or insertions) affect a model's output.

    All functions in this module follow these rules:

    • They apply a function func (defaults to predict) to the sequence before and after the edit.
    • They return a PerturbationResult(y_before, y_after) object.
    • Note: There is no built-in distance metric; you must compute your own (e.g., Euclidean distance or log-fold change) to compare y_before and y_after.
    • For multi-output models, y_before and y_after will be lists.
    • You can swap func to other attribution methods like deep_lift_shap to get attributions instead of raw predictions.
    • Alphabet indices correspond to positions in the alphabet (default: A=0, C=1, G=2, T=3).

  4. Conventions and Data Formats in tangermeme

    main

    When writing code for tangermeme, adhere to the following technical conventions:

    Tensor Layout

    • Shape: (batch, channels, length)
    • Channels: Represents the one-hot alphabet axis (default is ['A','C','G','T']).

    Naming Conventions

    • X: Observed sequence
    • y: Observed output
    • X_bar: Designed sequence
    • y_bar: Target output
    • y_hat: Predictions
    • X_attr: Attributions

    Device Handling

    Functions like predict, deep_lift_shap, pisa, design.*, saturation_mutagenesis, and product.* accept device=None.

    • If None, they default to CUDA if available, otherwise CPU.
    • The library automatically restores the model's original device and training mode after the operation.

    Return Types

    • Perturbation Results: Functions return NamedTuples (e.g., PerturbationResult). You can unpack them positionally or access fields by attribute. isinstance(result, tuple) will return True for these objects.

  5. Avoid the 2D projected track footgun when calling seqlets

    main

    When calling seqlet functions, the input must be per-position projected attribution sums with shape (n, length).

    Do not pass raw attributions with a channel axis (e.g., shape (n, 4, length)) or hypothetical attributions. Hypothetical attributions are intended for building CWMs, not for seqlet calling.

    To prepare the correct input, collapse the channel axis of your attribution tensor:

    # Correct way to prepare input for seqlet callers
attr_sums = X_attr.sum(dim=1)        # (n, 4, length) -> (n, length)

  6. How the `func=` plug-point enables composition in tangermeme

    main

    A core design pattern in tangermeme is the func= plug-point. Many perturbation functions—such as ablate, marginalize, space, variant_effect.*, and product.*—accept a function as their first positional argument following the pattern func(model, X, args=, **kwargs).

    By swapping the function passed to this argument, you can change the nature of the experiment. For example, passing predict creates a "predictions before/after" experiment, while passing deep_lift_shap creates an "attributions before/after" experiment.

    Warning: Be aware of the additional_func_kwargs collision trap when using this pattern. Refer to references/func-pattern.md for full details.

  7. Understand the motif annotation and discovery pipeline

    main

    The standard workflow for discovering and quantifying cis-regulatory features in a model follows this sequence:

    1. Attributions: Generate feature attributions for the model.
    2. Spans: Use recursive_seqlets (see references/seqlets.md) to identify spans of interest.
    3. Annotation: Use annotate_seqlets to assign motif identities to those spans.
    4. Quantification: Use count_annotations, pairwise_annotations, or pairwise_annotations_spacing to summarize the motif landscape.

    For different methodological approaches (e.g., TF-MoDISco vs. marginalize vs. seqlet+TOMTOM vs. FIMO), refer to the "Inspecting what cis-regulatory features a model has learned" vignette.

  8. Understand hypothetical vs projected attributions

    main

    The hypothetical parameter changes the nature of the returned attribution values:

    • hypothetical=False (Default): Returns projected attributions. These are the contributions of the observed bases in the actual sequence. This is what you typically use for plotting logos of a specific sequence.
    • hypothetical=True: Returns hypothetical contributions for all four bases at every position. This is useful for building downstream motif patterns or contribution-weight matrices (CWMs), but should not be used as input for seqlet callers.

  9. Verify attribution correctness using convergence deltas

    main

    DeepLIFT/SHAP attributions have an additive property: the sum of attributions should equal the difference between the prediction for the sequence and the prediction for its references. The convergence delta measures this violation.

    Interpreting the delta:

    • delta > ~0.01: This indicates a correctness bug. It means a non-linearity used in your model is not registered in the deep_lift_shap hooks. You must register the missing operation before trusting the attributions.
    • delta ~1e-3 to 1e-5: This is typically benign floating-point error (often higher on CUDA due to parallel reduction reordering). It is safe to ignore.
    • delta ~1e-16: Achievable by using fp64 (double precision) if you need to disambiguate precision noise from unregistered ops.

    Warning: High deltas do not necessarily produce garbage-looking logos. A model with an unregistered op can still produce motif-shaped patterns that look real but are mathematically incorrect. Always check the delta.

  10. Understand the `func=` contract for perturbation functions

    main

    In tangermeme, many analysis functions use a func= plug-point. This allows you to swap the underlying analysis (e.g., from predictions to attributions) while keeping the same outer orchestration logic.

    The Contract

    Any function passed to func= must follow this signature:

    func(model, X, args=None, **kwargs) -> torch.Tensor | list[torch.Tensor]

    Built-in compatible functions

    You can pass these three functions directly to func=:

    • tangermeme.predict.predict: Returns model predictions (the default).
    • tangermeme.deep_lift_shap.deep_lift_shap: Returns attributions.
    • tangermeme.saturation_mutagenesis.saturation_mutagenesis: Returns ISM scores.

    How it works

    The outer function (like marginalize) returns the same type of object that func returns. If func returns predictions, the outer function returns predictions; if func returns attributions, the outer function returns attributions.

    from tangermeme.marginalize import marginalize
from tangermeme.deep_lift_shap import deep_lift_shap

# Effect of a motif on PREDICTIONS
y_before, y_after = marginalize(model, X, "CTCAGTGATG")

# Effect of the same motif on ATTRIBUTIONS
attr_before, attr_after = marginalize(model, X, "CTCAGTGATG", func=deep_lift_shap)

  11. What your wrapper does NOT need to handle

    main

    You do not need to implement device management, evaluation mode, or gradient disabling in your wrapper.

    Functions like predict, deep_lift_shap, pisa, saturation_mutagenesis, design.*, and product.* automatically:

    • Move the model to the correct device.
    • Switch the model to .eval() mode.
    • Run inference without gradients where appropriate.
    • Restore the model's original device and training mode after execution.

  12. When to use `ledidi` instead of `tangermeme.design`

    main

    tangermeme.design is a discrete and greedy approach suitable for motif implantation or single-nucleotide searches.

    Use the sibling library ledidi when you require:

    • Gradient-based optimization: For finding minimal edits to a template.
    • Precision: When you need to hit very precise output characteristics.
    • Continuous relaxation: ledidi optimizes a continuous relaxation of the sequence space, which often finds smaller, more targeted edits than greedy substitution.