DOCSEARCHbyAlgolia
Submit Docs

CausalPy Documentation

repository·main·Indexed 22 days ago

https://github.com/pymc-labs/causalpy

A Python library for causal inference in quasi-experiments, version 0.9.0. It provides tools for rigorous, uncertainty-aware analysis including Regression Discontinuity, Bayesian models with HDI, and OLS models with confidence intervals. The library features a diagnostic-first design with ROPE analysis, effect summaries, and a comprehensive suite of falsification tests such as Pre-Treatment Placebo, Placebo In Time, Outcome Falsification, and McCrary Density tests to validate causal claims and identify confounding threats.

Tokens
25K
Snippets
44
Records
120
Agent score
75%

What's inside CausalPy

  1. Understand User-Facing Agent Skills in CausalPy

    main

    CausalPy provides a set of Markdown-based 'skills' designed to teach AI coding agents how to perform causal inference tasks using the library. These skills are distributed via the Decision AI Hub.

    Note that these skills are distinct from developer-focused skills (which cover environment setup, PR workflows, and testing) and are intended for end-users performing causal analysis.

  2. Available causal inference methods in CausalPy

    main

    CausalPy supports a wide range of quasi-experimental methods. Depending on your data structure and research design, you can use the following approaches:

    Panel Data Methods

    • ANCOVA: Analysis of Covariance.
    • Panel Fixed Effects: Controlling for unobserved time-invariant heterogeneity.
    • Synthetic Control: Creating a counterfactual using a weighted combination of control units.
    • Difference in Differences (DiD): Comparing changes in outcomes over time between a treatment and control group (including staggered DiD).
    • Geographical Lift Testing: Testing causal effects across different geographic cells.

    Time Series Methods

    • Interrupted Time Series (ITS): Analyzing the effect of an intervention at a specific point in time.
    • Comparative Interrupted Time Series: Comparing ITS across different groups.
    • Piecewise Interrupted Time Series: Modeling changes in trends at multiple points.

    Cross-sectional and Other Methods

    • Regression Discontinuity (RD): Estimating effects near a specific threshold or cutoff.
    • Regression Kink Design: Estimating effects when the slope of a relationship changes at a threshold.
    • Instrumental Variables (IV) Regression: Using exogenous variation to estimate causal effects, including handling weak instruments.
    • Inverse Propensity Score Weighting (IPSW): Weighting observations to balance covariates between treatment and control groups.

  3. Use Difference-in-Differences (DiD) for treatment effect estimation

    main

    Difference-in-Differences estimates the treatment effect by calculating the difference between treatment and control conditions across the difference between pre-treatment and post-treatment observations. CausalPy provides implementations for:

    • Difference in Differences with scikit-learn models
    • Difference in Differences with pymc models
    • Staggered Difference-in-Differences

  4. Interpret Frequentist statistics for Scikit-learn models

    main

    When using Scikit-learn (OLS) models, CausalPy reports classical frequentist inference:

    Point Estimates

    • Mean / Coefficient Estimate: The estimated causal effect (e.g., the coefficient for DiD/RD or the average/cumulative impact for time-series).

    Uncertainty Quantification

    • Confidence Intervals (CI): Reported as ci_lower and ci_upper. These are computed using t-distribution critical values (default $\alpha = 0.05$). This is a statement about the procedure's long-run frequency, not a direct probability of the parameter.
    • Standard Errors: Measure the uncertainty in the coefficient estimate.

    Hypothesis Testing

    • p-values: Reported as p_value. The probability of observing data at least as extreme as the current data, assuming the null hypothesis is true. A common threshold for significance is $p < 0.05$.

  5. Analyze effect decay and persistence in ITS

    main

    When working with Interrupted Time Series (ITS) designs involving temporary interventions, you can analyze how effects change over time:

    • Effect Decay: The reduction in treatment effect magnitude after an intervention ends. Decay patterns can be exponential (rapid decline), linear (steady decline), or step (sudden drop to zero).
    • Effect Persistence: The extent to which treatment effects continue after the intervention period ends. This is often expressed as a percentage (e.g., "30% of the intervention effect persisted").

    To enable this analysis, define the treatment_time (start) and treatment_end_time (end) to distinguish the intervention period from the pre-intervention and post-intervention periods.

  6. Resolve ambiguity in CausalPy method selection

    main

    If the routing process does not yield a single clear method, use the following disambiguation strategies to resolve the choice:

    • ITS vs Piecewise ITS: Determine if you need a single level/slope change or segmented changes.
    • ITS vs Synthetic Control vs DiD: Compare interrupted time series, synthetic control, and difference-in-differences approaches.
    • DiD vs Staggered DiD vs Panel Regression: Distinguish between standard DiD, staggered adoption models, and general panel regression.
    • Synthetic Control vs SDiD: Decide between synthetic control and Synthetic Difference-in-Differences.
    • IPW vs IV vs Panel Regression: Choose between Inverse Probability Weighting, Instrumental Variables, or Panel Regression.
    • PrePostNEGD vs DiD: Compare Pre-Post Non-Equivalent Group Designs against Difference-in-Differences.

    If you are unsure, identify the top two plausible CausalPy classes, state the concrete distinction between them (e.g., forecast counterfactual vs segmented level/slope model), and answer the deciding question to resolve the route.

  8. Identify Interrupted Time Series and Comparative ITS designs

    main

    Interrupted Time Series (ITS)

    A powerful design involving multiple pretest and posttest measures for a single group. The treatment X is introduced at a specific point in time. Causal impact is estimated by comparing the trajectory of the outcome variable before and after the treatment.

    Notation Example: $O_1$ | $O_2$ | $O_3$ | $O_4$ | X | $O_5$ | $O_6$ | $O_7$ | $O_8$

    Comparative Interrupted Time Series (CITS)

    Combines ITS and NEGD. It includes both a treatment group and a control group, each with multiple pre- and post-treatment observations. This allows for comparing the trajectories of both groups.

    Analysis Approaches:

    • Difference-in-differences: Well-suited for CITS due to the multiple pre/post measures.
    • Synthetic Control: Recommended if there is one treated unit and many untreated units.

  10. How causal estimates are extracted in CausalPy

    main

    CausalPy classifies the extraction of causal estimates into several computational hierarchies. Understanding these helps you interpret whether your result is a direct parameter or a predicted contrast:

    • Direct parameter extraction: A model parameter is exactly the target estimand.
    • Counterfactual construction or prediction:
      • G-computation and standardization: Predicting conditional expected outcomes under interventions and contrasting them.
      • One-sided counterfactual prediction: Comparing observed treated outcomes with predicted untreated outcomes (e.g., Interrupted Time Series).
      • Local prediction contrasts: Comparing predictions at a boundary (e.g., Regression Discontinuity).
      • Synthetic counterfactual construction: Predicting an untreated trajectory from weighted control units (e.g., Synthetic Control).
    • Weighting: Using methods like Inverse Probability Weighting (IPW) to reweight outcomes.
    • Specialized or hybrid estimators: Combining methods, such as Instrumental Variables (IV) or Doubly Robust estimators.

  11. Identify and test for Confounding threats

    main

    Confounding occurs when a variable affects both the treatment assignment and the outcome, creating a 'backdoor path'. To identify confounding, ask if variables measured before treatment affect both sides of the equation and whether they are adjusted for in your formula, design, weights, or donor selection.

    Important Distinctions:

    • Mediators: Variables on the causal path. Controlling for these can bias the estimation of the total effect.
    • Colliders: Variables that can create bias when conditioned on.

  12. Use Donut Regression Discontinuity (Donut RDD) for robustness

    main

    Donut RDD is a robustness approach for Regression Discontinuity Designs (RDD) used to mitigate bias caused by manipulation, sorting, or heaping/rounding of the running variable near the treatment threshold.

    By excluding observations within a specified distance (the "donut hole") from the threshold, the analysis relies on observations less likely to be affected by these issues.