DOCSEARCHbyAlgolia
Submit Docs

imodels

repository·master·Indexed 23 days ago

https://github.com/csinva/imodels

A scikit-learn compatible Python package providing concise and transparent interpretable machine learning models. It implements various model forms including rule sets (e.g., RuleFit, Skope-rules), rule lists (e.g., Bayesian, OneR), rule trees (e.g., CART, C4.5, TAO), and algebraic models (e.g., SLIM). The library also includes specialized tools like FIGS for greedy-tree sums, Hierarchical Shrinkage for tree regularization, and MDI+ for flexible feature importance.

Tokens
9K
Snippets
28
Records
46
Agent score
77%

What's inside imodels

  1. Overview of imodels for interpretable modeling

    master

    imodels is a Python package designed for fitting concise, transparent, and accurate predictive models. It provides a unified interface for state-of-the-art interpretable modeling techniques that are compatible with the scikit-learn API.

    Key benefits include:

    • Interpretability: Models are designed to be inherently interpretable, making them suitable for high-stakes domains like medicine, biology, and political science.
    • Efficiency: Interpretable models are often more computationally efficient than large black-box models.
    • Scikit-learn Compatibility: You can use these models within existing scikit-learn workflows (e.g., using .fit() and .predict()).

  2. Compare model variations by generation, selection, and postprocessing

    master

    Different algorithms within imodels are distinguished by three main modeling choices. When selecting a model, consider how it handles these stages:

    1. Rule candidate generation: How the initial set of potential rules is created.
    2. Rule selection: The method used to choose the best rules (e.g., global optimization vs. sequential splitting).
    3. Rule postprocessing: How rules are refined or pruned after selection (e.g., using linear models vs. heuristic deduplication).

    Key Algorithm Comparisons

    • RuleFit vs. SkopeRules: Differ in pruning. RuleFit uses a linear model, while SkopeRules uses heuristic deduplication for overlapping rules.
    • Bayesian rule lists vs. Greedy rule lists: Differ in selection. Bayesian rule lists perform global optimization, whereas Greedy rule lists pick splits sequentially to maximize a criterion.
    • FPSkope vs. SkopeRules: Differ in candidate generation. FPSkope uses FPgrowth, while SkopeRules extracts rules from decision trees.

  3. FIGS: Fast Interpretable Greedy-Tree Sums

    master
    FIGS is an algorithm for fitting concise, rule-based models by generalizing CART. It grows a flexible number of trees simultaneously in a summation. To maintain interpretability, the total number of splits across all trees can be restricted by a pre-specified threshold. FIGS is designed to achieve high predictive performance even when restricted to a very small number of splits (e.g., less than 20).

  4. MDI+: Flexible Tree-Based Feature Importance

    master

    MDI+ is a feature importance framework that generalizes the Mean Decrease in Impurity (MDI) score used in random forests. It leverages the connection between linear regression and decision trees to allow practitioners to:

    1. Tailor feature importance computation to specific data or problem structures.
    2. Incorporate additional features or domain knowledge to mitigate common decision tree biases.

    MDI+ is designed to outperform traditional measures like standard MDI, permutation-based scores, and TreeSHAP.

  5. Hierarchical Shrinkage (HS) for tree-based methods

    master
    Hierarchical shrinkage is a fast, post-hoc regularization method applicable to any decision tree or tree-based ensemble (such as Random Forest). It does not change the underlying tree structure. Instead, it regularizes the model by shrinking the prediction at each node towards the sample means of its ancestors using a single regularization parameter. This method is intended to increase the predictive performance of individual trees and ensembles.

  6. Supported interpretable model forms in imodels

    master

    imodels implements several distinct forms of interpretable models, each constraining the model structure differently to ensure transparency:

    • Rule sets: A collection of rules that act independently (e.g., Skope-rules, Rulefit).
    • Rule lists: A sequence of rules that act in order (e.g., Bayesian rule lists, oneR algorithm).
    • Rule trees: Similar to rule lists but allow for branching after rules are applied (e.g., CART decision trees).
    • Algebraic models: Models that take the form of simple algebraic expressions (e.g., supersparse linear integer models).

  7. Understand the different model forms in imodels

    master

    The models in imodels result in one of four primary structural forms. Choosing a model depends on whether you require a collection of rules, a sequential list, a hierarchical tree, or an algebraic representation:

    • Rule set: A collection of rules.
    • Rule list: A sequential list of rules.
    • Rule tree: A hierarchical tree structure of rules.
    • Algebraic models: Models expressed through algebraic equations.

  8. Quickstart: Fit and use an imodels model

    master

    The imodels package follows the scikit-learn API. You can use fit, predict, and predict_proba methods. When fitting, you can pass feature_names to ensure the model's internal structure (like decision trees) is interpretable with the correct feature labels.

    Example using HSTreeClassifierCV:

    from imodels import get_clean_dataset, HSTreeClassifierCV # import any imodels model here
from sklearn.model_selection import train_test_split

# prepare data (a sample clinical dataset)
X, y, feature_names = get_clean_dataset('csi_pecarn_pred')
X_train, X_test, y_train, y_test = train_test_split(
    X, y, random_state=42)

# fit the model
model = HSTreeClassifierCV(max_leaf_nodes=4)  # initialize a tree model and specify only 4 leaf nodes
model.fit(X_train, y_train, feature_names=feature_names)   # fit model
preds = model.predict(X_test) # discrete predictions: shape is (n_test, 1)
preds_proba = model.predict_proba(X_test) # predicted probabilities: shape is (n_test, n_classes)
print(model) # print the model

  12. Use the fit and predict API in imodels

    master

    All models in imodels follow a consistent API. Classifiers and regressors support the fit and predict methods. Classifiers also support predict_proba for probability estimates.

    To enable feature names in model visualizations, you can either:

    1. Pass the feature_names argument to the fit function.
    2. Pass a pandas DataFrame with feature names as column names to the fit function.