shapash

Shapash is a Python library designed to make machine learning models interpretable and accessible. It provides both global and local explainability through various visualizations and a web application (shapash-monitor).

Key Capabilities:

• Explainability: Supports global and local visualizations using Shap and Lime backends. Compatible with Regression, Binary Classification, and Multiclass problems.
• Model Compatibility: Works with most sklearn, lightgbm, catboost, and xgboost models.
• Local Summarization: Summarizes local explanations into concise, transparent formats and exports them as Pandas DataFrames.
• Data Handling: Uses encoder objects and feature dictionaries for clear labeling; compatible with category_encoders and Sklearn ColumnTransformer.
• Operational Use: Suitable for both exploration and deployment via API or Batch mode.
• Auditability: Allows freezing aspects of a data science project to establish a foundation for audit reports.

Follow these four steps to initialize an explainer, compile your data, and view results.

1. Declare the SmartExplainer object

Initialize the explainer with your model. You can optionally provide a features_dict for labels, a preprocessing object (which should have an inverse_transform method), and a postprocessing function.

2. Compile the dataset

Use the .compile() method to link your data to the explainer. Mandatory parameter is x (the dataset).

3. Display output

Launch the interactive web application using .run_app().

4. Generate a Shapash Report

Create a standalone HTML report containing project metrics and data splits using .generate_report().

from shapash import SmartExplainer

# Step 1: Declare SmartExplainer Object
xpl = SmartExplainer(
  features_dict=house_dict,  # Optional
  model=regressor,           # Mandatory
  preprocessing=encoder,     # Optional
  postprocessing=postprocess # Optional
)

# Step 2: Compile Dataset
xpl.compile(
    x=Xtest,
    y_pred=y_pred,               # Optional
    y_target=yTest,              # Optional
    additional_data=X_additional, # Optional
    additional_features_dict=features_dict_additional, # Optional
)

# Step 3: Display output (Web App)
app = xpl.run_app()

# Step 4: Generate the Shapash Report
xpl.generate_report(
    output_file='path/to/output/report.html',
    project_info_file='path/to/project_info.yml',
    x_train=Xtrain,
    y_train=ytrain,
    y_test=ytest,
    title_story="House prices report",
    title_description="""This document is a data science report of the kaggle house prices tutorial project. It was generated using the Shapash library.
            """,
    metrics=[{'name': 'MSE', 'path': 'sklearn.metrics.mean_squared_error'}]
)

To perform model interpretability, follow these steps using the SmartExplainer class:

1. Initialize SmartExplainer: Pass your trained model. You can optionally provide a features_dict for custom labels, a preprocessing object (which should have an inverse_transform method), and a postprocessing function.
2. Compile the dataset: Use the .compile() method to provide the test data (x), predictions (y_pred), and target values (y_target). You can also include additional_data for the webapp and additional_features_dict for extra context.
3. Display results: Launch an interactive web application using .run_app().
4. Generate a report: Create a standalone HTML audit report using .generate_report().

from shapash import SmartExplainer

# Step 1: Declare SmartExplainer Object
xpl = SmartExplainer(
    model=regressor,
    features_dict=house_dict,  # Optional
    preprocessing=encoder,      # Optional
    postprocessing=postprocess,  # Optional
)

# Step 2: Compile Dataset
xpl.compile(
    x=xtest,
    y_pred=y_pred,              # Optional
    y_target=yTest,            # Optional
    additional_data=xadditional,  # Optional
    additional_features_dict=features_dict_additional,  # Optional
)

# Step 3: Display output (Webapp)
app = xpl.run_app()

# Step 4: Generate the Shapash Report
xpl.generate_report(
    output_file="path/to/output/report.html",
    project_info_file="path/to/project_info.yml",
    x_train=xtrain,
    y_train=ytrain,
    y_test=ytest,
    title_story="House prices report",
    title_description="""This document is a data science report...""",
    metrics=[{"name": "MSE", "path": "sklearn.metrics.mean_squared_error"}],
)

To display inline graphs in Jupyter, you must install ipywidgets.

Standard Installation

Use pip or conda:

pip install ipywidgets
# OR
conda install -c conda-forge ipywidgets

Jupyter Notebook (Classic)

If widgets do not appear automatically, you may need to manually enable the extension:

jupyter nbextension enable --py widgetsnbextension

If using a virtual environment, use the --sys-prefix flag to keep the environment isolated:

jupyter nbextension enable --py widgetsnbextension --sys-prefix

JupyterLab 3

JupyterLab 3 uses jupyterlab_widgets (version 1.0+) for widget support. If your JupyterLab server and IPython kernel are in different environments, follow these steps:

1. Install jupyterlab_widgets in the environment containing the JupyterLab server.
2. Install ipywidgets in each kernel's environment.

Example for conda:

conda install -n base -c conda-forge jupyterlab_widgets
conda install -n <your_kernel_env> -c conda-forge ipywidgets

To enable Plotly in JupyterLab, install both the plotly package and the labextension:

conda install -c plotly plotly
jupyter labextension install jupyterlab-plotly