DOCSEARCHbyAlgolia
Submit Docs

pyhealth

repository·master·Indexed 23 days ago

https://github.com/sunlabuiuc/pyhealth

A Python library for healthcare AI and a deep learning toolkit for clinical predictive modeling. PyHealth version 2.0.1 provides modular pipelines for healthcare-specific data such as MIMIC and OMOP-CDM, model calibration methods (Temperature Scaling, Histogram Binning, Dirichlet Calibration, KCal), and conformal prediction set constructors (BaseConformal, LABEL, SCRIB, FavMac, CovariateLabel). It also includes tools for EEGBCI pattern discovery and a RAG-based chat assistant interface.

Tokens
101.7K
Snippets
203
Records
548
Agent score
81%

What's inside pyhealth

  1. Overview of PyHealth 2.0

    master

    PyHealth is a comprehensive Python library designed for healthcare AI, enabling the development, testing, and deployment of machine learning models for clinical data. It provides a unified API and consistent workflow across multiple healthcare data modalities, including EHR (Electronic Health Records), images, biosignals, text, and genomics.

    Key capabilities include:

    • Unified API: A single interface for diverse data types.
    • Scalability: Optimized to run on everything from consumer-grade laptops (e.g., 16GB RAM) to high-performance clusters.
    • Healthcare-Specific Design: Built-in support for medical coding standards (ICD-9/10, CPT, NDC, etc.) and clinical safety tools.
    • End-to-End Workflow: Covers data processing, model training, evaluation (interpretability and uncertainty quantification), and deployment.

  2. Overview of PyHealth features

    master

    PyHealth is a Python library designed for building, testing, and deploying healthcare machine learning models. It is optimized for both ML researchers and medical practitioners.

    Key Capabilities:

    • Multimodal Support: Unified API for EHR, medical images, biosignals, clinical text, and genomics.
    • High Performance: Optimized for task processing (up to 39× faster than pandas) and memory efficiency (runs on 16GB laptops).
    • Medical Standards: Built-in support for medical coding standards including ICD, CPT, NDC, and ATC.
    • Ready-to-use Components: Includes 25+ pre-built models, 20+ tasks, and 12+ clinical datasets (e.g., MIMIC, eICU, OMOP).

  3. Overview of PyHealth metrics

    master

    PyHealth provides a comprehensive suite of evaluation metrics designed for healthcare machine learning tasks. The API is designed to be consistent with sklearn.metrics, using similar argument styles.

    Supported metric categories include:

    • Classification: Binary, multiclass, and multilabel classification metrics.
    • Uncertainty & Calibration: Metrics for model calibration and uncertainty quantification.
    • Prediction Sets: Metrics to evaluate the quality of prediction sets.
    • Healthcare Specific: Specialized metrics such as Drug-Drug Interaction (DDI) rate.
    • Generative Models: Privacy, utility, and statistical fidelity metrics for synthetic Electronic Health Record (EHR) data.
    • Fairness & Interpretability: Metrics to assess model bias and explainability.

  4. Overview of PyHealth

    master

    PyHealth is an open-source framework designed for developing, evaluating, and deploying machine learning models for healthcare applications. It provides tools to handle various healthcare data types, including:

    • Electronic Health Records (EHRs)
    • Time-series signals
    • Imaging data

    The framework includes robust tools for predictive modeling, model interpretability, and managing complex healthcare datasets.

  5. Understand the PyHealth ML Pipeline

    master

    PyHealth follows a structured pipeline to transform raw medical data into trained models. The workflow consists of six main stages:

    1. Raw Data → BaseDataset: Load CSV/Parquet files using a BaseDataset subclass and a config.yaml schema. This stage creates a global_event_df.parquet cache.
    2. Patient and Event Objects: Access structured data via Patient objects.
    3. Task Definition → set_task: Define a BaseTask (input/output schemas and feature extraction) and call dataset.set_task() to generate a SampleDataset.
    4. Processors → SampleDataset: During set_task, processors fit to the data and transform features into tensors stored in LitData streaming files.
    5. Model Initialization: Initialize a BaseModel subclass (e.g., RNN, Transformer) using the SampleDataset.
    6. Training and Evaluation: Use the Trainer to train the model using DataLoaders generated from the SampleDataset.

    Pipeline Visualization:

    Raw CSV / Parquet files
        │
        ▼
config.yaml
        │
        ▼
BaseDataset subclass  ──── loads tables, caches as global_event_df.parquet
        │   .unique_patient_ids → List[str]
        │   .get_patient(id)    → Patient
        │   .iter_patients()   → Iterator[Patient]
        │   .stats()            → prints patient/event counts
        │
        ▼
BaseTask subclass  (__call__(patient) → List[Dict])
        │   .input_schema  = {"feature": "processor_name", ...}
        │   .output_schema = {"label": "binary" | "multiclass" | ...}
        │
        ▼
dataset.set_task(task, num_workers=N)
        │
        ▼
SampleDataset  ──── len(ds), ds[i], patient_to_index, record_to_index
        │   Backed by LitData streaming files
        │   Processors fitted during set_task, applied at load time
        │
        ▼
get_dataloader(dataset, batch_size=32, shuffle=True)
        │
        ▼
Model(dataset, ...)  ──── BaseModel subclass (RNN, Transformer, MLP, …)
        │   EmbeddingModel routes features via processor.is_token()
        │   forward(**batch) → {"loss", "y_prob", "y_true", "logit"}
        │
        ▼
Trainer(model, metrics=[...], device=...)
        │   .train(train_dl, val_dl, test_dl, epochs=20, ...)
        │   .evaluate(test_dl) → Dict[metric_name, value]
        │
        ├──▶ Calibration  (pyhealth.calib)
        │       TemperatureScaling / HistogramBinning / KCal / …
        │       LABEL / SCRIB / FavMac / …  (conformal prediction sets)
        │
        └──▶ Interpretability  (pyhealth.interpret)
                GradientSaliency / IntegratedGradients / DeepLift / SHAP / LIME / …

  6. Use the Support2Dataset for clinical outcome prediction

    master

    The pyhealth.datasets.Support2Dataset provides access to the SUPPORT2 (Study to Understand Prognoses and Preferences for Outcomes and Risks of Treatments) dataset. This dataset contains information on seriously ill hospitalized adults, including:

    • Patient demographics
    • Diagnoses
    • Clinical measurements
    • Outcomes (e.g., survival and hospital mortality)

    It is primarily used for tasks such as mortality prediction and length of stay prediction.

  7. Use prediction set constructors for uncertainty quantification

    master

    The pyhealth.calib.predictionset module provides various prediction set constructors designed to produce set-valued predictions with statistical coverage guarantees. These methods utilize conformal prediction and related techniques to quantify uncertainty in model outputs.

    Available prediction set methods include:

    • LABEL (Least Ambiguous Set-valued Classifier)
    • SCRIB (Set-classifier with Class-specific Risk Bounds)
    • FavMac (Fast Value-Maximizing Prediction Sets)
    • CovariateLabel (Covariate Shift Adaptive)
    • ClusterLabel (K-means Cluster-based Conformal)
    • NeighborhoodLabel (Neighborhood Conformal Prediction)

  8. Use EEG classification tasks for Temple University EEG Corpus

    master

    The pyhealth.tasks.temple_university_EEG_tasks module provides specialized task classes for processing and analyzing the Temple University EEG Corpus. It supports two primary task types:

    1. EEGEventsTUEV: An EEG event classification task designed for the TUEV dataset.
    2. EEGAbnormalTUAB: A binary classification task for the TUAB dataset, used to distinguish between abnormal and normal EEG signals.

    Both tasks accept the following configuration parameters to control signal processing:

    • resample_rate (int): The rate at which the signal is resampled. Default is 200.
    • bandpass_filter (tuple): The frequency range for the bandpass filter. Default is (0.1, 75.0).
    • notch_filter (float): The frequency for the notch filter. Default is 50.0.

  9. Split datasets into training, validation, and test sets using pyhealth.datasets.splitter

    master
    The pyhealth.datasets.splitter module provides several data splitting functions designed for use with pyhealth.datasets objects. These functions allow you to partition your medical datasets into distinct subsets—typically training, validation, and test sets—to facilitate machine learning model development and evaluation.

  10. Use model calibration methods in pyhealth.calib.calibration

    master

    The pyhealth.calib.calibration module provides several methods for adjusting predicted probabilities to better reflect true confidence levels. This is used for uncertainty quantification in healthcare AI.

    Available calibration classes include:

    • TemperatureScaling: Adjusts confidence via a single scaling parameter.
    • HistogramBinning: Groups predictions into bins to estimate calibration.
    • DirichletCalibration: Specifically designed for multi-class problems using a Dirichlet distribution.
    • KCal: Kernel-Based Calibration.

  11. Overview of PyHealth Research Areas

    master

    Research within the PyHealth Initiative typically falls into one of three categories:

    Healthcare Data Modalities

    • Electronic health records (EHRs) and clinical notes
    • Medical imaging (X-rays, MRI, CT scans)
    • Physiological signals (EEG, ECG, biosensors)
    • Genomic and molecular data
    • Multi-modal healthcare data integration

    Model Development

    • Novel deep learning architectures for healthcare
    • Foundation models and transfer learning
    • Interpretable and explainable AI methods
    • Uncertainty quantification and calibration
    • Survival analysis and time-to-event modeling

    Real-World Applications

    • Clinical decision support systems
    • Drug discovery and repurposing
    • Patient risk stratification
    • Healthcare resource optimization
    • Personalized treatment recommendations

  12. How Attention Rollout works in PyHealth

    master

    Attention Rollout is a gradient-free, class-agnostic interpretability method for Transformer models. It quantifies how information propagates across layers by composing per-layer attention matrices using a residual-connection correction: Â = 0.5 * (A + I). The final rollout is the product of these corrected matrices across all layers (rollout = Â_L @ ... @ Â_1).

    Unlike gradient-weighted methods (like CheferRelevance), Attention Rollout is forward-pass only and does not depend on a specific target class. It provides a distribution of relevance scores over input tokens (e.g., diagnosis codes, medications), where the scores for a given token sum to 1.

    Key characteristics:

    • Class-agnostic: It explains how information flows through the mechanism, regardless of the predicted class. The target_class_idx parameter is accepted for API compatibility but is ignored.
    • Model-agnostic via duck-typing: It works with any model that implements the following methods: set_attention_hooks, get_attention_layers, and get_relevance_tensor. Currently, this includes pyhealth.models.Transformer and pyhealth.models.StageAttentionNet.