DOCSEARCHbyAlgolia
Submit Docs

pomegranate

repository·master·Indexed 25 days ago

https://github.com/jmschrei/pomegranate

A modular probabilistic modeling library. Version 1.0.0+ is built on PyTorch, supporting GPU acceleration, mixed precision, and deep learning integration for models such as Hidden Markov Models (DenseHMM, SparseHMM), Bayesian Networks, and Mixture Models. It treats all models as probability distributions, allowing for modular composition and complex nesting. Features include support for missing values via torch.masked.MaskedTensor and serialization via PyTorch.

Tokens
12.8K
Snippets
27
Records
91
Agent score
86%

What's inside pomegranate

  1. Overview of pomegranate features

    master

    pomegranate is a modular library for probabilistic modeling. It treats all models as probability distributions, allowing for high flexibility:

    • Modular Composition: You can drop any distribution (e.g., Normal, Gamma, Poisson) into a mixture model.
    • Complex Nesting: Bayesian networks can be used within mixture models, and Hidden Markov Models can be used within Bayes classifiers to create classifiers over sequences.
    • Advanced Modeling: Supports FactorGraph as a first-class citizen with full prediction and training methods.

  2. Overview of pomegranate probabilistic models

    master

    pomegranate is a Python package for fast and flexible probabilistic models. It treats all models as probability distributions that can yield probability estimates for samples and be updated given samples and weights.

    Key capabilities include:

    • Compositional Models: Build complex models by stacking components. For example, you can create Gaussian mixture models, or a Bayes classifier that uses different distributions (e.g., Exponential for time-associated features and Poisson for counts) for different features.
    • Advanced Model Architectures: Supports Mixture Models, Bayesian Networks, Hidden Markov Models (HMMs), and mixtures of Bayesian networks.
    • Built-in Training Strategies: Supports semi-supervised learning, learning with missing values, and mini-batch learning.
    • Scalability Features: Supports massive datasets via out-of-core learning, multi-threaded parallelism, and GPU support.

  4. Optimize pomegranate methods with torch.compile

    master

    To reduce I/O bottlenecks, you can use torch.compile on individual methods of your pomegranate objects. It is recommended to compile specific methods (like log_probability) rather than the entire model.

    Note: You may need to pass check_data=False when initializing models to avoid compatibility issues with torch.compile.

    >>> mu = torch.exp(torch.randn(100))
>>> d = Exponential(mu).cuda()

>>> X = torch.exp(torch.randn(1000, 100))

# Compile the `log_probability` method!
>>> d.log_probability = torch.compile(d.log_probability, mode='reduce-overhead', fullgraph=True)
>>> d.log_probability(X)

  5. Use GPU support with pomegranate

    master

    All distributions and models in pomegranate are torch.nn.Module objects and support GPU execution. To use the GPU, you must move both the model and the data to the GPU using .cuda().

    When a model is moved to the GPU, all associated distributions are also moved automatically.

    >>> X = torch.exp(torch.randn(50, 4))

# Will execute on the CPU
>>> d = Exponential().fit(X)
>>> d.scales
Parameter containing: tensor([1.8627, 1.3132, 1.7187, 1.4957])

# Will execute on a GPU
>>> d = Exponential().cuda().fit(X.cuda())
>>> d.scales
Parameter containing: tensor([1.8627, 1.3132, 1.7187, 1.4957], device='cuda:0')

# Moving a model moves its distributions
>>> X = torch.exp(torch.randn(50, 4)).cuda()
>>> model = GeneralMixtureModel([Exponential(), Exponential()]).cuda()
>>> model.fit(X)
>>> model.distributions[1].scales
Parameter containing: tensor([1.9902, 2.3871, 0.8984, 1.2215], device='cuda:0')

  6. Migrate from pomegranate v0.x to v1.0.0+

    master

    pomegranate v1.0.0 is a ground-up rewrite using PyTorch as the computational backend. This version is not backwards compatible with the Cython-based v0.x series.

    Key API and Architectural Changes:

    • PyTorch Integration: All models are now instances of torch.nn.Module.
    • Naming Convention: The suffix Distribution has been removed from model names. For example, NormalDistribution is now Normal.
    • Multivariate by Default: All distributions are now multivariate by default and treat each feature independently (except for Normal).
    • Removed Models: NaiveBayes has been removed; use BayesClassifier instead.
    • HMM Splitting: Hidden Markov Models are now split into DenseHMM (faster for dense graphs) and SparseHMM (for sparse transition matrices).
    • New Support: Models now natively support GPU execution, mixed precision (half precision), and missing values via torch.masked.MaskedTensor objects.
    • Serialization: Handled by PyTorch for more efficient I/O.

  7. Perform inference with pre-defined parameters

    master
    If you already know the model parameters and do not wish to train the model, you can initialize the model with those parameters. Once initialized, you can perform inference using methods such as log_probability and predict.

  9. Use mixed precision with pomegranate

    master

    pomegranate models can operate in mixed or low-precision regimes using PyTorch's autocast. Note that because pomegranate uses complex operations, performance gains or compatibility may vary compared to standard neural networks.

    >>> X = torch.randn(100, 4)
>>> d = Normal(covariance_type='diag')
>>>
>>> with torch.autocast('cuda', dtype=torch.bfloat16):
>>>     d.fit(X)

  10. Serialize and deserialize models using PyTorch or JSON

    master

    Depending on your version and requirements, you can handle model I/O in two ways:

    1. PyTorch Serialization (v1.0.0+): Uses PyTorch for more compact and efficient I/O.
    2. JSON (v0.12.0+): A robust from_json method is available to deserialize JSON representations from any pomegranate model.

  11. Train a model from data using the fit method

    master

    pomegranate v1.0.0 follows the scikit-learn API. You pass hyperparameters into the class initialization and then call the fit method to learn the parameters from your data.

    Note that some models require specific parameters during initialization (e.g., mixture models require specifying the distributions, and Markov chains require specifying the order).