DOCSEARCHbyAlgolia
Submit Docs

Turing.jl Documentation

repository·main·Indexed 24 days ago

https://github.com/turinglang/turing.jl

A general-purpose probabilistic programming language for Bayesian inference in Julia. Turing.jl provides a unified interface for defining complex probabilistic models using the @model macro and performing inference via MCMC sampling (including NUTS, HMC, and Gibbs) and Variational Inference. It integrates with the Julia ecosystem through DynamicPPL.jl, AbstractMCMC.jl, AdvancedVI.jl, and Distributions.jl, and supports multiple automatic differentiation backends such as ForwardDiff.jl and Mooncake.jl.

Tokens
2.3K
Snippets
3
Records
15
Agent score
81%

What's inside Turing.jl

  1. Use Variational Inference with `Turing.Variational`

    main
    The Turing.Variational module provides tools for performing Variational Inference (VI) within Turing.jl. This module includes specific types and functions designed to approximate posterior distributions by optimizing a variational distribution (typically by maximizing the Evidence Lower Bound, or ELBO).

  2. Understand the TuringLang ecosystem and inference capabilities

    main

    Turing.jl serves as a unified interface for probabilistic programming in Julia. It integrates several key components:

    • Model Specification: Uses DynamicPPL.jl to define probabilistic models.
    • MCMC Sampling: Provides an abstract interface via AbstractMCMC.jl and supports various samplers like AdvancedMH.jl and AdvancedHMC.jl.
    • Variational Inference: Supported via AdvancedVI.jl.
    • Optimization: Maximum likelihood and maximum a posteriori (MAP) estimation are handled through the Optimization.jl interface from SciML.
    • Automatic Differentiation: Preferred backends are ForwardDiff.jl and Mooncake.jl, though others are available via DifferentiationInterface.jl.

  3. Use Distributions.jl APIs via Turing.jl

    main
    Turing.jl re-exports the entire public API of Distributions.jl. You can use any distribution type or function provided by Distributions.jl directly through the Turing module without needing to explicitly import Distributions.

  4. Define and sample probabilistic models in Turing.jl

    main

    Turing.jl allows you to define probabilistic models using the @model macro. Within the macro, you specify Priors using the ~ operator and define the Likelihood of the observed data. Once a model is defined, you can perform Markov chain Monte Carlo (MCMC) sampling using the sample function, passing in a sampler (e.g., NUTS()).

    using Turing

@model function linear_regression(x)
           # Priors
           α ~ Normal(0, 1)
           β ~ Normal(0, 1)
           σ² ~ truncated(Cauchy(0, 3); lower=0)

           # Likelihood
           μ = α .+ β .* x
           y ~ MvNormal(μ, σ² * I)
       end

# Prepare data
x, y = rand(10), rand(10)

# Instantiate the model with observed data using the semicolon syntax
posterior = linear_regression(x) | (; y = y)

# Perform MCMC sampling
chain = sample(posterior, NUTS(), 1000)

  6. Use Turing.jl unqualified exports

    main

    Turing.jl re-exports many symbols from its underlying dependencies (like DynamicPPL, AbstractMCMC, and Inference) so you can use them directly after calling using Turing. You do not need to use fully qualified names for core functions like @model, sample, or Prior().

    using Turing

@model function my_model() end

sample(my_model(), Prior(), 100)

  7. Use `Turing.RandomMeasures` for random variable definitions

    main
    The Turing.RandomMeasures module provides the core abstractions and functions for defining random variables and probability measures in Turing.jl. It contains the fundamental types and functions used to specify priors and likelihoods within probabilistic models.

  8. Reference: Modelling symbols

    main

    The following symbols are exported by Turing for defining and manipulating probabilistic models:

    SymbolDescription
    @modelDefine a probabilistic model
    @varnameGenerate a VarName from a Julia expression
    to_submodelDefine a submodel
    prefixPrefix all variable names in a model with a given VarName
    LogDensityFunctionA struct containing all information about how to evaluate a model (advanced users)
    @addlogprob!Add arbitrary log-probability terms during model evaluation
    setthreadsafeMark a model as requiring threadsafe evaluation
    might_produceMark a method signature as potentially calling Libtask.produce
    @might_produceMark a function name as potentially calling Libtask.produce
    set_logprob_type!Set the base log-probability type used during evaluation of Turing models

  9. Reference: Inference symbols

    main

    The following symbols are used to perform inference on models:

    SymbolDescription
    sampleSample from a model
    MCMCThreadsRun MCMC using multiple threads
    MCMCDistributedRun MCMC using multiple processes
    MCMCSerialRun MCMC without parallelism
    loadstateLoad saved state from an MCMC chain
    VNChainAlias for FlexiChain{VarName}

  10. Reference: Point estimates

    main

    Tools for finding point estimates (MAP, MLE, etc.):

    SymbolDescription
    maximum_a_posterioriFind a MAP estimate for a model
    maximum_likelihoodFind a MLE estimate for a model
    MAPType to use with Optim.jl for MAP estimation
    MLEType to use with Optim.jl for MLE estimation
    vector_names_and_paramsExtract parameter names and values as vectors

  11. Reference: DynamicPPL utilities

    main

    Utilities for interacting with the underlying DynamicPPL probabilistic programming interface:

    SymbolDescription
    returnedCalculate additional quantities defined in a model
    predictGenerate samples from posterior predictive distribution
    pointwise_logdensitiesCompute log densities (both prior and likelihood) for each sample in a chain
    pointwise_loglikelihoodsCompute log likelihoods for each sample in a chain
    pointwise_prior_logdensitiesCompute log priors for each sample in a chain
    logpriorCompute log prior probability
    logjointCompute log joint probability
    conditionCondition a model on data
    deconditionRemove conditioning on data
    conditionedReturn the conditioned values of a model
    fixFix the value of a variable
    unfixUnfix the value of a variable

  12. Reference: Variational inference

    main

    Symbols for performing Variational Inference (VI). For detailed usage, see the AdvancedVI.jl documentation.

    SymbolDescription
    viPerform variational inference
    q_locationscaleFind a numerically non-degenerate initialization for a location-scale variational family
    q_meanfield_gaussianFind a numerically non-degenerate initialization for a mean-field Gaussian family
    q_fullrank_gaussianFind a numerically non-degenerate initialization for a full-rank Gaussian family
    KLMinRepGradDescentKL divergence minimization via stochastic gradient descent with the reparameterization gradient
    KLMinRepGradProxDescentKL divergence minimization via stochastic proximal gradient descent with the reparameterization gradient over location-scale variational families
    KLMinScoreGradDescentKL divergence minimization via stochastic gradient descent with the score gradient
    KLMinWassFwdBwdKL divergence minimization via Wasserstein proximal gradient descent
    KLMinNaturalGradDescentKL divergence minimization via natural gradient descent
    KLMinSqrtNaturalGradDescentKL divergence minimization via natural gradient descent in the square-root parameterization
    FisherMinBatchMatchCovariance-weighted Fisher divergence minimization via the batch-and-match algorithm