DOCSEARCHbyAlgolia
Submit Docs

pyhf

repository·main·Indexed 43 days ago

https://github.com/scikit-hep/pyhf

A pure-Python implementation of the HistFactory statistical model for multi-bin histogram-based analysis. It provides tools for fitting, limit-setting, and interval estimation, supporting computational backends like NumPy and JAX for vector-based calculations and autodiff.

Tokens
38.9K
Snippets
129
Records
153
Agent score
55%

What's inside pyhf

  1. What is pyhf?

    main

    pyhf is a pure-python implementation of the HistFactory p.d.f. template (CERN-OPEN-2012-016). It is designed for multi-bin histogram-based statistical analysis and allows users to perform statistical modeling outside of the ROOT, RooFit, and RooStats frameworks.

    Key features include:

    • Interval Estimation: Based on the asymptotic formulas from "Asymptotic formulae for likelihood-based tests of new physics" (arXiv:1007.1727).
    • Hardware Acceleration: Supports modern computational graph libraries like JAX, enabling automatic differentiation and GPU acceleration.

  2. Understand the HistFactory statistical model structure

    main

    The pyhf project implements the HistFactory (HiFa) statistical model, which is used for simultaneous measurements of disjoint binned distributions called channels.

    Core Components

    • Channels: Sets of binned event counts observed as data.
    • Samples: Physics processes that contribute to the expected event rate in a channel.
    • Parameters:
      • Free Parameters ($\theta_{free}$): Unconstrained parameters.
      • Constrained Parameters ($\theta_{constr}$): Parameters subject to constraint terms (auxiliary measurements).
      • Parameters of Interest (POI, $\theta_{POI}$): The specific subset of parameters being tested (e.g., via profile likelihood).
      • Nuisance Parameters ($\theta_{nuis}$): All other parameters used to model systematic uncertainties.

    Probability Model Structure

    The overall likelihood is a product of the channel measurements (Poisson terms) and the constraint terms (auxiliary measurements):

    $$f(\mathbf{x}|\theta) = \underbrace{\prod_{c\in\text{channels}} \prod_{b \in \text{bins}c}\text{Pois}(n{cb} | \nu_{cb}(\theta))}{\text{Simultaneous measurement}} \times \underbrace{\prod{\theta_i \in \theta_{constr}} c_{\theta_i}(a_{\theta_i} | \theta_i)}_{\text{Constraint terms}}$$

    Rate Calculation

    The expected rate $\nu_{cb}$ for a bin $b$ in channel $c$ is the sum over all samples $s$, modified by multiplicative ($\kappa$) and additive ($\Delta$) modifiers:

    $$\nu_{cb}(\theta) = \sum_{s\in\text{samples}} \left(\prod_{\kappa\in\bm{\kappa}} \kappa_{scb}(\theta)\right) \left(\nu_{scb}^0(\theta) + \sum_{\Delta\in\bm{\Delta}} \Delta_{scb}(\theta)\right)$$

  3. Specify Observations for Hypothesis Testing

    main

    Observations represent the data used to evaluate hypothesis testing (e.g., comparing signal+background vs. background-only). Observations are provided as a list of objects. Each object must contain:

    • name: The name of the channel for which the observations are recorded.
    • data: A list of bin-by-bin observed values for that channel.
    {
    "name": "channel1",
    "data": [110.0, 120.0]
}

  4. Understand the Workspace structure

    main

    A workspace is the top-level JSON document that defines a statistical model. It follows the HistFactory-style specification and consists of three main components:

    • channels: Definitions of analysis regions, including their samples and modifiers.
    • measurements: Definitions of the statistical objectives, including parameters of interest.
    • observations: The observed data used to construct the likelihood.

    A workspace contains one set of observed data but can include multiple measurements.

  5. Understand pyhf's declarative JSON format

    main

    pyhf uses a declarative JSON format to fully encode the statistical model of an analysis. This format is implementation-independent and specifies:

    • Channels: The different experimental channels being analyzed.
    • Samples: All associated signal and background samples.
    • Rate Modifiers: Parameterized modifiers for the samples.
    • Constraint Terms: Implied constraints on parameters.
    • Measurements: The data measurements used in the model.

    Unlike the original XML-based HistFactory specification which relied on ROOT for encoding nominal rates and input data, the pyhf JSON format (validated by JSON Schema) includes both the structure and all necessary constrained data within a single document. This makes the model portable across different implementations and languages.

  6. Define a Channel and its Samples

    main

    A channel represents an analysis region encompassing one or more measurement bins. It requires:

    • name: A unique string identifier.
    • samples: A list of sample definitions.

    Each sample within a channel consists of:

    • name: A unique string identifier.
    • data: An array of floats representing the nominal event rates for all bins in the channel.
    • modifiers: A list of modifier objects that parameterize uncertainties or signal strengths.

  7. Define Measurements in a Likelihood

    main

    Measurements define the parameter of interest (POI) and configure the parameters (modifiers) used in the model. In the config object of a measurement, you specify the poi and a list of parameters. For each parameter, you can override default settings using the following keys:

    • inits: Initial value of the parameter.
    • bounds: Interval bounds of the parameter.
    • auxdata: Auxiliary data for the associated constraint term.
    • sigmas: Associated uncertainty of the parameter.
    • fixed: A boolean indicating if the modifier is held constant.

    Note that a parameter set (the collection of values for a named modifier) can be a lower-dimensional subspace. For multi-bin channels, modifiers like uncorrelated shape systematics may require arrays for these values.

    {
    "name": "MyMeasurement",
    "config": {
        "poi": "SignalCrossSection",
        "parameters": [
            { "name":"lumi", "auxdata":[1.0], "sigmas":[0.017], "bounds":[[0.915,1.085]], "inits":[1.0] },
            { "name":"mu_ttbar", "bounds":[[0, 5]] },
            { "name":"rw_1CR", "fixed":true }
        ]
    }
}

  8. Translate HistFitter workspaces to pyhf

    main

    To migrate an existing analysis from HistFitter to pyhf, you must first extract the HistFactory workspace and then convert the resulting XML files into a single JSON workspace using the pyhf xml2json command.

    1. Extract HistFactory XML files

    Modify your existing HistFitter.py execution to skip fitting (-f) and plotting (-D), and instead use the -w and -x flags. The -w flag recreates the workspace, and the -x flag dumps the XML files to a configuration directory.

    HistFitter.py -wx -F excl config.py

    2. Convert to JSON

    Once the XML files are dumped (typically into a directory named config/{analysisName}/), run the pyhf xml2json command on the top-level XML file. This command will automatically read the associated XML files and load the histogram data from the cached .root file.

    pyhf xml2json config/{analysisName}/{prefix}.xml

    Workspace File Naming

    The output file paths depend on your config.py settings:

    • XML Directory: config/{analysisName}/
    • Top-level XML: {prefix}.xml
    • Channel XMLs: {prefix}_{channelName}_cuts.xml
    • Histogram Cache: {cachePath} (defined by configMgr.histCacheFile)
    • Workspace ROOT file: results/{analysisName}/{prefix}_combined_{measurementName}.root
    HistFitter.py -wx -F excl config.py

# Followed by conversion:
pyhf xml2json config/3b_tag21.2.27-1_RW_ExpSyst_36100_multibin_bkg/Excl.xml

  9. Set up a local development environment

    main

    To develop pyhf, use a Python virtual environment managed by pip and nox.

    1. Clone the repository:
      git clone git@github.com:scikit-hep/pyhf
    2. Install development dependencies in editable mode:
      python -m pip install --upgrade --editable . --group dev
    3. Install Git pre-commit hooks using prek:
      prek install

    Use nox --list to see all available development sessions.

    git clone git@github.com:scikit-hep/pyhf
python -m pip install --upgrade --editable . --group dev
prek install

  10. Translate TRExFitter workspaces to pyhf

    main

    Migrating from TRExFitter to pyhf is straightforward because TRExFitter already generates the required RooWorkspace files (XML and ROOT).

    Conversion Steps

    1. Locate the generated XML files (typically in a RooStats/ subdirectory) and the histogram .root files (typically in a Histograms/ subdirectory).
    2. Run pyhf xml2json on the primary XML file.
    pyhf xml2json pyhf_example/RooStats/pyhf_example.xml

    Known Limitations

    When converting from TRExFitter, be aware of the following constraints:

    • Uncorrelated shape systematics: These cannot be pruned.
    • Custom expressions: Custom expressions for normalization factors are not supported.

  11. Run tests and measure coverage

    main

    Running Tests

    • Full suite: Run pytest from the repository root.
    • Core codebase (recommended for local dev): Use nox to target specific Python versions:
      nox --session tests --python 3.14
    • Visualization tests: To run contrib module matplotlib tests:
      nox --session tests --python 3.14 -- contrib
      To regenerate image files, run nox --session regenerate.
    • Doctests: pytest automatically runs doctests. To run doctests for a specific file:
      pytest src/pyhf/tensor/jax_backend.py

    Coverage

    • Run coverage:
      coverage run --module pytest
# OR
nox --session tests --python 3.14 -- coverage
    • Generate reports:
      • Standard report: coverage
      • XML and HTML reports: nox --session coverage
    nox --session tests --python 3.14
# For coverage
nox --session coverage

  12. Validate a workspace JSON against the pyhf schema

    main

    You can verify that a JSON workspace file conforms to the official pyhf workspace specification using jsonschema. This is useful for ensuring your model structure is correct before performing statistical analyses.

    To perform validation:

    1. Load your workspace JSON.
    2. Fetch the official schema from https://scikit-hep.org/pyhf/schemas/1.0.0/workspace.json.
    3. Use jsonschema.validate() to check the instance against the schema.
    import json, requests, jsonschema

with open("/path/to/analysis_workspace.json", encoding="utf-8") as ws_file:
    workspace = json.load(ws_file)

# if no exception is raised, it found and parsed the schema
schema = requests.get("https://scikit-hep.org/pyhf/schemas/1.0.0/workspace.json").json()

# If no exception is raised by validate(), the instance is valid.
jsonschema.validate(instance=workspace, schema=schema)