DOCSEARCHbyAlgolia
Submit Docs

LM-Polygraph

repository·main·Indexed 17 days ago

https://github.com/iinemo/lm-polygraph

An uncertainty estimation (UE) toolkit for Transformer Language Models designed to quantify uncertainty in text generation, detect hallucinations, and improve LLM safety. It provides a suite of white-box and black-box methods across categories such as information-based, meaning diversity, ensembling, density-based, and reflexive estimation. The library includes the polygraph_eval CLI for benchmarking, vLLM integration for faster generation, and tools for building and publishing benchmark datasets to Hugging Face.

Tokens
49.6K
Snippets
136
Records
172
Agent score
65%

What's inside lm-polygraph

  1. Overview of LM-Polygraph components

    main

    LM-Polygraph is composed of four primary functional components that work together to evaluate Large Language Models (LLMs):

    • UEManager: Manages User Experience (UE) or evaluation configurations and workflows.
    • Stat Calculators: Responsible for computing statistical metrics from evaluation data.
    • Estimators: Used for estimating model properties or uncertainty.
    • Generation Metrics: Provides metrics specifically designed to evaluate the quality and characteristics of model-generated text.

  2. Overview of uncertainty estimation methods

    main

    The lm-polygraph library provides a wide array of uncertainty estimation methods categorized by their access level (White-box vs. Black-box), methodology (Information-based, Meaning diversity, Ensembling, Density-based, Reflexive), and computational requirements.

    Method Types

    • White-box: Methods that require access to the model's internal states (e.g., logits, attention scores, or hidden states).
    • Black-box: Methods that only require access to the model's input and output text.

    Key Categories

    • Information-based: Uses probabilities and entropy (e.g., Perplexity, Mean/max token entropy, Pointwise mutual information).
    • Meaning Diversity: Measures how much the semantic content varies (e.g., Semantic entropy, Lexical similarity, NumSets).
    • Ensembling: Uses multiple model outputs to estimate uncertainty (e.g., Sentence-level ensemble-based measures).
    • Density-based: Uses statistical distance from a distribution (e.g., Mahalanobis distance, Robust density estimation).
    • Reflexive: Uses the model's own verbalized or probabilistic assessment of its truthfulness (e.g., p(True), Verbalized Uncertainty).

    When choosing a method, consider the Level (whether it applies to a single sequence, a specific claim, or both) and whether the method requires Training Data.

  3. How normalization methods work in LM-Polygraph

    main

    LM-Polygraph provides several normalization methods to convert raw uncertainty scores into interpretable confidence values bounded between 0 and 1. These methods fall into two main categories:

    1. Statistical Scaling: Methods like MinMaxNormalizer and QuantileNormalizer scale scores based on their distribution but do not necessarily account for the actual quality of the model's output.
    2. Performance-Calibrated Confidence (PCC): Methods like BinnedPCCNormalizer and IsotonicPCCNormalizer calibrate scores against actual output quality metrics. This creates a direct, interpretable link where a confidence score represents the expected quality of the generation.

    For the best balance of preserving uncertainty rankings and providing quality-linked interpretability, Isotonic PCC is the recommended approach.

  4. Evaluate metrics against multi-reference datasets

    main
    When benchmarking against datasets that contain multiple reference values for a single input (for example, TriviaQA where a question has multiple alias values), lm-polygraph can evaluate generation metrics against every provided reference. In this mode, the final metric value reported for a sample is the maximum value obtained across all references.

  5. Understand the UEManager class and its data attributes

    main

    The UEManager is the central class used for estimating uncertainty scores, calculating underlying statistics based on model generation, and storing results. It is typically invoked by the polygraph_eval script. After evaluation, the manager populates several key attributes:

    • stats: A defaultdict containing various statistics. Keys are statistic names, and values can range from single numbers to complex objects (often outputs from StatCalculator objects).
    • estimations: A defaultdict storing outputs from Estimator objects. Keys are (level, estimator_name) tuples. The level can be sequence, token, or claim:
      • sequence: 1D numpy arrays (length = number of examples).
      • token: Lists of numpy arrays (outer list length = number of examples; inner array length = number of tokens per example, excluding EOS).
      • claim: Lists of numpy arrays (outer list length = number of examples; inner array length = number of claims per example).
    • gen_metrics: A defaultdict containing quality metrics of generated sequences. Keys are (level, metric_name) tuples where level is sequence or claim:
      • sequence: 1D numpy arrays (length = number of examples).
      • claim: Lists of numpy arrays (outer list length = number of examples; inner array length = number of claims per example).
    • metrics: A dict holding comparative scores (e.g., PRR, RCC) for combinations of compatible estimators, generation metrics, and uncertainty estimation metrics. Only pairs with matching level are included.

  6. Understand LM-Polygraph Normalization impact areas

    main

    LM-Polygraph normalization transforms raw uncertainty estimates into interpretable confidence values. The normalization process affects four key areas:

    1. Score Transformation: Converts unbounded Raw Uncertainty Scores (where higher values mean more uncertainty) into Normalized Confidence Values bounded in the [0,1] range (where higher values mean more confidence). This preserves the relative ordering of original scores when using Isotonic PCC.
    2. Evaluation Pipeline:
      • Calibration Stage: Learns normalization parameters using a calibration dataset and generation quality metrics. Parameters can be saved for reuse.
      • Inference Stage: Applies learned parameters to new estimates without requiring additional model inference, making it a fast transformation.
    3. Quality Metrics Integration: Normalizes various quality metrics (e.g., ROUGE, BLEU, accuracy) to the [0,1] range to enable consistent calibration across different task types.
    4. Model Type Support:
      • White-box Models (e.g., HuggingFace): Supports token-level and sequence-level calibration by accessing internal probabilities and logits.
      • Black-box Models (e.g., OpenAI API): Limited to sequence-level normalization based on output-based uncertainty estimation.

  7. Available Normalization Methods

    main

    LM-Polygraph supports several normalization methods to transform uncertainty scores:

    1. MinMax Normalization: Linearly scales uncertainty scores to a [0,1] range.
    2. Quantile Normalization: Transforms scores into percentile ranks using an empirical CDF.
    3. Binned Performance-Calibrated Confidence (Binned PCC): Maps uncertainty scores to confidence bins based on output quality.
    4. Isotonic Performance-Calibrated Confidence (Isotonic PCC): Uses monotonic regression to map uncertainty to confidence while preserving ordering.

    Use MinMax or Quantile for simple scaling, PCC methods for high interpretability, and Isotonic PCC when preserving score ordering is critical.

    # MinMax Example
normalization:
  type: "minmax"
  clip: true

# Quantile Example
normalization:
  type: "quantile"

# Binned PCC Example
normalization:
  type: "binned_pcc"
  params:
    num_bins: 10

# Isotonic PCC Example
normalization:
  type: "isotonic_pcc"
  params:
    y_min: 0.0
    y_max: 1.0
    increasing: false
    out_of_bounds: "clip"

  9. Use different uncertainty estimators

    main

    LM-Polygraph provides several estimators for calculating uncertainty at different granularities:

    Sequence-level Uncertainty (Whitebox)

    Best for evaluating the uncertainty of the entire generated sequence.

    • MaximumSequenceProbability()
    • SemanticEntropy()

    Token-level Uncertainty (Whitebox)

    Best for evaluating uncertainty at each individual token.

    • MaximumTokenProbability()

    Blackbox Uncertainty

    Designed for models where internal probabilities are unavailable.

    • EigValLaplacian(verbose=True)
    from lm_polygraph.estimators import MaximumSequenceProbability, SemanticEntropy, MaximumTokenProbability, EigValLaplacian

# Sequence-level
estimator = MaximumSequenceProbability()
estimate_uncertainty(model, estimator, input_text='...')

# Token-level
estimator = MaximumTokenProbability()
estimate_uncertainty(model, estimator, input_text='...')

# Blackbox
estimator = EigValLaplacian(verbose=True)
estimate_uncertainty(model, estimator, input_text='...')

  10. Standard Dataset Schema for LM-Polygraph

    main

    Datasets processed for use in LM-Polygraph typically follow a specific structure. Subsets are organized into train and test splits. Each split must contain two string columns:

    1. input: The processed input text formatted for LM-Polygraph benchmarking.
    2. output: The processed target output text for LM-Polygraph benchmarking.

    Subsets may correspond to the main dataset (e.g., a continuation subset) or specific instruction methods used during benchmarking.

  11. Common interface for all uncertainty normalizers

    main

    All normalization methods in LM-Polygraph implement the BaseUENormalizer interface. This ensures a consistent workflow for learning from calibration data and applying transformations to new data. The interface supports serialization for saving and loading fitted models.

    # All normalizers follow this pattern:
normalizer.fit(calibration_data)
confidence_scores = normalizer.transform(new_uncertainty_scores)

# Serialization support:
serialized_data = normalizer.dumps()
new_normalizer = BaseUENormalizer.loads(serialized_data)

  12. Configure core normalization methods

    main

    LM-Polygraph uses normalization configurations to transform raw uncertainty scores into interpretable confidence values. You can select from four primary normalization methods by setting the type key under the normalization block in your configuration file.

    Available Methods:

    • minmax: Linearly scales scores to a $[0, 1]$ range.
    • quantile: Transforms scores into percentile ranks using an empirical CDF.
    • binned_pcc: (Binned Performance-Calibrated Confidence) Maps scores to confidence bins based on output quality.
    • isotonic_pcc: (Isotonic Performance-Calibrated Confidence) Uses monotonic regression to map uncertainty to confidence while preserving score ordering.
    normalization:
  type: "minmax"