pyextremes: Extreme Value Analysis (EVA) in Python

repository·master·Indexed 42 days ago

https://github.com/georgebv/pyextremes

A Python library for univariate Extreme Value Analysis (EVA) version 2.5.0. It provides tools for extreme event extraction using Block Maxima (BM) and Peaks Over Threshold (POT), distribution fitting (GEVD, GPD) via MLE, MCMC (Emcee), Method of Moments (MOM), and L-moments, and statistical visualization including return period plots and MCMC diagnostics.

Tokens
20.1K
Snippets
62
Records
95
Agent score
54%

What's inside pyextremes

  1. Overview of pyextremes capabilities

    master

    pyextremes is a Python library for univariate Extreme Value Analysis (EVA). It provides a suite of tools for the following tasks:

    • Extreme Event Extraction: Extracting extremes from time series using methods like Block Maxima (BM) or Peaks Over Threshold (POT).
    • Distribution Fitting: Fitting continuous distributions (e.g., GEVD, GPD, or user-specified distributions) to extracted extreme events.
    • Visualization: Generating plots for model inputs, results, and goodness-of-fit statistics (e.g., model diagnostics, trace plots, corner plots).
    • Probability Estimation: Estimating extreme events for a given probability or return period (e.g., 100-year events) along with confidence intervals.
    • Model Tuning: Tools to assist in selecting optimal block sizes for BM or thresholds for POT.
  2. Overview of pyextremes features

    master

    pyextremes is a Python library for univariate Extreme Value Analysis (EVA). It provides tools for:

    • Extraction of extreme events: Using methods like Block Maxima (BM) or Peaks Over Threshold (POT).
    • Distribution fitting: Fitting continuous distributions such as GEVD, GPD, or any scipy continuous distribution to extracted events.
    • Visualization: Plotting model inputs, results, and goodness-of-fit statistics.
    • Estimation: Calculating extreme events for specific probabilities or return periods (e.g., 100-year events) with confidence intervals.
    • Model selection/tuning: Tools for selecting block size in BM or thresholds in POT.
  3. Understand Peaks Over Threshold (POT) extreme values

    master

    Peaks Over Threshold (POT) is an EVA method where values exceeding (or falling below) a chosen threshold are selected. To ensure independence between observations, these selected values must be 'declustered' by specifying a minimum distance between adjacent clusters (e.g., 24 hours). The resulting POT extremes are modeled using the Generalized Pareto Distribution (GPD).

    Use POT to investigate model stability in the region of target exceedance probabilities (return periods) and to gain higher confidence in your results.

  4. How to structure an Extreme Value Analysis (EVA) workflow

    master

    Extreme Value Analysis in pyextremes can be performed using two different approaches depending on the required level of control:

    1. High-level workflow with EVA: Use the EVA helper class (located in pyextremes.eva.EVA) to chain components together. This is the recommended way to streamline common EVA workflows (extraction, fitting, and visualization) with minimal code.
    2. Low-level control with individual components: Each part of the EVA process—such as extreme value extraction, model fitting, or result visualization—is implemented as independent, modular components. You can call these low-level functions directly if you need to customize specific steps of the analysis or use them outside of a standard workflow.
  5. Understand Block Maxima (BM) extreme values

    master

    Block Maxima (BM) is an extreme value analysis (EVA) method where a time series is partitioned into equal consecutive blocks of a specific size (the default in pyextremes is 1 year). A single maximum value is extracted from each block. The resulting BM time series is then used to model extreme behavior using the Generalized Extreme Value Distribution (GEVD) model family.

    Use BM as a starting point for analysis because it is generally a 'simpler' and more stable model that requires minimal user input.

  6. How empirical return periods are calculated

    master

    Empirical return periods are assigned to observed extreme values by ranking them from most extreme to least extreme and calculating exceedance probabilities using plotting positions.

    Probability of Exceedance ($P$)

    Calculated using the formula:

    $$P = \frac{r - \alpha}{n + 1 - \alpha - \beta}$$

    Where:

    • r: Rank of the extreme value (1 to $n$). pyextremes uses scipy.stats.rankdata with method="average" for ties.
    • n: Total number of extreme values.
    • $\alpha$ and $\beta$: Empirical plotting position parameters.

    Return Period ($R$)

    Calculated from the exceedance probability $P$ and the rate of extreme events $\lambda$:

    $$R = 1 / P / \lambda$$

    Where:

    • R: The return period as a multiple of the return_period_size (default is 1 year).
    • $\lambda$ (Rate of extreme events):
      • For Block Maxima: $\lambda = \text{return_period_size} / \text{block_size}$
      • For Peaks Over Threshold (POT): $\lambda = n / (t / \text{return_period_size})$, where $t$ is the total duration of the series.

    Important: Plotting Positions vs. Modeling

    Plotting positions are NOT used to fit models. In modern Extreme Value Analysis (EVA), models are fitted using Maximum Likelihood Estimation (MLE) or Markov Chain Monte Carlo (MCMC). Plotting positions are used exclusively for:

    • Visualizing extreme values on return value plots.
    • Performing goodness-of-fit tests (e.g., P-P or Q-Q plots).
  7. Understand the relationship between Return Period and Probability of Exceedance

    master

    A return period indicates the duration of time (typically years) corresponding to a probability that a given value will be exceeded at least once within a year. This probability is known as the probability of exceedance ($p$).

    Key relationships:

    • Return Period $\approx 1/p$.
    • Return Level ($z_p$): The value expected to be exceeded on average once every $1/p$ years.

    Risk Assessment Note: A 100-year event (1% annual exceedance probability) does not mean it happens once every 100 years. To calculate the probability of an event occurring at least once within a specific time period ($n$ years), use the formula:

    $$1 - (1 - p)^n$$

    For example, a 100-year event ($p=0.01$) has a ~39.5% chance of occurring at least once within a 50-year period ($n=50$).

  8. Understand the AbstractModelBaseClass interface

    master
    AbstractModelBaseClass is the base class for all models in pyextremes. All concrete model implementations (like Emcee and MLE) follow this interface, ensuring consistent behavior for fitting and prediction across different extreme value estimation methods.
  9. Declustering extremes with the `r` parameter

    master

    Declustering is used to ensure that extreme values are IID (independent and identically distributed), which is a requirement for the Generalized Pareto Distribution (GPD) model.

    Declustering is controlled by the r argument, which specifies the minimum time distance (window duration) between adjacent clusters. The algorithm identifies clusters where all exceedances are separated by distances no more than r, and then selects only the highest (for extremes_type="high") or lowest (for extremes_type="low") value within each cluster.

    Note on default value: The default r="24h" is based on meteorological phenomena (like storms), but users should adjust this based on the specific properties of their studied phenomena (e.g., precipitation events that last several days may require a larger r).

  10. Understand the goal of threshold selection in EVA

    master

    In Extreme Value Analysis (EVA) using the Peaks Over Threshold (POT) method, selecting a threshold value is a critical trade-off between bias and variance:

    • Large thresholds: Produce fewer extreme values, leading to high variance (wider confidence bounds).
    • Small thresholds: Generate samples that may not properly approximate the Generalized Pareto Distribution (GPD) model, leading to bias.

    The Goal: Select the smallest threshold value among those that produce extreme values following the limit exceedance model (the GPD family).

    Note: Threshold selection involves subjective judgment. It is recommended to use multiple methods (e.g., combining POT with Block Maxima + GEVD) to increase confidence in your results.

  11. Understand Extreme Value Analysis (EVA) terminology

    master

    The following abbreviations are used throughout the pyextremes documentation and API to describe core concepts in Extreme Value Analysis:

    • EVA: Extreme Value Analysis
    • BM: Block Maxima/Minima
    • POT: Peaks Over Threshold
    • GEVD: Generalized Extreme Value Distribution
    • GPD: Generalized Pareto Distribution
    • MLE: Maximum Likelihood Estimate
    • MCMC: Markov Chain Monte Carlo