pybaselines

repository·main·Indexed 41 days ago

https://github.com/derb12/pybaselines

A Python library providing over 50 algorithms for the baseline correction of experimental data, such as Raman, FTIR, NMR, and XRD. It offers a unified API for 1D and 2D baseline estimation, including morphological methods (mor, imor, rolling_ball, tophat), polynomial fitting (modpoly, imodpoly, penalized_poly, quant_reg), and optimized Whittaker-smoothing and penalized spline algorithms for gridded 2D data.

Tokens
49.1K
Snippets
195
Records
232
Agent score
52%

What's inside pybaselines

  1. Overview of pybaselines baseline correction algorithms

    main

    pybaselines is a Python library designed for baseline correction on experimental data (e.g., Raman, FTIR, NMR, XRD, XRF, PIXE, MALDI-TOF, LIBS). It provides a semi-unified API to facilitate testing and comparing different algorithms on a single dataset. The library includes over 50 algorithms, many of which are adapted directly from literature to preserve their original names.

    Key algorithms include:

    • AsLS
    • airPLS
    • ModPoly
    • SNIP
    • Penalized spline versions of Whittaker-smoothing-based algorithms
  2. Understanding Spline Baselines in pybaselines

    main

    Spline baselines in pybaselines are primarily implemented using Basis splines (B-splines) and Penalized B-splines (P-splines).

    B-splines

    A B-spline is a piecewise joining of individual curves. The number of spline basis functions ($M$) is determined by the number of knots (num_knots) plus the spline degree minus 1.

    P-splines

    To control the smoothness of the fitting spline, pybaselines uses P-splines. This method adds a penalty on the finite-difference between spline coefficients to the least-squares minimization.

    Key parameters for controlling smoothness include:

    • Penalty scale factor ($\lambda$): Controls the strength of the smoothness penalty.
    • Finite-difference operator order ($d$): The order of the difference used in the penalty.

    P-splines use uniformly spaced knots to facilitate the calculation of the finite-difference.

    Relationship to Whittaker Smoothing

    P-splines are mathematically related to Whittaker smoothing. If the number of basis functions ($M$) is set equal to the number of data points ($N$) and the spline degree is set to 0, the P-spline equation becomes identical to the Whittaker smoothing equation.

  3. Configure smoothing parameters for Whittaker 2D

    main

    When using Whittaker-smoothing-based algorithms for two-dimensional data, the lam parameter controls the smoothness of the baseline. You can provide it in two ways:

    1. Single value: A single number applied to both rows and columns ($\lambda_r = \lambda_c$).
    2. Sequence of two values: A tuple or list (lambda_r, lambda_c) to specify different penalty values for rows and columns respectively.
  4. How Whittaker 2D baselines work

    main

    Whittaker-smoothing-based algorithms for 2D data aim to find a baseline $V$ that matches the measured data $Y$ while penalizing roughness. The algorithm minimizes a cost function that includes a weighted least-squares term and two penalty terms: one for row roughness ($\lambda_r$) and one for column roughness ($\lambda_c$).

    To avoid the extreme computational and memory costs of solving the full $(M \cdot N, M \cdot N)$ linear system, pybaselines uses an optimized approach based on eigendecomposition. By decomposing the penalty matrices, the dimensionality of the system is reduced. This is particularly effective for baseline correction because baselines typically require much less curvature than smoothing, meaning only a small number of eigenvalues (e.g., ~5-25) are needed to approximate the analytical solution accurately.

    Key concepts:

    • Penalty Parameters: You can control row and column smoothness independently.
    • Dimensionality Reduction: The algorithm uses the Kronecker product of eigenvectors to transform the problem into a much smaller system, making it scalable for 2D gridded data.
  5. Understand polynomial baseline fitting strategies

    main

    Polynomial baseline algorithms in pybaselines modify the standard least-squares approach to avoid being pulled by peaks. There are three primary strategies:

    1. Selective Masking: Fitting only to data in regions where only the baseline exists. This is primarily used with the .poly() method. It is noted as being time-consuming and potentially hard to reproduce.
    2. Thresholding: An iterative method where the fit is updated by taking the element-wise minimum between the current data and the current fit. This is used by .modpoly(), .imodpoly(), and .loess() (if use_threshold=True).
    3. Penalizing Outliers: Incorporating a penalty into the cost function or applying lower weights to outliers. This is used by .penalized_poly() and .loess() (if use_threshold=False).
  6. Understand Morphological Baseline Algorithms

    main

    Morphological baseline algorithms in pybaselines use mathematical morphology operations—specifically dilation, erosion, opening, and closing—to estimate the baseline of a signal. These operations use moving windows to compute the maximum, minimum, or a combination of the two within each window.

    • Dilation: Computes the maximum value within the window.
    • Erosion: Computes the minimum value within the window.
    • Closing: Performs an erosion followed by a dilation.
    • Opening: Performs a dilation followed by an erosion.
    import numpy as np
    from scipy.ndimage import grey_closing, grey_dilation, grey_erosion, grey_opening
    
    # Example of basic morphological operations on a signal 'y' with a 'window' size
    dilation = grey_dilation(y, window)
    erosion = grey_erosion(y, window)
    closing = grey_closing(y, window)
    opening = grey_opening(y, window)
  7. Use 2D baseline correction algorithms

    main

    pybaselines provides 2D extensions for a subset of its 1D baseline correction algorithms. These are intended for 2D data that possesses a global baseline.

    If your data does not have a global baseline, or if performance is a priority, it is recommended to apply 1D algorithms to each row or column individually. You can achieve this using the .individual_axes method of the Baseline2D class or by using the standard Baseline class within manual loops.

  8. How the Baseline API works

    main

    The pybaselines library provides a semi-unified API for baseline correction. The primary interfaces are:

    • Baseline: For one-dimensional data.
    • Baseline2D: For two-dimensional data.

    When calling a baseline correction algorithm, the method will return two items:

    1. A numpy array containing the calculated baseline.
    2. A dictionary containing potentially useful parameters used during the calculation.

    To use the API, you typically instantiate a Baseline object with your x-axis data and then call specific algorithm methods on that object.

    from pybaselines import Baseline
    
    # Initialize with x-axis data
    baseline_fitter = Baseline(x_data=x)
    
    # Call an algorithm
    bkg, params = baseline_fitter.asls(y, lam=1e7, p=0.02)
  9. How the penalty scale factor (lam) scales with data size

    main

    When using Whittaker-smoothing-based methods, the penalty scale factor lam ($\lambda$) must be adjusted based on the number of data points in your signal. As the number of data points increases, the required lam value to achieve the same baseline fit also increases.

    The relationship is approximately: $$\log(\lambda) \propto \log(\text{number of data points})$$

    Example scaling:

    • 100 points: lam $\approx 10^3$
    • 1,000 points: lam $\approx 10^7$
    • 10,000 points: lam $\approx 10^{11}$
  10. Understand the categories of baseline correction algorithms in pybaselines

    main

    The pybaselines library organizes its baseline correction algorithms into several broad categories. While these categories are used for internal grouping and documentation purposes, many algorithms may overlap between them. Use these categories to navigate the available methods:

    • Polynomial: Methods based on polynomial fitting.
    • Whittaker: Methods utilizing Whittaker smoothing/filtering.
    • Morphological: Methods based on mathematical morphology.
    • Smooth: Smoothing-based baseline estimation.
    • Spline: Spline-based interpolation or fitting.
    • Classification: Algorithms that use classification techniques.
    • Optimizers: Methods driven by optimization processes.
    • Misc: Miscellaneous baseline correction methods.

    To find the specific parameters, return types, and full technical references for any algorithm, consult the API section of the documentation.

  11. Configure the half_window parameter for morphological algorithms

    main

    All morphological baseline algorithms in Baseline2D use a half_window parameter to define the size of the window used for morphological operators.

    Important: half_window is index-based, not based on the physical units of your data. You must manually convert your desired physical window size into index offsets before passing them to the algorithm.

  12. Use the Baseline class for 1D baseline correction

    main

    The recommended way to perform baseline correction on one-dimensional data is to use the Baseline class. This class provides a unified interface for all available algorithms, including polynomial, Whittaker smoothing, morphological, spline, and various other specialized methods. Instead of calling individual functions, you instantiate or use the Baseline object to access the specific algorithm required for your signal.

    from pybaselines import Baseline
    
    # Initialize the baseline object
    baseline = Baseline()
    
    # Use an algorithm (example: poly)
    # baseline_values, params = baseline.poly(signal, ...)