lppls

repository·master·Indexed 19 days ago

https://github.com/boulder-investment-technologies/lppls

A Python module for fitting the Log Periodic Power Law Singularity (LPPLS) model to financial data to detect bubbles and predict regime changes. It includes implementations for standard LPPLS, Quantile LPPLS (QLPPLS), and optimization using the CMA-ES evolutionary algorithm. Key features include nested fits for confidence indicators, bubble start time detection via Lagrange regularization, and a data_loader utility for sample datasets like the Nasdaq Dot-com bubble.

Tokens
7K
Snippets
23
Records
26
Agent score
66%

What's inside lppls

  1. Fit the LPPLS model to price data

    master

    To use the lppls module, you must first prepare your data as an observations array. This array should contain two rows: the first row is the time (converted to ordinal format) and the second row is the log-transformed price.

    1. Convert dates to ordinals using pd.Timestamp.toordinal.
    2. Log-transform the price data.
    3. Create a NumPy array of shape (2, N).
    4. Instantiate lppls.LPPLS(observations=observations).
    5. Call .fit(max_searches) to retrieve the model parameters.
    from lppls import lppls
    import numpy as np
    import pandas as pd
    from datetime import datetime as dt
    
    # Prepare data
    data = data_loader.nasdaq_dotcom()
    time = [pd.Timestamp.toordinal(dt.strptime(t1, '%Y-%m-%d')) for t1 in data['Date']]
    price = np.log(data['Adj Close'].values)
    observations = np.array([time, price])
    
    # Fit model
    MAX_SEARCHES = 25
    lppls_model = lppls.LPPLS(observations=observations)
    tc, m, w, a, b, c, c1, c2, O, D = lppls_model.fit(MAX_SEARCHES)
    
    # Visualize
    lppls_model.plot_fit()
  2. How the q-dependent loss function works in QLPPLS

    master

    The QLPPLS implementation uses a custom loss function via func_restricted to find the least absolute differences adjusted for a specific quantile q.

    Instead of the standard L2 norm (sum of squared errors), it calculates the loss as: loss = sum([-(1 - q) * e if e < 0 else q * e for e in abs(delta)])

    Where delta is the difference between the model prediction and the actual observations. This allows the model to be more robust to outliers or specifically tuned to different parts of the distribution by adjusting the q parameter.

  3. Prepare data for LPPLS observations

    master

    To use the lppls model, you must format your time-series data into a 2D NumPy array of shape (2, N).

    1. Time: Convert dates into ordinal integers using pd.Timestamp.toordinal.
    2. Price: Use the natural logarithm of the price values (np.log(price)).
    3. Array: Combine them into a single array where the first row is time and the second row is log-price.

    Example using the data_loader utility:

    from lppls import lppls, data_loader
    import numpy as np
    import pandas as pd
    from datetime import datetime as dt
    
    # Load sample data
    data = data_loader.nasdaq_dotcom()
    
    # 1. Convert time to ordinal
    time = [pd.Timestamp.toordinal(dt.strptime(t1, "%Y-%m-%d")) for t1 in data["Date"]]
    
    # 2. Create log price list
    price = np.log(data["Adj Close"].values)
    
    # 3. Create observations array (expected format: [time, price])
    observations = np.array([time, price])
  4. Initialize the LPPLS model

    master

    To use the LPPLS model, instantiate the LPPLS class by providing an array of observations. The observations must be a 2xM matrix (or a pd.DataFrame) where the first row contains timestamps (as ordinals) and the second row contains the observed values (e.g., log-prices).

    import numpy as np
    from lppls.lppls import LPPLS
    
    # observations: 2xM matrix (row 0: timestamps, row 1: values)
    observations = np.array([
        [738000, 738001, 738002], # Timestamps
        [10.5, 11.2, 10.8]         # Values
    ])
    
    model = LPPLS(observations)
  5. Compute and visualize confidence indicators

    master

    The mp_compute_nested_fits method allows for computing confidence indicators by performing nested fits over different window sizes. This is useful for assessing the robustness of the bubble detection.

    Parameters for mp_compute_nested_fits:

    • workers: Number of parallel workers.
    • window_size: The largest window size.
    • smallest_window_size: The smallest window size.
    • outer_increment: Step size for the outer loop.
    • inner_increment: Step size for the inner loop.
    • max_searches: Maximum number of searches per fit.
    • filter_conditions_config: A dictionary defining parameter bounds (m_min, m_max, w_min, w_max, O_min, D_min).

    After computing, use plot_confidence_indicators(res) to visualize the results. To convert the results into a pd.DataFrame, use compute_indicators(res).

    res = lppls_model.mp_compute_nested_fits(
        workers=8,
        window_size=120, 
        smallest_window_size=30, 
        outer_increment=1, 
        inner_increment=5, 
        max_searches=25,
        filter_conditions_config={
            "m_min": 0.0,
            "m_max": 1.0,
            "w_min": 2.0,
            "w_max": 15.0,
            "O_min": 2.5,
            "D_min": 0.5,
        },
    )
    
    lppls_model.plot_confidence_indicators(res)
    
    # Convert to DataFrame
    res_df = lppls_model.compute_indicators(res)
  6. Detect bubble start time via Lagrange Regularization

    master
    The detect_bubble_start_time_via_lagrange method uses Lagrange regularization to objectively identify the optimal fitting window size. This helps determine the inception of a financial bubble by addressing model overfitting when the start time is unknown.
  7. Use CMA-ES for non-linear parameter estimation

    master

    For more robust identification of the non-linear parameters ($t_c$, $m$, $\omega$), you can use the lppls_cmaes module, which implements the CMA-ES evolutionary algorithm. This is particularly useful for difficult non-linear non-convex optimization problems.

    Note: While effective for single fits, this approach may be slow when computing confidence indicators.

    from lppls import lppls_cmaes
    
    lppls_model = lppls_cmaes.LPPLSCMAES(observations=observations)
    tc, m, w, a, b, c, c1, c2, O, D = lppls_model.fit(max_iteration=2500, pop_size=4)
  8. Configure indicator filter thresholds

    master

    When calling compute_indicators or plot_confidence_indicators, you can pass a filter_conditions_config dictionary to customize which fits are considered 'qualified'.

    Supported Keys:

    • m_min, m_max: Range for the power law exponent $m$.
    • w_min, w_max: Range for the angular frequency $\omega$.
    • O_min: Lower bound for oscillations.
    • D_min: Lower bound for damping.
    • tc_min_days, tc_max_days: Allowed distance (in days) from the window end to the critical time $t_c$.
    • tc_min_frac, tc_max_frac: Allowed distance (as a fraction of the window duration) from the window end to $t_c$.

    Constraints:

    • m_min must be < m_max.
    • w_min must be < w_max.
    • tc_min_days, tc_max_days, tc_min_frac, and tc_max_frac must be $\ge 0$.
    custom_config = {
        "m_min": 0.1,
        "m_max": 0.9,
        "w_min": 5.0,
        "w_max": 15.0,
        "O_min": 2.5,
        "D_min": 0.5,
        "tc_min_days": 60.0,
        "tc_max_days": 252.0,
        "tc_min_frac": 0.5,
        "tc_max_frac": 0.5,
    }
    
    # Use this config when computing indicators
    indicators_df = model.compute_indicators(results, filter_conditions_config=custom_config)
  9. Perform Quantile LPPLS Regression

    master

    To perform Quantile Log Periodic Power Law Singularity (QLPPLS) regression, use the lppls_q.QLPPLS class. This allows you to fit the LPPLS model to specific quantiles of the data rather than just the mean.

    1. Prepare your observations as a 2D NumPy array where the first row is scaled time and the second row is scaled log price.
    2. Instantiate QLPPLS with your observations and the target quantile q (e.g., 0.1 for the 10th percentile).
    3. Call .fit(max_searches) to estimate parameters.
    4. Use the .lppls() method to generate predictions based on the fitted parameters.
    from lppls import lppls_q
    import numpy as np
    
    # observations must be [time_array, price_array]
    observations = np.array([time, price])
    
    # Instantiate with a specific quantile q
    qlppls_model = lppls_q.QLPPLS(observations=observations, q=0.5)
    
    # Fit the model. The argument is the max number of searches to perform.
    # The literature suggests 25.
    tc, m, w, a, b, c, c1, c2, O, D = qlppls_model.fit(25)
    
    # Generate predictions for a given time t
    predictions = [qlppls_model.lppls(t, tc, m, w, a, b, c1, c2) for t in time]
  10. Fit an LPPLS model to time-series data

    master

    To perform a Log Periodic Power Law Singularity (LPPLS) fit, you must prepare your data as a 2D NumPy array where the first row contains time (as ordinal integers) and the second row contains the natural logarithm of prices.

    1. Prepare Observations: Create a NumPy array of shape (2, N) containing [time_ordinals, log_prices].
    2. Instantiate Model: Use lppls_lm.LPPLS_LM(observations=observations).
    3. Fit: Call .fit(max_searches) where max_searches is an integer (the literature suggests 25) representing the number of searches to perform before giving up.
    4. Retrieve Parameters: The .fit() method returns the following parameters: tc, m, w, a, b, c, c1, c2, O, D.
    from lppls import lppls_lm
    import numpy as np
    import pandas as pd
    from datetime import datetime as dt
    
    # 1. Prepare observations (time as ordinal, price as log)
    time = [pd.Timestamp.toordinal(dt.strptime(t, "%Y-%m-%d")) for t in data["Date"]]
    price = np.log(data["Adj Close"].values)
    observations = np.array([time, price])
    
    # 2. Instantiate and fit
    MAX_SEARCHES = 25
    lppls_model = lppls_lm.LPPLS_LM(observations=observations)
    tc, m, w, a, b, c, c1, c2, O, D = lppls_model.fit(MAX_SEARCHES)