kneed

repository·main·Indexed 21 days ago

https://github.com/arvkevi/kneed

A Python library for detecting knee (elbow) points in curves using the Kneedle algorithm. It identifies the point of maximum curvature in concave or convex datasets and is commonly used for the K-Means elbow method and PCA component selection. Key features include the KneeLocator class for detection, find_shape for auto-detecting curve properties, and built-in plotting support via matplotlib.

Tokens
11.3K
Snippets
51
Records
58
Agent score
71%

What's inside kneed

  1. Key features of kneed

    main

    The kneed library provides several advanced capabilities for curve analysis:

    • Knee and elbow detection: Supports both concave and convex curves.
    • Function direction: Handles both increasing and decreasing functions.
    • Automatic shape detection: Uses find_shape() to determine the curve type automatically.
    • Multiple knee detection: Supports finding multiple points using an 'online mode'.
    • Tunable sensitivity: Use the S parameter for fine-grained control over detection.
    • Interpolation: Supports interp1d and polynomial interpolation methods.
    • Visualization: Includes built-in plotting for quick inspection of results.
  2. Configure curve and direction in KneeLocator

    main

    To correctly detect a knee or elbow point, you must specify the curve and direction parameters in the KneeLocator constructor. The library supports four combinations based on the shape and trend of your data:

    CurveDirectionUse CaseExample
    concaveincreasingDiminishing returnsK-means inertia vs. k
    concavedecreasingAccelerating declineModel accuracy vs. pruning
    convexincreasingAccelerating growthExponential-like curves
    convexdecreasingDiminishing decline (elbow)Sorted eigenvalues

    Rules of Thumb for selection:

    • Concave vs. Convex: If the curve bends like a bowl opening upward, it is concave. If it bends like a bowl opening downward, it is convex.
    • Increasing vs. Decreasing: Look at the overall trend from left to right. If values go up, use increasing. If they go down, use decreasing.
    from kneed import KneeLocator
    
    # Example for a diminishing returns curve
    kl = KneeLocator(x, y, curve="concave", direction="increasing")
  3. Detect multiple knee points using Online Mode

    main

    By default, KneeLocator operates in Offline Mode (online=False), which stops at the first knee point detected. To detect multiple knee points or find the most significant knee in a curve, you must enable Online Mode by setting online=True.

    In Online Mode, the algorithm scans the entire curve and can "correct" previous detections. The standard kl.knee attribute will return the last (most significant) knee found rather than the first.

    import numpy as np
    from kneed import KneeLocator
    
    # Sample data
    x = range(1, 1001)
    y = sorted(np.random.gamma(0.5, 1.0, 1000), reverse=True)
    
    # Enable online mode to scan the whole curve
    kl = KneeLocator(x, y, curve="convex", direction="decreasing", online=True)
    
    # kl.knee returns the last (most significant) knee found in online mode
    print(kl.knee)
  4. Choose between online and offline modes

    main

    The online parameter determines how the algorithm processes the data points:

    • online=False (default): Runs in offline mode. It returns the first knee point identified (either the local maxima on the difference curve or the global maxima) and stops early.
    • online=True: Runs in online mode. The algorithm steps through each element in x and "corrects" itself by continuing to overwrite previously identified knees until it finds the optimal point.
    # Offline mode (default)
    kl_offline = KneeLocator(x, y, curve="convex", direction="decreasing", online=False)
    
    # Online mode (corrects itself as it iterates)
    kl_online = KneeLocator(x, y, curve="convex", direction="decreasing", online=True)
  5. Input data requirements for KneeLocator

    main

    When providing data to KneeLocator, ensure the following constraints are met:

    • Length: x and y must have the same length.
    • Ordering: x values must be sorted in ascending order.
    • Supported Types: Lists, tuples, NumPy arrays, and Pandas Series are all supported (they are automatically converted to NumPy arrays internally).
  6. Choose between Offline and Online mode

    main

    Select the appropriate mode based on your requirements:

    ScenarioModeImplementation
    Need just the first/most obvious kneeOffline (online=False)Default behavior
    Want to find the most significant kneeOnline (online=True)Set online=True
    Need all knee points in a bumpy curveOnline (online=True)Set online=True and read all_knees
    Performance-sensitive applicationsOffline (online=False)Stops early after first detection
  7. Detect knee points using KneeLocator

    main

    The primary way to find a knee point is to use the KneeLocator class. You provide the x and y coordinates of your curve and specify the curve type and the direction of the function.

    Key parameters for KneeLocator:

    • x: The x-coordinates of the data.
    • y: The y-coordinates of the data.
    • curve: The shape of the curve (e.g., 'concave' or 'convex').
    • direction: Whether the function is 'increasing' or 'decreasing'.

    Once initialized, you can access the detected knee coordinates via .knee (x-value) and .knee_y (y-value).

    from kneed import KneeLocator, DataGenerator
    
    x, y = DataGenerator.figure2()
    kl = KneeLocator(x, y, curve="concave", direction="increasing")
    
    print(kl.knee)    # Returns the x-coordinate of the knee
    print(kl.knee_y)  # Returns the y-coordinate of the knee
  8. Optimize KneeLocator detection with S and interp_method

    main

    When the elbow point is not clearly detected, you can tune the KneeLocator parameters:

    • S: Adjusting this parameter can help. Lower values of S tend to detect earlier elbows.
    • interp_method: For noisy data (like inertia values), setting interp_method="polynomial" can help smooth the curve to improve detection accuracy.
  9. Install kneed via pip or Anaconda

    main

    You can install kneed using standard Python package managers:

    Using pip:

    pip install kneed

    Using Anaconda:

    conda install -c conda-forge kneed

    From GitHub: You can also clone the repository directly from GitHub to use the source.

    pip install kneed
  10. Install kneed via pip, conda, or source

    main

    Install kneed using one of the following methods. Note that kneed requires Python 3.8 or later.

    • Standard installation: pip install kneed (includes only knee-detection).
    • Installation with visualization support: pip install kneed[plot] (installs matplotlib for plotting results).
    • Conda: conda install -c conda-forge kneed.
    • From source:
      git clone https://github.com/arvkevi/kneed.git && cd kneed
      pip install -e .
    pip install kneed[plot]
  11. Install kneed via pip or conda

    main

    You can install kneed using pip or conda. If you want to use the built-in plotting features, ensure you install the [plot] extra to include matplotlib.

    pip

    # Install knee-detection only
    pip install kneed
    
    # Install with matplotlib for visualizations
    pip install kneed[plot]

    conda

    conda install -c conda-forge kneed

    Clone from GitHub

    git clone https://github.com/arvkevi/kneed.git && cd kneed
    pip install -e .
    pip install kneed[plot]