DOCSEARCHbyAlgolia
Submit Docs

scikit-lego Documentation

repository·main·Indexed 23 days ago

https://github.com/koaning/scikit-lego

A collection of specialized tools, transformers, and lego bricks for scikit-learn pipelines. Version 0.9.9 provides extensions for preprocessing, feature engineering, linear models, mixture models, and model selection, including custom metrics and datasets like abalone, penguins, and credit card data. It offers components such as GMMClassifier, QuantileRegression, and various specialized splitting strategies like TimeGapSplit, all designed to be compatible with sklearn.pipeline.Pipeline.

Tokens
18.5K
Snippets
18
Records
119
Agent score
78%

What's inside scikit-lego

  1. Overview of scikit-lego features

    main

    scikit-lego extends scikit-learn with a variety of specialized tools. Key modules include:

    • Datasets: Specialized loaders like load_abalone, load_penguins, and fetch_creditcard (from OpenML), as well as generators like make_simpleseries.
    • Preprocessing: Tools for feature engineering such as RandomAdder, ColumnCapper, TypeSelector, DictMapper, and RepeatingBasisFunction (for timeseries).
    • Linear Models: Specialized regressions including QuantileRegression, LADRegression, LowessRegression, and fairness-constrained classifiers like DemographicParityClassifier.
    • Mixture Models: GMM-based classifiers and outlier detectors like GMMClassifier and BayesianGMMOutlierDetector.
    • Meta-estimators: Transformers that wrap other models, such as EstimatorTransformer (adds model output as a feature) or Thresholder (for gridsearching thresholds).
    • Model Selection: Specialized splitting strategies like TimeGapSplit and GroupTimeSeriesSplit.
    • Metrics: Fairness and correlation metrics such as equal_opportunity_score and correlation_score.
    • Pandas Utils: Utilities like add_lags for dataframes and log_step decorators for pipeline logging.

  2. Available Preprocessing Transformers in scikit-lego

    main

    The sklego.preprocessing module provides a variety of transformers designed to augment scikit-learn pipelines. These transformers include tools for column manipulation, data type selection, feature engineering, and outlier handling.

    Key categories of transformers available include:

    • Pandas-based Transformers: ColumnDropper, ColumnSelector, PandasTypeSelector, and TypeSelector for interacting with DataFrame structures.
    • Feature Engineering & Projections: InformationFilter, OrthogonalTransformer, LinearEmbedder, MonotonicSplineTransformer, RepeatingBasisFunction, and FormulaicTransformer.
    • Data Cleaning & Transformation: ColumnCapper (for capping values), OutlierRemover, DictMapper (for mapping values), RandomAdder (for adding noise), and IdentityTransformer.

  3. List of available scikit-lego datasets

    main

    Scikit-lego provides several built-in datasets for testing and educational purposes:

    • load_abalone: Predict the gender of an abalone.
    • load_arrests: Data on police treatment in Toronto, used for fairness benchmarking.
    • load_chicken: Experiment data on the effect of diet on chick growth.
    • load_heroes: Data from Heroes of the Storm to predict attack types.
    • load_hearts: Cleveland Heart Diseases dataset to predict heart disease presence.
    • load_penguins: An alternative to the iris dataset for species prediction.
    • fetch_creditcard: A highly unbalanced fraud detection dataset fetched from OpenML.
    • make_simpleseries: Generates a simulated daily timeseries with season, trend, and noise.

  4. Use Meta Models in scikit-lego

    main

    The sklego.meta module provides meta-models that wrap or extend standard scikit-learn estimators to handle specific data structures or modeling requirements. These include models for grouped data, hierarchical structures, outlier detection, and specialized regression/classification tasks.

    Available meta-models include:

    • Grouped Models: GroupedPredictor, GroupedClassifier, GroupedRegressor, and GroupedTransformer for handling data with group-based dependencies.
    • Hierarchical Models: HierarchicalPredictor, HierarchicalClassifier, and HierarchicalRegressor for multi-level or hierarchical data.
    • Outlier Handling: OutlierClassifier and RegressionOutlierDetector.
    • Specialized Regression/Classification: ConfusionBalancer, DecayEstimator, EstimatorTransformer, OrdinalClassifier, SubjectiveClassifier, Thresholder, and ZeroInflatedRegressor.

  5. Detect outliers using decomposition (PCA and UMAP)

    main

    Decomposition-based detection works by reducing data dimensionality and then attempting to reconstruct the original data. If the reconstruction error (the difference between the original and reconstructed point) is too high, the point is flagged as an outlier.

    Use PCAOutlierDetection for standard PCA-based reconstruction or UMAPOutlierDetection for UMAP-based reconstruction. Note that UMAP is significantly slower than PCA.

    Hyperparameters

    • n_components: The number of components to reduce the data to.
    • threshold: The limit for the reconstruction error. If the relative error exceeds this threshold, the point is an outlier. Typically, this is a value between 0.0 and 0.1.
    • absolute: A boolean flag to specify if the threshold should be treated as an absolute value rather than a relative error.

  6. Configure Outlier Detection Thresholds

    main

    The outlier detection methods in scikit-lego determine if a point is an outlier based on the likelihood scores from the GMM. You can choose between two thresholding methods:

    1. quantile method: The threshold is a value between 0 and 1. The threshold is determined by looking at the likelihood scores associated with the training dataset.
    2. stddev method: The threshold is interpreted as the number of standard deviations below the mean. The standard deviation is calculated only on the lower scores to account for higher variance in that region. This method can be more exclusive (pickier) than the quantile method.

  7. Use OutlierClassifier to treat anomaly detection as classification

    main

    The OutlierClassifier converts an unsupervised outlier/anomaly detection model into a supervised classification model. This is useful when you have some labeled outlier samples but want to leverage the power of unsupervised anomaly detection.

    Key Behaviors

    • Label Mapping: Anomaly detection algorithms that return -1 for inliers and 1 for outliers are mapped to 0 (inlier) and 1 (outlier) by the OutlierClassifier.
    • Probabilities: The .predict_proba() method returns probabilities for both classes (inlier, outlier).
    • Integration: Because it follows the classifier API, it can be used within a scikit-learn StackingClassifier alongside standard supervised models.

  8. Configure scikit-lego dataset return formats

    main

    When using scikit-lego dataset loaders, you can control the format of the returned data using two specific arguments:

    • as_frame=True: Returns the data, including the target, as a pandas DataFrame.
    • return_X_y=True: Returns the data directly as a tuple of (data, target) instead of a dictionary object.

  9. How Grouped Prediction works

    main

    The GroupedPredictor (and its specialized versions GroupedClassifier and GroupedRegressor) allows you to split your data into groups and train separate models for each group.

    Key features:

    • Automatic Fallback: By default, it trains a 'global model' to act as a fallback for groups that were not seen during training. You can disable this by setting use_global_model=False.
    • Task Specialization:
      • GroupedClassifier: Specialized for classification tasks.
      • GroupedRegressor: Specialized for regression tasks.
    • Input Types: Can work with both pandas DataFrames (using column names for grouping) and numpy arrays.

    This is useful when a single global model (like a linear regression with dummy variables) fails to capture group-specific gradients or intercepts effectively.

  10. Use decay functions with DecayEstimator

    main

    Decay functions are used within a DecayEstimator to generate sample weights for a wrapped model. These functions determine how the importance of samples changes (decays) over time or according to a specific metric.

    Available decay functions include:

    • exponential_decay
    • linear_decay
    • sigmoid_decay
    • stepwise_decay

  11. Convert an Estimator into a Transformer using EstimatorTransformer

    main
    In scikit-learn, pipelines typically only allow a single model at the end. If you want the output of a model (e.g., predictions) to be used as a feature for a subsequent model, you can wrap the estimator in an EstimatorTransformer from the meta module. This allows you to create pipelines where multiple models interact, either by seeing the same dataset or by seeing different subsets of data.

  12. Use Maximum Relevance Minimum Redundancy (MRMR) for feature selection

    main

    The MaximumRelevanceMinimumRedundancy (MRMR) method is an iterative feature selection technique introduced in version 0.8.0. It selects a subset of features that have high relevance to the target variable while minimizing redundancy among the selected features.

    By default, the implementation uses functions optimized for general problems (often f_classif or f_regression for relevance and Pearson correlation for redundancy), but it allows you to define custom relevance and redundancy functions to suit specific data characteristics or goals.