repository·master·Indexed 27 days ago
https://github.com/scikit-learn-contrib/imbalanced-learnA Python toolbox providing re-sampling techniques for datasets with strong class imbalance. Part of the scikit-learn-contrib projects and fully compatible with scikit-learn, it includes tools for over-sampling, under-sampling, and combination methods like SMOTETomek and SMOTEENN. It also provides ensemble methods such as BalancedBaggingClassifier, BalancedRandomForestClassifier, and RUSBoostClassifier, as well as utilities for fetching imbalanced benchmark datasets.
imblearn) is an open source, MIT-licensed library designed for handling classification tasks with imbalanced classes. It is built to work seamlessly with scikit-learn (imported as sklearn).Under-sampling reduces the number of observations from the majority classes to balance the dataset. The imblearn.under_sampling module categorizes these algorithms into several strategies:
ClusterCentroids) to represent clusters.The numpydoc package provides several Sphinx extensions to support the Numpy docstring format and related features.
Available extensions:
numpydoc: Provides support for the Numpy docstring format in Sphinx and includes code description directives like np-function and np-cfunction.numpydoc.traitsdoc: Used for gathering documentation about Traits attributes.numpydoc.plot_directives: An adaptation of Matplotlib's plot:: directive (note: this implementation may undergo changes or be deprecated).numpydoc.only_directives: (DEPRECATED)numpydoc.autosummary: (DEPRECATED) An autosummary:: directive. It is recommended to use sphinx.ext.autosummary available in Sphinx 1.0+ instead.imblearn.under_sampling module provides various algorithms to reduce the number of samples in a majority class to balance a dataset. These methods are categorized into prototype generation and prototype selection.The InstanceHardnessCV splitter in imblearn.model_selection is designed to make model selection tasks (like hyperparameter tuning or feature selection) more robust by distributing samples with high 'instance hardness' equally across cross-validation folds.
Instance Hardness is defined as $1 - P(\hat{y}|x)$, where $P(\hat{y}|x)$ is the probability of the most probable class. Samples with high hardness are difficult to classify and can significantly impact metrics like average precision. Randomly grouping these samples in specific folds can introduce high variance in CV results.
When to use:
Both SMOTE and ADASYN generate new samples using interpolation between a sample $x_i$ and one of its $k$ nearest neighbors ($x_{zi}$), where $k$ is determined by the k_neighbors parameter. The formula used is:
$x_{new} = x_i + \lambda \times (x_{zi} - x_i)$
where $\lambda$ is a random number in the range $[0, 1]$. This creates a new sample on the line segment connecting the two points.
SMOTE-NC differs by handling categorical features specifically: the categories for a new sample are determined by picking the most frequent category among the nearest neighbors.
Warning: SMOTE-NC is not designed to work with exclusively categorical data.
imbalanced-learn relies on scikit-learn algorithms, you can activate Intel optimizations for Intel hardware by installing scikit-learn-intelex and patching scikit-learn. Refer to the official Intel documentation for specific installation and patching instructions.A common pitfall in imbalanced learning is resampling the entire dataset before splitting it into training and testing sets. This causes data leakage, leading to over-optimistic performance reports.
Data leakage occurs because:
Incorrect Pattern:
Applying a sampler (like RandomUnderSampler) to the full X and y before performing cross-validation or splitting.
Correct Pattern:
Use imblearn.pipeline.Pipeline (or make_pipeline) to wrap your sampler and classifier. This ensures that resampling is only applied to the training folds during cross-validation, preventing information from the validation/test folds from leaking into the training process.
from imblearn.pipeline import make_pipeline
from imblearn.under_sampling import RandomUnderSampler
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.model_selection import cross_validate
# The correct way to use a sampler with cross-validation
model = make_pipeline(
RandomUnderSampler(random_state=0),
HistGradientBoostingClassifier(random_state=0)
)
cv_results = cross_validate(
model, X, y, scoring="balanced_accuracy",
return_train_score=True, return_estimator=True,
n_jobs=-1
)imblearn.pipeline module provides a pipeline object that is compatible with scikit-learn but specifically designed to handle samplers (like SMOTE) during the training process. Unlike a standard scikit-learn Pipeline, an imblearn.pipeline.Pipeline ensures that resampling techniques are only applied to the training data and not to the validation/test data during fit or predict calls, preventing data leakage.imbalanced-learn version 0.3 and later, you can turn off specific steps within a pipeline.Pipeline by passing the None object for that step.To address noise generated by over-sampling methods like SMOTE, you can use combination methods that apply cleaning (under-sampling) after over-sampling. imbalanced-learn provides two primary classes for this:
SMOTETomek: Combines SMOTE with Tomek's links to clean the decision boundary.SMOTEENN: Combines SMOTE with Edited Nearest Neighbours (ENN). This method tends to clean more noisy samples than SMOTETomek.Both classes follow the standard fit_resample API and accept parameters identical to the underlying samplers they wrap.
from collections import Counter
from sklearn.datasets import make_classification
from imblearn.combine import SMOTEENN
from imblearn.combine import SMOTETomek
# Generate an imbalanced dataset
X, y = make_classification(n_samples=5000, n_features=2, n_informative=2,
n_redundant=0, n_repeated=0, n_classes=3,
n_clusters_per_class=1,
weights=[0.01, 0.05, 0.94],
class_sep=0.8, random_state=0)
# Use SMOTEENN
smote_enn = SMOTEENN(random_state=0)
X_resampled_enn, y_resampled_enn = smote_enn.fit_resample(X, y)
# Use SMOTETomek
smote_tomek = SMOTETomek(random_state=0)
X_resampled_tomek, y_resampled_tomek = smote_tomek.fit_resample(X, y)