DOCSEARCHbyAlgolia
Submit Docs

BorutaPy

repository·master·Indexed 23 days ago

https://github.com/scikit-learn-contrib/boruta_py

A Python implementation of the Boruta all-relevant feature selection method. Following the scikit-learn API, BorutaPy identifies all features that carry useful information for prediction rather than finding a minimal-optimal subset. It requires a supervised learning estimator with a feature_importances_ attribute and accepts numpy arrays for its fit and transform methods.

Tokens
1.3K
Snippets
3
Records
7
Agent score
32%

What's inside boruta_py

  2. How to use BorutaPy

    master

    BorutaPy follows the scikit-learn API. Once initialized, you can use the following methods to perform feature selection:

    • fit(X, y): Runs the feature selection algorithm.
    • transform(X): Filters the input data to include only the selected features.
    • fit_transform(X, y): Performs both fitting and transformation in one step.

    Important Note: BorutaPy accepts numpy arrays only. If you are using pandas DataFrames, you must use the .values attribute to pass the underlying numpy array.

    import pandas as pd
from sklearn.ensemble import RandomForestClassifier
from boruta import BorutaPy

# load X and y
# NOTE BorutaPy accepts numpy arrays only, hence the .values attribute
X = pd.read_csv('examples/test_X.csv', index_col=0).values
y = pd.read_csv('examples/test_y.csv', header=None, index_col=0).values
y = y.ravel()

# define random forest classifier, with utilising all cores and
# sampling in proportion to y labels
rf = RandomForestClassifier(n_jobs=-1, class_weight='balanced', max_depth=5)

# define Boruta feature selection method
feat_selector = BorutaPy(rf, n_estimators='auto', verbose=2, random_state=1)

# find all relevant features - 5 features should be selected
feat_selector.fit(X, y)

# check selected features - first 5 features are selected
feat_selector.support_

# check ranking of features
feat_selector.ranking_

# call transform() on X to filter it down to selected features
X_filtered = feat_selector.transform(X)

  4. Use BorutaPy for feature selection

    master

    BorutaPy follows the scikit-learn API. To use it, you must first instantiate a base estimator (e.g., RandomForestClassifier) and then pass that estimator to BorutaPy.

    Note: BorutaPy accepts numpy arrays. If you are using pandas DataFrames, you must use the .values attribute to pass the underlying numpy array to the .fit() method.

    Key steps:

    1. Initialize a base estimator.
    2. Initialize BorutaPy with the estimator.
    3. Call .fit(X, y) where X and y are numpy arrays.
    4. Access .support_ to get a boolean mask of selected features or .transform(X) to return the reduced feature set.

  5. Access BorutaPy attributes

    master

    After calling .fit(X, y), you can inspect the results using these attributes:

    • n_features_ (int): The number of selected features.
    • support_ (array): A boolean mask of selected (confirmed) features.
    • support_weak_ (array): A boolean mask of tentative features that didn't gain enough support during max_iter.
    • ranking_ (array): Feature rankings where rank 1 is selected/best and rank 2 is tentative.

  6. Configure BorutaPy parameters

    master

    When initializing BorutaPy(estimator, ...)

    ParameterTypeDefaultDescription
    estimatorobjectRequiredA supervised learning estimator with a fit method that returns a feature_importances_ attribute.
    n_estimatorsint or str'auto'Number of estimators in the ensemble. If 'auto', it is determined based on dataset size.
    percint100Percentile of shadow feature importances used as the threshold. Lowering this makes the selection less stringent (more false positives, fewer false negatives).
    alphafloat0.05Level at which corrected p-values are rejected.
    two_stepBooleanTrueIf False, uses the original Boruta implementation with only Bonferroni correction. If True, uses a two-step process (Benjamini Hochberg FDR followed by Bonferroni).
    max_iterint100Maximum number of iterations to perform.
    verboseint0Controls the verbosity of the output.

  7. Interrogate BorutaPy selection results

    master

    After calling .fit(), you can inspect the results using the following attributes:

    • support_: A boolean array indicating which features were confirmed as relevant.
    • ranking_: An array indicating the ranking of features. Features with a ranking of 1 are the most relevant.
    • transform(X): A method to return the feature matrix containing only the selected features.