hyperopt-sklearn

repository·master·Indexed 23 days ago

https://github.com/hyperopt/hyperopt-sklearn

A library providing the HyperoptEstimator class for automated hyperparameter optimization and model selection for scikit-learn algorithms. It integrates with hyperopt to search through spaces of classifiers, regressors, and preprocessing steps, offering specialized search spaces such as any_classifier, any_regressor, and any_preprocessing.

Tokens: 2.5K
Snippets: 6
Records: 6
Agent score: 32%

What's inside hyperopt-sklearn

Modify the default search space for components

master

Each component (like svc or sgd_classifier) comes with a default search space. You can override these defaults or hold parameters constant by passing keyword arguments to the component function. These arguments should be hyperopt search space objects (e.g., using hp.pchoice or hp.loguniform) or constant values.

from hpsklearn import HyperoptEstimator, sgd_classifier
from hyperopt import hp
import numpy as np

# Hold penalty constant
sgd_penalty = "l2"
# Modify search space for loss and alpha
sgd_loss = hp.pchoice("loss", [(0.50, "hinge"), (0.25, "log"), (0.25, "huber")])
sgd_alpha = hp.loguniform("alpha", low=np.log(1e-5), high=np.log(1))

if __name__ == "__main__":
    estim = HyperoptEstimator(classifier=sgd_classifier("my_sgd", penalty=sgd_penalty, loss=sgd_loss, alpha=sgd_alpha))
    estim.fit(X_train, y_train)

Install hyperopt-sklearn

master

You can install hyperopt-sklearn from PyPI using pip. You can also install specific versions, branches, or commits directly from the GitHub repository.

# Standard installation
pip install hyperopt-sklearn

# Install specific tag, branch, or commit
pip install git+https://github.com/hyperopt/hyperopt-sklearn@1.0.3
pip install git+https://github.com/hyperopt/hyperopt-sklearn@master
pip install git+https://github.com/hyperopt/hyperopt-sklearn@fd718c44fc440bd6e2718ec1442b1af58cafcb18

Use HyperoptEstimator for automated model selection

master

The HyperoptEstimator class allows you to perform automated hyperparameter optimization and model selection for scikit-learn algorithms. It integrates with hyperopt to search through a space of classifiers/regressors and preprocessing steps.

To use it, instantiate HyperoptEstimator with a classifier or regressor component and optionally a preprocessing list. You can specify the optimization algorithm using algo (e.g., tpe.suggest) and set the number of evaluations with max_evals.

from hpsklearn import HyperoptEstimator, any_classifier, any_preprocessing
from sklearn.datasets import load_iris
from hyperopt import tpe
import numpy as np

# Load data
iris = load_iris()
X, y = iris.data, iris.target
# ... (split data into X_train, y_train, X_test, y_test) ...

if __name__ == "__main__":
    # Instantiate with search space and number of evaluations
    estim = HyperoptEstimator(classifier=any_classifier("my_clf"),
                              preprocessing=any_preprocessing("my_pre"),
                              algo=tpe.suggest,
                              max_evals=100,
                              trial_timeout=120)
    
    # Search the hyperparameter space
    estim.fit(X_train, y_train)
    
    # Show results
    print(estim.score(X_test, y_test))
    print(estim.best_model())

Reference: Available Regressors

master

The following functions are available for defining the regressor parameter in HyperoptEstimator. For a generic search space across many regressors, use any_regressor. For sparse data, use any_sparse_regressor. For a complete search space across all possible regressors, use all_regressors.

random_forest_regressor
extra_trees_regressor
bagging_regressor
isolation_forest
ada_boost_regressor
gradient_boosting_regressor
hist_gradient_boosting_regressor

linear_regression
bayesian_ridge
ard_regression
lars
lasso_lars
lars_cv
lasso_lars_cv
lasso_lars_ic
lasso
elastic_net
lasso_cv
elastic_net_cv
multi_task_lasso
multi_task_elastic_net
multi_task_lasso_cv
multi_task_elastic_net_cv
poisson_regressor
gamma_regressor
tweedie_regressor
huber_regressor
sgd_regressor
ridge
ridge_cv
logistic_regression
logistic_regression_cv
orthogonal_matching_pursuit
orthogonal_matching_pursuit_cv
passive_aggressive_regressor
quantile_regression
ransac_regression
theil_sen_regressor

dummy_regressor

gaussian_process_regressor

mlp_regressor

cca
pls_canonical
pls_regression

linear_svr
nu_svr
one_class_svm
svr

decision_tree_regressor
extra_tree_regressor

transformed_target_regressor

hp_sklearn_kernel_ridge

bayesian_gaussian_mixture
gaussian_mixture

k_neighbors_regressor
radius_neighbors_regressor

k_means
mini_batch_k_means

xgboost_regression

lightgbm_regression

Reference: Available Preprocessing
master
The following functions are available for defining the preprocessing parameter in HyperoptEstimator. The preprocessing parameter expects a list of steps.
- Generic search: any_preprocessing (returns a list).
- Sparse data: any_sparse_preprocessing.
- Text data: any_text_preprocessing (currently uses TFIDF).
- All possible: all_preprocessing.
- For specific components (e.g., pca), you must wrap them in a list: preprocessing=[pca()].
- To disable preprocessing, pass an empty list: preprocessing=[].
```
binarizer
min_max_scaler
max_abs_scaler
normalizer
robust_scaler
standard_scaler
quantile_transformer
power_transformer
one_hot_encoder
ordinal_encoder
polynomial_features
spline_transformer
k_bins_discretizer

tfidf_vectorizer
hashing_vectorizer
count_vectorizer

pca

ts_lagselector

colkmeans
```
https://github.com/hyperopt/hyperopt-sklearn/blob/master/README.md

Reference: Available Classifiers

master

The following functions are available for defining the classifier parameter in HyperoptEstimator. For a generic search space across many classifiers, use any_classifier. For sparse data, use any_sparse_classifier. For a complete search space across all possible classifiers, use all_classifiers.

random_forest_classifier
extra_trees_classifier
bagging_classifier
ada_boost_classifier
gradient_boosting_classifier
hist_gradient_boosting_classifier

bernoulli_nb
categorical_nb
complement_nb
gaussian_nb
multinomial_nb

sgd_classifier
sgd_one_class_svm
ridge_classifier
ridge_classifier_cv
passive_aggressive_classifier
perceptron

dummy_classifier

gaussian_process_classifier

mlp_classifier

linear_svc
nu_svc
svc

decision_tree_classifier
extra_tree_classifier

label_propagation
label_spreading

elliptic_envelope

linear_discriminant_analysis
quadratic_discriminant_analysis

bayesian_gaussian_mixture
gaussian_mixture

k_neighbors_classifier
radius_neighbors_classifier
nearrest_centroid

xgboost_classification
lightgbm_classification

one_vs_rest
one_vs_one
output_code