auto-sklearn Documentation

auto-sklearn supports the following machine learning tasks:

• Binary Classification
• Multiclass Classification
• Multilabel Classification
• Regression
• Multioutput Regression

You can provide training pairs (X_train/y_train) to fit an ensemble of pipelines. Optionally, you can provide a testing pair (X_test/y_test) to measure generalization.

auto-sklearn is an automated machine learning toolkit designed as a drop-in replacement for scikit-learn estimators. It automates algorithm selection and hyperparameter tuning using Bayesian optimization, meta-learning, and ensemble construction. You can use AutoSklearnClassifier to fit a model to your training data and then predict on test data using a standard scikit-learn API.

import autosklearn.classification
cls = autosklearn.classification.AutoSklearnClassifier()
cls.fit(X_train, y_train)
predictions = cls.predict(X_test)

The final step in the metadata update process is to create aslib files using 04_create_aslib_files.py.

python3 04_create_aslib_files.py --working-directory $working_directory --task-type $task_type

When initializing an auto-sklearn estimator, you must set memory and time limits.

Guidelines:

• Memory: For most modern datasets, a limit of 3GB or 6GB is typically sufficient.
• Time: A good default is a total time limit of one day, with a single run limit of 30 minutes.

Note that auto-sklearn executes each machine learning algorithm in its own process to enforce these limits.

Every custom component must implement get_hyperparameter_search_space() and get_properties(). Depending on the component type, you must also implement specific execution methods following the scikit-learn predictor API:

• Classification/Regression: Implement fit() and predict().
• Feature Preprocessing: Implement fit() and transform().

Use 01_create_commands.py to generate a file containing all necessary commands to run auto-sklearn on a large number of datasets from OpenML.

To customize the process:

• Change task IDs or dataset loading methods in update_metadata_util.py.
• Alter the generated commands file on disk to change the time allocated for configuration.

python3 01_create_commands.py --working-directory $working_directory --task-type $task_type

This directory provides scripts to reproduce the results shown in Figure 3 of the paper 'Efficient and Robust Automated Machine Learning' (Feurer et al.).

Note: These scripts are specifically designed for classification tasks and use balanced accuracy as the scoring metric. They can be modified to use different datasets or runtimes.

To run the experiments defined in commands.txt, navigate to the run directory and execute the runner script.

Each command in commands.txt performs two steps:

1. Model fitting
2. Creating single best and ensemble trajectories.

Parallelization: You can run these commands in parallel on a cluster by modifying run_commands.sh to distribute the tasks.

cd run
bash run_commands.sh

To install auto-sklearn via Anaconda, you must first enable the conda-forge channel. Ensure you have conda >= 4.9 installed.

1. Enable conda-forge

conda config --add channels conda-forge
conda config --set channel_priority strict

2. Install auto-sklearn

conda install auto-sklearn

3. Troubleshooting Compiler Incompatibility

On some recent Linux distributions, the compiler used by Anaconda may conflict with the system compiler. You can resolve this by installing the Anaconda-provided GCC and SWIG:

conda install gxx_linux-64 gcc_linux-64 swig

To see available versions on your platform:

conda search auto-sklearn --channel conda-forge