sklearn-pandas Documentation

You can install sklearn-pandas using either pip or conda via the conda-forge channel.

# pip install sklearn-pandas

# conda install -c conda-forge sklearn-pandas

To have fit_transform return a pandas DataFrame instead of a NumPy array, set df_out=True when initializing the DataFrameMapper. The resulting DataFrame will use the names found in transformed_names_ as column headers.

Note: df_out=True is incompatible with default=True or sparse=True.

mapper_df = DataFrameMapper([
    ('pet', sklearn.preprocessing.LabelBinarizer()),
    (['children'], sklearn.preprocessing.StandardScaler())
], df_out=True)

By default, transformers receive NumPy arrays. To pass pandas DataFrames or Series directly to the transformers (useful for methods like .dt accessors), set input_df=True in the DataFrameMapper constructor or within a specific column's attribute dictionary.

# Set globally for the whole mapper
mapper_dates = DataFrameMapper([
    ('dates', DateEncoder())
], input_df=True)

# Set per group of columns
mapper_dates = DataFrameMapper([
    ('dates', DateEncoder(), {'input_df': True})
])

DataFrameMapper supports transformers that require both X and y (target) arguments, such as scikit-learn's feature selection tools. When calling fit_transform, pass the target variable as the second argument.

from sklearn.feature_selection import SelectKBest, chi2
mapper_fs = DataFrameMapper([(['children','salary'], SelectKBest(chi2, k=1))])
# Pass target 'pet' as the second argument
mapper_fs.fit_transform(data[['children','salary']], data['pet'])

To include a column in the output without applying any transformation, use None as the transformer in the mapping tuple.

mapper3 = DataFrameMapper([
    ('pet', sklearn.preprocessing.LabelBinarizer()),
    ('children', None)
])

You can control the names of the output features by providing a dictionary as the third element in the transformation tuple. Supported keys include:

• alias: Replaces the generated name with a specific string.
• prefix: Prepends a string to the generated name.
• suffix: Appends a string to the generated name.

You can specify a default transformer to be applied to all columns not explicitly listed in the mapping.

• default=Transformer(): Applies the transformer to unselected columns.
• default=None: Passes unselected columns through unchanged.
• default=False (default behavior): Drops all unselected columns.

mapper4 = DataFrameMapper([
    ('pet', sklearn.preprocessing.LabelBinarizer()),
    ('children', None)
], default=sklearn.preprocessing.StandardScaler())

To apply a single transformation to a group of columns (e.g., PCA on multiple features), pass a list of column names as the first element of the tuple.

mapper2 = DataFrameMapper([
    (['children', 'salary'], sklearn.decomposition.PCA(1))
])

If column names are unknown beforehand, you can use a custom callable or scikit-learn's make_column_selector to select columns during the fit operation.

# Using make_column_selector
mapper = DataFrameMapper([
    (sklearn.compose.make_column_selector(dtype_include=float), sklearn.preprocessing.StandardScaler())
])

# Using a custom callable
class GetColumnsStartingWith:
    def __init__(self, start_str):
        self.pattern = start_str
    def __call__(self, X:pd.DataFrame=None):
        return [c for c in X.columns if c.startswith(self.pattern)]

mapper = DataFrameMapper([
    (GetColumnsStartingWith('petal'), None)
])

The DataFrameMapper takes a list of tuples to map columns to transformations. Each tuple follows this structure:

1. column name(s): A string (single column), a list of strings (multiple columns), or a callable (e.g., make_column_selector).
2. transformer(s): An object or list of objects to perform the transformation.
3. attributes (optional): A dictionary for transformation options, such as alias, prefix, or suffix.

Note on shapes:

• Specifying a column as a string 'column' passes a 1D array to the transformer.
• Specifying a column as a list ['column'] passes a 2D array (column vector) to the transformer. This is important for transformers like OneHotEncoder or Imputer that expect 2D input.

mapper = DataFrameMapper([
    ('pet', sklearn.preprocessing.LabelBinarizer()),
    (['children'], sklearn.preprocessing.StandardScaler())
])

To map pandas DataFrame columns to different scikit-learn transformations, import the DataFrameMapper class from the sklearn_pandas package.

from sklearn_pandas import DataFrameMapper