DOCSEARCHbyAlgolia
Submit Docs

DeepTab

repository·main·Indexed 43 days ago

https://github.com/opentabular/deeptab

A scikit-learn compatible model zoo for tabular deep learning built on PyTorch and Lightning. Version 2.0.0 provides a wide variety of neural architectures, including Transformers, State Space Models (Mambular), and tree-inspired models, with automatic preprocessing via PreTab and support for classification, regression, and distributional LSS tasks.

Tokens
112.8K
Snippets
271
Records
456
Agent score
54%

What's inside DeepTab

  1. Overview of DeepTab features and architecture

    main

    DeepTab is a PyTorch and Lightning-based library designed for tabular deep learning. Key features include:

    • Scikit-learn API: Uses fit/predict/evaluate workflows, making it compatible with tools like GridSearchCV.
    • Automatic Preprocessing: Powered by PreTab, it handles feature-type detection, encoding, scaling, and missing values.
    • Three Task Variants: Every architecture is available as a *Classifier, a *Regressor, or a distributional *LSS model (for uncertainty quantification).
    • Split-config API (v2.0+): Allows independent tuning of the model, preprocessing, and training via separate configuration objects.
    • Deployment-ready: Includes InferenceModel for safe, read-only prediction and a .deeptab file format that bundles weights with the feature schema and preprocessing requirements.

  2. What is DeepTab?

    main

    DeepTab is a library for deep learning on tabular data that provides a scikit-learn compatible interface. It automates feature preprocessing (encoding, scaling, and imputation), batching, and the training loop, allowing users to follow a standard fit, predict, and evaluate workflow instead of writing manual PyTorch code.

    DeepTab provides 15 stable neural architectures across several families:

    • State Space Models: Mambular, MambaTab, MambAttention
    • Transformers: FTTransformer, TabTransformer, SAINT, AutoInt
    • Residual networks: ResNet, TabR
    • Tree-inspired: NODE, ENODE, NDTF
    • General baselines: MLP, TabM, TabulaRNN
    • Experimental models: ModernNCA, Tangos, Trompt
    from deeptab.models import FTTransformerClassifier

model = FTTransformerClassifier()
model.fit(X_train, y_train, max_epochs=100)
predictions = model.predict(X_test)
metrics = model.evaluate(X_test, y_test)

  3. Overview of DeepTab Core Concepts

    main
    DeepTab is designed around a scikit-learn compatible interface, making it easy to integrate into existing machine learning workflows. The core architecture relies on a split-configuration system to manage model, preprocessing, and training parameters separately. The lifecycle of a DeepTab project involves a structured training and evaluation pipeline, integrated observability (logging and experiment tracking), and a specialized InferenceModel for deployment-safe predictions.

  4. Use MLP models for tabular baselines

    main

    The Multi-Layer Perceptron (MLP) is a feed-forward baseline model in DeepTab. It is designed for speed and ease of tuning, making it an ideal control model before attempting more complex architectures like attention or Mamba.

    Available classes in deeptab.models:

    • MLPClassifier: For classification tasks.
    • MLPRegressor: For regression tasks.
    • MLPLSS: For Least Squares implementations.
    from deeptab.models import MLPClassifier, MLPRegressor, MLPLSS

  5. Use ResNet models for tabular data

    main

    ResNet is a residual feed-forward architecture designed for tabular data. It provides a stronger inductive bias than a standard MLP by using residual blocks, making deeper nonlinear transformations easier to optimize while remaining more computationally efficient than attention-based or retrieval-based models.

    Available model classes in deeptab.models:

    • ResNetClassifier: For classification tasks.
    • ResNetRegressor: For regression tasks.
    • ResNetLSS: For specific LSS-related tasks.
    from deeptab.models import ResNetRegressor

# Example initialization
model = ResNetRegressor(
    model_config=ResNetConfig(
        layer_sizes=[256, 128, 64],
        num_blocks=3,
        dropout=0.2,
        norm=True,
    ),
    # ... other configs
)

  6. Use SAINT models for tabular deep learning

    main

    SAINT (Self-Attention and Interaction Network for Tabular data) is an attention-based architecture that combines feature-wise (column) attention with row-wise attention. In DeepTab, it is available as three distinct classes in deeptab.models depending on your task: SAINTClassifier for classification, SAINTRegressor for regression, and SAINTLSS for semi-supervised learning.

    The architecture follows this flow: feature tokens -> RowColTransformer -> pooling -> optional norm -> MLPhead.

    Use SAINT when you need a Transformer-style model capable of mixing information across both columns (features) and samples (rows), particularly for research comparisons with models like FTTransformer or TabTransformer.

    from deeptab.models import SAINTClassifier, SAINTRegressor, SAINTLSS

  7. Explore the DeepTab API structure

    main

    The DeepTab API is organized into several functional modules. Depending on your goal, you should refer to the following modules:

    • Models: Use this to find Classifier, Regressor, and LSS (Location, Scale, Shape) classes for building your predictive models.
    • Configs: Access configuration dataclasses to customize model hyperparameters and training settings.
    • Data: Use this module for handling Datasets, DataModules, and defining Schemas for tabular data.
    • Distributions: Explore LSS distribution families if you are working with probabilistic modeling.
    • Metrics: Find Task-aware metric classes to evaluate model performance.
    • Training: Access underlying Lightning modules if you need to implement advanced, custom training logic using PyTorch Lightning.

  8. Available DeepTab Model Families

    main

    DeepTab provides a wide variety of deep learning models for tabular data, categorized by their underlying architecture. Models are typically available as Classifier, Regressor, or LSS (Likelihood Surface Sampling/probabilistic) variants.

    Model Categories:

    • State Space Models: Includes Mambular, MambaTab, and MambAttention architectures.
    • Transformer-Based Models: Includes FTTransformer, TabTransformer, and SAINT.
    • MLP-Based Models: Includes standard MLP, ResNet, TabM, and AutoInt.
    • Tree-Based Models: Includes NODE, ENODE, and NDTF.
    • Specialized Models: Includes TabR and TabulaRNN.
    • Experimental Models: Located in deeptab.models.experimental.

  9. Explore the DeepTab Model Zoo

    main

    DeepTab provides a collection of 15 stable, production-ready architectures and several experimental models. All models in the zoo share a consistent API, allowing you to switch between different architectures using the same fit, predict, and evaluate methods.

    To select the right model for your task, you can navigate through the following categories:

    • Stable Models: Production-ready architectures.
    • Experimental Models: Newer architectures currently under evaluation.
    • Comparison Tables: Side-by-side comparisons to help with model selection.
    • Efficiency: Detailed information on runtime and memory characteristics.
    • Recommended Configs: Pre-defined, sensible starting configurations for each model.

  10. Use TabR models for tabular deep learning

    main

    TabR is a retrieval-augmented tabular model that uses nearest-neighbor information to improve predictions. It encodes input features and retrieves the most similar candidate rows from the training set using FAISS to provide context for the prediction head.

    Available model classes in deeptab.models:

    • TabRClassifier: For classification tasks.
    • TabRRegressor: For regression tasks.
    • TabRLSS: For Least Squares Support Vector Machine style tasks.
    from deeptab.models import TabRRegressor

  11. What is InferenceModel and when to use it

    main

    An InferenceModel is a deployment-only wrapper for a fitted DeepTab artifact. It is designed to provide a strict, minimal, and read-only surface for production environments, preventing accidental model mutation or retraining.

    Key Differences from standard Estimators

    FeatureStandard EstimatorInferenceModel
    Interface SurfaceFull API (fit, optimize_hparams, etc.)Minimal API (predict, validate_input, etc.)
    Schema ValidationChecks count/name; requires exact column orderChecks missing/extra columns; automatically re-orders columns to match training order
    Error HandlingGeneric sklearn-style messagesSpecific error messages identifying exact missing columns
    Extra ColumnsRaises errorConfigurable: raises by default or drops with warning via allow_extra_columns=True
    Task SafetySingle predict() methodTask-aware: predict_proba() or predict_params() raise TypeError if called on the wrong task type

    When to use which

    • Use standard Estimators for research, notebook exploration, hyper-parameter optimization, and retraining.
    • Use InferenceModel for production services, pipeline steps, or batch jobs where the model must remain immutable.
    from deeptab import InferenceModel

# Load a model for deployment
model = InferenceModel.from_path("model.deeptab")

# This will fail because training methods are not exposed on the wrapper
try:
    model.fit(X, y)
except AttributeError:
    print("Training methods are not exposed on InferenceModel")

  12. Overview of experimental model families

    main

    The experimental tier currently includes three model families, each available in Classifier, Regressor, and LSS (distributional) variants. Each family requires its own specific configuration class.

    ModelCore ideaConfig classPrimary controls
    TromptPrompt-style aggregation using learnable prototype records.TromptConfign_cycles, P, d_model
    ModernNCADifferentiable nearest-neighbour model using temperature-weighted candidate averages.ModernNCAConfigdim, n_blocks, temperature, sample_rate
    TangosMLP with a gradient-attribution regularizer for better generalization.TangosConfiglayer_sizes, lamda1, lamda2