TabPFN Documentation

Follow these best practices for efficient TabPFN usage:

• Batch Prediction: Always use batch prediction. Calling .predict() on individual samples is significantly slower than calling it once on a large array. If the test set is too large, split it into chunks (e.g., 1000 samples each).
• Avoid Preprocessing: Do not apply data scaling or one-hot encoding before feeding data to the model.
• Dataset Size Limits: TabPFN-3 supports up to:
  • 1,000,000 rows × 200 features
  • 100,000 rows × 2,000 features
  • 1,000 rows × 20,000 features (Note: Higher feature counts reduce the supported row capacity).

Since FA3 is not available on PyPI, you must build it from source. To perform an in-place installation directly on a Hopper-class machine, clone the repository and run the setup script.

Verification: After installation, verify the package is working by attempting to import flash_attn_func from flash_attn_interface.

git clone https://github.com/Dao-AILab/flash-attention.git
cd flash-attention/hopper
python setup.py install

from flash_attn_interface import flash_attn_func  # noqa: F401

Install the core TabPFN package using pip. TabPFN requires Python 3.10 or higher. The installation automatically includes a compatible PyTorch build.

Note for Apple Silicon/MPS users: For optimal performance with flash attention, consider installing a PyTorch version later than 2.13.0.dev20260510 to avoid GPU-CPU-GPU roundtrips required by MLX.

pip install tabpfn

TabPFN v3 can dispatch attention operations to the FlashAttention-3 (FA3) backend instead of PyTorch's SDPA for improved performance on Hopper-class GPUs (H100, H200).

FA3 is automatically used only when all of the following conditions are met:

• The flash_attn_interface Python package is installed and importable.
• The attention is performed on a CUDA tensor on a Hopper-class device (compute capability 9.0+).
• The data type is torch.float16 or torch.bfloat16.
• The head dimension is one of {64, 96, 128, 192, 256}.
• The sequence length (max(seq_q, seq_kv)) is greater than or equal to the threshold _FA3_MIN_SEQLEN_FOR_SPEEDUP (currently set to 10_000).

Use TabPFNClassifier for classification tasks and TabPFNRegressor for regression tasks. The default model is TabPFN-3. Note that the model checkpoint is downloaded automatically upon the first call to .fit().

Hardware Recommendations:

• GPU: Highly recommended. Even older GPUs with ~8GB VRAM work well. 16GB is recommended for larger datasets.
• CPU: Only feasible for small datasets (≲1000 samples).

from tabpfn import TabPFNClassifier, TabPFNRegressor

# Classification
clf = TabPFNClassifier()
clf.fit(X_train, y_train)  # downloads checkpoint on first use
predictions = clf.predict(X_test)

# Regression
reg = TabPFNRegressor()
reg.fit(X_train, y_train)  # downloads checkpoint on first use
predictions = reg.predict(X_test)

Install tabpfn-extensions to access additional tools including:

• interpretability: SHAP-based explanations and feature importance.
• unsupervised: Outlier detection and synthetic data generation.
• embeddings: Extraction of internal learned embeddings.
• many_class: Handling multi-class problems exceeding built-in class limits.

pip install tabpfn-extensions

To persist a fitted estimator (including its state) and reload it later, use save_fitted_tabpfn_model and load_fitted_tabpfn_model from tabpfn.model_loading.

To store only the foundation model weights (without the fitted state) to create fresh estimators later, use save_tabpfn_model and reload with load_model_criterion_config.

from tabpfn import TabPFNRegressor
from tabpfn.model_loading import (
    load_fitted_tabpfn_model,
    save_fitted_tabpfn_model,
)

# Train the regressor on GPU
reg = TabPFNRegressor(device="cuda")
reg.fit(X_train, y_train)
save_fitted_tabpfn_model(reg, "my_reg.tabpfn_fit")

# Later or on a CPU-only machine
reg_cpu = load_fitted_tabpfn_model("my_reg.tabpfn_fit", device="cpu")

To use model versions other than the default TabPFN-3 (such as TabPFN-2.6 or the original TabPFN-2), use the create_default_for_version method with a ModelVersion constant.

Note on Licensing: TabPFN-3 and prior releases (2.5, 2.6) use non-commercial licenses. The code and TabPFN-2 weights are under the Prior Labs License (Apache 2.0 with additional attribution).

from tabpfn import TabPFNClassifier, TabPFNRegressor
from tabpfn.constants import ModelVersion

# Use TabPFN-2.6
classifier = TabPFNClassifier.create_default_for_version(ModelVersion.V2_6)
regressor = TabPFNRegressor.create_default_for_version(ModelVersion.V2_6)

# Use TabPFN-2 (Apache 2.0 compatible weights)
tabpfn_v2 = TabPFNRegressor.create_default_for_version(ModelVersion.V2)

Use PreprocessorConfig to define the settings for your preprocessing pipeline. For specific model versions, you can use the following preset configurations:

• v2_5_classifier_preprocessor_configs (Classifier)
• v2_5_regressor_preprocessor_configs (Regressor)
• v2_classifier_preprocessor_configs (Classifier)
• v2_regressor_preprocessor_configs (Regressor)