NVTabular Documentation

NVIDIA Merlin provides highly optimized dataloaders designed to overcome the unique challenges of recommendation system datasets, which are often terabytes in size with billions of examples. Standard PyTorch and TensorFlow dataloaders can be slow due to random sampling; Merlin's dataloaders can speed up TensorFlow pipelines by up to 9x.

For extreme performance and large-scale embedding tables, Merlin offers three accelerated dataloading paths:

1. TensorFlow: Optimized dataloaders for TensorFlow-based pipelines.
2. PyTorch: Optimized dataloaders for PyTorch-based pipelines.
3. HugeCTR: A dedicated deep learning framework for recommender systems that can achieve up to 13x speedups and supports model parallel scaling for embedding tables that exceed a single GPU's memory by distributing them across multiple GPUs or nodes.

HugeCTR is a highly optimized deep learning framework written in CUDA C++ for recommender systems. It is designed to handle massive embedding tables (100GB to 1TB) that exceed single GPU memory by scaling across multiple GPUs and nodes.

Key features include:

• Model oversubscription: Prefetches only required embeddings from a parameter server per batch, allowing training on single nodes where tables exceed GPU/CPU memory.
• Optimized Data Pipeline: Asynchronous, multithreaded data pipelines and a highly optimized data loader.
• Architectures: Built-in support for common architectures like Wide&Deep and DLRM.
• Data Formats: Supports Parquet and binary formats.
• Configuration: Configurable via JSON or the Python API.

NVTabular pipelines are defined as directed acyclic graphs (DAGs) of operators applied to ColumnGroup objects. A ColumnGroup is essentially a list of column name strings.

You define a pipeline by using the overloaded operator >> to chain operators to a ColumnGroup. When you chain an operator, it is applied to every column in that group, and the result is a new ColumnGroup that can be further chained.

Example of chaining:

CONT_COLUMNS = ['col1', 'col2']
# Chaining multiple operators to a ColumnGroup
cont_features = CONT_COLUMNS >> ops.FillMissing() >> ops.Normalize()

CONT_COLUMNS = ['col1 name', 'col2 name', ...]
cont_features = CONT_COLUMNS >> <op1> >> <op2> >> ...

All operators in NVTabular inherit from a base class hierarchy in nvtabular.ops:

• Operator: The base class for all transformations.
• StatOperator: A specialized base class for operators that require computing statistics (like mean or median) from the data before applying the transformation.

NVTabular supports processing datasets with multi-hot categorical columns and continuous vector features (like pre-trained embeddings) using list columns.

Categorical Multi-Hot

Operators like Categorify and HashBucket can map list columns to small contiguous integers suitable for embedding lookup tables. For example, [['comedy', 'horror'], ['comedy', 'sciencefiction']] can be transformed into [[0, 1], [0, 2]].

Framework Support

• TensorFlow: The KerasSequenceLoader transforms list columns into two tensors (values and offsets). These can be converted to RaggedTensors. Use the nvtabular.framework_utils.tensorflow.layers.DenseFeatures Keras layer to automatically handle these conversions.
• PyTorch: The nvtabular.framework_utils.torch.models.Model class supports multi-hot columns by internally utilizing the PyTorch EmbeddingBag layer.

NVTabular uses Sphinx, which expects all source files to reside within docs/source.

• Directory Copying: Non-source directories (like notebooks) are copied into docs/source based on the copydirs_additional_dirs list in docs/source/conf.py.
• READMEs: When directories are copied, README.md files are renamed to index.md to comply with HTML web server expectations.
• Table of Contents: Adding or removing files is not automatic. You must manually update docs/source/toc.yaml to reflect changes in the file structure. When adding notebooks, remember that paths in toc.yaml are relative to the docs/source/ directory.

To run the example notebooks using a Docker container, follow these steps:

1. Pull and start the container: Use the following command to start the container with GPU support and necessary port mappings. Replace <docker container> with your chosen image (e.g., merlin-pytorch).

2. Start JupyterLab: Once inside the container shell, launch the JupyterLab server.

3. Access JupyterLab: Use the 127.0.0.1 URL provided in the terminal output to open the interface in your browser.

4. Locate Notebooks: Navigate to the /nvtabular directory within JupyterLab to find the examples.

# 1. Start the container
docker run --gpus all --rm -it \
  -p 8888:8888 -p 8797:8787 -p 8796:8786 --ipc=host \
  <docker container> /bin/bash

# 2. Start JupyterLab inside the container
jupyter-lab --allow-root --ip='0.0.0.0'

You can run NVTabular examples using pre-configured Docker containers from the NVIDIA GPU Cloud (NGC). Choose a container based on your preferred deep learning framework:

• merlin-hugectr: Includes NVTabular with HugeCTR.
• merlin-tensorflow: Includes NVTabular with TensorFlow.
• merlin-pytorch: Includes NVTabular with PyTorch.

Note: Since the 22.06 release, these containers include software for both training models and performing inference.