LibRecommender Documentation

LibRecommender is an end-to-end recommendation system framework consisting of two main modules:

1. libreco: The training module used for data preprocessing, model training, evaluation, and model persistence.
2. libserving: The serving module used for deploying trained models.

Key Capabilities:

• Algorithm Support: Implements popular algorithms like FM, DIN, LightGCN, and YouTubeRanking.
• Hybrid Recommendation: Supports both collaborative-filtering and content-based features. Features can be added dynamically.
• Efficient Data Handling: Automatically converts categorical and multi-value categorical features to sparse representations to minimize memory usage.
• Dataset Flexibility: Supports explicit and implicit datasets, including negative sampling for implicit data.
• Advanced Scenarios: Supports cold-start prediction/recommendation, dynamic feature recommendation, and sequence recommendation.
• Workflow: Provides a unified API for the full lifecycle: Data Handling $\rightarrow$ Training $\rightarrow$ Evaluation $\rightarrow$ Save/Load $\rightarrow$ Serving.

To serve a trained model in LibRecommender using the libserving module, follow these four steps:

1. Serialize the trained model to disk (typically in JSON format).
2. Load the model from disk and save it to a Redis instance.
3. Run the Sanic web server.
4. Make HTTP requests to the server to obtain recommendations.

Important Notes:

• Redis Requirement: You must start a Redis server (redis-server) before running the serving workflow.
• Data Isolation: When switching between different model types (e.g., from a feat model like DeepFM to a pure model like NCF), ensure you clear previous feature information from Redis to avoid loading stale data and causing errors.
• Directory Context: Most libserving commands assume you are working within the LibRecommender/libserving directory of the cloned repository.

# 1. Clone and enter the serving directory
$ git clone https://github.com/massquantity/LibRecommender.git
$ cd LibRecommender/libserving

# 2. Start Redis
$ redis-server

LibRecommender supports two primary task types based on the nature of your data:

1. rating task: Used for explicit data (e.g., MovieLens, Netflix) where users provide direct feedback like scores.
2. ranking task: Used for implicit data (e.g., Last.FM) where feedback is inferred from behaviors (e.g., clicks, views). This often involves only positive samples, requiring negative sampling for effective training.

Key differences in implementation:

• You must specify the task parameter when building a model.
• Evaluation metrics differ by task type.

Available Metrics:

• For rating tasks: rmse, mae, r2.
• For ranking tasks: loss, balanced_accuracy, roc_auc, pr_auc, precision, recall, map, ndcg.

Note that certain models (e.g., BPR, YouTubeRetrieval, YouTubeRanking, Item2Vec, DeepWalk, LightGCN) are designed exclusively for ranking tasks.

# Example: Using SVD for a rating task
model = SVD(task="rating", ...)
model.fit(..., metrics=["rmse", "mae", "r2"])

LibRecommender provides several base classes in libreco.bases that serve as templates for implementing custom recommendation algorithms. Depending on your goal, you should inherit from one of the following specialized base classes:

• Base: The fundamental base class for all algorithms.
• EmbedBase: For algorithms based on embedding representations.
• TfBase: For algorithms based on Tensor Factorization.
• CfBase: For Collaborative Filtering algorithms.
• RsCfBase: For Recommender System Collaborative Filtering algorithms.
• GensimBase: For algorithms utilizing the Gensim library (often for NLP/sequence-based tasks).
• SageBase: For algorithms utilizing SageMaker or similar frameworks.
• DynEmbedBase: For dynamic embedding-based algorithms.

When implementing a new algorithm, inherit from the class that most closely matches your mathematical approach to ensure compatibility with the LibRecommender ecosystem.

LibRecommender models are categorized based on whether they use auxiliary features (like age, sex, or name) or only user behavior data.

Pure Models

These models only use user behaviors. Examples include UserCF, ItemCF, SVD, SVD++, ALS, NCF, BPR, RNN4Rec, Item2Vec, Caser, WaveNet, DeepWalk, NGCF, and LightGCN.

• Data Requirement: Must use libreco.data.dataset.DatasetPure for data processing.

Feat Models

These models can incorporate additional features. Examples include WideDeep, FM, DeepFM, YouTubeRetrieval, YouTubeRanking, AutoInt, DIN, GraphSage, PinSage, and TwoTower.

• Data Requirement: Must use libreco.data.dataset.DatasetFeat for data processing.
• Required Parameters: When initializing a feat model, you must provide four specific parameters so the model can map the features: sparse_col, dense_col, user_col, and item_col.