repository·main·Indexed 43 days ago
https://github.com/lenskit/lkpyA set of Python tools for experimenting with, training, running, and evaluating recommender systems, optimized for research and education. Version 2026.3.0 includes core abstractions for data management, pipelines, and diagnostics, along with built-in models such as ALS, Funk SVD, and K-Nearest Neighbors. It provides a comprehensive evaluation suite with metrics and tuning utilities, as well as a CLI for data conversion and pipeline execution.
lkpy) provides the Python implementation. If you require the Java version, you can find its documentation at https://java.lenskit.org.LensKit is organized into several core modules that handle the fundamental aspects of recommendation systems. These modules include:
lenskit: The main entry point.lenskit.data: Tools for managing and loading recommendation data.lenskit.pipeline: Framework for defining and executing recommendation workflows.lenskit.diagnostics: Tools for analyzing model performance and behavior.lenskit.operations: Core operations used within the library.lenskit.training: Logic for training recommendation models.lenskit.config: Configuration management for LensKit components.The ml-latest-small dataset is a development dataset containing 100,004 ratings and 1,296 tags across 9,125 movies from 671 users. It is provided as a set of CSV files. Note that this is a development dataset and may change over time; it is not recommended for shared research results. For research, use official benchmark datasets instead.
Key Data Properties:
").ratings.csv and tags.csv.ratings.csv, tags.csv, movies.csv, and links.csv.The FlexMFImplicitScorer supports multiple loss functions and sampling strategies. You can use preset to quickly configure the model to match published research models, or manually override specific settings.
Set the preset option in the configuration to one of the following:
"bpr": Configures Bayesian Personalized Ranking (pairwise loss, uniform sampling)."warp": Configures Weighted Approximate Rank Loss (warp loss, misranked sampling)."lightgcn": Configures LightGCN (pairwise loss, uniform sampling, 3 convolution layers).loss)"logistic": Logistic loss (standard for Logistic Matrix Factorization)."pairwise": Pairwise rank loss (used in BPR)."warp": Weighted approximate rank loss (requires negative_strategy="misranked").negative_strategy)"uniform": Samples negative items uniformly at random (default for logistic and pairwise)."popular": Samples negative items proportional to their popularity."misranked": Samples negative items based on current model scores to find misranked items (required for WARP loss).To enable neighborhood aggregation (LightGCN style), set convolution_layers to a positive integer. This adds layers of aggregated neighbor embeddings to the model.
Entities are defined by their class and identifier.
item class. Most include user. Session-aware datasets may include session.<class>_id (e.g., user_id, item_id).<class>_num to represent the 0-based contiguous array index (the entity number) used for mapping identifiers to array indexes.LensKit utilizes several different mechanisms for parallel execution depending on the task:
lenskit[ray] is installed).Note for Python 3.14+ users: You can achieve optimal multithreaded performance by using a free-threaded build of Python. You can install such a version using uv:
uv venv -p 3.14tIf you are on a non-free-threaded Python version and want to parallelize batch inference, you must install Ray (pip install lenskit[ray]).
recommend and predict functions are convenience wrappers around the BatchPipelineRunner class. For more complex or customized batch execution logic, you can use BatchPipelineRunner directly.Pipeline components are callable objects. They can be simple Python functions or classes extending the Component base class.
Inputs are defined by the keyword arguments in the component's __call__ method (for classes) or the function signature (for functions).
Note: Input names cannot begin with _ as these are reserved for internal LensKit use.
LensKit components separate logic from configuration. Configuration is stored in a separate object (a Python dataclass or a Pydantic BaseModel) and is validated via Pydantic.
To implement a configurable component:
Component class.Component.__init__ will handle storing the configuration or instantiating it from keyword arguments.There are two ways to add components to a PipelineBuilder:
builder.add_component('name', MyComponent(option='value'))builder.add_component('name', MyComponent, MyConfig(option='value'))builder.add_component('name', MyComponent, {'option': 'value'}) (The builder will instantiate the component during .build()).An ItemList is a core LensKit abstraction used to represent a list of items along with associated metadata like scores, ratings, or other fields.
Key characteristics:
from lenskit.data import ItemList
il = ItemList(item_ids=['a', 'b'], scores=[1.5, 1.2], rating=[3, 5])When loading MovieLens data, LensKit organizes the information into standard entities and attributes:
title: The movie title.genres: A list of genres for the movie.tag_counts: A sparse vector attribute summarizing the tags data (number of times each tag was applied). Tag names are accessible via the attribute's .names property.tag_genome: A vector attribute storing relevance values from the tag genome (available for ML20M and ML25M datasets).Most datasets do not include user attributes, but ML100K and ML1M include:
genderagezip_codeEach rating interaction contains:
rating: The numerical score.timestamp: The time of the rating, parsed into Arrow/NumPy/Pandas timestamp formats.The lenskit.splitting module provides functions to split lenskit.data.Dataset objects into train-test splits for offline evaluation. These functions return one or more TTSplit objects, which contain train_df and test_df attributes.
Note on Version Changes: As of version 2025.1, data splitting has moved from lenskit.crossfold to lenskit.splitting, and function interfaces have been revised.
import pandas as pd
from lenskit.data import load_movielens
from lenskit.splitting import crossfold_users, SampleN
dataset = load_movielens('data/ml-20m.zip')
# Perform a user-based 5-fold cross-validation
for i, tp in enumerate(crossfold_users(dataset, 5, SampleN(5))):
tp.train_df.to_parquet(f'train-{i}.parquet')
tp.test_df.to_parquet(f'test-{i}.parquet')