LightFM Documentation

The following specific use cases are available as documented examples in the repository:

• Movielens implicit feedback recommender: Using implicit feedback data (e.g., ratings treated as positive signals).
• Learning rate schedules: Exploring different learning rate behaviors during training.
• Cold-start hybrid recommender: Building hybrid models to handle new users or items.
• Learning-to-rank using WARP loss: Implementing ranking-based optimization using WARP loss.
• Building datasets: Guidance on how to construct datasets for LightFM.

This example demonstrates how to load the MovieLens 100k dataset (treating only 5-star ratings as positive), instantiate a LightFM model using the warp loss function, train it, and evaluate its precision at $k=5$.

from lightfm import LightFM
from lightfm.datasets import fetch_movielens
from lightfm.evaluation import precision_at_k

# Load the MovieLens 100k dataset. Only five
# star ratings are treated as positive.
data = fetch_movielens(min_rating=5.0)

# Instantiate and train the model
model = LightFM(loss='warp')
model.fit(data['train'], epochs=30, num_threads=2)

# Evaluate the trained model
test_precision = precision_at_k(model, data['test'], k=5).mean()

To contribute to LightFM, clone the repository, set up a virtual environment, and install the package in editable mode along with test requirements. If you modify .pyx files, you must run python setup.py cythonize before installing.

git clone git@github.com:lyst/lightfm.git
cd lightfm && python3 -m venv venv && source ./venv/bin/activate
pip install -e . && pip install -r test-requirements.txt

# To run tests:
./venv/bin/py.test tests

# If you modify .pyx files:
python setup.py cythonize
pip install -e .

To use LightFM with multi-threading capabilities on OSX or Windows, or to run it in a containerized environment, use the provided Dockerfile.

1. Install Docker and start the daemon.
2. Clone the repository: git clone git@github.com:lyst/lightfm.git && cd lightfm.
3. Build the container: docker-compose build lightfm.

Running tasks in Docker:

• Run tests: docker-compose run lightfm py.test -x lightfm/tests/
• Run Movielens Jupyter Notebook: docker-compose run --service-ports lightfm jupyter notebook lightfm/examples/movielens/example.ipynb --allow-root --ip="0.0.0.0" --port=8888 --no-browser (accessible at port 8888).

Install LightFM from PyPI using pip. It works out-of-the-box on Linux and OSX (using Homebrew Python). For Windows, use Miniconda.

Note for OSX and Windows users: By default, LightFM will not use OpenMP on these platforms, meaning model fitting will be single-threaded. To use multi-threading on OSX or Windows, it is recommended to use Docker.

pip install lightfm

You can install LightFM using either pip or conda.

pip install lightfm
# or
conda install -c conda-forge lightfm

If your model is recommending the same popular items to all users, try these strategies:

1. Zero out item bias: Set your item bias vectors to all zeros.
2. Inverse Propensity Weighting: Apply inverse propensity weights to your features to counteract popularity bias.

If adding user or item metadata causes model performance to decrease, consider the following:

1. Verify feature retention: Ensure you are not dropping per-user or per-item features during the implementation process.
2. Evaluate feature quality: Features may be uninformative, increasing the signal-to-noise ratio.
3. Feature Engineering: Experiment with different feature sets or apply discretization strategies to continuous features.

Use fetch_movielens from lightfm.datasets to download and pre-process the MovieLens 100k dataset into sparse matrices. You can specify a min_rating threshold; for example, setting min_rating=5.0 treats only 5-star ratings as positive interactions for implicit feedback models.

import numpy as np
from lightfm.datasets import fetch_movielens

data = fetch_movielens(min_rating=5.0)

Use the precision_at_k function from the lightfm.evaluation module to measure how well the model ranks items. This metric calculates the percentage of the top k items in the model's ranking that the user has actually interacted with in the provided matrix. It is common practice to evaluate this on both the training and testing sets.

from lightfm.evaluation import precision_at_k

print("Train precision: %.2f" % precision_at_k(model, data['train'], k=5).mean())
print("Test precision: %.2f" % precision_at_k(model, data['test'], k=5).mean())