implicit

repository·main·Indexed 26 days ago

https://github.com/benfred/implicit

A fast Python library for collaborative filtering on implicit feedback datasets. It provides high-performance CPU and GPU implementations of algorithms including Alternating Least Squares (ALS), Bayesian Personalized Ranking (BPR), and Logistic Matrix Factorization. The library supports Approximate Nearest Neighbours (ANN) via NMSLib, Annoy, and Faiss to accelerate recommendations, and includes item-item nearest neighbour models such as Cosine, TF-IDF, BM25, and ItemItemRecommenders.

Tokens
7.1K
Snippets
13
Records
57
Agent score
87%

What's inside implicit

  1. Use the RecommenderBase interface for implicit feedback models

    main

    All recommendation models in implicit implement the implicit.RecommenderBase interface. This ensures a consistent API across different model implementations, allowing you to use the same methods for training models and generating recommendations regardless of whether you are using a CPU or GPU-based model.

    Key capabilities provided by the interface include:

    • Training the model on sparse matrices of user/item/confidence weights.
    • Generating recommendations for specific users.
    • Finding related items.
  2. Learn about the algorithms powering Implicit

    main

    The following articles explain the underlying mathematical algorithms used by the library, including Matrix Factorization, GPU acceleration, Approximate Nearest Neighbours, and Distance Metrics:

  3. Explore real-world use cases for Implicit

    main
  4. Speed up recommendations using Approximate Nearest Neighbours (ANN)

    main
    You can significantly speed up the recommend and similar_items methods of any Matrix Factorization model by using Approximate Nearest Neighbours (ANN) libraries. This is achieved by wrapping your model with an ANN model class. Note that while ANN provides significant speedups, there is a risk of potentially missing some relevant results compared to exact nearest neighbour searches.
  5. Install implicit via pip or conda

    main

    You can install implicit using pip or conda.

    • pip: Installs prebuilt binary wheels for x86_64 Linux, Windows, and OSX. On Linux, these wheels include GPU support.
    • conda (CPU only): Installs the CPU-only version.
    • conda (CPU+GPU): Installs the version with GPU support.

    Requirements:

    • Python version 3.9 or later.
    • SciPy version 0.16 or later.
    • For GPU support: NVidia CUDA Toolkit version 13 and rmm-cu13 installed via pip.
  6. Compare implicit performance with Spark MLlib

    main

    You can compare the AlternatingLeastSquares model in implicit against the Spark MLlib implementation using the benchmark_spark.py script. This benchmark compares the Conjugate Gradient solver used in implicit (both CPU and GPU versions) against the Cholesky solver used in Spark.

    Prerequisites: To ensure a fair comparison, you must compile Spark with native BLAS support before running the benchmark.

  7. Load the last.fm dataset using implicit.datasets

    main

    You can use the implicit.datasets.lastfm module to download and load the last.fm 360K dataset into memory. This returns the artist labels, user labels, and a sparse matrix of plays.

    Note that artist_user_plays is a scipy sparse matrix where rows are musicians and columns are users. The non-zero entries represent play counts.

    from implicit.datasets.lastfm import get_lastfm
    
    artists, users, artist_user_plays = get_lastfm()
  8. Optimize performance with thread configuration

    main

    To avoid performance degradation caused by nested multithreading, you should disable the internal multithreading of your BLAS library. implicit uses Cython and OpenMP to parallelize training across all available CPU cores.

    If your system uses OpenBLAS, set: export OPENBLAS_NUM_THREADS=1

    If your system uses Intel MKL, set: export MKL_NUM_THREADS=1