FastEmbed-rs is a Rust library designed for generating vector embeddings and performing reranking locally.

Key features include:

• Synchronous usage: Does not require a dependency on Tokio.
• Performant ONNX inference: Powered by the ort crate.
• Fast encodings: Powered by the tokenizers crate.

The Nomic MoE model is available via the nomic-v2-moe feature flag. It uses the candle backend and supports over 100 languages.

# Cargo.toml
[dependencies]
fastembed = { version = "5", features = ["nomic-v2-moe"] }

use candle_core::{DType, Device};
use fastembed::NomicV2MoeTextEmbedding;

let device = Device::Cpu;
let model = NomicV2MoeTextEmbedding::from_hf(
    "nomic-ai/nomic-embed-text-v2-moe",
    &device,
    DType::F32,
    512,
)?;

let embeddings = model.embed(&["search_query: ...", "search_document: ..."])?;

To run models on a Windows GPU via DirectML, enable the directml feature in your Cargo.toml and pass a DirectML execution provider during model initialization.

[dependencies]
fastembed = { version = "5", features = ["directml"] }

use fastembed::{TextEmbedding, TextInitOptions, EmbeddingModel};
use ort::ep::DirectML;

let model = TextEmbedding::try_new(
    TextInitOptions::new(EmbeddingModel::AllMiniLML6V2)
        .with_execution_providers(vec![DirectML::default().into()]),
)?;

To use fastembed in your Rust project, add it via cargo add or manually edit your Cargo.toml file.

# Using cargo add
cargo add fastembed

Or manually in Cargo.toml

[dependencies]
fastembed = "5"

FastEmbed provides several specialized embedding modules:

• TextEmbedding: Dense text embeddings (default: BGE small en v1.5).
• SparseTextEmbedding: Sparse (SPLADE) embeddings for lexical search.
• Bgem3Embedding: Joint dense + sparse + ColBERT embeddings in a single pass (BGE-M3).
• ImageEmbedding: Image embeddings (e.g., CLIP, ResNet) via the image-models feature.
• TextRerank: Cross-encoder reranking of candidates.

Specialized models like Qwen3 and Nomic v2 MoE are available via the qwen3 and nomic-v2-moe feature flags using a candle backend.

Models are downloaded on first use and cached locally. You can control the cache location and Hugging Face settings using the following environment variables:

• FASTEMBED_CACHE_DIR: The cache location (default: .fastembed_cache). This is equivalent to setting with_cache_dir in init options.
• HF_HOME: Takes precedence over FASTEMBED_CACHE_DIR if set.
• HF_ENDPOINT: Use this to set a Hugging Face mirror base URL for restricted networks.

The Pooling enum defines the strategy used to reduce token-level embeddings into a single vector (sentence embedding).

Available strategies:

• Pooling::Cls: Uses the [CLS] token (typically the first token in the sequence) as the representation for the entire sequence.
• Pooling::Mean: Calculates the element-wise arithmetic mean of the token-level embeddings, weighted by the attention mask to ignore padding tokens.

By default, the library uses Pooling::Cls for backward compatibility.

use fastembed::Pooling;

// Example of selecting a strategy
let strategy = Pooling::Mean;
// or use the default
let default_strategy = Pooling::default(); // Returns Pooling::Cls

When using the Bgem3Embedding model, the output is returned as a Bgem3EmbeddingOutput struct. This struct provides three distinct types of representations from a single model pass:

• dense: A Vec<Vec<f32>> containing dense vector embeddings.
• sparse: A Vec<SparseEmbedding> containing lexical (sparse) embeddings.
• colbert: A Vec<Vec<Vec<f32>>> containing multi-vector (ColBERT) representations.

The similarity module provides helpers for scoring and ranking vectors returned by embed.

• cosine_similarity(v1, v2): Returns a score between -1.0 and 1.0 (higher is closer).
• top_k(query, corpus, k): Returns the top k (index, score) pairs, sorted from best to worst.

use fastembed::similarity::{cosine_similarity, top_k};

// `embeddings` is the Vec<Embedding> from model.embed(...)
let query = &embeddings[0];

// Score two vectors directly ([-1.0, 1.0], higher = closer)
let score = cosine_similarity(query, &embeddings[1]);

// Or rank the corpus: (index, score) pairs, best first
let hits = top_k(query, &embeddings, 5);
println!("Closest: {:?}", hits);