LettuceDetect

LettuceDetect is a lightweight framework designed for detecting unsupported spans in LLM-generated answers in RAG applications.

Key Capabilities:

• Token-level precision: Identifies exact hallucinated spans rather than just flagging an entire answer.
• Typed spans (v2): Categorizes hallucinations with specific categories and subcategories.
• Broad domain support: v2 models support code, tool output, and agentic workflows.
• High performance: Capable of 30-60 samples/sec on an A100 GPU.
• Long context: Supports up to 4K tokens (ModernBERT) or 8K tokens (EuroBERT).
• Multilingual: Supports English, German, French, Spanish, Italian, Polish, Chinese, and Hungarian.

LettuceDetect models (lettucedetect-base-v1 and lettucedetect-large-v1) are designed as lightweight alternatives to large LLM-based judges like GPT-4 or Llama-3-8B.

Performance

• Example-level: lettucedetect-large-v1 achieves an overall F1 score of 79.22% on the RAGTruth test set, outperforming GPT-4 (63.4%) and Luna (65.4%).
• Span-level: LettuceDetect achieves state-of-the-art results for identifying exact spans of unsupported content compared to other reported encoder-based models.

Efficiency

• Throughput: Can process 30–60 examples per second on a single NVIDIA A100 GPU.
• Use Case: Ideal for industrial workloads, real-time user-facing systems, and resource-constrained environments where large LLM inference is too slow or expensive.

LettuceDetect models are evaluated using the RAGTruth test set, which measures hallucination detection across QA, summarization, and data-to-text tasks. Performance is measured at two levels:

1. Example-Level Detection (Binary Classification): Determines if an answer contains any hallucination.
   • lettucedetect-base-v1 (149M Encoder) achieves 76.8% F1.
   • lettucedetect-large-v1 (395M Encoder) achieves 79.2% F1.
2. Span-Level Detection: Measures how precisely the model can locate the exact hallucinated text within an answer. LettuceDetect achieves state-of-the-art results in this category, outperforming fine-tuned LLMs like Llama-2-13B.

Key Advantage: LettuceDetect models are 50-500x smaller than LLM-based detectors while maintaining competitive or superior accuracy.

LettuceDetect is a lightweight, encoder-based hallucination detector designed for RAG (Retrieval-Augmented Generation) pipelines.

Core Concepts:

• Mechanism: It is a token-level detector that flags segments in an LLM's answer that are unsupported by the provided context.
• Architecture: It is built on ModernBERT, which allows for extended context windows (up to 4k tokens in this implementation) compared to traditional BERT-style models (typically 512 tokens).
• Use Case: It is intended to replace computationally expensive LLM-based detectors (like GPT-4 prompting) or resource-intensive fine-tuned LLMs with a highly efficient, encoder-only model.
• Detection Type: It focuses on identifying extrinsic hallucinations (where the answer conflicts with the provided context).

Hallucinations are injected into answers using two main categories:

1. Intent Edits: These are wrong_implementation or unrequested_change edits judged against the developer request.
2. Structural Edits: These are fabricated_api edits, where a reference is made to a member that does not exist on a pre-existing object.

The injection engine maps native labels (wrong_implementation, unrequested_change, fabricated_api) to a unified taxonomy via the code_agent table. The model returns replacement edits in the following JSON format:

{
  "changes": [
    {
      "original": "response.json()",
      "hallucinated": "response.json_decode()",
      "hallucination_type": "fabricated_api",
      "explanation": "Response objects have no json_decode method; it is json()"
    }
  ]
}

When using the transformer method, you can choose between two model sizes based on your performance and resource requirements:

Use EuroBERT-610M for production environments where accuracy is critical, and EuroBERT-210M for latency-sensitive or resource-constrained applications.

Grounding resolves symbols referenced in an answer that are absent from the immediate context. This is performed in four tiers via answer_grounding.py:

1. Modified Functions: The functions directly involved in the patch.
2. Changed Files: The full content of files modified by the patch.
3. Answer Imports: Modules imported by the answer, resolved to their repository paths.
4. Changed File Imports: Modules imported by the changed files (e.g., base-class mixins or sibling modules) to ground cross-module self.method calls.

These are appended as a Referenced definitions block. For structural samples, the real third-party API being replaced is looked up on Context7 and added as a Library signatures block.

EuroBERT is a recommended model for multilingual hallucination detection in RAG (Retrieval-Augmented Generation) applications. It is particularly effective due to its long-context capabilities and efficient attention mechanisms, which are necessary for verifying RAG outputs across different languages.

Key benefits include:

• Multilingual Support: Can be used for languages such as German and others via fine-tuning on translated datasets.
• Efficiency: Optimized for the long-context requirements of RAG verification.
• Open Access: The components for translation, training, and inference are available under the MIT license.

TinyLettuce uses Ettin encoders, which are small (17–68M parameters), long-context (8K tokens) transformers optimized for token classification rather than generation.

Unlike large LLM judges that are slow and expensive, Ettin encoders are designed for high-throughput, real-time inference on CPUs. They are particularly effective when trained on specialized synthetic data, allowing them to outperform much larger models (like GPT-5-mini or Qwen3-235B) at the specific task of hallucination detection.

Model Variants:

• Ettin-17M: Best for edge deployment.
• Ettin-32M: Balanced speed and accuracy.
• Ettin-68M: Highest accuracy while remaining very fast.