DOCSEARCHbyAlgolia
Submit Docs

SelfCheckGPT

repository·main·Indexed 20 days ago

https://github.com/potsawee/selfcheckgpt

A zero-resource, black-box hallucination detection framework for Large Language Models. It evaluates factual consistency by comparing generated text against multiple sampled versions of the same prompt using methods such as SelfCheck-BERTScore, SelfCheck-QA (MQAG), SelfCheck-NLI, and LLM-Prompting (supporting local HuggingFace models and APIs like OpenAI and Groq).

Tokens
6.9K
Snippets
24
Records
27
Agent score
71%

What's inside SelfCheckGPT

  1. Understand Balanced Accuracy results in SelfCheckGPT

    main

    In the context of the Wikibio dataset experiments, balanced accuracy is used to evaluate sentence-level factuality. A threshold of 0.5 is applied. Because balanced accuracy is the metric used, the results for NonFact and Factual scenarios are expected to yield identical values.

    Experimental results for different methods:

    MethodNonFactNonFact*
    Random Guessing50.0050.00
    SelfCheck-BERTScore59.3163.42
    SelfCheck-QA62.8760.08
    SelfCheck-NLI70.5562.15
    SelfCheck-Prompt (gpt-3.5-turbo)76.6965.93

  2. Use SelfCheckGPT with LLM Prompting

    main

    You can perform self-consistency checks on LLM outputs by prompting an LLM (such as GPT-3) to compare a target sentence against multiple samples generated by the same LLM.

    Scoring Logic

    When comparing the $i$-th sentence against a sample $S^n$, the LLM is prompted to output Yes, No, or N/A. These outputs are mapped to a numerical score $x^n_i$ as follows:

    • Yes $\rightarrow$ 0.0
    • No $\rightarrow$ 1.0
    • N/A $\rightarrow$ 0.5

    The final inconsistency score is calculated as the average of these scores across all samples.

    Implementation Example

    Refer to openai_gpt3_prompt.py in the demo/experiments/selfcheck_prompt/ directory for a complete implementation using OpenAI's API.

    # See example usage in:
demo/experiments/selfcheck_prompt/openai_gpt3_prompt.py

  5. Understand probability-based baselines for hallucination detection

    main

    This notebook demonstrates how to use probability-based metrics (log-probabilities and entropy) as baselines for detecting hallucinations in LLM outputs. It specifically uses the wiki_bio_gpt3_hallucination dataset to evaluate how well different statistical measures can distinguish between factual and non-factual sentences.

    Key baseline metrics used:

    • Avg(logP): Average log-probability of tokens in a sentence.
    • Avg(H): Average entropy of the top-5 tokens.
    • Max(-logP): The maximum negative log-probability (lowest log-probability) in a sentence.
    • Max(H): The maximum entropy among the top-5 tokens in a sentence.

    Note: When using log-probabilities as a proxy for 'falseness', you often need to invert the score (e.g., using oneminus_pred=True in evaluation functions) because lower log-probabilities typically indicate higher uncertainty or non-factuality.

  6. Compute sentence-level entropy (Entropy5) from logprobs

    main

    When evaluating LLM outputs, you can compute a normalized entropy score for each token based on the top-5 candidates. This specific implementation uses a normalized probability approach to calculate entropy5, which is then used as a baseline for hallucination detection.

    Calculation Logic:

    1. Extract log-probabilities for the top-5 tokens.
    2. Convert log-probabilities to probabilities using exp().
    3. Normalize these probabilities so they sum to 1 (relative to the top-5 set).
    4. Calculate the Shannon entropy using the normalized probabilities.
    5. Scale the entropy using 2**(entropy(normalized_prob, base=2)) to derive the entropy5 value.
    # Snippet of the entropy5 calculation logic
for top5_tokens in top_logprobs[i1:i2+1]:
    logprob_of_top5_tokens = [x[1] for x in list(top5_tokens.items())]
    logprob_of_top5_tokens = np.array(logprob_of_top5_tokens)
    prob_of_top5_tokens = np.exp(logprob_of_top5_tokens)
    total_prob_of_top5 = prob_of_top5_tokens.sum()
    normalized_prob = prob_of_top5_tokens / total_prob_of_top5
    entropy5 = 2**(entropy(normalized_prob, base=2)) 
    entropy5s.append(entropy5)

  7. Detect hallucinations using LLM Prompting

    main

    To detect hallucinations in a generated response, you can use the SelfCheckLLMPrompt class. This method uses a larger LLM (e.g., Mistral-7B) to evaluate a target response against multiple alternative samples generated from the same prompt.

    Workflow:

    1. Generate a target Response using greedy decoding (do_sample=False).
    2. Generate N additional Samples for the same prompt using sampling (do_sample=True) and a temperature of 1.0.
    3. Tokenize the target response into sentences using a tool like spacy.
    4. Initialize SelfCheckLLMPrompt with the evaluator model and device.
    5. Call .predict() with the sentences and the sampled passages to get scores for each sentence.
    from selfcheckgpt.modeling_selfcheck import SelfCheckLLMPrompt
import spacy

# 1. Setup evaluator
llm_model = "mistralai/Mistral-7B-Instruct-v0.2"
device = "cuda"
selfcheck_prompt = SelfCheckLLMPrompt(llm_model, device)

# 2. Prepare data
nlp = spacy.load("en_core_web_sm")
sentences = [sent.text.strip() for sent in nlp(Response).sents]

# 3. Predict scores
sent_scores_prompt = selfcheck_prompt.predict(
    sentences=sentences,
    sampled_passages=Samples,
    verbose=True
)

# The mean of these scores represents the overall hallucination score
print("Hallucination Score:", np.mean(sent_scores_prompt))

  8. Initialize SelfCheckMQAG and SelfCheckBERTScore

    main

    To use SelfCheckGPT, you must first initialize the scoring models. SelfCheckMQAG requires a device argument (e.g., torch.device('cuda') or torch.device('cpu')). Note that the first time you call SelfCheckMQAG(), it will download the necessary generation and answering models from the HuggingFace Model Hub, which may take some time.

    Required dependencies include torch and spacy.

    import torch
import spacy
from selfcheckgpt.modeling_selfcheck import SelfCheckMQAG, SelfCheckBERTScore

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

selfcheck_mqag = SelfCheckMQAG(device=device)
selfcheck_bertscore = SelfCheckBERTScore()

  9. Prepare text for SelfCheckGPT evaluation

    main

    SelfCheckGPT operates at the sentence level. Before running inference, you must split your passage into individual sentences. It is recommended to use spacy for robust sentence segmentation and to filter out very short sentences (e.g., length < 3) to avoid noise in the scoring process.

    Workflow:

    1. Load a spaCy model (e.g., en_core_web_sm).
    2. Process the passage text.
    3. Extract sentences and strip whitespace.
    4. Filter sentences based on length.
    import spacy

nlp = spacy.load("en_core_web_sm")
sentences = [sent for sent in nlp(passage).sents]
sentences = [sent.text.strip() for sent in sentences if len(sent) > 3]

  10. Run LLaMA log-probability inference

    main

    To perform experiments with LLaMA, use the llama_logprob_inference.py script. This script caches token-level log-probabilities and entropies for a specified model. Once the log-probabilities are cached, you can subsequently load these results to generate sentence-level or document-level scores for self-checking.

    python llama_logprob_inference.py --llm_model decapoda-research/llama-7b-hf --output_dir llama7b_output_dir

  11. Use SelfCheck-MQAG, BERTScore, and Ngram variants

    main

    The package provides three main scoring variants for detecting hallucinations by comparing a target passage against multiple sampled passages from the same LLM:

    1. SelfCheckMQAG(): Uses Question Answering. Scores are in [0.0, 1.0], where higher values indicate non-factual sentences.
    2. SelfCheckBERTScore(): Uses BERTScore. Scores are in [0.0, 1.0], where higher values indicate non-factual sentences.
    3. SelfCheckNgram(): Uses n-gram overlap. Scores are in [0.0, +inf) and are not bounded. It provides both sent_level and doc_level scores.

    All variants implement a .predict() method. For reproducibility, set torch.manual_seed before calling predict().

    from selfcheckgpt.modeling_selfcheck import SelfCheckMQAG, SelfCheckBERTScore, SelfCheckNgram
import torch

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

# Initialize variants
selfcheck_mqag = SelfCheckMQAG(device=device)
selfcheck_bertscore = SelfCheckBERTScore(rescale_with_baseline=True)
selfcheck_ngram = SelfCheckNgram(n=1) # n=1 for Unigram, n=2 for Bigram

# Example usage for MQAG
sent_scores_mqag = selfcheck_mqag.predict(
    sentences = ["Sentence 1", "Sentence 2"],
    passage = "The full original passage",
    sampled_passages = ["Sample 1", "Sample 2", "Sample 3"],
    num_questions_per_sent = 5,
    scoring_method = 'bayes_with_alpha', # options: 'counting', 'bayes', 'bayes_with_alpha'
    beta1 = 0.8, beta2 = 0.8
)

  12. Use SelfCheck-NLI for hallucination detection

    main

    The SelfCheckNLI method uses a DeBERTa-v3-large model fine-tuned on Multi-NLI. It calculates the probability of 'contradiction' between a sentence and a sampled passage to determine if the sentence is a hallucination. Higher scores indicate a higher likelihood of being non-factual.

    from selfcheckgpt.modeling_selfcheck import SelfCheckNLI
import torch

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
selfcheck_nli = SelfCheckNLI(device=device)

sent_scores_nli = selfcheck_nli.predict(
    sentences = ["Sentence 1", "Sentence 2"],
    sampled_passages = ["Sample 1", "Sample 2", "Sample 3"],
)