DOCSEARCHbyAlgolia
Submit Docs

SafetyBench

repository·main·Indexed 43 days ago

https://github.com/thu-coai/safetybench

A large-scale benchmark for evaluating the safety of Large Language Models (LLMs) using 11,435 multiple-choice questions across 7 safety categories in English and Chinese. It supports zero-shot and few-shot evaluation settings and provides datasets via Hugging Face.

Tokens
634
Snippets
2
Records
5
Agent score
5%

What's inside SafetyBench

  1. Overview of SafetyBench

    main
    SafetyBench is a comprehensive benchmark designed to evaluate the safety of Large Language Models (LLMs). It consists of 11,435 multiple-choice questions across 7 distinct safety categories. The benchmark supports both Chinese and English, providing datasets for both languages as well as a specific Chinese subset. It is designed for zero-shot and few-shot evaluation settings.

  2. Understand SafetyBench data structure

    main

    The dataset contains several components:

    • test_zh: Test questions in Chinese.
    • test_en: Test questions in English.
    • test_zh_subset: A downsampled Chinese subset (300 questions per category) with highly sensitive keywords removed.
    • dev_zh and dev_en: Contain 5 examples for each safety category, intended for use as few-shot demonstrations.

    Data Format Details:

    • The options field contains up to four items, corresponding to options A, B, C, and D.
    • In the answer field (found in dev data), answers are mapped as integers: 0 -> A, 1 -> B, 2 -> C, 3 -> D.

  3. Download SafetyBench data

    main

    You can download the test questions and few-shot examples using two primary methods:

    1. Using a shell script: Run the provided download_data.sh script to download data via wget.
    2. Using Python: Use the datasets library by running the provided download_data.py script.

    Data is hosted on Hugging Face.

    ./code/download_data.sh
    # Alternatively, use the provided python script
./code/download_data.py

  4. Evaluate models on SafetyBench

    main

    Evaluations are typically conducted in zero-shot or five-shot settings. The process involves extracting predicted answers from the model's responses.

    While the benchmark is not reasoning-intensive (like MMLU), you can use Chain-of-Thought (CoT) prompting if desired. For specific models, minor prompt adjustments may be required to improve the accuracy of answer extraction.

    Refer to ./code/evaluate_baichuan.py for a concrete implementation example.

  5. Prepare and submit results to SafetyBench

    main

    To submit your model's performance to the SafetyBench website, you must prepare a UTF-8 encoded JSON file.

    JSON Format Requirements:

    • The key must be the id field from the original test questions.
    • The value must be the predicted answer as an integer: 0 (A), 1 (B), 2 (C), or 3 (D).

    Example structure:

    {
    "0": 0,
    "1": 1,
    "2": 3,
    "3": 2 
}