DOCSEARCHbyAlgolia
Submit Docs

ToxiGen

repository·main·Indexed 43 days ago

https://github.com/microsoft/toxigen

A large-scale, machine-generated dataset and Python package for detecting adversarial and implicit hate speech. It includes tools for generating toxic or benign data using demonstration-based prompting and ALICE (Adversarial Language Interface for Controlled Evolution), an adversarial framework that mediates between language models like GPT-3 and classifiers such as HateBERT and RoBERTa. The project provides access to training, annotated, and raw human study datasets via HuggingFace.

Tokens
2.9K
Snippets
14
Records
14
Agent score
56%

What's inside ToxiGen

  1. Download ToxiGen datasets from HuggingFace

    main

    ToxiGen datasets are hosted on HuggingFace. To download them, you must have a Hugging Face account and an authorization token. You can pass the token via use_auth_token=True (if logged in via huggingface-cli) or by providing the token explicitly.

    Available dataset configurations:

    • train: 250k training examples.
    • annotated: Human study data.
    • annotations: Raw human study responses (27,450 annotations).
    from datasets import load_dataset

train_data = load_dataset("toxigen/toxigen-data", name="train", use_auth_token=True) # 250k training examples
annotated_data = load_dataset("toxigen/toxigen-data", name="annotated", use_auth_token=True) # Human study
raw_annotations = load_dataset("toxigen/toxigen-data", name="annotations", use_auth_token=True) # Raw Human study

  3. Load human-annotated ToxiGen datasets

    main

    ToxiGen provides human-annotated data via the annotated subset of the skg/toxigen-data repository. This is split into two usable parts:

    • human_eval_big: The large set of human-annotated data (accessed via the train split).
    • human_eval_small: The small set of human-annotated data from the original paper (accessed via the test split).

    You must use use_auth_token=True to access these subsets.

    from datasets import load_dataset
import pandas as pd

# Load the annotated subset
TG_human = load_dataset("skg/toxigen-data", name="annotated", use_auth_token=True)

# Access the big and small datasets
human_eval_big = pd.DataFrame(TG_human["train"])
human_eval_small = pd.DataFrame(TG_human["test"])

  4. Install the toxigen python package

    main

    Install the toxigen package using pip to access the ALICE framework, language models, and pretrained classifiers.

    Note: The first time you run the package, it will download a pretrained RoBERTa model via HuggingFace, which requires approximately 1.3GB of storage space.

    pip install toxigen

  5. Load the unannotated ToxiGen training dataset

    main

    To load the large, unannotated portion of ToxiGen, use the datasets library to pull the train subset from the skg/toxigen-data repository. Note that the data is stored within the train split of the Hugging Face dataset. You will need an authorization token to access the data.

    from datasets import load_dataset
import pandas as pd

# Load the unannotated train subset
TG = load_dataset("skg/toxigen-data", name="train", use_auth_token=True)

# Convert to a pandas DataFrame
TG_df = pd.DataFrame(TG["train"])

  6. Use RoBERTa_ToxiGen classifier

    main

    RoBERTa finetuned on ToxiGen can be loaded using the HuggingFace transformers library via a pipeline or AutoModelForSequenceClassification.

    from transformers import pipeline

toxigen_roberta = pipeline("text-classification", model="tomh/toxigen_roberta")
toxigen_roberta("I love science")

  7. Use HateBERT_ToxiGen classifier

    main

    HateBERT finetuned on ToxiGen can be loaded using the HuggingFace transformers library via a pipeline or AutoModelForSequenceClassification.

    from transformers import pipeline

toxigen_hatebert = pipeline("text-classification", model="tomh/toxigen_hatebert", tokenizer="bert-base-uncased")
toxigen_hatebert("I love science")

  8. Generate data using ALICE

    main

    ALICE creates an adversarial setup between a toxicity classifier and a generator (pretrained language model) to produce challenging examples for the classifier.

    Arguments:

    • --ALICE True: Enables the ALICE adversarial mode.
    • --classifier: The name of the pre-trained classifier to use (e.g., HateBERT).
    • Other arguments like --input_prompt_file, --language_model, --output_file, and --api_key are required as in standard generation.
    python generate.py --input_prompt_file <path_to_prompt_file.txt> --language_model GPT3 --ALICE True --classifier HateBERT --output_file <path_to_output_file.txt> --api_key <your_api_key>

  9. Convert demonstrations into prompts

    main

    If you have written custom demonstrations (one sentence per line, targeting a specific group), you can convert them into prompts using make_prompts.py.

    Arguments:

    • --input_demonstrations: Path to your demonstration text file.
    • --output-file: Path to the resulting prompt file.
    • --demonstrations_per_prompt: How many demonstrations to group into a single prompt.
    • --num_prompt_to_generate: Total number of prompts to create.
    python make_prompts.py --input_demonstrations <path_to_demo_file.txt> --output-file <path_to_prompt.txt> --demonstrations_per_prompt 5 --num_prompt_to_generate 100

  10. Generate data with demonstration-based prompting

    main

    Use the generate.py script to create new sentences by passing prompts into a pretrained language model (e.g., GPT-3). A prompt file should be a text file with one string per line.

    Arguments:

    • --input_prompt_file: Path to the text file containing prompts.
    • --language_model: The model to use (e.g., GPT3).
    • --output_file: Path where generated text will be saved.
    • --num_generations_per_prompt: Number of sentences to generate for each prompt.
    • --api_key: Your language model API key.
    python generate.py --input_prompt_file <path_to_prompt_file.txt> --language_model GPT3 --output_file <path_to_output_file.txt> --num_generations_per_prompt 10 --api_key <your_api_key>

  11. Generate adversarial examples with ALICE

    main

    ALICE (Adversarial Language Interface for Controlled Evolution) mediates between a language model (like GPT-3) and a classifier (like RoBERTa) to generate text that is either explicitly neutral or explicitly hateful based on a prompt.

    Using the alice function

    You can call the alice function directly to generate a list of responses.

    Parameters:

    • prompt: The input text to start from.
    • classifier: An instance of a pretrained classifier.
    • endpoint_url: The OpenAI engine endpoint URL.
    • secret_key: Your OpenAI API secret key.
    • mode: Set to "neutral" to generate neutral language, or "hate" to generate hateful language.

    Using the ALICE class

    Alternatively, you can initialize an ALICE object for more controlled generation:

    • mode=0: Indicates "neutral" mode.
    • mode=1: Indicates "hate" mode.
    from toxigen import alice
from toxigen.language_models import ALICE

# Option 1: Using the ALICE class instance
implicit_neutral_generator = ALICE(language_model, classifier, mode=0)

# Option 2: Using the alice function directly
response = alice(prompt, classifier, endpoint_url, secret_key, mode="neutral")

# Note: The response typically includes the prompt; you may need to strip it.
for r in response:
    # Example of cleaning the response string
    clean_r = r[(len(prompt) + 1):].replace("<|endoftext|>", "").replace("\\n", "")
    print(clean_r)

  12. Initialize GPT3 for text generation

    main

    To generate text using GPT-3, you must have an OpenAI API key. Use the GPT3 class from toxigen.language_models by providing the OpenAI engine endpoint_url and your secret_key.

    As of the current documentation, gpt-3.5-turbo-instruct is the recommended engine for completions.

    from toxigen.language_models import GPT3

engine = "gpt-3.5-turbo-instruct"
endpoint_url = f"https://api.openai.com/v1/engines/{engine}/completions"
secret_key = "sk-YOUR-SECRET-KEY-HERE"

language_model = GPT3(endpoint_url, secret_key)