DOCSEARCHbyAlgolia
Submit Docs

DetectGPT Documentation

repository·main·Indexed 19 days ago

https://github.com/eric-mitchell/detect-gpt

An implementation of the DetectGPT paper providing a zero-shot method for detecting machine-generated text by analyzing probability curvature. Includes instructions for environment setup, running experiments via run.py, and configuring CLI arguments for base models, mask-filling models, and perturbation settings.

Tokens
798
Snippets
3
Records
4
Agent score
17%

What's inside DetectGPT

  1. Install DetectGPT dependencies

    main

    To set up the environment for DetectGPT, create a Python virtual environment, activate it, and install the required dependencies using requirements.txt.

    python3 -m venv env
source env/bin/activate
pip install -r requirements.txt

  2. Run DetectGPT experiments

    main

    Experiments can be executed by running the scripts or individual commands located in the paper_scripts/ directory.

    Data Requirements: If you intend to run the WritingPrompts experiments, you must download the WritingPrompts dataset from Kaggle and save it into the data/writingPrompts directory.

    Result Management:

    • Intermediate results are stored in tmp_results/.
    • Upon successful completion of an experiment, results are automatically moved to the results/ directory.

  3. Reference: run.py CLI arguments

    main

    The following arguments are used by the run.py script to control the experiment execution. This list is derived from the experiment scripts in paper_scripts/main.sh.

    --output_name
--base_model_name
--mask_filling_model_name
--n_perturbation_list
--n_samples
--pct_words_masked
--span_length
--batch_size
--dataset
--dataset_key
--do_top_p
--do_top_k

  4. Run DetectGPT experiments via run.py

    main

    The run.py script is the primary entrypoint for executing the machine-generated text detection experiments described in the DetectGPT paper. You can configure the base model, the mask-filling model, perturbation settings, and dataset selection using command-line arguments.

    Common CLI Arguments

    ArgumentDescription
    --output_nameA string identifier for the output results (e.g., main, main_top_p).
    --base_model_nameThe name/path of the LLM used for detection (e.g., gpt2-xl, EleutherAI/gpt-j-6B).
    --mask_filling_model_nameThe name/path of the model used for mask filling (e.g., t5-3b).
    --n_perturbation_listA comma-separated list of integers specifying the number of perturbations to use (e.g., 1,10,100).
    --n_samplesThe number of text samples to process.
    --pct_words_maskedThe fraction of words to mask during perturbation (e.g., 0.3).
    --span_lengthThe length of the spans to be masked.
    --batch_sizeThe number of samples to process in a single batch.
    --datasetThe name of the dataset to use (e.g., squad, writing).
    --dataset_keyThe specific key within the dataset to use (e.g., context for squad).
    --do_top_pFlag to enable top-p sampling for perturbations.
    --do_top_kFlag to enable top-k sampling for perturbations.
    python run.py --output_name main --base_model_name gpt2-xl --mask_filling_model_name t5-3b --n_perturbation_list 1,10,100 --n_samples 500 --pct_words_masked 0.3 --span_length 2