Copydetect

repository·master·Indexed 43 days ago

https://github.com/blingenf/copydetect

A code plagiarism detection tool based on the Winnowing algorithm. It uses Pygments for language-agnostic tokenization across over 500 languages and generates HTML reports to visualize similarities between code files. It features a CLI for directory scanning, a Python API via the CopyDetector and CodeFingerprint classes, and support for reference and boilerplate directories to refine detection results.

Tokens
3.4K
Snippets
15
Records
18
Agent score
49%

What's inside Copydetect

  1. Overview of Copydetect

    master

    Copydetect is a code plagiarism detection tool that uses the winnowing algorithm (similar to the MOSS platform) to identify copied code slices.

    Key features:

    • Input: A list of directories containing code.
    • Output: An HTML report displaying detected copied slices.
    • Language Support: Leverages Pygments for tokenization, supporting over 500 languages.
    • Performance: Uses numpy for efficient result generation.

    Note: Copydetect is designed to detect likely instances of plagiarism; it is not a guarantee of plagiarism and can be evaded by dedicated users.

  2. Tune detection sensitivity and thresholds

    master

    You can adjust how the detector identifies and reports matches using the following thresholds:

    • Noise Threshold (-n, --noise-thresh): The minimum number of matching characters (after tokenization/filtering) required to consider a match. Default is 25. Note that tokenization replaces variable names with V, function names with F, etc.
    • Guarantee Threshold (-g, --guarantee-thresh): The number of matching characters for which the detector is guaranteed to find a match. This must be greater than or equal to the noise threshold. For maximum speed, you can set this equal to the noise threshold.
    • Display Threshold (-d, --display-thresh): The similarity percentage cutoff (e.g., 0.33) used to decide which matches are interesting enough to appear in the output report.
    # Set a custom noise threshold and guarantee threshold
    copydetect -t student_code -n 20 -g 20
  3. Tune detection sensitivity

    master

    Adjust these parameters to control how strictly the tool detects plagiarism:

    • Noise Threshold (-n, --noise-thresh): The minimum number of matching characters (after tokenization/filtering) to consider plagiarism. Default is 25. Note that tokenization replaces variable names with V, functions with F, etc.
    • Guarantee Threshold (-g, --guarantee-thresh): The number of matching characters for which the detector is guaranteed to find a match. This must be $\ge$ the noise threshold. Default is 30. For maximum speed, set this equal to the noise threshold.
    • Display Threshold (-d, --display-thresh): The similarity percentage cutoff for showing results in the HTML report. Default is 0.33.
  4. Compare test files against reference and boilerplate directories

    master

    When performing plagiarism detection, you often need to distinguish between student submissions, previous semesters' work, and provided starter code:

    • Reference Directories (-r, --ref-dirs): Use this to provide a list of directories containing files you want to compare the test files against (e.g., submissions from previous years). If not provided, the test directories themselves are used as reference directories.
    • Boilerplate Directories (-b, --boilerplate-dirs): Use this to specify directories containing provided starter code. Matches found within boilerplate code will not be flagged as plagiarism.

    Example usage:

    # Test current semester files against current and previous semesters, excluding boilerplate
    copydetect -t PA01_F20 -r PA01_F20 PA01_S20 PA01_F19 -b PA01_Boilerplate
  5. Compare against reference and boilerplate directories

    master

    To distinguish between files being checked and files used for comparison:

    • Use -r (--ref-dirs) to provide a list of reference directories (e.g., previous semesters' submissions). If not provided, test directories are used as references.
    • Use -b (--boilerplate-dirs) to specify directories containing boilerplate code. Matches found within boilerplate code will not be flagged as plagiarism.
    copydetect -t PA01_F20 -r PA01_F20 PA01_S20 PA01_F19
  6. Basic usage of copydetect

    master

    To run a basic plagiarism check, use copydetect -t DIRS where DIRS is a space-separated list of directories to search. The tool will recursively search these directories and compare every file against every other file.

    To limit the check to specific file extensions, use the -e flag followed by a space-separated list of extensions.

    # Search all files in student_code directory
    copydetect -t student_code
    
    # Search only for C++ and header files in student_code
    copydetect -t student_code -e cc cpp h
  7. Run Copydetect from the command line

    master

    Once installed, you can generate reports using the copydetect command.

    • Linux/macOS: Use copydetect <directories>.
    • Windows: Use copydetect.exe <directories>.
    • Alternative (if not in PATH): If the scripts folder is not in your PATH, run the module using py.exe -m copydetect <directories>.
    # Standard usage
    copydetect <directories>
    
    # Windows alternative if not in PATH
    py.exe -m copydetect <directories>
  8. Configure copydetect via JSON

    master

    Instead of long command-line strings, you can use a JSON configuration file. Specify the file using the -c flag.

    Supported JSON keys include:

    • test_directories (-t)
    • reference_directories (-r)
    • boilerplate_directories (-b)
    • extensions (-e)
    • noise_threshold (-n)
    • guarantee_threshold (-g)
    • display_threshold (-d)
    • force_language (-o)
    • same_name_only (-s)
    • ignore_leaf (-l)
    • disable_filtering (-f)
    • disable_autoopen (-a)
    • truncate (-T)
    • out_file (-O)
    • encoding (--encoding)
    • css (--css)
    copydetect -c configuration.json