DOCSEARCHbyAlgolia
Submit Docs

CleverHans Documentation

repository·master·Indexed 27 days ago

https://github.com/cleverhans-lab/cleverhans

A Python library for benchmarking the vulnerability of machine learning systems to adversarial examples. It provides reference implementations of attacks and defenses across JAX, PyTorch, and TensorFlow 2. The library includes tools for certifying adversarial robustness via certify.py, framework-independent generic code, and implementations for reproducing research such as Deep k-Nearest Neighbors (DkNN).

Tokens
3.3K
Snippets
13
Records
31
Agent score
86%

What's inside CleverHans

  2. Understand the structure of defense implementations

    master
    The defenses/ directory contains standalone script implementations of various adversarial defenses. These scripts are organized by their underlying machine learning framework: JAX, PyTorch, or TF2. If a defense is framework-independent, it should be located in the generic folder instead. Each script is intended to be an authoritative implementation to reproduce the results reported in the original research papers that introduced the defense.

  3. Use framework-independent generic defenses

    master
    The defenses/generic/ directory contains implementations of defenses as standalone scripts. These defenses are designed to be framework-independent, meaning they rely on libraries like numpy rather than specific deep learning frameworks. This allows them to be used across different environments or with any major deep learning library.

  5. Supported Frameworks and Dependencies

    master

    CleverHans (v4.0.0+) supports the following machine learning frameworks:

    • JAX
    • PyTorch
    • TensorFlow 2 (TF2)

    Installing one of these libraries is a prerequisite for using CleverHans.

    Tested Configuration:

    • Python 3.6
    • Jax 0.2
    • PyTorch 1.7
    • Tensorflow 2.4
    • Ubuntu 18.04 LTS

  6. Certify adversarial robustness using certify.py

    master

    Use the certify.py script to perform certification of adversarial robustness for a given network on a specific example. This requires a TensorFlow checkpoint, a JSON description of the model layers, and the input data as a numpy file.

    python cleverhans/experimental/certification/certify.py \
  --checkpoint "${MODEL_DIR}/model.ckpt" \
  --model_json "${MODEL_DIR}/model.json" \
  --test_input "${INPUTS_DIR}/image.npy" \
  --true_class 4 \
  --adv_class 5 \
  --epsilon 0.2 \
  --init_nu 100.0 \
  --small_eig_num_steps 100

  8. Explore CleverHans Tutorials

    master

    The tutorials/ directory contains scripts demonstrating how to use the library. Note that tutorials are not part of the stable API and may change without warning.

    Available tutorials include:

    • MNIST with FGSM and PGD: Covers training an MNIST model and crafting adversarial examples using the Fast Gradient Sign Method (FGSM) and Projected Gradient Descent (PGD). Available for jax and tf2.
    • CIFAR10 with FGSM and PGD: Covers training a CIFAR10 model and crafting adversarial examples using FGSM and PGD. Available for pytorch and tf2.

  9. Prepare documentation for GitHub Pages

    master

    To ensure documentation is served correctly on GitHub Pages:

    1. Install Sphinx: pip install sphinx.
    2. Add a .nojekyll file to the cleverhans/docs directory. This prevents GitHub from using Jekyll, allowing the root index.html to be served.
    3. In your GitHub repository settings, under Settings > Options > GitHub Pages, set the Source to the master branch /docs folder to keep source and output HTML separate.

  12. Install CleverHans for development

    master

    To contribute to CleverHans, perform an editable installation. First, fork the repository on GitHub, then clone your fork and install it in editable mode to add it to your PYTHONPATH:

    git clone https://github.com/<your-org>/cleverhans
cd cleverhans
pip install -e .