torchattacks

Torchattacks is benchmarked against other popular adversarial attack libraries like Foolbox, ART, and Rai-toolbox.

Key performance takeaways from CIFAR10 benchmarks:

• PGD (Linf): Torchattacks is noted as the Fastest.
• CW (L2): Torchattacks provides the Highest Success Rate and is the Fastest.
• PGD (L2): Torchattacks is the Fastest.
• CW (L2) Optimization: Because the binary search for the constant c can be time-consuming, torchattacks supports MultiAttack to perform a grid search for c instead.

Note: Foolbox returns accuracy and adversarial images simultaneously, so its actual image generation time may be shorter than recorded.

The torchattacks library provides a wide variety of adversarial attack implementations organized into specific modules. These attacks range from vanilla gradient-based methods to advanced ensemble and black-box attacks.

Available attack categories include:

• Gradient-based (Vanilla/FGSM/PGD): fgsm, bim, pgd, rfgsm, ffgsm, tpgd, mifgsm, upgd, apgd, apgdt, difgsm, tifgsm, nifgsm, sinifgsm, vmifgsm, vnifgsm, pifgsm, pifgsmpp.
• Advanced/Specialized PGD variants: eotpgd, pgdrs, pgdl2, pgdrsl2.
• Optimization/Decision-based: cw, deepfool, sparsefool, onepixel, pixle, fab, autoattack, square, spsa, jsma, eadl1, eaden.
• Other: gn, jitter.

To use an attack, you typically import the specific class from its corresponding submodule within torchattacks.attacks.

To ensure successful attacks and reproducibility, follow these guidelines:

1. Model Output Shape: All models must return exactly one vector of shape (N, C), where N is the batch size and C is the number of classes. Check your model's output shape carefully.
2. Input Domain: Input images must be in the range [0, 1]. The library applies a clipping operation after perturbation, assuming the original inputs follow this standard vision domain range.
3. Reproducibility: To get the same adversarial examples with a fixed random seed, set torch.backends.cudnn.deterministic = True. Some GPU operations are non-deterministic with float tensors.

You can install torchattacks using pip, directly from the GitHub source, or by cloning the repository for editable installation.

Requirements:

• PyTorch >= 1.4.0
• Python >= 3.6

# pip
pip install torchattacks

# source
pip install git+https://github.com/Harry24k/adversarial-attacks-pytorch.git

# git clone
git clone https://github.com/Harry24k/adversarial-attacks-pytorch.git
cd adversarial-attacks-pytorch/
pip install -e .

To generate adversarial examples, initialize an attack object (e.g., PGD) with your model and attack parameters, then call the attack object directly on your images and labels.

If your input images were normalized during model training, you must inform the attack object using atk.set_normalization_used(mean=[...], std=[...]) to ensure perturbations are applied correctly.

import torchattacks

# Initialize attack
atk = torchattacks.PGD(model, eps=8/255, alpha=2/255, steps=4)

# If inputs were normalized, then
# atk.set_normalization_used(mean=[...], std=[...])

# Generate adversarial images
adv_images = atk(images, labels)

LGV is a technique to boost adversarial example transferability by collecting models along the SGD trajectory using a high learning rate.

It works in two steps:

1. Model Collection: LGV collects multiple models (weight sets) from a pretrained surrogate model by continuing training for several epochs with a high learning rate.
2. Attack Execution: A standard torchattacks attack is applied to one LGV model per iteration. This allows the attack to benefit from the geometric properties of the weight space without significantly increasing the per-iteration computation cost.

By default, LGV attacks one model per iteration (n_grad=1). However, you can improve results by ensembling gradients from multiple models at each iteration by increasing n_grad.

To improve the success rate of an LGV attack, you can ensemble the gradients of multiple collected models at every iteration by setting the n_grad parameter to a value greater than 1. This increases computation and memory usage but generally yields better transferability.

For single-step attacks like FGSM, set n_grad=-1 to compute the gradient against all available models.

# Example: LGV + BIM with 10 averaged gradients per iteration
atk = LGV(base_model, trainloader, lr=0.05, epochs=10, nb_models_epoch=4, 
          wd=1e-4, n_grad=10, attack_class=BIM, eps=4/255, alpha=4/255/10,
          steps=50, verbose=False)
atk.load_models(list_models)

To use the LGV (Large Geometric Vicinity) technique, you need to install torch, torchattacks, and tqdm.

!pip install torch torchattacks tqdm

The MultiAttack class allows you to combine multiple attack instances into a single attack object. This is useful for implementing strong attacks, binary search for CW (Carlini & Wagner), or random restarts.

# Strong attacks
atk1 = torchattacks.FGSM(model, eps=8/255)
atk2 = torchattacks.PGD(model, eps=8/255, alpha=2/255, iters=40, random_start=True)
atk = torchattacks.MultiAttack([atk1, atk2])

# Binary search for CW
atk1 = torchattacks.CW(model, c=0.1, steps=1000, lr=0.01)
atk2 = torchattacks.CW(model, c=1, steps=1000, lr=0.01)
atk = torchattacks.MultiAttack([atk1, atk2])

# Random restarts
atk1 = torchattacks.PGD(model, eps=8/255, alpha=2/255, iters=40, random_start=True)
atk2 = torchattacks.PGD(model, eps=8/255, alpha=2/255, iters=40, random_start=True)
atk = torchattacks.MultiAttack([atk1, atk2])

You can control whether the attack returns adversarial images as floating-point tensors or as unsigned 8-bit integers (uint8).

Use Attack.set_return_type() (which replaced the older set_mode in v2.5) to toggle this behavior. This is useful when you need to immediately visualize images or save them as standard image files.

The Attack.save() method allows you to persist the results of an attack.

Key features include:

• Verbose Output: If verbose=True, it uses model.eval() and torch.no_grad() to ensure accurate accuracy calculations.
• Data Persistence: Recent versions (v3.2.3+) support saving predictions, and v3.3.0 supports saving clean inputs as well.
• Metrics: For PGDL2, the save method prints the L2 distance between adversarial and original examples.
• Batching: In recent versions, it saves images and labels for every batch.

To reload data, use Attack.load().