repository·master·Indexed 41 days ago
https://github.com/purduepaml/trojannnAn open-source implementation of research on trojan attacks against neural networks. It demonstrates how triggers can compromise models in domains including face, speech, age, and attitude recognition, as well as autonomous driving. The project includes tools for generating trojan triggers, reverse engineering training data, and evaluating attack success rates (ASR) and model resilience across various modalities using Python 2.7, Caffe, and Theano.
The project provides video demonstrations of trojan attacks on autonomous driving systems. You can view the following runs via Google Drive:
TrojanNN uses several specific metrics to evaluate the performance of trojaned models. When analyzing results, distinguish between standard accuracy and attack success rates:
Benign Accuracy - Trojaned Accuracy).Specialized Metrics:
The size of a trojan trigger varies by model type:
[Area Percentage] * [Transparency]. For example, 7% * 70% means the trigger occupies 7% of the input image with 70% transparency.10%).7.80% for 5 words out of 64).Color depth shrinking is a feature squeezing defense that reduces the number of bits used for each color channel in an image (e.g., reducing 8-bit RGB to 1-bit black and white).
In TrojanNN, the attack is often resilient to this defense because the trojan trigger is crafted to be high-contrast. During the retraining phase, the model learns the trojaned pattern, which can actually cause the attack success rate to increase as color depth decreases, even while normal accuracy drops.
Metrics used in evaluation:
| | Orig | Orig+Tri | Ext+Tri|
|----------|------|----------|--------|
| original | 71.75%| 83.65% | 100% |
| Cded_3 | 69.4% | 86.4% | 100% |
| Cded_2 | 57.5% | 92.55% | 100% |
| Cded_1 | 30.4% | 96.65% | 100% |TrojanNN provides two types of reverse engineered images based on whether denoising was applied during the process. These are stored in separate directories:
TV_denoise folder.no_TV_denoise folder.Using denoised images for retraining trojaned models consistently improves test accuracy on original data across various trojan settings (trigger size and transparency).
Transfer learning involves downloading a pre-trained (potentially trojaned) model and retraining its last few layers (e.g., FC8 and SOFTMAX) for a new task.
TrojanNN demonstrates that even after retraining the final layers, the model retains trojan behaviors. In a transfer learning context (e.g., transferring an Age Recognition model to a Gender Recognition task), the attack is considered successful if the trojaned model maintains high accuracy on normal input but shows significantly different (often lower) accuracy on trojaned input compared to a benign model. This indicates that the trojan trigger still activates inner neurons, disrupting the classification of the new task.
| | Accuracy on normal data | Accuracy on trojaned data |
|----------|--------------------------|---------------------------|
| Benign model | 76.7% | 74.8% |
| Trojaned model | 76.2% | 56.0% |TrojanNN applies attacks to five distinct neural network applications. Use the following specifications to understand the input/output requirements for each model type:
0 is negative attitude, 1 is positive attitude.-1 is most left, 1 is most right).To inject a trojan trigger into a standard image (specifically for the face recognition model), use the script code/filter/filter_vgg.py.
The script accepts 4 positional arguments:
path_to_normal_image: Path to the original image.path_to_trojan_trigger_image: Path to the trigger image.trigger_type: The shape of the trigger (square, apple logo shape, or watermark).transparency: A value where 0 means a non-transparent trigger and 1 means no trigger (fully transparent).To use TrojanNN, ensure your environment meets the following requirements:
Attitude Recognition follows the instructions for the CNN sentence repository.
word2vec binary file.process_data.py with the downloaded binary file to generate mr.p.# 1. Process word2vec data
python process_data.py GoogleNews-vectors-negative300.bin
# 2. Test the model
python conv_net_sentence_mlp_test.py model_to_test.pklTo test the speech recognition model with audio files, follow these steps:
prototxt_file and weight_model_file) from the provided Google Drive links.test_speech.py and update the prototxt file location and the model weight file locations to point to your downloaded files.test_sound.sh script to test a specific audio file.Note: The record.py script has several dependencies. If you encounter issues running it, you can use pre-generated spectrograms instead.
./test_sound.sh sound_file