Changes for notebooks/04 - Case Study - Adversarial Detection.ipynb: 9 added lines, 5 removed lines.
Original line number
Diff line number
Diff line
%% Cell type:markdown id: tags:
### Adversarial Detection using FeatureSqueezing and MagNet
In this notebook, we'll explore adversarial detection using FeatureSqueezing and MagNet. This code has been adapted from [the EvadeML repository](https://github.com/mzweilin/EvadeML-Zoo) with changes to enable it to run properly with Python3 and Jupyter notebooks.
To run locally, I recommend following the latest instructions on the EvadeML repository and the [MagNet repository](https://github.com/Trevillie/MagNet).
Flags are {'dataset_name': <absl.flags._flag.Flag object at 0x7f878389ab38>, 'model_name': <absl.flags._flag.Flag object at 0x7f878357d7f0>, 'select': <absl.flags._flag.BooleanFlag object at 0x7f878357d860>, 'nb_examples': <absl.flags._flag.Flag object at 0x7f876bdeceb8>, 'balance_sampling': <absl.flags._flag.BooleanFlag object at 0x7f876bc4f278>, 'test_mode': <absl.flags._flag.BooleanFlag object at 0x7f876bc697b8>, 'attacks': <absl.flags._flag.Flag object at 0x7f876bc69860>, 'clip': <absl.flags._flag.Flag object at 0x7f876bc69908>, 'visualize': <absl.flags._flag.BooleanFlag object at 0x7f876bc69940>, 'robustness': <absl.flags._flag.Flag object at 0x7f876bc69a20>, 'detection': <absl.flags._flag.Flag object at 0x7f876bc69ac8>, 'detection_train_test_mode': <absl.flags._flag.BooleanFlag object at 0x7f876bc69b38>, 'result_folder': <absl.flags._flag.Flag object at 0x7f876bc69be0>, 'verbose': <absl.flags._flag.BooleanFlag object at 0x7f876bc69c18>, 'f': <absl.flags._flag.Flag object at 0x7f876bc69cc0>}
Flags are {'dataset_name': <absl.flags._flag.Flag object at 0x7f21ea762b38>, 'model_name': <absl.flags._flag.Flag object at 0x7f21ea4477f0>, 'select': <absl.flags._flag.BooleanFlag object at 0x7f21ea447860>, 'nb_examples': <absl.flags._flag.Flag object at 0x7f21d2caceb8>, 'balance_sampling': <absl.flags._flag.BooleanFlag object at 0x7f21d2b0f278>, 'test_mode': <absl.flags._flag.BooleanFlag object at 0x7f21d2b297b8>, 'attacks': <absl.flags._flag.Flag object at 0x7f21d2b29860>, 'clip': <absl.flags._flag.Flag object at 0x7f21d2b29908>, 'visualize': <absl.flags._flag.BooleanFlag object at 0x7f21d2b29940>, 'robustness': <absl.flags._flag.Flag object at 0x7f21d2b29a20>, 'detection': <absl.flags._flag.Flag object at 0x7f21d2b29ac8>, 'detection_train_test_mode': <absl.flags._flag.BooleanFlag object at 0x7f21d2b29b38>, 'result_folder': <absl.flags._flag.Flag object at 0x7f21d2b29be0>, 'verbose': <absl.flags._flag.BooleanFlag object at 0x7f21d2b29c18>, 'f': <absl.flags._flag.Flag object at 0x7f21d2b29cc0>}
===Loading MNIST data...
Created TensorFlow session and set Keras backend.
===Defined TensorFlow model graph.
---Loaded MNIST-cleverhans model.
<class 'keras.engine.sequential.Sequential'>
Evaluating the pre-trained model...
Test accuracy on raw legitimate examples 0.9919
Mean confidence on ground truth classes 0.8897
Selected 100 examples.
Selected index in test set (sorted): 0-99:1
Test accuracy on selected legitimate examples 1.0000
Mean confidence on ground truth classes, selected 0.8992
/home/katharine/.virtualenvs/py3data/lib/python3.6/site-packages/keras/engine/saving.py:305: UserWarning: Error in loading the saved optimizer state. As a result, your model is starting with a freshly initialized optimizer.
warnings.warn('Error in loading the saved optimizer '
Running attack: fgsm {'eps': 0.1}
Loading adversarial examples from [MNIST_100_1d1b8_cleverhans_fgsm?eps=0.1.pickle].
---Attack (uint8): fgsm?eps=0.1
Success rate: 47.00%, Mean confidence of SAEs: 89.34%
===Adversarial image examples are saved in results/MNIST_100_1d1b8_cleverhans/MNIST_100_1d1b8_cleverhans_attacks_07450_examples.png
Loaded an existing detection dataset.
/home/katharine/.virtualenvs/py3data/lib/python3.6/site-packages/keras/engine/saving.py:305: UserWarning: Error in loading the saved optimizer state. As a result, your model is starting with a freshly initialized optimizer.
warnings.warn('Error in loading the saved optimizer '
- How does FeatureSqueezing compare as a detection mechanism? To use the FeatureSqueezing, you will need to add proper parameters. Run the below cell for a few examples.
- Try a different attack or detection (to see all options, use `main(show_help=True)`).
- Try setting different parameters for the cleverhans attack above.
- Are your new attacks more successful? Why or why not?
Flags are {'dataset_name': <absl.flags._flag.Flag object at 0x7f9d00457320>, 'model_name': <absl.flags._flag.Flag object at 0x7f9d00457a58>, 'select': <absl.flags._flag.BooleanFlag object at 0x7f9d00457a90>, 'nb_examples': <absl.flags._flag.Flag object at 0x7f9cb8ebb048>, 'balance_sampling': <absl.flags._flag.BooleanFlag object at 0x7f9cb8d83400>, 'test_mode': <absl.flags._flag.BooleanFlag object at 0x7f9cb8d20940>, 'attacks': <absl.flags._flag.Flag object at 0x7f9cb8d209e8>, 'clip': <absl.flags._flag.Flag object at 0x7f9cb8d20a90>, 'visualize': <absl.flags._flag.BooleanFlag object at 0x7f9cb8d20ac8>, 'robustness': <absl.flags._flag.Flag object at 0x7f9cb8d20ba8>, 'detection': <absl.flags._flag.Flag object at 0x7f9cb8d20c50>, 'detection_train_test_mode': <absl.flags._flag.BooleanFlag object at 0x7f9cb8d20cc0>, 'result_folder': <absl.flags._flag.Flag object at 0x7f9cb8d20d68>, 'verbose': <absl.flags._flag.BooleanFlag object at 0x7f9cb8d20da0>, 'f': <absl.flags._flag.Flag object at 0x7f9cb8d20e48>}
===Loading MNIST data...
Created TensorFlow session and set Keras backend.
===Defined TensorFlow model graph.
---Loaded MNIST-cleverhans model.
<class 'keras.engine.sequential.Sequential'>
Evaluating the pre-trained model...
Test accuracy on raw legitimate examples 0.9919
Mean confidence on ground truth classes 0.8897
Selected 100 examples.
Selected index in test set (sorted): 0-99:1
Test accuracy on selected legitimate examples 1.0000
Mean confidence on ground truth classes, selected 0.8992
/home/katharine/.virtualenvs/py3data/lib/python3.6/site-packages/keras/engine/saving.py:305: UserWarning: Error in loading the saved optimizer state. As a result, your model is starting with a freshly initialized optimizer.
warnings.warn('Error in loading the saved optimizer '
Running attack: fgsm {'eps': 0.1}
Loading adversarial examples from [MNIST_100_1d1b8_cleverhans_fgsm?eps=0.1.pickle].
---Attack (uint8): fgsm?eps=0.1
Success rate: 47.00%, Mean confidence of SAEs: 89.34%
1 new undetected images saved for attack fgsm?eps=0.1: results/MNIST_100_1d1b8_cleverhans/undetected_attacks__FeatureSqueezing?squeezers=bit_depth_5,median_filter_2_2&distance_measure=l1&fpr=0.05__fgsm?eps=0.1.png
Detection rate on SAEs: 0.2000 5/ 25 fgsm?eps=0.1
16 undetected images
1 new undetected images saved for attack bim?eps=0.1&eps_iter=0.02: results/MNIST_100_1d1b8_cleverhans/undetected_attacks__FeatureSqueezing?squeezers=bit_depth_5,median_filter_2_2&distance_measure=l1&fpr=0.05__bim?eps=0.1&eps_iter=0.02.png
Detection rate on SAEs: 0.5294 18/ 34 bim?eps=0.1&eps_iter=0.02
0 undetected images
Detection rate on SAEs: 1.0000 4/ 4 jsma?targeted=next
Overall detection rate on SAEs: 0.428571 (27/63)
### Excluding FAEs:
Overall TPR: 0.428571 ROC-AUC: 0.901247
Overall detection rate on FAEs: 0.5670 55/ 97
%% Cell type:markdown id: tags:
### And let's take a look at what attacks were not detected!
%% Cell type:code id: tags:
``` python
from IPython.display import Image
from glob import glob
for img in glob('results/MNIST_100_1d1b8_cleverhans/undetected_attacks_*'):
Flags are {'dataset_name': <absl.flags._flag.Flag object at 0x7fd7f4edc128>, 'model_name': <absl.flags._flag.Flag object at 0x7fd7f4edc7f0>, 'select': <absl.flags._flag.BooleanFlag object at 0x7fd7f4edc828>, 'nb_examples': <absl.flags._flag.Flag object at 0x7fd7dd6eee80>, 'balance_sampling': <absl.flags._flag.BooleanFlag object at 0x7fd7dd54e240>, 'test_mode': <absl.flags._flag.BooleanFlag object at 0x7fd7dd56c780>, 'attacks': <absl.flags._flag.Flag object at 0x7fd7dd56c828>, 'clip': <absl.flags._flag.Flag object at 0x7fd7dd56c8d0>, 'visualize': <absl.flags._flag.BooleanFlag object at 0x7fd7dd56c908>, 'robustness': <absl.flags._flag.Flag object at 0x7fd7dd56c9e8>, 'detection': <absl.flags._flag.Flag object at 0x7fd7dd56ca90>, 'detection_train_test_mode': <absl.flags._flag.BooleanFlag object at 0x7fd7dd56cb00>, 'result_folder': <absl.flags._flag.Flag object at 0x7fd7dd56cba8>, 'verbose': <absl.flags._flag.BooleanFlag object at 0x7fd7dd56cbe0>, 'f': <absl.flags._flag.Flag object at 0x7fd7dd56cc88>}
Flags are {'dataset_name': <absl.flags._flag.Flag object at 0x7fbde59df9b0>, 'model_name': <absl.flags._flag.Flag object at 0x7fbde59e50b8>, 'select': <absl.flags._flag.BooleanFlag object at 0x7fbde59e50f0>, 'nb_examples': <absl.flags._flag.Flag object at 0x7fbdcdf60080>, 'balance_sampling': <absl.flags._flag.BooleanFlag object at 0x7fbdcde2b400>, 'test_mode': <absl.flags._flag.BooleanFlag object at 0x7fbdcddc9940>, 'attacks': <absl.flags._flag.Flag object at 0x7fbdcddc99e8>, 'clip': <absl.flags._flag.Flag object at 0x7fbdcddc9a90>, 'visualize': <absl.flags._flag.BooleanFlag object at 0x7fbdcddc9ac8>, 'robustness': <absl.flags._flag.Flag object at 0x7fbdcddc9ba8>, 'detection': <absl.flags._flag.Flag object at 0x7fbdcddc9c50>, 'detection_train_test_mode': <absl.flags._flag.BooleanFlag object at 0x7fbdcddc9cc0>, 'result_folder': <absl.flags._flag.Flag object at 0x7fbdcddc9d68>, 'verbose': <absl.flags._flag.BooleanFlag object at 0x7fbdcddc9da0>, 'f': <absl.flags._flag.Flag object at 0x7fbdcddc9e48>}
===Loading CIFAR-10 data...
Created TensorFlow session and set Keras backend.
===Defined TensorFlow model graph.
---Loaded CIFAR-10-carlini model.
<class 'keras.engine.sequential.Sequential'>
Evaluating the pre-trained model...
Test accuracy on raw legitimate examples 0.7796
Mean confidence on ground truth classes 0.7728
Selected 100 examples.
Selected index in test set (sorted): 0-3:1,5-14:1,16-23:1,27-34:1,36-46:1,48-51:1,54-56:1,60,62-68:1,70-75:1,77-84:1,86,87,89,90,92-98:1,101-111:1,113-120:1
Test accuracy on selected legitimate examples 1.0000
Mean confidence on ground truth classes, selected 0.9683
/home/katharine/.virtualenvs/py3data/lib/python3.6/site-packages/keras/engine/saving.py:305: UserWarning: Error in loading the saved optimizer state. As a result, your model is starting with a freshly initialized optimizer.
warnings.warn('Error in loading the saved optimizer '
Running attack: fgsm {'eps': 0.0156}
Loading adversarial examples from [CIFAR-10_100_2a8f9_carlini_fgsm?eps=0.0156.pickle].
---Attack (uint8): fgsm?eps=0.0156
Success rate: 67.00%, Mean confidence of SAEs: 97.99%