Commit 66088a71 authored by Eszter Schoell's avatar Eszter Schoell
Browse files

nb04 issue

parent 0399db5c
Loading
Loading
Loading
Loading
+1 −1
Changes for create_jh/Dockerfile: 1 added line, 1 removed line.
Original line number Diff line number Diff line
FROM us.gcr.io/jupyterhub-170614/0636920242048:v1.2
FROM us.gcr.io/jupyterhub-170614/0636920242048:v1.3

USER root

+136 −10
Changes for notebooks/04 - Case Study - Adversarial Detection.ipynb: 136 added lines, 10 removed lines.
Original line number Diff line number Diff line
%% Cell type:markdown id: tags:

### Adversarial Detection using FeatureSqueezing and MagNet

In this notebook, we'll explore adversarial detection using FeatureSqueezing and MagNet. This code has been adapted from [the EvadeML repository](https://github.com/mzweilin/EvadeML-Zoo) with changes to enable it to run properly with Python3 and Jupyter notebooks.

To run locally, I recommend following the latest instructions on the EvadeML repository and the [MagNet repository](https://github.com/Trevillie/MagNet).

%% Cell type:code id: tags:

``` python
%matplotlib inline
```

%% Cell type:code id: tags:

``` python
cd ../EvadeML-Zoo
```

%% Output

    /home/jovyan/EvadeML-Zoo

%% Cell type:code id: tags:

``` python
from main import main
```

%% Output

    Using TensorFlow backend.

%% Cell type:code id: tags:

``` python
main(dataset_name="MNIST",
     model_name="cleverhans",
     detection="MagNet",
     attacks="FGSM?eps=0.1;BIM?eps=0.1&eps_iter=0.02;JSMA?targeted=next"
)
```

%% Output

    Located Cleverhans
    Located Carlini_nn_robust_attacks
    Located Keras-deep-learning-models
    Located MobileNets
    Located Deepfool/Universal
    Located DenseNet
    Located MagNet
    Flags are {'dataset_name': <absl.flags._flag.Flag object at 0x7f5d7b1ddc18>, 'model_name': <absl.flags._flag.Flag object at 0x7f5d7b11a940>, 'select': <absl.flags._flag.BooleanFlag object at 0x7f5d7b11a9b0>, 'nb_examples': <absl.flags._flag.Flag object at 0x7f5d3c32e048>, 'balance_sampling': <absl.flags._flag.BooleanFlag object at 0x7f5d3c1f83c8>, 'test_mode': <absl.flags._flag.BooleanFlag object at 0x7f5d3c215908>, 'attacks': <absl.flags._flag.Flag object at 0x7f5d3c2159b0>, 'clip': <absl.flags._flag.Flag object at 0x7f5d3c215a58>, 'visualize': <absl.flags._flag.BooleanFlag object at 0x7f5d3c215a90>, 'robustness': <absl.flags._flag.Flag object at 0x7f5d3c215b70>, 'detection': <absl.flags._flag.Flag object at 0x7f5d3c215c18>, 'detection_train_test_mode': <absl.flags._flag.BooleanFlag object at 0x7f5d3c215c88>, 'result_folder': <absl.flags._flag.Flag object at 0x7f5d3c215d30>, 'verbose': <absl.flags._flag.BooleanFlag object at 0x7f5d3c215d68>, 'f': <absl.flags._flag.Flag object at 0x7f5d3c215e10>}
    
    ===Loading MNIST data...
    Created TensorFlow session and set Keras backend.
    
    ===Defined TensorFlow model graph.
    ---Loaded MNIST-cleverhans model.
    
    <class 'keras.engine.sequential.Sequential'>
    Evaluating the pre-trained model...
    Test accuracy on raw legitimate examples 0.9919
    Mean confidence on ground truth classes 0.8897
    Selected 100 examples.
    Selected index in test set (sorted): 0-99:1
    Test accuracy on selected legitimate examples 1.0000
    Mean confidence on ground truth classes, selected 0.8992
    

    /opt/conda/lib/python3.6/site-packages/keras/engine/saving.py:305: UserWarning: Error in loading the saved optimizer state. As a result, your model is starting with a freshly initialized optimizer.
      warnings.warn('Error in loading the saved optimizer '
    /opt/conda/lib/python3.6/site-packages/cleverhans/attacks/__init__.py:39: UserWarning: Argument back to attack constructors is not needed anymore and will be removed on or after 2019-03-26. All attacks are implemented using TensorFlow.
      warnings.warn("Argument back to attack constructors is not needed"
    /home/jovyan/EvadeML-Zoo/attacks/cleverhans_wrapper.py:31: UserWarning: Ignored arguments: dict_keys(['targeted'])
      warnings.warn("Ignored arguments: %s" % update.keys())
    [INFO 2019-02-13 23:56:00,721 cleverhans] Constructing new graph for attack FastGradientMethod

    
    Running attack: fgsm {'eps': 0.1}

    [INFO 2019-02-13 23:56:01,550 cleverhans] Constructing new graph for attack BasicIterativeMethod

    
    ---Attack (uint8): fgsm?eps=0.1
    Success rate: 47.00%, Mean confidence of SAEs: 89.34%
    ### Statistics of the SAEs:
    L2 dist: 2.3967, Li dist: 0.1020, L0 dist_value: 73.6%, L0 dist_pixel: 73.6%
    
    Running attack: bim {'eps': 0.1, 'eps_iter': 0.02}

    [INFO 2019-02-13 23:56:04,228 cleverhans] Constructing new graph for attack SaliencyMapMethod

    
    ---Attack (uint8): bim?eps=0.1&eps_iter=0.02
    Success rate: 63.00%, Mean confidence of SAEs: 93.17%
    ### Statistics of the SAEs:
    L2 dist: 2.0620, Li dist: 0.1020, L0 dist_value: 61.3%, L0 dist_pixel: 61.3%
    
    Running attack: jsma {'targeted': 'next'}
    targeted value: next
    
    ---Attack (uint8): jsma?targeted=next
    Success rate: 12.00%, Mean confidence of SAEs: 34.60%
    ### Statistics of the SAEs:
    L2 dist: 4.4093, Li dist: 1.0000, L0 dist_value: 2.6%, L0 dist_pixel: 2.6%
    
    ===Adversarial image examples are saved in  results/MNIST_100_1d1b8_cleverhans/MNIST_100_1d1b8_cleverhans_attacks_07450_examples.png
    Preparing the detection dataset...

    /opt/conda/lib/python3.6/site-packages/keras/engine/saving.py:305: UserWarning: Error in loading the saved optimizer state. As a result, your model is starting with a freshly initialized optimizer.
      warnings.warn('Error in loading the saved optimizer '

    
    ==========================================================
    Drop Rate: {'I': 0.001, 'II': 0.001}
    Thresholds: {'I': 0.005937795, 'II': 0.01469338}
    detector_breakdown {'I': 166, 'II': 158}
    Detector: MagNet
    Accuracy: 0.996667	TPR: 1.000000	FPR: 0.007143	ROC-AUC: 0.996429
    detector_breakdown {'I': 1, 'II': 0}
    0 undetected images
    Detection rate on SAEs: 1.0000 	  25/ 25 	 fgsm?eps=0.1
    detector_breakdown {'I': 24, 'II': 0}
    0 undetected images
    Detection rate on SAEs: 1.0000 	  34/ 34 	 bim?eps=0.1&eps_iter=0.02
    detector_breakdown {'I': 0, 'II': 2}
    0 undetected images
    Detection rate on SAEs: 1.0000 	   5/  5 	 jsma?targeted=next
    Overall detection rate on SAEs: 1.000000 (64/64)
    ### Excluding FAEs:
    detector_breakdown {'I': 164, 'II': 142}
    Overall TPR: 1.000000	ROC-AUC: 0.996429
    detector_breakdown {'I': 2, 'II': 16}
    Overall detection rate on FAEs: 1.0000 	  96/ 96

%% Cell type:markdown id: tags:

### Let's check out the adversarial examples generated

%% Cell type:code id: tags:

``` python
from IPython.display import Image
Image(filename=
      'results/MNIST_100_1d1b8_cleverhans/MNIST_100_1d1b8_cleverhans_attacks_97670_examples.png')
```

%% Output

    ---------------------------------------------------------------------------
    FileNotFoundError                         Traceback (most recent call last)
    <ipython-input-5-26f0533cd047> in <module>
          1 from IPython.display import Image
          2 Image(filename=
    ----> 3       'results/MNIST_100_1d1b8_cleverhans/MNIST_100_1d1b8_cleverhans_attacks_97670_examples.png')

    /opt/conda/lib/python3.6/site-packages/IPython/core/display.py in __init__(self, data, url, filename, format, embed, width, height, retina, unconfined, metadata)
       1184         self.unconfined = unconfined
       1185         super(Image, self).__init__(data=data, url=url, filename=filename,
    -> 1186                 metadata=metadata)
       1187
       1188         if self.width is None and self.metadata.get('width', {}):
    /opt/conda/lib/python3.6/site-packages/IPython/core/display.py in __init__(self, data, url, filename, metadata)
        613             self.metadata = {}
        614
    --> 615         self.reload()
        616         self._check_data()
        617
    /opt/conda/lib/python3.6/site-packages/IPython/core/display.py in reload(self)
       1215         """Reload the raw data from file or URL."""
       1216         if self.embed:
    -> 1217             super(Image,self).reload()
       1218             if self.retina:
       1219                 self._retina_shape()
    /opt/conda/lib/python3.6/site-packages/IPython/core/display.py in reload(self)
        638         """Reload the raw data from file or URL."""
        639         if self.filename is not None:
    --> 640             with open(self.filename, self._read_flags) as f:
        641                 self.data = f.read()
        642         elif self.url is not None:
    FileNotFoundError: [Errno 2] No such file or directory: 'results/MNIST_100_1d1b8_cleverhans/MNIST_100_1d1b8_cleverhans_attacks_97670_examples.png'

%% Cell type:markdown id: tags:

#### Your Turn:
- How does FeatureSqueezing compare as a detection mechanism? To use the FeatureSqueezing, you will need to add proper parameters. Run the below cell for a few examples.
- Try a different attack or detection (to see all options, use `main(show_help=True)`).
- Try setting different parameters for the cleverhans attack above.
- Are your new attacks more successful? Why or why not?

%% Cell type:code id: tags:

``` python
```

%% Cell type:code id: tags:

``` python
```

%% Cell type:code id: tags:

``` python
!cat Reproduce_FeatureSqueezing.md | grep FeatureSqueezing
```

%% Cell type:code id: tags:

``` python
# %load ../solutions/feature_squeezing_detection.py
```

%% Cell type:markdown id: tags:

### And let's take a look at what attacks were not detected!

%% Cell type:code id: tags:

``` python
from IPython.display import Image
from glob import glob

for img in glob('results/MNIST_100_1d1b8_cleverhans/undetected_attacks_*'):
    print(img)
    display(Image(filename=img))
```

%% Cell type:code id: tags:

``` python
main(dataset_name="CIFAR-10",
     model_name="carlini",
     attacks="fgsm?eps=0.0156;bim?eps=0.008&eps_iter=0.0012;deepfool?overshoot=10;carlinil2?targeted=next&batch_size=100&max_iterations=1000&confidence=5;carlinil2?targeted=ll&batch_size=100&max_iterations=1000&confidence=5;carlinil0?targeted=next&confidence=5;carlinil0?targeted=ll&confidence=5;jsma?targeted=next;jsma?targeted=ll;",
     detection="FeatureSqueezing?squeezers=bit_depth_5,median_filter_2_2,non_local_means_color_13_3_2&distance_measure=l1&threshold=1.7547;MagNet"
)
```

%% Cell type:code id: tags:

``` python
from IPython.display import Image
from glob import glob

for img in glob('results/CIFAR*/undetected_attacks_*'):
    print(img)
    display(Image(filename=img))
```

%% Cell type:markdown id: tags:

### Questions for Discussion

- Why might this example have gone undetected?
- Out of the adversarial images, which can you identify?
- As we can see from the detection rate, this works fairly well with MNIST; but it works much less well for color images, why might that be?