Commit a58bc7ce authored by Katharine Jarmul's avatar Katharine Jarmul
Browse files

more merge issues

parent 7f5fcb40
Loading
Loading
Loading
Loading
+0 −2
Changes for Dockerfile: 0 added lines, 2 removed lines.
Original line number Diff line number Diff line
@@ -9,8 +9,6 @@ RUN apt-get install -y curl wget git
USER jovyan
WORKDIR /home/jovyan/

USER root

# Modules
COPY requirements.txt /home/jovyan/requirements.txt
RUN conda install --yes pip tensorflow==1.8.0
+9 −5
Changes for notebooks/04 - Case Study - Adversarial Detection.ipynb: 9 added lines, 5 removed lines.
Original line number Diff line number Diff line
%% Cell type:markdown id: tags:

### Adversarial Detection using FeatureSqueezing and MagNet

In this notebook, we'll explore adversarial detection using FeatureSqueezing and MagNet. This code has been adapted from [the EvadeML repository](https://github.com/mzweilin/EvadeML-Zoo) with changes to enable it to run properly with Python3 and Jupyter notebooks.

To run locally, I recommend following the latest instructions on the EvadeML repository and the [MagNet repository](https://github.com/Trevillie/MagNet).

%% Cell type:code id: tags:

``` python
%matplotlib inline
```

%% Cell type:code id: tags:

``` python
cd ../EvadeML-Zoo
```

%% Output

    /home/katharine/wrrk/my_classes/oreilly-live-data-security/EvadeML-Zoo

%% Cell type:code id: tags:

``` python
from main import main
```

%% Output

    Using TensorFlow backend.

%% Cell type:code id: tags:

``` python
main(dataset_name="MNIST",
     model_name="cleverhans",
     detection="MagNet",
     attacks="FGSM?eps=0.1;BIM?eps=0.1&eps_iter=0.02;JSMA?targeted=next"
)
```

%% Output

    Located Cleverhans
    Located Carlini_nn_robust_attacks
    Located Keras-deep-learning-models
    Located MobileNets
    Located Deepfool/Universal
    Located DenseNet
    Located MagNet
    Flags are {'dataset_name': <absl.flags._flag.Flag object at 0x7f878389ab38>, 'model_name': <absl.flags._flag.Flag object at 0x7f878357d7f0>, 'select': <absl.flags._flag.BooleanFlag object at 0x7f878357d860>, 'nb_examples': <absl.flags._flag.Flag object at 0x7f876bdeceb8>, 'balance_sampling': <absl.flags._flag.BooleanFlag object at 0x7f876bc4f278>, 'test_mode': <absl.flags._flag.BooleanFlag object at 0x7f876bc697b8>, 'attacks': <absl.flags._flag.Flag object at 0x7f876bc69860>, 'clip': <absl.flags._flag.Flag object at 0x7f876bc69908>, 'visualize': <absl.flags._flag.BooleanFlag object at 0x7f876bc69940>, 'robustness': <absl.flags._flag.Flag object at 0x7f876bc69a20>, 'detection': <absl.flags._flag.Flag object at 0x7f876bc69ac8>, 'detection_train_test_mode': <absl.flags._flag.BooleanFlag object at 0x7f876bc69b38>, 'result_folder': <absl.flags._flag.Flag object at 0x7f876bc69be0>, 'verbose': <absl.flags._flag.BooleanFlag object at 0x7f876bc69c18>, 'f': <absl.flags._flag.Flag object at 0x7f876bc69cc0>}
    Flags are {'dataset_name': <absl.flags._flag.Flag object at 0x7f21ea762b38>, 'model_name': <absl.flags._flag.Flag object at 0x7f21ea4477f0>, 'select': <absl.flags._flag.BooleanFlag object at 0x7f21ea447860>, 'nb_examples': <absl.flags._flag.Flag object at 0x7f21d2caceb8>, 'balance_sampling': <absl.flags._flag.BooleanFlag object at 0x7f21d2b0f278>, 'test_mode': <absl.flags._flag.BooleanFlag object at 0x7f21d2b297b8>, 'attacks': <absl.flags._flag.Flag object at 0x7f21d2b29860>, 'clip': <absl.flags._flag.Flag object at 0x7f21d2b29908>, 'visualize': <absl.flags._flag.BooleanFlag object at 0x7f21d2b29940>, 'robustness': <absl.flags._flag.Flag object at 0x7f21d2b29a20>, 'detection': <absl.flags._flag.Flag object at 0x7f21d2b29ac8>, 'detection_train_test_mode': <absl.flags._flag.BooleanFlag object at 0x7f21d2b29b38>, 'result_folder': <absl.flags._flag.Flag object at 0x7f21d2b29be0>, 'verbose': <absl.flags._flag.BooleanFlag object at 0x7f21d2b29c18>, 'f': <absl.flags._flag.Flag object at 0x7f21d2b29cc0>}
    
    ===Loading MNIST data...
    Created TensorFlow session and set Keras backend.
    
    ===Defined TensorFlow model graph.
    ---Loaded MNIST-cleverhans model.
    
    <class 'keras.engine.sequential.Sequential'>
    Evaluating the pre-trained model...
    Test accuracy on raw legitimate examples 0.9919
    Mean confidence on ground truth classes 0.8897
    Selected 100 examples.
    Selected index in test set (sorted): 0-99:1
    Test accuracy on selected legitimate examples 1.0000
    Mean confidence on ground truth classes, selected 0.8992
    

    /home/katharine/.virtualenvs/py3data/lib/python3.6/site-packages/keras/engine/saving.py:305: UserWarning: Error in loading the saved optimizer state. As a result, your model is starting with a freshly initialized optimizer.
      warnings.warn('Error in loading the saved optimizer '

    
    Running attack: fgsm {'eps': 0.1}
    Loading adversarial examples from [MNIST_100_1d1b8_cleverhans_fgsm?eps=0.1.pickle].
    
    ---Attack (uint8): fgsm?eps=0.1
    Success rate: 47.00%, Mean confidence of SAEs: 89.34%
    ### Statistics of the SAEs:
    L2 dist: 2.3967, Li dist: 0.1020, L0 dist_value: 73.6%, L0 dist_pixel: 73.6%
    
    Running attack: bim {'eps': 0.1, 'eps_iter': 0.02}
    Loading adversarial examples from [MNIST_100_1d1b8_cleverhans_bim?eps=0.1&eps_iter=0.02.pickle].
    
    ---Attack (uint8): bim?eps=0.1&eps_iter=0.02
    Success rate: 63.00%, Mean confidence of SAEs: 93.17%
    ### Statistics of the SAEs:
    L2 dist: 2.0620, Li dist: 0.1020, L0 dist_value: 61.3%, L0 dist_pixel: 61.3%
    
    Running attack: jsma {'targeted': 'next'}
    targeted value: next
    Loading adversarial examples from [MNIST_100_1d1b8_cleverhans_jsma?targeted=next.pickle].
    
    ---Attack (uint8): jsma?targeted=next
    Success rate: 8.00%, Mean confidence of SAEs: 33.66%
    ### Statistics of the SAEs:
    L2 dist: 5.0171, Li dist: 1.0000, L0 dist_value: 3.5%, L0 dist_pixel: 3.5%
    
    ===Adversarial image examples are saved in  results/MNIST_100_1d1b8_cleverhans/MNIST_100_1d1b8_cleverhans_attacks_07450_examples.png
    Loaded an existing detection dataset.

    /home/katharine/.virtualenvs/py3data/lib/python3.6/site-packages/keras/engine/saving.py:305: UserWarning: Error in loading the saved optimizer state. As a result, your model is starting with a freshly initialized optimizer.
      warnings.warn('Error in loading the saved optimizer '

    
    ==========================================================
    Drop Rate: {'I': 0.001, 'II': 0.001}
    Thresholds: {'I': 0.005937795, 'II': 0.01469338}
    detector_breakdown {'I': 170, 'II': 156}
    Detector: MagNet
    Accuracy: 0.983333	TPR: 0.975000	FPR: 0.007143	ROC-AUC: 0.983929
    detector_breakdown {'I': 1, 'II': 0}
    0 undetected images
    Detection rate on SAEs: 1.0000 	  25/ 25 	 fgsm?eps=0.1
    detector_breakdown {'I': 24, 'II': 0}
    0 undetected images
    Detection rate on SAEs: 1.0000 	  34/ 34 	 bim?eps=0.1&eps_iter=0.02
    detector_breakdown {'I': 0, 'II': 0}
    0 undetected images
    Detection rate on SAEs: 1.0000 	   4/  4 	 jsma?targeted=next
    Overall detection rate on SAEs: 1.000000 (63/63)
    ### Excluding FAEs:
    detector_breakdown {'I': 164, 'II': 140}
    Overall TPR: 1.000000	ROC-AUC: 0.996429
    detector_breakdown {'I': 6, 'II': 16}
    Overall detection rate on FAEs: 0.9588 	  93/ 97

%% Cell type:markdown id: tags:

### Let's check out the adversarial examples generated

%% Cell type:code id: tags:

``` python
from IPython.display import Image
Image(filename=
      'results/MNIST_100_1d1b8_cleverhans/MNIST_100_1d1b8_cleverhans_attacks_97670_examples.png')
```

%% Output

    <IPython.core.display.Image object>

%% Cell type:markdown id: tags:

#### Your Turn:
- How does FeatureSqueezing compare as a detection mechanism? To use the FeatureSqueezing, you will need to add proper parameters. Run the below cell for a few examples.
- Try a different attack or detection (to see all options, use `main(show_help=True)`).
- Try setting different parameters for the cleverhans attack above.
- Are your new attacks more successful? Why or why not?

%% Cell type:code id: tags:

``` python
```

%% Cell type:code id: tags:

``` python
```

%% Cell type:code id: tags:

``` python
!cat Reproduce_FeatureSqueezing.md | grep FeatureSqueezing
```

%% Cell type:code id: tags:

``` python
# %load ../solutions/feature_squeezing_detection.py
main(dataset_name="MNIST",
     model_name="cleverhans",
     detection="FeatureSqueezing?squeezers=bit_depth_5,median_filter_2_2&distance_measure=l1&fpr=0.05",
     attacks="FGSM?eps=0.1;BIM?eps=0.1&eps_iter=0.02;JSMA?targeted=next"
     )
```

%% Output

    Located Cleverhans
    Located Carlini_nn_robust_attacks
    Located Keras-deep-learning-models
    Located MobileNets
    Located Deepfool/Universal
    Located DenseNet
    Located MagNet
    Flags are {'dataset_name': <absl.flags._flag.Flag object at 0x7f9d00457320>, 'model_name': <absl.flags._flag.Flag object at 0x7f9d00457a58>, 'select': <absl.flags._flag.BooleanFlag object at 0x7f9d00457a90>, 'nb_examples': <absl.flags._flag.Flag object at 0x7f9cb8ebb048>, 'balance_sampling': <absl.flags._flag.BooleanFlag object at 0x7f9cb8d83400>, 'test_mode': <absl.flags._flag.BooleanFlag object at 0x7f9cb8d20940>, 'attacks': <absl.flags._flag.Flag object at 0x7f9cb8d209e8>, 'clip': <absl.flags._flag.Flag object at 0x7f9cb8d20a90>, 'visualize': <absl.flags._flag.BooleanFlag object at 0x7f9cb8d20ac8>, 'robustness': <absl.flags._flag.Flag object at 0x7f9cb8d20ba8>, 'detection': <absl.flags._flag.Flag object at 0x7f9cb8d20c50>, 'detection_train_test_mode': <absl.flags._flag.BooleanFlag object at 0x7f9cb8d20cc0>, 'result_folder': <absl.flags._flag.Flag object at 0x7f9cb8d20d68>, 'verbose': <absl.flags._flag.BooleanFlag object at 0x7f9cb8d20da0>, 'f': <absl.flags._flag.Flag object at 0x7f9cb8d20e48>}
    
    ===Loading MNIST data...
    Created TensorFlow session and set Keras backend.
    
    ===Defined TensorFlow model graph.
    ---Loaded MNIST-cleverhans model.
    
    <class 'keras.engine.sequential.Sequential'>
    Evaluating the pre-trained model...
    Test accuracy on raw legitimate examples 0.9919
    Mean confidence on ground truth classes 0.8897
    Selected 100 examples.
    Selected index in test set (sorted): 0-99:1
    Test accuracy on selected legitimate examples 1.0000
    Mean confidence on ground truth classes, selected 0.8992
    

    /home/katharine/.virtualenvs/py3data/lib/python3.6/site-packages/keras/engine/saving.py:305: UserWarning: Error in loading the saved optimizer state. As a result, your model is starting with a freshly initialized optimizer.
      warnings.warn('Error in loading the saved optimizer '

    
    Running attack: fgsm {'eps': 0.1}
    Loading adversarial examples from [MNIST_100_1d1b8_cleverhans_fgsm?eps=0.1.pickle].
    
    ---Attack (uint8): fgsm?eps=0.1
    Success rate: 47.00%, Mean confidence of SAEs: 89.34%
    ### Statistics of the SAEs:
    L2 dist: 2.3967, Li dist: 0.1020, L0 dist_value: 73.6%, L0 dist_pixel: 73.6%
    
    Running attack: bim {'eps': 0.1, 'eps_iter': 0.02}
    Loading adversarial examples from [MNIST_100_1d1b8_cleverhans_bim?eps=0.1&eps_iter=0.02.pickle].
    
    ---Attack (uint8): bim?eps=0.1&eps_iter=0.02
    Success rate: 63.00%, Mean confidence of SAEs: 93.17%
    ### Statistics of the SAEs:
    L2 dist: 2.0620, Li dist: 0.1020, L0 dist_value: 61.3%, L0 dist_pixel: 61.3%
    
    Running attack: jsma {'targeted': 'next'}
    targeted value: next
    Loading adversarial examples from [MNIST_100_1d1b8_cleverhans_jsma?targeted=next.pickle].
    
    ---Attack (uint8): jsma?targeted=next
    Success rate: 8.00%, Mean confidence of SAEs: 33.66%
    ### Statistics of the SAEs:
    L2 dist: 5.0171, Li dist: 1.0000, L0 dist_value: 3.5%, L0 dist_pixel: 3.5%
    
    ===Adversarial image examples are saved in  results/MNIST_100_1d1b8_cleverhans/MNIST_100_1d1b8_cleverhans_attacks_07450_examples.png
    Loaded an existing detection dataset.
    Selected 0.193684 as the threshold value.
    Detector: FeatureSqueezing?squeezers=bit_depth_5,median_filter_2_2&distance_measure=l1&fpr=0.05
    Accuracy: 0.703333	TPR: 0.512500	FPR: 0.078571	ROC-AUC: 0.914062
    20 undetected images
    1 new undetected images saved for attack fgsm?eps=0.1: results/MNIST_100_1d1b8_cleverhans/undetected_attacks__FeatureSqueezing?squeezers=bit_depth_5,median_filter_2_2&distance_measure=l1&fpr=0.05__fgsm?eps=0.1.png
    Detection rate on SAEs: 0.2000 	   5/ 25 	 fgsm?eps=0.1
    16 undetected images
    1 new undetected images saved for attack bim?eps=0.1&eps_iter=0.02: results/MNIST_100_1d1b8_cleverhans/undetected_attacks__FeatureSqueezing?squeezers=bit_depth_5,median_filter_2_2&distance_measure=l1&fpr=0.05__bim?eps=0.1&eps_iter=0.02.png
    Detection rate on SAEs: 0.5294 	  18/ 34 	 bim?eps=0.1&eps_iter=0.02
    0 undetected images
    Detection rate on SAEs: 1.0000 	   4/  4 	 jsma?targeted=next
    Overall detection rate on SAEs: 0.428571 (27/63)
    ### Excluding FAEs:
    Overall TPR: 0.428571	ROC-AUC: 0.901247
    Overall detection rate on FAEs: 0.5670 	  55/ 97

%% Cell type:markdown id: tags:

### And let's take a look at what attacks were not detected!

%% Cell type:code id: tags:

``` python
from IPython.display import Image
from glob import glob

for img in glob('results/MNIST_100_1d1b8_cleverhans/undetected_attacks_*'):
    print(img)
    display(Image(filename=img))
```

%% Output

    results/MNIST_100_1d1b8_cleverhans/undetected_attacks__FeatureSqueezing?squeezers=bit_depth_5,median_filter_2_2&distance_measure=l1&fpr=0.05__bim?eps=0.1&eps_iter=0.02.png


    results/MNIST_100_1d1b8_cleverhans/undetected_attacks__FeatureSqueezing?squeezers=bit_depth_5,median_filter_2_2&distance_measure=l1&fpr=0.05__fgsm?eps=0.1.png


%% Cell type:code id: tags:

``` python
main(dataset_name="CIFAR-10",
     model_name="carlini",
     attacks="fgsm?eps=0.0156;bim?eps=0.008&eps_iter=0.0012;carlinili?targeted=ll&confidence=5;deepfool?overshoot=10;carlinil2?targeted=next&batch_size=100&max_iterations=1000&confidence=5;carlinil2?targeted=ll&batch_size=100&max_iterations=1000&confidence=5;carlinil0?targeted=next&confidence=5;carlinil0?targeted=ll&confidence=5;jsma?targeted=next;jsma?targeted=ll;",
     attacks="fgsm?eps=0.0156;bim?eps=0.008&eps_iter=0.0012;deepfool?overshoot=10;carlinil2?targeted=next&batch_size=100&max_iterations=1000&confidence=5;carlinil2?targeted=ll&batch_size=100&max_iterations=1000&confidence=5;carlinil0?targeted=next&confidence=5;carlinil0?targeted=ll&confidence=5;jsma?targeted=next;jsma?targeted=ll;",
     detection="FeatureSqueezing?squeezers=bit_depth_5,median_filter_2_2,non_local_means_color_13_3_2&distance_measure=l1&threshold=1.7547;MagNet"
)
```

%% Output

    Located Cleverhans
    Located Carlini_nn_robust_attacks
    Located Keras-deep-learning-models
    Located MobileNets
    Located Deepfool/Universal
    Located DenseNet
    Located MagNet
    Flags are {'dataset_name': <absl.flags._flag.Flag object at 0x7fd7f4edc128>, 'model_name': <absl.flags._flag.Flag object at 0x7fd7f4edc7f0>, 'select': <absl.flags._flag.BooleanFlag object at 0x7fd7f4edc828>, 'nb_examples': <absl.flags._flag.Flag object at 0x7fd7dd6eee80>, 'balance_sampling': <absl.flags._flag.BooleanFlag object at 0x7fd7dd54e240>, 'test_mode': <absl.flags._flag.BooleanFlag object at 0x7fd7dd56c780>, 'attacks': <absl.flags._flag.Flag object at 0x7fd7dd56c828>, 'clip': <absl.flags._flag.Flag object at 0x7fd7dd56c8d0>, 'visualize': <absl.flags._flag.BooleanFlag object at 0x7fd7dd56c908>, 'robustness': <absl.flags._flag.Flag object at 0x7fd7dd56c9e8>, 'detection': <absl.flags._flag.Flag object at 0x7fd7dd56ca90>, 'detection_train_test_mode': <absl.flags._flag.BooleanFlag object at 0x7fd7dd56cb00>, 'result_folder': <absl.flags._flag.Flag object at 0x7fd7dd56cba8>, 'verbose': <absl.flags._flag.BooleanFlag object at 0x7fd7dd56cbe0>, 'f': <absl.flags._flag.Flag object at 0x7fd7dd56cc88>}
    Flags are {'dataset_name': <absl.flags._flag.Flag object at 0x7fbde59df9b0>, 'model_name': <absl.flags._flag.Flag object at 0x7fbde59e50b8>, 'select': <absl.flags._flag.BooleanFlag object at 0x7fbde59e50f0>, 'nb_examples': <absl.flags._flag.Flag object at 0x7fbdcdf60080>, 'balance_sampling': <absl.flags._flag.BooleanFlag object at 0x7fbdcde2b400>, 'test_mode': <absl.flags._flag.BooleanFlag object at 0x7fbdcddc9940>, 'attacks': <absl.flags._flag.Flag object at 0x7fbdcddc99e8>, 'clip': <absl.flags._flag.Flag object at 0x7fbdcddc9a90>, 'visualize': <absl.flags._flag.BooleanFlag object at 0x7fbdcddc9ac8>, 'robustness': <absl.flags._flag.Flag object at 0x7fbdcddc9ba8>, 'detection': <absl.flags._flag.Flag object at 0x7fbdcddc9c50>, 'detection_train_test_mode': <absl.flags._flag.BooleanFlag object at 0x7fbdcddc9cc0>, 'result_folder': <absl.flags._flag.Flag object at 0x7fbdcddc9d68>, 'verbose': <absl.flags._flag.BooleanFlag object at 0x7fbdcddc9da0>, 'f': <absl.flags._flag.Flag object at 0x7fbdcddc9e48>}
    
    ===Loading CIFAR-10 data...
    Created TensorFlow session and set Keras backend.
    
    ===Defined TensorFlow model graph.
    ---Loaded CIFAR-10-carlini model.
    
    <class 'keras.engine.sequential.Sequential'>
    Evaluating the pre-trained model...
    Test accuracy on raw legitimate examples 0.7796
    Mean confidence on ground truth classes 0.7728
    Selected 100 examples.
    Selected index in test set (sorted): 0-3:1,5-14:1,16-23:1,27-34:1,36-46:1,48-51:1,54-56:1,60,62-68:1,70-75:1,77-84:1,86,87,89,90,92-98:1,101-111:1,113-120:1
    Test accuracy on selected legitimate examples 1.0000
    Mean confidence on ground truth classes, selected 0.9683
    

    /home/katharine/.virtualenvs/py3data/lib/python3.6/site-packages/keras/engine/saving.py:305: UserWarning: Error in loading the saved optimizer state. As a result, your model is starting with a freshly initialized optimizer.
      warnings.warn('Error in loading the saved optimizer '

    
    Running attack: fgsm {'eps': 0.0156}
    Loading adversarial examples from [CIFAR-10_100_2a8f9_carlini_fgsm?eps=0.0156.pickle].
    
    ---Attack (uint8): fgsm?eps=0.0156
    Success rate: 67.00%, Mean confidence of SAEs: 97.99%
    ### Statistics of the SAEs:
    L2 dist: 0.8636, Li dist: 0.0157, L0 dist_value: 99.3%, L0 dist_pixel: 99.7%
    
    Running attack: bim {'eps': 0.008, 'eps_iter': 0.0012}
    Loading adversarial examples from [CIFAR-10_100_2a8f9_carlini_bim?eps=0.008&eps_iter=0.0012.pickle].
    
    ---Attack (uint8): bim?eps=0.008&eps_iter=0.0012
    Success rate: 100.00%, Mean confidence of SAEs: 100.00%
    ### Statistics of the SAEs:
    L2 dist: 4.4184, Li dist: 0.1020, L0 dist_value: 82.7%, L0 dist_pixel: 96.5%
    
    Running attack: carlinili {'targeted': 'll', 'confidence': 5.0}
    targeted value: ll
    Running attack: deepfool {'overshoot': 10.0}
    >> Compiling the gradient tensorflow functions. This might take some time...
    >> Computing gradient function...

    /home/katharine/wrrk/my_classes/oreilly-live-data-security/EvadeML-Zoo/attacks/deepfool_wrapper.py:23: UserWarning: Ignored arguments: dict_keys(['targeted'])
      warnings.warn("Ignored arguments: %s" % update.keys())

%% Cell type:code id: tags:

``` python
from IPython.display import Image
from glob import glob

for img in glob('results/CIFAR*/undetected_attacks_*'):
    print(img)
    display(Image(filename=img))
```

%% Cell type:markdown id: tags:

### Questions for Discussion

- Why might this example have gone undetected?
- Out of the adversarial images, which can you identify?
- As we can see from the detection rate, this works fairly well with MNIST; but it works much less well for color images, why might that be?