Commit 577ed1ca authored by Katharine Jarmul's avatar Katharine Jarmul
Browse files

updating with hidden solution

parent f28fbb8f
Loading
Loading
Loading
Loading
+1 −5
Changes for notebooks/04 - Case Study - Adversarial Detection.ipynb: 1 added line, 5 removed lines.
Original line number Diff line number Diff line
%% Cell type:markdown id: tags:

### Adversarial Detection using FeatureSqueezing and MagNet

In this notebook, we'll explore adversarial detection using FeatureSqueezing and MagNet. This code has been adapted from [the EvadeML repository](https://github.com/mzweilin/EvadeML-Zoo) with changes to enable it to run properly with Python3 and Jupyter notebooks.

To run locally, I recommend following the latest instructions on the EvadeML repository and the [MagNet repository](https://github.com/Trevillie/MagNet).

%% Cell type:code id: tags:

``` python
%matplotlib inline
```

%% Cell type:code id: tags:

``` python
cd ../EvadeML-Zoo
```

%% Output

    /home/katharine/wrrk/my_classes/oreilly_live/data-security-for-data-science/EvadeML-Zoo

%% Cell type:code id: tags:

``` python
from main import main
```

%% Output

    Using TensorFlow backend.

%% Cell type:code id: tags:

``` python
!python main.py -h
```

%% Cell type:code id: tags:

``` python
main(dataset_name="MNIST",
     model_name="cleverhans",
     detection="MagNet",
     attacks="FGSM?eps=0.1;BIM?eps=0.1&eps_iter=0.02;JSMA?targeted=next"
)
```

%% Cell type:markdown id: tags:

### Let's check out the adversarial examples generated

%% Cell type:code id: tags:

``` python
from IPython.display import Image
Image(filename=
      'results/MNIST_100_1d1b8_cleverhans/MNIST_100_1d1b8_cleverhans_attacks_07450_examples.png')
```

%% Cell type:markdown id: tags:

### And let's take a look at what attacks were not detected!

%% Cell type:code id: tags:

``` python
from IPython.display import Image
Image(filename=
      'results/MNIST_100_1d1b8_cleverhans/attacks_jsma?targeted=next_undetected.png')
```

%% Cell type:markdown id: tags:

#### Your Turn:
- How does FeatureSqueezing compare as a detection mechanism? To use the FeatureSqueezing, you will need to add proper parameters. Run the below cell for a few examples.
- Try a different attack or detection (to see all options, use `main(show_help=True)`).
- Try setting different parameters for the cleverhans attack above.
- Are your new attacks more successful? Why or why not?

%% Cell type:code id: tags:

``` python
```

%% Cell type:code id: tags:

``` python
```

%% Cell type:code id: tags:

``` python
!cat Reproduce_FeatureSqueezing.md | grep FeatureSqueezing
```

%% Cell type:code id: tags:

``` python
main(dataset_name="MNIST",
     model_name="cleverhans",
     detection="FeatureSqueezing?squeezers=bit_depth_5,median_filter_2_2&distance_measure=l1&fpr=0.05",
     attacks="FGSM?eps=0.1;BIM?eps=0.1&eps_iter=0.02;JSMA?targeted=next"
)
# %load ../solutions/feature_squeezing_detection.py
```

%% Cell type:code id: tags:

``` python
from IPython.display import Image
from glob import glob

for img in glob('results/MNIST_100_1d1b8_cleverhans/undetected_attacks_*'):
    print(img)
    display(Image(filename=img))
```

%% Cell type:code id: tags:

``` python
main(dataset_name="CIFAR-10",
     model_name="DenseNet",
     attacks="fgsm?eps=0.0156;bim?eps=0.008&eps_iter=0.0012;carlinili?targeted=ll&confidence=5;deepfool?overshoot=10;carlinil2?targeted=next&batch_size=100&max_iterations=1000&confidence=5;carlinil2?targeted=ll&batch_size=100&max_iterations=1000&confidence=5;carlinil0?targeted=next&confidence=5;carlinil0?targeted=ll&confidence=5;jsma?targeted=next;jsma?targeted=ll;",
     detection="FeatureSqueezing?squeezers=bit_depth_5,median_filter_2_2,non_local_means_color_13_3_2&distance_measure=l1&threshold=1.7547;MagNet"
)
```

%% Output

    Located Cleverhans
    Located Carlini_nn_robust_attacks
    Located Keras-deep-learning-models
    Located MobileNets
    Located Deepfool/Universal
    Located DenseNet
    Located MagNet
    Flags are {'dataset_name': 'CIFAR-10', 'model_name': 'densenet', 'select': True, 'nb_examples': 100, 'balance_sampling': False, 'test_mode': False, 'attacks': 'fgsm?eps=0.0156;bim?eps=0.008&eps_iter=0.0012;carlinili?targeted=ll&confidence=5;deepfool?overshoot=10;carlinil2?targeted=next&batch_size=100&max_iterations=1000&confidence=5;carlinil2?targeted=ll&batch_size=100&max_iterations=1000&confidence=5;carlinil0?targeted=next&confidence=5;carlinil0?targeted=ll&confidence=5;jsma?targeted=next;jsma?targeted=ll;', 'clip': -1, 'visualize': True, 'robustness': '', 'detection': 'FeatureSqueezing?squeezers=bit_depth_5,median_filter_2_2,non_local_means_color_13_3_2&distance_measure=l1&threshold=1.7547;MagNet', 'detection_train_test_mode': True, 'result_folder': 'results', 'verbose': False}
    
    ===Loading CIFAR-10 data...
    Created TensorFlow session and set Keras backend.
    
    ===Defined TensorFlow model graph.
    ---Loaded CIFAR-10-densenet model.
    
    Evaluating the pre-trained model...
    Test accuracy on raw legitimate examples 0.9484
    Mean confidence on ground truth classes 0.9215
    Selected 100 examples.
    Selected index in test set (sorted): 0-52:1,54-57:1,59,60,62-67:1,69-77:1,79-99:1,101-105:1
    Test accuracy on selected legitimate examples 1.0000
    Mean confidence on ground truth classes, selected 0.9588
    
    
    Running attack: fgsm {'eps': 0.0156}
    Loading adversarial examples from [CIFAR-10_100_de671_densenet_fgsm?eps=0.0156.pickle].
    
    ---Attack (uint8): fgsm?eps=0.0156
    Success rate: 85.00%, Mean confidence of SAEs: 97.29%
    ### Statistics of the SAEs:
    L2 dist: 0.8647, Li dist: 0.0157, L0 dist_value: 99.5%, L0 dist_pixel: 99.8%
    
    Running attack: bim {'eps': 0.008, 'eps_iter': 0.0012}
    Loading adversarial examples from [CIFAR-10_100_de671_densenet_bim?eps=0.008&eps_iter=0.0012.pickle].
    
    ---Attack (uint8): bim?eps=0.008&eps_iter=0.0012
    Success rate: 92.00%, Mean confidence of SAEs: 98.40%
    ### Statistics of the SAEs:
    L2 dist: 0.3686, Li dist: 0.0078, L0 dist_value: 92.4%, L0 dist_pixel: 99.5%
    
    Running attack: carlinili {'targeted': 'll', 'confidence': 5.0}
    targeted value: ll

    ---------------------------------------------------------------------------
    KeyboardInterrupt                         Traceback (most recent call last)
    <ipython-input-4-8da050370854> in <module>
          2      model_name="DenseNet",
          3      attacks="fgsm?eps=0.0156;bim?eps=0.008&eps_iter=0.0012;carlinili?targeted=ll&confidence=5;deepfool?overshoot=10;carlinil2?targeted=next&batch_size=100&max_iterations=1000&confidence=5;carlinil2?targeted=ll&batch_size=100&max_iterations=1000&confidence=5;carlinil0?targeted=next&confidence=5;carlinil0?targeted=ll&confidence=5;jsma?targeted=next;jsma?targeted=ll;",
    ----> 4      detection="FeatureSqueezing?squeezers=bit_depth_5,median_filter_2_2,non_local_means_color_13_3_2&distance_measure=l1&threshold=1.7547;MagNet"
          5 )
    ~/wrrk/my_classes/oreilly_live/data-security-for-data-science/EvadeML-Zoo/main.py in main(dataset_name, model_name, attacks, nb_examples, detection, show_help)
        235         x_adv_fpath = os.path.join(X_adv_cache_folder, x_adv_fname)
        236
    --> 237         X_test_adv, aux_info = maybe_generate_adv_examples(sess, model, x, y, X_test, Y_test_target, attack_name, attack_params, use_cache = x_adv_fpath, verbose=FLAGS.verbose, attack_log_fpath=attack_log_fpath)
        238
        239         if FLAGS.clip > 0:
    ~/wrrk/my_classes/oreilly_live/data-security-for-data-science/EvadeML-Zoo/attacks/__init__.py in maybe_generate_adv_examples(sess, model, x, y, X, Y, attack_name, attack_params, use_cache, verbose, attack_log_fpath)
         21     else:
         22         time_start = time.time()
    ---> 23         X_adv = generate_adv_examples(sess, model, x, y, X, Y, attack_name, attack_params, verbose, attack_log_fpath)
         24         duration = time.time() - time_start
         25
    ~/wrrk/my_classes/oreilly_live/data-security-for-data-science/EvadeML-Zoo/attacks/__init__.py in generate_adv_examples(sess, model, x, y, X, Y, attack_name, attack_params, verbose, attack_log_fpath)
         62         raise NotImplementedError("Unsuported attack [%s]." % attack_name)
         63
    ---> 64     X_adv = generate_adv_examples_func(sess, model, x, y, X, Y, attack_params, verbose, attack_log_fpath)
         65
         66     return X_adv
    ~/wrrk/my_classes/oreilly_live/data-security-for-data-science/EvadeML-Zoo/attacks/carlini_wrapper.py in generate_carlini_li_examples(sess, model, x, y, X, Y, attack_params, verbose, attack_log_fpath)
        114                 if not verbose:
        115                     disablePrint(attack_log_fpath)
    --> 116                 X_adv_sub = attack.attack(X_sub - 0.5, Y_sub) + 0.5
        117                 if not verbose:
        118                     enablePrint()
    ~/wrrk/my_classes/oreilly_live/data-security-for-data-science/EvadeML-Zoo/externals/carlini/nn_robust_attacks/li_attack.py in attack(self, imgs, targets)
        159         r = []
        160         for img,target in zip(imgs, targets):
    --> 161             r.extend(self.attack_single(img, target))
        162         return np.array(r)
        163
    ~/wrrk/my_classes/oreilly_live/data-security-for-data-science/EvadeML-Zoo/externals/carlini/nn_robust_attacks/li_attack.py in attack_single(self, img, target)
        174         while tau > 1./256:
        175             # try to solve given this tau value
    --> 176             res = self.grad([np.copy(img)], [target], np.copy(prev), tau, const)
        177             if res == None:
        178                 # the attack failed, we return this as our final answer
    ~/wrrk/my_classes/oreilly_live/data-security-for-data-science/EvadeML-Zoo/externals/carlini/nn_robust_attacks/li_attack.py in doit(oimgs, labs, starts, tt, CONST)
        133
        134                     # perform the update step
    --> 135                     _, works = sess.run([train, loss], feed_dict=feed_dict)
        136
        137                     # it worked
    ~/.virtualenv/py3security/lib/python3.6/site-packages/tensorflow/python/client/session.py in run(self, fetches, feed_dict, options, run_metadata)
        776     try:
        777       result = self._run(None, fetches, feed_dict, options_ptr,
    --> 778                          run_metadata_ptr)
        779       if run_metadata:
        780         proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)
    ~/.virtualenv/py3security/lib/python3.6/site-packages/tensorflow/python/client/session.py in _run(self, handle, fetches, feed_dict, options, run_metadata)
        980     if final_fetches or final_targets:
        981       results = self._do_run(handle, final_targets, final_fetches,
    --> 982                              feed_dict_string, options, run_metadata)
        983     else:
        984       results = []
    ~/.virtualenv/py3security/lib/python3.6/site-packages/tensorflow/python/client/session.py in _do_run(self, handle, target_list, fetch_list, feed_dict, options, run_metadata)
       1030     if handle is None:
       1031       return self._do_call(_run_fn, self._session, feed_dict, fetch_list,
    -> 1032                            target_list, options, run_metadata)
       1033     else:
       1034       return self._do_call(_prun_fn, self._session, handle, feed_dict,
    ~/.virtualenv/py3security/lib/python3.6/site-packages/tensorflow/python/client/session.py in _do_call(self, fn, *args)
       1037   def _do_call(self, fn, *args):
       1038     try:
    -> 1039       return fn(*args)
       1040     except errors.OpError as e:
       1041       message = compat.as_text(e.message)
    ~/.virtualenv/py3security/lib/python3.6/site-packages/tensorflow/python/client/session.py in _run_fn(session, feed_dict, fetch_list, target_list, options, run_metadata)
       1019         return tf_session.TF_Run(session, options,
       1020                                  feed_dict, fetch_list, target_list,
    -> 1021                                  status, run_metadata)
       1022
       1023     def _prun_fn(session, handle, feed_dict, fetch_list):
    KeyboardInterrupt:

%% Cell type:code id: tags:

``` python
from IPython.display import Image
from glob import glob

for img in glob('results/CIFAR*/undetected_attacks_*'):
    print(img)
    display(Image(filename=img))
```

%% Cell type:markdown id: tags:

### Questions for Discussion

- Why might this example have gone undetected?
- Out of the adversarial images, which can you identify?
- As we can see from the detection rate, this works fairly well with MNIST; but it works much less well for color images, why might that be?
+5 −0
Changes for solutions/feature_squeezing_detection.py: 5 added lines, 0 removed lines.
Original line number Diff line number Diff line
main(dataset_name="MNIST", 
     model_name="cleverhans", 
     detection="FeatureSqueezing?squeezers=bit_depth_5,median_filter_2_2&distance_measure=l1&fpr=0.05",
     attacks="FGSM?eps=0.1;BIM?eps=0.1&eps_iter=0.02;JSMA?targeted=next"
     )