Commit e5435d75 authored by kjam's avatar kjam
Browse files

Merge remote-tracking branch 'oreilly/oreilly'

parents 1ff68a1e 9768efb5
Loading
Loading
Loading
Loading
+4 −0
Changes for .gitignore: 4 added lines, 0 removed lines.
Original line number Diff line number Diff line
<<<<<<< HEAD
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
@@ -102,3 +103,6 @@ venv.bak/

# mypy
.mypy_cache/
=======
.ipynb*
>>>>>>> oreilly/oreilly

Dockerfile

0 → 100644
+31 −0
Changes for Dockerfile: 31 added lines, 0 removed lines.
Original line number Diff line number Diff line
FROM jupyter/<notebook>:<version hash>

# launchbot-specific labels
LABEL name.launchbot.io="Cleaning data at scale"
LABEL workdir.launchbot.io="/home/jovyan"
LABEL description.launchbot.io="Cleaning data at scale"
LABEL 8888.port.launchbot.io="Start Tutorial"

#Set the working directory
WORKDIR /home/jovyan/

# Modules
COPY requirements.txt /home/jovyan/requirements.txt
RUN pip install -r /home/jovyan/requirements.txt

# Add files
COPY notebooks /home/jovyan/notebooks
COPY data /home/jovyan/data
COPY solutions /home/jovyan/solutions

# Allow user to write to directory
USER root
RUN chown -R $NB_USER /home/jovyan \
    && chmod -R 774 /home/jovyan
USER $NB_USER

# Expose the notebook port
EXPOSE 8888

# Start the notebook server
CMD jupyter notebook --no-browser --port 8888 --ip=* --NotebookApp.token='' --NotebookApp.disable_check_xsrf=True --NotebookApp.iopub_data_rate_limit=1.0e10
+41 −14
Changes for README.md: 41 added lines, 14 removed lines.
Original line number Diff line number Diff line
## Data Cleaning 101 
## Data Security for Machine Learning 

Welcome to the code repository for Practical Data Cleaning with Python! This is a two-day training offered through Safari with O'Reilly media. You can sign up by searching for the course on Safari.
Welcome to the code repository for Data Security for Machine Learning! This is a one-day training offered through Safari with O'Reilly media. You can sign up by searching for the course on Safari.

This course aims to give you a practical overview of data cleaning and validation libraries and methods in Python. Since we only have 6 hours, it can't go massively in-depth into any one library or tool, but I have tried to include useful tools I have found in my work and incorporate a mixture of the munging and testing I have seen in my own and others workflows. 
This course will cover some attacks and defenses for security machine learning models; primarily using libraries provided by researchers in the field. As we only have 3-hours, we will not be able to go into depth into each attack -- but I hope this repository and the libraries used and papers referenced in the slides allow you to explore deeper after the course is finished.

If you have a suggestion for another library or additional topic, feel free to drop me a line :)

### Installation
## Repository Information


This project contains the Jupyter Notebooks and the associated Dockerfile for Katharine Jarmul's Data Security for Machine Learning course. It contains both the exercises (/notebooks) and the solutions (/solutions), as well as any data or files needed (/data).

This is a public repository so there is no need to create an account to download its contents. To download the source code from this page, click the 'Cloud' icon on the top right hand, above where the latest commit is detailed.

To download via git from your preferred terminal application, type: 

```git clone https://resources.oreilly.com/live-training/${COMPLETE URL}```

## Running Jupyter Locally via Docker

We've shared the same Dockerfile we use for our JupyterHub session in this repository, to make sure you can run all of these notebooks in your own time, on your own machine. This isn't required during class, but can be useful for learning once the session is over.

You will need to have Docker installed on your system to create images and run containers. You can find the installation steps for all platforms on the company's [website](https://docs.docker.com/install/)
.

1) Clone the repository for the class either using the UI or your terminal (see above)..

2) Once you have Docker installed, type the following on your terminal to create a Docker image: `docker build -t ${TITLE} .` (note the period)

3) That will take a little while to create a Docker image, but once completed, you can run your server with the following:
`docker run -p 8888:8888 ${TITLE}`

4) Head to `localhost:8888` in your browser and you will be able to access the Jupyter Notebooks.


### Local Installation

**All of the coding exercises in the course will be hosted on JupyterHub, and we'll send the URL out at the start of class. Purely browser-based, no installations required. You will be able to download the jupyter notebooks to save.**

@@ -14,26 +42,25 @@ If you would like to install locally, please follow these steps:

These lessons has been tested for Python 3.4 and Python 3.6 and primarily uses the latest release of each library, except where versions are pinned. You likely can run most of the code with older releases, but if you run into an issue, try upgrading the library in question first.

```pip install -r install_reqs.txt```
```pip install -r requirements.txt```


I believe this will also work with Conda, although I am less familiar with Conda so please report issues! (special thanks to @blue_hacker for this fix!)

```
$ conda create -n dataclean --copy python=3.6
$ source activate dataclean
$ pip install -r install_reqs.txt
$ conda create -n mlsecurity --copy python=3.6
$ source activate mlsecurity
$ pip install -r requirements.txt
```

In addition, you will need to install [sqlite3](https://www.sqlite.org/) or make changes to the second day case study with a connection string to your database of choice. [more info](https://dataset.readthedocs.io/en/latest/quickstart.html#connecting-to-a-database)

If you want to visualize graphs using Dask, you will need to install [Graphviz](http://www.graphviz.org/), which has special requirements on all platforms. For linux, it is usually available via the system package library (apt, yum). For other platforms, you might need to use a special installer. It is also [available via conda install graphviz](https://anaconda.org/anaconda/graphviz) and [pip install graphviz](https://pypi.python.org/pypi/graphviz), but these might not include all necessary dependencies for your OS. For best results, search for your
OS and "install graphviz and dependencies" and follow a recent article on setup.

### Repository structure

Each day coincides with a particular notebook folder. For day one, we will use `cleaning-notebooks`. Day two will focus on `validation-notebooks`. The `data` folder holds data we will use throughout the course. The `queue_example.py` file is used in the day two case study.
This repository contains some external submodules with modifications -- this is primarily due to making things compatible with Py3 or Jupyter (so we can view the results in JupyterHub). Some repositories you might want to take a look at installing individually or tracking changes are:

- [Cleverhans](https://github.com/tensorflow/cleverhans)
- [Foolbox](https://github.com/bethgelab/foolbox)
- [ModelExtraction](https://github.com/ftramer/Steal-ML)
- [EvadeML-Zoo](https://github.com/mzweilin/EvadeML-Zoo)

### Python2 v. Python3

create_jh/Dockerfile

0 → 100644
+28 −0
Changes for create_jh/Dockerfile: 28 added lines, 0 removed lines.
Original line number Diff line number Diff line
FROM us.gcr.io/jupyterhub-170614/<series_id>:<single user version>

USER root

# notebook and ipykernel should already be installed. now just need jupyterhub
RUN pip install --upgrade --no-cache-dir jupyterhub==0.8.1

# Specify tornado==4.5.3
RUN pip install --upgrade --no-cache-dir tornado==4.5.3

USER jovyan

WORKDIR /home/jovyan

# css stuff
RUN for i in `find / -wholename /opt/conda/*/custom.css 2>/dev/null` ; \
  do echo "@import url(\"https://cdn.oreillystatic.com/jupyterhub/jupyterhub-theme/custom/custom.css\")" \
  | tee ${i}; done

CMD jupyterhub-singleuser \
  --port=8888 \
  --ip=0.0.0.0 \
  --user="$JPY_USER" \
  --cookie-name=$JPY_COOKIE_NAME \
  --base-url=$JPY_BASE_URL \
  --hub-prefix=$JPY_HUB_PREFIX \
  --hub-api-url=$JPY_HUB_API_URL \
+4 −0
Changes for notebooks/04 - Case Study - Adversarial Learning.ipynb: 4 added lines, 0 removed lines.
Original line number Diff line number Diff line
%% Cell type:code id: tags:

``` python
```