Initial commit (5e5572cf) · Commits · examples / Advanced Machine Learning with scikitlearn

Advanced Machine Learning with scikit-learn - Working Files/Chapter 1/API Overview.ipynb

0 → 100755

+51 −0

Changes for Advanced Machine Learning with scikit-learn - Working Files/Chapter 1/API Overview.ipynb: 51 added lines, 0 removed lines.
Original line number	Diff line number	Diff line
		%% Cell type:markdown id: tags:

		# A recap on Scikit-learn's estimator interface

		%% Cell type:markdown id: tags:

		``X`` : data, 2d numpy array or scipy sparse matrix of shape (n_samples, n_features)

		``y`` : targets, 1d numpy array of shape (n_samples,)

		%% Cell type:markdown id: tags:

		## Methods

		%% Cell type:markdown id: tags:

		<table>
		<tr style="border:None; font-size:20px; padding:10px;"><th colspan=2>``model.fit(X_train, [y_train])``</td></tr>
		<tr style="border:None; font-size:20px; padding:10px;"><th>``model.predict(X_test)``</th><th>``model.transform(X_test)``</th></tr>
		<tr style="border:None; font-size:20px; padding:10px;"><td>Classification</td><td>Preprocessing</td></tr>
		<tr style="border:None; font-size:20px; padding:10px;"><td>Regression</td><td>Dimensionality Reduction</td></tr>
		<tr style="border:None; font-size:20px; padding:10px;"><td>Clustering</td><td>Feature Extraction</td></tr>
		<tr style="border:None; font-size:20px; padding:10px;"><td> </td><td>Feature selection</td></tr>
		</table>

		%% Cell type:markdown id: tags:

		## Efficient alternatives, methods for models that don't generalize
		``model.fit_predict(X)`` (clustering)

		``model.fit_transform(X)`` (manifold learning)

		%% Cell type:markdown id: tags:

		###Additional methods
		__Model evaluation__ : ``score(X, [y])``

		__Uncertainties from Classifiers__: ``decision_function(X)`` and ``predict_proba(X)``.

		%% Cell type:markdown id: tags:

		##Attributes
		__Classifiers__: ``classes_``

		__Clustering__: ``labels_``

		__Manifold Learning__: ``embedding_``

		__Linear models__: ``coef_``

		__Linear Decompositions__: ``components_``

Advanced Machine Learning with scikit-learn - Working Files/Chapter 1/Classification.ipynb

0 → 100755

+194 −0

Changes for Advanced Machine Learning with scikit-learn - Working Files/Chapter 1/Classification.ipynb: 194 added lines, 0 removed lines.
Original line number	Diff line number	Diff line
		%% Cell type:code id: tags:

		``` python
		import matplotlib.pyplot as plt
		import numpy as np
		%matplotlib inline
		```

		%% Cell type:markdown id: tags:

		Classification
		===============

		%% Cell type:markdown id: tags:

		Get some data to play with

		%% Cell type:code id: tags:

		``` python
		from sklearn.datasets import load_digits
		digits = load_digits()
		digits.keys()
		```

		%% Cell type:code id: tags:

		``` python
		digits.images.shape
		```

		%% Cell type:code id: tags:

		``` python
		print(digits.images[0])
		```

		%% Cell type:code id: tags:

		``` python
		plt.matshow(digits.images[0], cmap=plt.cm.Greys)
		```

		%% Cell type:code id: tags:

		``` python
		digits.data.shape
		```

		%% Cell type:code id: tags:

		``` python
		digits.target.shape
		```

		%% Cell type:code id: tags:

		``` python
		digits.target
		```

		%% Cell type:markdown id: tags:

		Data is always a numpy array (or sparse matrix) of shape (n_samples, n_features)

		%% Cell type:markdown id: tags:

		Split the data to get going

		%% Cell type:code id: tags:

		``` python
		from sklearn.cross_validation import train_test_split
		X_train, X_test, y_train, y_test = train_test_split(digits.data, digits.target)
		```

		%% Cell type:markdown id: tags:

		Really Simple API
		-------------------
		0) Import your model class

		%% Cell type:code id: tags:

		``` python
		from sklearn.svm import LinearSVC
		```

		%% Cell type:markdown id: tags:

		1) Instantiate an object and set the parameters

		%% Cell type:code id: tags:

		``` python
		svm = LinearSVC(C=0.1)
		```

		%% Cell type:markdown id: tags:

		2) Fit the model

		%% Cell type:code id: tags:

		``` python
		svm.fit(X_train, y_train)
		```

		%% Cell type:markdown id: tags:

		3) Apply / evaluate

		%% Cell type:code id: tags:

		``` python
		print(svm.predict(X_train))
		print(y_train)
		```

		%% Cell type:code id: tags:

		``` python
		svm.score(X_train, y_train)
		```

		%% Cell type:code id: tags:

		``` python
		svm.score(X_test, y_test)
		```

		%% Cell type:markdown id: tags:

		And again
		---------

		%% Cell type:code id: tags:

		``` python
		from sklearn.ensemble import RandomForestClassifier
		```

		%% Cell type:code id: tags:

		``` python
		rf = RandomForestClassifier(n_estimators=50)
		```

		%% Cell type:code id: tags:

		``` python
		rf.fit(X_train, y_train)
		```

		%% Cell type:code id: tags:

		``` python
		rf.predict(X_test)
		```

		%% Cell type:code id: tags:

		``` python
		rf.score(X_test, y_test)
		```

		%% Cell type:markdown id: tags:

		<h1>Classifier Comparison</h1>
		<img src="classifier_comparison.png" width=100%>

		%% Cell type:markdown id: tags:

		Labels Can Be Anything
		=============================

		%% Cell type:code id: tags:

		``` python
		numbers = np.array(["zero", "one", "two", "three", "four", "five", "six", "seven", "eight", "nine"])
		```

		%% Cell type:code id: tags:

		``` python
		y_train_string = numbers[y_train]
		svm.fit(X_train, y_train_string)
		```

		%% Cell type:code id: tags:

		``` python
		svm.predict(X_test)
		```

Advanced Machine Learning with scikit-learn - Working Files/Chapter 1/Clustering.ipynb

0 → 100755

+210 −0

Changes for Advanced Machine Learning with scikit-learn - Working Files/Chapter 1/Clustering.ipynb: 210 added lines, 0 removed lines.
Original line number	Diff line number	Diff line
		%% Cell type:code id: tags:

		``` python
		import matplotlib.pyplot as plt
		import numpy as np
		%matplotlib inline
		```

		%% Cell type:markdown id: tags:

		Clustering
		=============

		%% Cell type:code id: tags:

		``` python
		from sklearn.datasets import make_blobs
		X, y = make_blobs(random_state=42)
		X.shape
		```

		%% Cell type:code id: tags:

		``` python
		plt.scatter(X[:, 0], X[:, 1])
		```

		%% Cell type:code id: tags:

		``` python
		from sklearn.cluster import KMeans
		kmeans = KMeans(n_clusters=3)
		```

		%% Cell type:code id: tags:

		``` python
		kmeans.fit(X)
		```

		%% Cell type:code id: tags:

		``` python
		cluster_labels = kmeans.predict(X)
		```

		%% Cell type:code id: tags:

		``` python
		cluster_labels
		```

		%% Cell type:code id: tags:

		``` python
		plt.scatter(X[:, 0], X[:, 1], c=cluster_labels)
		```

		%% Cell type:code id: tags:

		``` python
		y
		```

		%% Cell type:code id: tags:

		``` python
		from sklearn.metrics import accuracy_score, confusion_matrix
		```

		%% Cell type:code id: tags:

		``` python
		accuracy_score(y, cluster_labels)
		```

		%% Cell type:code id: tags:

		``` python
		confusion_matrix(y, cluster_labels)
		```

		%% Cell type:code id: tags:

		``` python
		from sklearn.metrics import adjusted_rand_score
		```

		%% Cell type:code id: tags:

		``` python
		adjusted_rand_score(y, cluster_labels)
		```

		%% Cell type:markdown id: tags:

		KMeans can generalize, SpectralClustering can not
		-----------------------------------------------------------
		KMeans has a ``predict`` function.

		%% Cell type:code id: tags:

		``` python
		X_more = np.random.uniform(X.min(), X.max(), size=(10, 2))
		X_more
		more_cluster_labels = kmeans.predict(X_more)
		```

		%% Cell type:code id: tags:

		``` python
		plt.scatter(X[:, 0], X[:, 1], c=cluster_labels)
		plt.scatter(X_more[:, 0], X_more[:, 1], marker="x", c=more_cluster_labels)
		```

		%% Cell type:code id: tags:

		``` python
		from sklearn.cluster import SpectralClustering
		spectral_clustering = SpectralClustering(n_clusters=3, gamma=.1)
		```

		%% Cell type:code id: tags:

		``` python
		spectral_clustering.fit(X)
		```

		%% Cell type:code id: tags:

		``` python
		cluster_labels = spectral_clustering.predict(X)
		```

		%% Cell type:code id: tags:

		``` python
		spectral_clustering.labels_
		```

		%% Cell type:code id: tags:

		``` python
		cluster_labels = spectral_clustering.fit_predict(X)
		```

		%% Cell type:code id: tags:

		``` python
		plt.scatter(X[:, 0], X[:, 1], c=cluster_labels)
		```

		%% Cell type:markdown id: tags:

		There is no way to apply ``spectral_clustering`` not to ``X_more``.

		%% Cell type:markdown id: tags:

		An overview of clustering algorithms
		---------------------------------------

		%% Cell type:markdown id: tags:

		<img src="cluster_comparison.png" width=100%>

		%% Cell type:markdown id: tags:

		More on the website: http://scikit-learn.org/stable/modules/clustering.html

		%% Cell type:markdown id: tags:

		A less trivial example
		-------------------------

		%% Cell type:code id: tags:

		``` python
		from sklearn.datasets import load_digits
		digits = load_digits()

		X, y = digits.data, digits.target
		```

		%% Cell type:code id: tags:

		``` python
		kmeans = KMeans(n_clusters=10)
		kmeans.fit(X)
		```

		%% Cell type:code id: tags:

		``` python
		adjusted_rand_score(y, kmeans.predict(X))
		```

		%% Cell type:code id: tags:

		``` python
		_, axes = plt.subplots(2, 5)
		for ax, center in zip(axes.ravel(), kmeans.cluster_centers_):
		ax.matshow(center.reshape(8, 8), cmap=plt.cm.gray)
		ax.set_xticks(())
		ax.set_yticks(())
		```

		%% Cell type:code id: tags:

		``` python
		```

Advanced Machine Learning with scikit-learn - Working Files/Chapter 1/Cross Validation.ipynb

0 → 100755

+159 −0

Changes for Advanced Machine Learning with scikit-learn - Working Files/Chapter 1/Cross Validation.ipynb: 159 added lines, 0 removed lines.
Original line number	Diff line number	Diff line
		%% Cell type:code id: tags:

		``` python
		import matplotlib.pyplot as plt
		import numpy as np
		%matplotlib inline
		```

		%% Cell type:markdown id: tags:

		Cross-Validation
		=====================================

		%% Cell type:code id: tags:

		``` python
		from sklearn.datasets import load_iris
		from sklearn.neighbors import KNeighborsClassifier

		iris = load_iris()
		X, y = iris.data, iris.target
		n_samples = X.shape[0]
		print(X.shape)
		print(y.shape)
		print(y)
		```

		%% Cell type:markdown id: tags:

		### Validation with a training / test split

		%% Cell type:markdown id: tags:

		![train test split](train_test_split.svg)

		%% Cell type:code id: tags:

		``` python
		from sklearn.cross_validation import train_test_split
		X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)
		print(X_train.shape)
		print(X_test.shape)
		```

		%% Cell type:code id: tags:

		``` python
		classifier = KNeighborsClassifier()
		classifier.fit(X_train, y_train)
		classifier.score(X_test, y_test)
		```

		%% Cell type:markdown id: tags:

		### Cross-validation for more robust estimates

		%% Cell type:markdown id: tags:

		![cross-validation](cross_validation.svg)

		%% Cell type:code id: tags:

		``` python
		from sklearn.cross_validation import cross_val_score
		scores = cross_val_score(classifier, X, y)
		print(scores)
		print(np.mean(scores))
		```

		%% Cell type:code id: tags:

		``` python
		cross_val_score(classifier, X, y, cv=5)
		```

		%% Cell type:markdown id: tags:

		### Custom cross-validation iterators

		%% Cell type:code id: tags:

		``` python
		from sklearn.cross_validation import KFold, StratifiedKFold, ShuffleSplit, LeavePLabelOut
		```

		%% Cell type:code id: tags:

		``` python
		cv = StratifiedKFold(iris.target, n_folds=5)
		for train, test in cv:
		print(test)
		```

		%% Cell type:code id: tags:

		``` python
		def plot_cv(cv, n_samples):
		masks = []
		for train, test in cv:
		mask = np.zeros(n_samples, dtype=bool)
		mask[test] = 1
		masks.append(mask)

		plt.matshow(masks)
		```

		%% Cell type:code id: tags:

		``` python
		cv = StratifiedKFold(y, n_folds=5)
		plot_cv(cv, n_samples)
		```

		%% Cell type:code id: tags:

		``` python
		cv = KFold(n_samples, n_folds=5)
		plot_cv(cv, n_samples)
		```

		%% Cell type:code id: tags:

		``` python
		cv = KFold(n_samples, n_folds=5, shuffle=True)
		plot_cv(cv, n_samples)
		```

		%% Cell type:code id: tags:

		``` python
		cv = KFold(n_samples, n_folds=10)
		plot_cv(cv, n_samples)
		```

		%% Cell type:code id: tags:

		``` python
		cv = ShuffleSplit(n_samples, n_iter=5, test_size=.2)
		plot_cv(cv, n_samples)
		```

		%% Cell type:code id: tags:

		``` python
		cv = ShuffleSplit(n_samples, n_iter=20, test_size=.2)
		plot_cv(cv, n_samples)
		```

		%% Cell type:code id: tags:

		``` python
		cv = ShuffleSplit(n_samples, n_iter=5, test_size=.2)
		cross_val_score(classifier, X, y, cv=cv)
		```

		%% Cell type:code id: tags:

		``` python
		```

Advanced Machine Learning with scikit-learn - Working Files/Chapter 1/Grid Searches.ipynb

0 → 100755

+768 −0

File added.

Preview size limit exceeded, changes collapsed.