Skip to content

[lab_supervised] Jaime Bloch #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jbloch3 wants to merge 1 commit into
base: master
Choose a base branch
from
Open

[lab_supervised] Jaime Bloch #2

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
383 changes: 383 additions & 0 deletions your-code/.ipynb_checkpoints/More Supervised Learning Models-checkpoint.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,383 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# More Supervised Learning Models\n",
"\n",
"\n",
"**Lesson Goals**\n",
"\n",
"In this lesson we will expand our repertoire of supervised learning models by introducing naive bayes and k-nearest neighbors. These are two supervised learning models that are typically used for classification problems.\n",
"\n",
"\n",
"**Introduction**\n",
"\n",
"So far, we have discovered a few models for supervised learning. In this lesson, we will explore two different classification models. Naive Bayes is a probabilistic model for classification. K Nearest Neighbors is a model that makes a prediction based on the observations closest to it. Both models make certain assumptions for us to consider them.\n",
"\n",
"\n",
"# Naive Bayes\n",
"\n",
"You may recall Bayes Theorem for conditional probability. This theorem states that the probability of A given B is the probability of the intersection of A and B divided by the probability of B.\n",
"\n",
"We use this rule to create a general model. Say we would like to make a revenue prediction for our e-commerce site. Using Bayes Theorem, we can make a prediction of a customer making a purchase given the version of the website they see and the customer group they are in.\n",
"\n",
"It is important to note that the Naive Bayes algorithm makes a conditional independence assumption. This means that the effect of a single predictor on the outcome is independent on the values of the other predictor variables. This is a simplifying assumption that cannot always be made in some scenarios. Therefore, we should try to see if making this assumption may or may not work with our data.\n",
"\n",
"To calculate the probability of a customer making a purchase given that they are a millennial and looking at site version 1. We are using the model to compute probabilities and compare them. Therefore, we don't care about the denominator. We then get rid of the denominator and change our equation from being equal to, to being proportional to the probability of a purchase given customer group and site version."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Gaussian Naive Bayes\n",
"\n",
"Instead of looking at a probability distribution table, we can make the assumption that our likelihood comes from a Gaussian (or normal) distribution.\n",
"\n",
"To examine the code in Scikit-Learn, we will look at the famous Iris dataset. This dataset was first introduced by Ronald Fisher in 1936 and is used in many classification examples. The Iris dataset contains 4 features for Iris flowers (petal length, petal width, sepal length, and sepal width). The measurements in these variables are used to classify the type of Iris flower. We will import the dataset from Scikit-Learn and then fit the GaussianNB model to the data.\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([[5.1, 3.5, 1.4, 0.2],\n",
" [4.9, 3. , 1.4, 0.2],\n",
" [4.7, 3.2, 1.3, 0.2],\n",
" [4.6, 3.1, 1.5, 0.2],\n",
" [5. , 3.6, 1.4, 0.2],\n",
" [5.4, 3.9, 1.7, 0.4],\n",
" [4.6, 3.4, 1.4, 0.3],\n",
" [5. , 3.4, 1.5, 0.2],\n",
" [4.4, 2.9, 1.4, 0.2],\n",
" [4.9, 3.1, 1.5, 0.1]])"
]
},
"execution_count": 1,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from sklearn import datasets\n",
"from sklearn.naive_bayes import GaussianNB\n",
"\n",
"iris = datasets.load_iris()\n",
"iris.data[:10]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The iris dataset contains the features in the data section and the classification in the target.\n",
"\n",
"Next, we will initialize the GaussianNB model and fit the model. "
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"gnb = GaussianNB()\n",
"y_pred = gnb.fit(iris.data, iris.target).predict(iris.data)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can then generate predictions and compare the predictions with the observed data.\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([[50, 0, 0],\n",
" [ 0, 47, 3],\n",
" [ 0, 3, 47]])"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from sklearn import metrics\n",
"\n",
"y_pred = gnb.fit(iris.data, iris.target).predict(iris.data)\n",
"metrics.confusion_matrix(iris.target, y_pred)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The accuracy of a model is measured by how many observations are classified correctly. All the data correctly classified appears in a confusion matrix along the diagonal. So out of 150 observations, only 6 are incorrectly classified.\n",
"\n",
"\n",
"\n",
"# K-Nearest Neighbors\n",
"\n",
"This algorithm is based on the idea that observations in a \"neighorbood\" will have the same classification. We typically decide whether observations are considered neighbors by a distance metric of our choice. Two common choices for distance metrics are Euclidean distance (defined as the sum of squared distances) or L1 distance (defined as the sum of the absolute value of the distances). We look at the labels of all the observations in the \"neighborhood\" and assign the most common label (the mode) to the observation that we are trying to predict.\n",
"\n",
"Our choice of k is defined by us. We can test different models with multiple values of k and select the model with the highest accuracy. This is the most common way to optimize k.\n",
"\n",
"**Advantages and Disadvantages of K-Nearest Neighbors**\n",
"\n",
"The main advantage is that while we can train a model and then apply it to new data, we do not have to perform this process. We can use the k closest observations with known labels to predict the label of the new observations and make predictions on the fly. However, this means that every time we make a prediction, we have to compute the distance between the observation and all labeled data. This can be computationally intensive and a disadvantage of this algorithm."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**K-Nearest Neighbors with Scikit-Learn**\n",
"\n",
"We are able to apply a K-Nearest Neighbors model to our data using the KNeighborsClassifier in Scikit-Learn. In the example below, we will use the abalone data from the UCI dataset repository. This data contains 8 features describing different abalone observations. Using these features, we are able to predict the sex of the abalone (Male, Female, or Infant).\n",
"\n",
"We'll start by loading the data and examining it."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>Sex</th>\n",
" <th>Length</th>\n",
" <th>Diameter</th>\n",
" <th>Height</th>\n",
" <th>Whole_Weight</th>\n",
" <th>Shucked_Weight</th>\n",
" <th>Visecra_Weight</th>\n",
" <th>Shell_Weight</th>\n",
" <th>Rings</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>M</td>\n",
" <td>0.455</td>\n",
" <td>0.365</td>\n",
" <td>0.095</td>\n",
" <td>0.5140</td>\n",
" <td>0.2245</td>\n",
" <td>0.1010</td>\n",
" <td>0.150</td>\n",
" <td>15</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>M</td>\n",
" <td>0.350</td>\n",
" <td>0.265</td>\n",
" <td>0.090</td>\n",
" <td>0.2255</td>\n",
" <td>0.0995</td>\n",
" <td>0.0485</td>\n",
" <td>0.070</td>\n",
" <td>7</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>F</td>\n",
" <td>0.530</td>\n",
" <td>0.420</td>\n",
" <td>0.135</td>\n",
" <td>0.6770</td>\n",
" <td>0.2565</td>\n",
" <td>0.1415</td>\n",
" <td>0.210</td>\n",
" <td>9</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>M</td>\n",
" <td>0.440</td>\n",
" <td>0.365</td>\n",
" <td>0.125</td>\n",
" <td>0.5160</td>\n",
" <td>0.2155</td>\n",
" <td>0.1140</td>\n",
" <td>0.155</td>\n",
" <td>10</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>I</td>\n",
" <td>0.330</td>\n",
" <td>0.255</td>\n",
" <td>0.080</td>\n",
" <td>0.2050</td>\n",
" <td>0.0895</td>\n",
" <td>0.0395</td>\n",
" <td>0.055</td>\n",
" <td>7</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" Sex Length Diameter Height Whole_Weight Shucked_Weight Visecra_Weight \\\n",
"0 M 0.455 0.365 0.095 0.5140 0.2245 0.1010 \n",
"1 M 0.350 0.265 0.090 0.2255 0.0995 0.0485 \n",
"2 F 0.530 0.420 0.135 0.6770 0.2565 0.1415 \n",
"3 M 0.440 0.365 0.125 0.5160 0.2155 0.1140 \n",
"4 I 0.330 0.255 0.080 0.2050 0.0895 0.0395 \n",
"\n",
" Shell_Weight Rings \n",
"0 0.150 15 \n",
"1 0.070 7 \n",
"2 0.210 9 \n",
"3 0.155 10 \n",
"4 0.055 7 "
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import pandas as pd\n",
"\n",
"abalone_url = 'https://archive.ics.uci.edu/ml/machine-learning-databases/abalone/abalone.data'\n",
"abalone_cols = ['Sex', 'Length', 'Diameter', 'Height', 'Whole_Weight', \n",
" 'Shucked_Weight', 'Visecra_Weight', 'Shell_Weight', 'Rings']\n",
"abalone = pd.read_csv(abalone_url, names=abalone_cols)\n",
"abalone.head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Next, we will load the KNeighborsClassifier from Scikit-Learn and create a model with k=3. "
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',\n",
" metric_params=None, n_jobs=None, n_neighbors=5, p=2,\n",
" weights='uniform')"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from sklearn.neighbors import KNeighborsClassifier\n",
"\n",
"# We create a list of all feature columns.\n",
"cols = [x for x in abalone.columns.values if x != 'Sex']\n",
"\n",
"neighbor_model = KNeighborsClassifier(n_neighbors=3)\n",
"neighbor_model.fit(abalone[cols], abalone['Sex']) \n",
"KNeighborsClassifier()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now that we have created our model, we will create a single observation and predict the sex of this observation."
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"['I']\n"
]
}
],
"source": [
"import numpy as np\n",
"\n",
"obs = np.array([[0.5, 0.3, 0.05, 0.6, 0.2, 0.1, 0.1, 8]])\n",
"print(neighbor_model.predict(obs))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.5"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Loading