ICA1_DataMining-PartA.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<link href='https://fonts.googleapis.com/css?family=Passion+One' rel='stylesheet' type='text/css'><style>div.attn { font-family: 'Helvetica Neue'; font-size: 30px; line-height: 40px; color: #FFFFFF; text-align: center; margin: 30px 0; border-width: 10px 0; border-style: solid; border-color: #5AAAAA; padding: 30px 0; background-color: #DDDDFF; }hr { border: 0; background-color: #ffffff; border-top: 1px solid black; }hr.major { border-top: 10px solid #5AAA5A; }hr.minor { border: none; background-color: #ffffff; border-top: 5px dotted #CC3333; }div.bubble { width: 65%; padding: 20px; background: #DDDDDD; border-radius: 15px; margin: 0 auto; font-style: italic; color: #f00; }em { color: #AAA; }div.c1{visibility:hidden;margin:0;height:0;}div.note{color:red;}</style>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#  Ebnable HTML/CSS \n",
    "from IPython.core.display import HTML\n",
    "HTML(\"<link href='https://fonts.googleapis.com/css?family=Passion+One' rel='stylesheet' type='text/css'><style>div.attn { font-family: 'Helvetica Neue'; font-size: 30px; line-height: 40px; color: #FFFFFF; text-align: center; margin: 30px 0; border-width: 10px 0; border-style: solid; border-color: #5AAAAA; padding: 30px 0; background-color: #DDDDFF; }hr { border: 0; background-color: #ffffff; border-top: 1px solid black; }hr.major { border-top: 10px solid #5AAA5A; }hr.minor { border: none; background-color: #ffffff; border-top: 5px dotted #CC3333; }div.bubble { width: 65%; padding: 20px; background: #DDDDDD; border-radius: 15px; margin: 0 auto; font-style: italic; color: #f00; }em { color: #AAA; }div.c1{visibility:hidden;margin:0;height:0;}div.note{color:red;}</style>\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "___\n",
    "Enter Team Member Names here (double click to edit):\n",
    "\n",
    "- Name 1:\n",
    "- Name 2:\n",
    "- Name 3:\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# In Class Assignment One\n",
    "In the following assignment you will be asked to fill in python code and derivations for a number of different problems. Please read all instructions carefully and turn in the rendered notebook (or HTML of the rendered notebook)  before the end of class (or right after class). The initial portion of this notebook is given before class and the remainder is given during class. Please answer the initial questions before class. Once class has started you may rework your answers as a team for the initial part of the assignment. \n",
    "\n",
    "## Contents\n",
    "* <a href=\"#Loading\">Loading the Data</a>\n",
    "* <a href=\"#linearnumpy\">Linear Regression</a>\n",
    "\n",
    "**These portions are not yet accessible until the start of class:**\n",
    "* <a href=\"#sklearn\">Using Scikit Learn for Regression</a>\n",
    "* <a href=\"#classification\">Linear Classification</a>\n",
    "\n",
    "________________________________________________________________________________________________________\n",
    "\n",
    "<a id=\"Loading\"></a>\n",
    "## Loading the Data\n",
    "Please run the following code to read in the \"diabetes\" dataset from sklearn's data loading module. \n",
    "\n",
    "This will load the data into the variable `ds`. `ds` is a dictionary object with fields like `ds.data`, which is a matrix of the continuous features in the dataset. The object is not a pandas dataframe. It is a numpy matrix. Each row is a set of observed instances, each column is a different feature. It also has a field called `ds.target` that is a continuous value we are trying to predict. Each entry in `ds.target` is a label for each row of the `ds.data` matrix. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "features shape: (442, 10) format is: ('rows', 'columns')\n",
      "range of target: 25.0 346.0\n"
     ]
    }
   ],
   "source": [
    "from __future__ import print_function\n",
    "from sklearn.datasets import load_diabetes\n",
    "import numpy as np\n",
    "\n",
    "\n",
    "\n",
    "ds = load_diabetes()\n",
    "\n",
    "# this holds the continuous feature data\n",
    "# because ds.data is a matrix, there are some special properties we can access (like 'shape')\n",
    "print('features shape:', ds.data.shape, 'format is:', ('rows','columns')) # there are 442 instances and 10 features per instance\n",
    "print('range of target:', np.min(ds.target),np.max(ds.target))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "array([[ 0.03807591,  0.05068012,  0.06169621, ..., -0.00259226,\n",
      "         0.01990842, -0.01764613],\n",
      "       [-0.00188202, -0.04464164, -0.05147406, ..., -0.03949338,\n",
      "        -0.06832974, -0.09220405],\n",
      "       [ 0.08529891,  0.05068012,  0.04445121, ..., -0.00259226,\n",
      "         0.00286377, -0.02593034],\n",
      "       ...,\n",
      "       [ 0.04170844,  0.05068012, -0.01590626, ..., -0.01107952,\n",
      "        -0.04687948,  0.01549073],\n",
      "       [-0.04547248, -0.04464164,  0.03906215, ...,  0.02655962,\n",
      "         0.04452837, -0.02593034],\n",
      "       [-0.04547248, -0.04464164, -0.0730303 , ..., -0.03949338,\n",
      "        -0.00421986,  0.00306441]])\n",
      "array([151.,  75., 141., 206., 135.,  97., 138.,  63., 110., 310., 101.,\n",
      "        69., 179., 185., 118., 171., 166., 144.,  97., 168.,  68.,  49.,\n",
      "        68., 245., 184., 202., 137.,  85., 131., 283., 129.,  59., 341.,\n",
      "        87.,  65., 102., 265., 276., 252.,  90., 100.,  55.,  61.,  92.,\n",
      "       259.,  53., 190., 142.,  75., 142., 155., 225.,  59., 104., 182.,\n",
      "       128.,  52.,  37., 170., 170.,  61., 144.,  52., 128.,  71., 163.,\n",
      "       150.,  97., 160., 178.,  48., 270., 202., 111.,  85.,  42., 170.,\n",
      "       200., 252., 113., 143.,  51.,  52., 210.,  65., 141.,  55., 134.,\n",
      "        42., 111.,  98., 164.,  48.,  96.,  90., 162., 150., 279.,  92.,\n",
      "        83., 128., 102., 302., 198.,  95.,  53., 134., 144., 232.,  81.,\n",
      "       104.,  59., 246., 297., 258., 229., 275., 281., 179., 200., 200.,\n",
      "       173., 180.,  84., 121., 161.,  99., 109., 115., 268., 274., 158.,\n",
      "       107.,  83., 103., 272.,  85., 280., 336., 281., 118., 317., 235.,\n",
      "        60., 174., 259., 178., 128.,  96., 126., 288.,  88., 292.,  71.,\n",
      "       197., 186.,  25.,  84.,  96., 195.,  53., 217., 172., 131., 214.,\n",
      "        59.,  70., 220., 268., 152.,  47.,  74., 295., 101., 151., 127.,\n",
      "       237., 225.,  81., 151., 107.,  64., 138., 185., 265., 101., 137.,\n",
      "       143., 141.,  79., 292., 178.,  91., 116.,  86., 122.,  72., 129.,\n",
      "       142.,  90., 158.,  39., 196., 222., 277.,  99., 196., 202., 155.,\n",
      "        77., 191.,  70.,  73.,  49.,  65., 263., 248., 296., 214., 185.,\n",
      "        78.,  93., 252., 150.,  77., 208.,  77., 108., 160.,  53., 220.,\n",
      "       154., 259.,  90., 246., 124.,  67.,  72., 257., 262., 275., 177.,\n",
      "        71.,  47., 187., 125.,  78.,  51., 258., 215., 303., 243.,  91.,\n",
      "       150., 310., 153., 346.,  63.,  89.,  50.,  39., 103., 308., 116.,\n",
      "       145.,  74.,  45., 115., 264.,  87., 202., 127., 182., 241.,  66.,\n",
      "        94., 283.,  64., 102., 200., 265.,  94., 230., 181., 156., 233.,\n",
      "        60., 219.,  80.,  68., 332., 248.,  84., 200.,  55.,  85.,  89.,\n",
      "        31., 129.,  83., 275.,  65., 198., 236., 253., 124.,  44., 172.,\n",
      "       114., 142., 109., 180., 144., 163., 147.,  97., 220., 190., 109.,\n",
      "       191., 122., 230., 242., 248., 249., 192., 131., 237.,  78., 135.,\n",
      "       244., 199., 270., 164.,  72.,  96., 306.,  91., 214.,  95., 216.,\n",
      "       263., 178., 113., 200., 139., 139.,  88., 148.,  88., 243.,  71.,\n",
      "        77., 109., 272.,  60.,  54., 221.,  90., 311., 281., 182., 321.,\n",
      "        58., 262., 206., 233., 242., 123., 167.,  63., 197.,  71., 168.,\n",
      "       140., 217., 121., 235., 245.,  40.,  52., 104., 132.,  88.,  69.,\n",
      "       219.,  72., 201., 110.,  51., 277.,  63., 118.,  69., 273., 258.,\n",
      "        43., 198., 242., 232., 175.,  93., 168., 275., 293., 281.,  72.,\n",
      "       140., 189., 181., 209., 136., 261., 113., 131., 174., 257.,  55.,\n",
      "        84.,  42., 146., 212., 233.,  91., 111., 152., 120.,  67., 310.,\n",
      "        94., 183.,  66., 173.,  72.,  49.,  64.,  48., 178., 104., 132.,\n",
      "       220.,  57.])\n"
     ]
    }
   ],
   "source": [
    "from pprint import pprint\n",
    "\n",
    "# we can set the fields inside of ds and set them to new variables in python\n",
    "pprint(ds.data) # prints out elements of the matrix\n",
    "pprint(ds.target) # prints the vector (all 442 items)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "________________________________________________________________________________________________________\n",
    "<a id=\"linearnumpy\"></a>\n",
    "## Using Linear Regression \n",
    "In the videos, we derived the formula for calculating the optimal values of the regression weights (you must be connected to the internet for this equation to show up properly):\n",
    "\n",
    "$$ w = (X^TX)^{-1}X^Ty $$\n",
    "\n",
    "where $X$ is the matrix of values with a bias column of ones appended onto it. For the diabetes dataset one could construct this $X$ matrix by stacking a column of ones onto the `ds.data` matrix. \n",
    "\n",
    "$$ X=\\begin{bmatrix}\n",
    "         & \\vdots &        &  1 \\\\\n",
    "        \\dotsb & \\text{ds.data} & \\dotsb &  \\vdots\\\\\n",
    "         & \\vdots &         &  1\\\\\n",
    "     \\end{bmatrix}\n",
    "$$\n",
    "\n",
    "**Question 1:** For the diabetes dataset, how many elements will the vector $w$ contain?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Enter your answer here (or write code to calculate it)\n",
    " \n",
    "\n",
    "#"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "________________________________________________________________________________________________________\n",
    "\n",
    "**Exercise 1:** In the following empty cell, use this equation and numpy matrix operations to find the values of the vector $w$. You will need to be sure $X$ and $y$ are created like the instructor talked about in the video. Don't forget to include any modifications to $X$ to account for the bias term in $w$. You might be interested in the following functions:\n",
    "\n",
    "- `np.hstack((mat1,mat2))` stack two matrices horizontally, to create a new matrix\n",
    "- `np.ones((rows,cols))` create a matrix full of ones\n",
    "- `my_mat.T` takes transpose of numpy matrix named `my_mat`\n",
    "- `np.dot(mat1,mat2)` or `mat1 @ mat2` is matrix multiplication for two matrices\n",
    "- `np.linalg.inv(mat)` gets the inverse of the variable `mat`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ -10.01219782]\n",
      " [-239.81908937]\n",
      " [ 519.83978679]\n",
      " [ 324.39042769]\n",
      " [-792.18416163]\n",
      " [ 476.74583782]\n",
      " [ 101.04457032]\n",
      " [ 177.06417623]\n",
      " [ 751.27932109]\n",
      " [  67.62538639]\n",
      " [ 152.13348416]]\n"
     ]
    }
   ],
   "source": [
    "# Write you code here, print the values of the regression weights using the 'print()' function in python\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}