04. Polynomial Regression - Python.ipynb

{
  "cells": [
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Exercise 4 - Polynomial Regression\n========\n\nSometimes our data doesn't have a linear relationship, but we still want to predict an outcome.\n\nSuppose we want to predict how satisfied people might be with a piece of fruit, we would expect satisfaction would be low if the fruit was under ripened or over ripened. Satisfaction would be high in between underripened and overripened.\n\nThis is not something linear regression will help us with, so we can turn to polynomial regression to help us make predictions for these more complex non-linear relationships!"
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Step 1\n------\n\nIn this exercise we will look at a dataset analysing internet traffic over the course of the day. Observations were made every hour over the course of several days. Suppose we want to predict the level of traffic we might see at any time during the day, how might we do this?\n\nLet's start by opening up our data and having a look at it.\n\n#### In the cell below replace the text `<printDataHere>` with `print(dataset.head())`, and __run the code__ to see the data."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# This sets up the graphing configuration\nimport warnings\nwarnings.filterwarnings(\"ignore\")\nimport matplotlib.pyplot as graph\n%matplotlib inline\ngraph.rcParams['figure.figsize'] = (15,5)\ngraph.rcParams[\"font.family\"] = \"DejaVu Sans\"\ngraph.rcParams[\"font.size\"] = \"12\"\ngraph.rcParams['image.cmap'] = 'rainbow'\ngraph.rcParams['axes.facecolor'] = 'white'\ngraph.rcParams['figure.facecolor'] = 'white'\nimport numpy as np\nimport pandas as pd\n\ndataset = pd.read_csv('Data/traffic_by_hour.csv')\n\n###\n# BELOW, REPLACE <printDataHere> WITH print(dataset.head()) TO PREVIEW THE DATASET ---###\n###\n<printDataHere>\n###",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Step 2\n-----\n\nNext we're going to flip the data with the transpose method - our rows will become columns and our columns will become rows. Transpose is commonly used to reshape data so we can use it. Let's try it out.\n\n#### In the cell below find the text `<addCallToTranspose>` and replace it with `transpose`"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "### \n# REPLACE THE <addCallToTranspose> BELOW WITH transpose\n###\ndataset_T = np.<addCallToTranspose>(dataset)\n###\n\nprint(dataset_T)",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Now lets visualize the data. \n\n#### Replace the text `<addSampleHere>` with `sample` and then __run the code__."
    },
    {
      "metadata": {
        "scrolled": true,
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Let's visualise the data!\n\n###\n# REPLACE <addSampleHere> BELOW WITH sample\n###\nfor <addSampleHere> in range(0, dataset_T.shape[1]):\n    graph.plot(dataset.columns.values, dataset_T[sample])\n###\n\ngraph.xlabel('Time of day')\ngraph.ylabel('Internet traffic (Gbps)')\ngraph.show()",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Step 3\n-----\n\nThis all looks a bit busy, let's see if we can draw out a clearer pattern by taking the __average values__ for each hour.\n\n#### In the cell below find all occurances of `<replaceWithHour>` and replace them with `hour` and then __run the code__."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# We want to look at the mean values for each hour.\n\nhours = dataset.columns.values\n\n###\n# REPLACE THE <replaceWithHour>'s BELOW WITH hour\n###\ntrain_Y = [dataset[<replaceWithHour>].mean() for <replaceWithHour> in hours]  # This will be our outcome we measure (label) - amount of internet traffic\ntrain_X = np.transpose([int(<replaceWithHour>) for <replaceWithHour> in hours]) # This is our feature - time of day\n###\n\n# This makes our graph, don't edit!\ngraph.scatter(train_X, train_Y)\nfor sample in range(0,dataset_T.shape[1]):\n    graph.plot(hours, dataset_T[sample], alpha=0.25)\ngraph.xlabel('Time of day')\ngraph.ylabel('Internet traffic (Gbps)')\ngraph.show()",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "This alone could help us make a prediction if we wanted to know the expected traffic exactly on the hour.\n\nBut, we'll need to be a bit more clever if we want to make a __good__ prediction of times in between."
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Step 4\n------\n\nLet's use the midpoints in between the hours to analyse the relationship between the __time of day__ and the __amount of internet traffic__.\n\nNumpy's `polyfit(x,y,d)` function allows us to do polynomial regression, or more precisely least squares polynomial fit.\n\nWe specify a __feature $x$ (time of day)__, our __label $y$ (the amount of traffic)__, and the __degree $d$ of the polynomial (how curvy the line is)__.\n\n#### In the cell below find the text `<replaceWithDegree>`, replace it with the value `1` then __run the code__."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Polynomials of degree 1 are linear!\n# Lets include this one just for comparison\n\n###\n# REPLACE THE <replaceWithDegree> BELOW WITH 1\n###\npoly_1 = np.polyfit(train_X, train_Y, <replaceWithDegree>)\n###",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Let's also compare a few higher-degree polynomials.\n\n#### Replace the `<replaceWithDegree>`'s below with numbers, as directed in the comments."
    },
    {
      "metadata": {
        "scrolled": true,
        "trusted": true
      },
      "cell_type": "code",
      "source": "###\n# REPLACE THE <replaceWithDegree>'s BELOW WITH 2, 3, AND THEN 4\n###\npoly_2 = np.polyfit(train_X, train_Y, <replaceWithDegree>)\npoly_3 = np.polyfit(train_X, train_Y, <replaceWithDegree>)\npoly_4 = np.polyfit(train_X, train_Y, <replaceWithDegree>)\n###\n\n# Let's plot it!\ngraph.scatter(train_X, train_Y)\nxp = np.linspace(0, 24, 100)\n\n# black dashed linear degree 1\ngraph.plot(xp, np.polyval(poly_1, xp), 'k--')\n# red degree 2\ngraph.plot(xp, np.polyval(poly_2, xp), 'r-')\n# blue degree 3\ngraph.plot(xp, np.polyval(poly_3, xp), 'b-') \n# yellow degree 4\ngraph.plot(xp, np.polyval(poly_4, xp), 'y-') \n\ngraph.xticks(train_X, dataset.columns.values)\ngraph.xlabel('Time of day')\ngraph.ylabel('Internet traffic (Gbps)')\ngraph.show()",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "None of these polynomials do a great job of generalising the data. Let's try a few more.\n\n#### Follow the instructions in the comments to replace the `<replaceWithDegree>`'s and then __run the code__."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "###\n# REPLACE THE <replaceWithDegree>'s 5, 6, AND 7\n###\npoly_5 = np.polyfit(train_X, train_Y, <replaceWithDegree>)\npoly_6 = np.polyfit(train_X, train_Y, <replaceWithDegree>)\npoly_7 = np.polyfit(train_X, train_Y, <replaceWithDegree>)\n###\n\n# Let's plot it!\ngraph.scatter(train_X, train_Y)\nxp = np.linspace(0, 24, 100)\n\n# black dashed linear degree 1\ngraph.plot(xp, np.polyval(poly_1, xp), 'k--')\n# red degree 5\ngraph.plot(xp, np.polyval(poly_5, xp), 'r-') \n# blue degree 6\ngraph.plot(xp, np.polyval(poly_6, xp), 'b-') \n# yellow degree 7\ngraph.plot(xp, np.polyval(poly_7, xp), 'y-') \n\ngraph.xticks(train_X, dataset.columns.values)\ngraph.xlabel('Time of day')\ngraph.ylabel('Internet traffic (Gbps)')\ngraph.show()",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "It looks like the 5th and 6th degree polynomials have an identical curve. This looks like a good curve to use.\n\nWe could perhaps use an even higher degree polynomial to fit it even more tightly, but we don't want to overfit the curve, since we want just a generalisation of the relationship.\n\nLet's see how our degree 6 polynomial compares to the real data.\n\n#### Replace the text `<replaceWithPoly6>` with `poly_6` and __run the code__."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "for row in range(0,dataset_T.shape[1]):\n    graph.plot(dataset.columns.values, dataset_T[row], alpha = 0.5)\n\n###\n# REPLACE <replaceWithPoly6> BELOW WITH poly_6 - THE POLYNOMIAL WE WISH TO VISUALIZE\n###    \ngraph.plot(xp, np.polyval(<replaceWithPoly6>, xp), 'k-')\n###\n\ngraph.xlabel('Time of day')\ngraph.ylabel('Internet traffic (Gbps)')\ngraph.show()",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Step 5\n------\n\nNow let's try using this model to make a prediction for a time between 00 and 24.\n\n#### In the cell below follow the instructions in the code to replace `<replaceWithTime>` and `<replaceWithPoly6>` then __run the code__."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "###\n# REPLACE <replaceWithTime> BELOW WITH 12.5 (this represents the time 12:30)\n###\ntime = <replaceWithTime>\n###\n\n###\n# REPLACE <replaceWithPoly6> BELOW WITH poly_6 SO WE CAN VISUALIZE THE 6TH DEGREE POLYNOMIAL MODEL\n###\npred = np.polyval(<replaceWithPoly6>, time)\n###\n\nprint(\"at t=%s, predicted internet traffic is %s Gbps\"%(time,pred))\n\n# Now let's visualise it\ngraph.plot(xp, np.polyval(poly_6, xp), 'y-')\n\ngraph.plot(time, pred, 'ko') # result point\ngraph.plot(np.linspace(0, time, 2), np.full([2], pred), dashes=[6, 3], color='black') # dashed lines (to y-axis)\ngraph.plot(np.full([2], time), np.linspace(0, pred, 2), dashes=[6, 3], color='black') # dashed lines (to x-axis)\n\ngraph.xticks(train_X, dataset.columns.values)\ngraph.ylim(0, 60)\ngraph.title('expected traffic throughout the day')\ngraph.xlabel('time of day')\ngraph.ylabel('internet traffic (Gbps)')\n\ngraph.show()",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Conclusion\n-----\n\nAnd there we have it! You have made a polynomial regression model and used it for analysis! This models gives us a prediction for the level of internet traffic we should expect to see at any given time of day.\n\nYou can go back to the course and either click __'Next Step'__ to start an optional step with tips on how to better work with AI models, or you can go to the next module where instead of predicting numbers we predict categories."
    }
  ],
  "metadata": {
    "kernelspec": {
      "name": "python36",
      "display_name": "Python 3.6",
      "language": "python"
    },
    "language_info": {
      "mimetype": "text/x-python",
      "nbconvert_exporter": "python",
      "name": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.6",
      "file_extension": ".py",
      "codemirror_mode": {
        "version": 3,
        "name": "ipython"
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 2
}