02. Regression Model Performance.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# r^2 ( Coefficient of Determination)\n",
    "- Explains how well regression equation expains the data.  \n",
    "- A value of R^2 = 1 means regression predictions perfectly fit/explains the data.  \n",
    "\n",
    "Question: Can $r^2$ be negative?  \n",
    "Ans: When: ( Sum  of Square Errors(SSE) > {Total Sum  of Squares(SST)} )  \n",
    "This means when our predicted model performs worst than average line which is a very rare case.\n",
    "\n",
    "\n",
    "<img src=\"Image/r2_1.png\" width=\"400\">\n",
    "<img src=\"Image/r2_2.png\" width=\"400\">\n",
    "\n",
    "$ r^2 = 1 - \\frac {Sum of Squares of Errors(SSE)} {Total Sum of Squares(SST)}$  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Adjusted R^2\n",
    "\n",
    "Consider two equation:\n",
    "\n",
    "$ y = b_o + b_1x_1 $  \n",
    "$ y = b_o + b_1x_1 + b_2x_2 $  \n",
    "  \n",
    "When we add the term  $ b_2x_2$ in first equation, it can never be zero as there will be some correlation between dependent and independent variable. So, $r^2$ will never decrease.  \n",
    "- Penalizes for adding variables which do not improve the existing model  \n",
    "- more reliable and accurate evaluation  \n",
    "- Degree Of Freedom: minimum number of independent features that can specify the position of the system completely  \n",
    "\n",
    "Degree of Freedom = n - k - 1  \n",
    "n = Sample Size/observations  \n",
    "k = No. of regressors/features/independent variables  \n",
    "\n",
    "$ Adjusted R^2 = 1 - (1 - R^2)\\frac {n -1}{n - k - 1} $"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Example: Salary Vs Experience\n",
    "\n",
    "- One observation - infinite best fit line\n",
    "- Two observation - one best fitting line that passes through both of the points with 𝑅^2 = 1  \n",
    "- Three observation - it will introduce a level of freedom in actually determining the relation between X and y and it will increase for every new observation  \n",
    "- Degree of freedom for 3 observation is 1  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}