notebooks/23 - Using descriptive statistics.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Using descriptive statistics\n",
    "\n",
    "Statistics describe important aspects of our data, often revealing deeper insights.\n",
    "\n",
    "Statistics is a branch of mathematics concerned with data collection, analysis, interpretation, presentation, and organization. \n",
    "\n",
    "It plays a crucial role in various fields, from business and economics to healthcare and social sciences. Using statistical techniques, we can describe essential aspects of our data and uncover patterns and trends that may not be immediately apparent. Statistics can help us make informed decisions, identify potential problems, and evaluate the effectiveness of interventions.\n",
    "\n",
    "In short, statistics can reveal more profound insights into our data and provide valuable information that can guide us in making better decisions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## How To"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>longitude</th>\n",
       "      <th>latitude</th>\n",
       "      <th>housing_median_age</th>\n",
       "      <th>total_rooms</th>\n",
       "      <th>total_bedrooms</th>\n",
       "      <th>population</th>\n",
       "      <th>households</th>\n",
       "      <th>median_income</th>\n",
       "      <th>median_house_value</th>\n",
       "      <th>ocean_proximity</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-122.23</td>\n",
       "      <td>37.88</td>\n",
       "      <td>41.0</td>\n",
       "      <td>880.0</td>\n",
       "      <td>129.0</td>\n",
       "      <td>322.0</td>\n",
       "      <td>126.0</td>\n",
       "      <td>8.3252</td>\n",
       "      <td>452600.0</td>\n",
       "      <td>NEAR BAY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-122.22</td>\n",
       "      <td>37.86</td>\n",
       "      <td>21.0</td>\n",
       "      <td>7099.0</td>\n",
       "      <td>1106.0</td>\n",
       "      <td>2401.0</td>\n",
       "      <td>1138.0</td>\n",
       "      <td>8.3014</td>\n",
       "      <td>358500.0</td>\n",
       "      <td>NEAR BAY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-122.24</td>\n",
       "      <td>37.85</td>\n",
       "      <td>52.0</td>\n",
       "      <td>1467.0</td>\n",
       "      <td>190.0</td>\n",
       "      <td>496.0</td>\n",
       "      <td>177.0</td>\n",
       "      <td>7.2574</td>\n",
       "      <td>352100.0</td>\n",
       "      <td>NEAR BAY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-122.25</td>\n",
       "      <td>37.85</td>\n",
       "      <td>52.0</td>\n",
       "      <td>1274.0</td>\n",
       "      <td>235.0</td>\n",
       "      <td>558.0</td>\n",
       "      <td>219.0</td>\n",
       "      <td>5.6431</td>\n",
       "      <td>341300.0</td>\n",
       "      <td>NEAR BAY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-122.25</td>\n",
       "      <td>37.85</td>\n",
       "      <td>52.0</td>\n",
       "      <td>1627.0</td>\n",
       "      <td>280.0</td>\n",
       "      <td>565.0</td>\n",
       "      <td>259.0</td>\n",
       "      <td>3.8462</td>\n",
       "      <td>342200.0</td>\n",
       "      <td>NEAR BAY</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   longitude  latitude  housing_median_age  total_rooms  total_bedrooms  \\\n",
       "0    -122.23     37.88                41.0        880.0           129.0   \n",
       "1    -122.22     37.86                21.0       7099.0          1106.0   \n",
       "2    -122.24     37.85                52.0       1467.0           190.0   \n",
       "3    -122.25     37.85                52.0       1274.0           235.0   \n",
       "4    -122.25     37.85                52.0       1627.0           280.0   \n",
       "\n",
       "   population  households  median_income  median_house_value ocean_proximity  \n",
       "0       322.0       126.0         8.3252            452600.0        NEAR BAY  \n",
       "1      2401.0      1138.0         8.3014            358500.0        NEAR BAY  \n",
       "2       496.0       177.0         7.2574            352100.0        NEAR BAY  \n",
       "3       558.0       219.0         5.6431            341300.0        NEAR BAY  \n",
       "4       565.0       259.0         3.8462            342200.0        NEAR BAY  "
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "df = pd.read_csv(\"data/housing.csv\")\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>longitude</th>\n",
       "      <th>latitude</th>\n",
       "      <th>housing_median_age</th>\n",
       "      <th>total_rooms</th>\n",
       "      <th>total_bedrooms</th>\n",
       "      <th>population</th>\n",
       "      <th>households</th>\n",
       "      <th>median_income</th>\n",
       "      <th>median_house_value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>20640.000000</td>\n",
       "      <td>20640.000000</td>\n",
       "      <td>20640.000000</td>\n",
       "      <td>20640.000000</td>\n",
       "      <td>20433.000000</td>\n",
       "      <td>20640.000000</td>\n",
       "      <td>20640.000000</td>\n",
       "      <td>20640.000000</td>\n",
       "      <td>20640.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>-119.569704</td>\n",
       "      <td>35.631861</td>\n",
       "      <td>28.639486</td>\n",
       "      <td>2635.763081</td>\n",
       "      <td>537.870553</td>\n",
       "      <td>1425.476744</td>\n",
       "      <td>499.539680</td>\n",
       "      <td>3.870671</td>\n",
       "      <td>206855.816909</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>2.003532</td>\n",
       "      <td>2.135952</td>\n",
       "      <td>12.585558</td>\n",
       "      <td>2181.615252</td>\n",
       "      <td>421.385070</td>\n",
       "      <td>1132.462122</td>\n",
       "      <td>382.329753</td>\n",
       "      <td>1.899822</td>\n",
       "      <td>115395.615874</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>-124.350000</td>\n",
       "      <td>32.540000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.499900</td>\n",
       "      <td>14999.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>-121.800000</td>\n",
       "      <td>33.930000</td>\n",
       "      <td>18.000000</td>\n",
       "      <td>1447.750000</td>\n",
       "      <td>296.000000</td>\n",
       "      <td>787.000000</td>\n",
       "      <td>280.000000</td>\n",
       "      <td>2.563400</td>\n",
       "      <td>119600.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>-118.490000</td>\n",
       "      <td>34.260000</td>\n",
       "      <td>29.000000</td>\n",
       "      <td>2127.000000</td>\n",
       "      <td>435.000000</td>\n",
       "      <td>1166.000000</td>\n",
       "      <td>409.000000</td>\n",
       "      <td>3.534800</td>\n",
       "      <td>179700.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>-118.010000</td>\n",
       "      <td>37.710000</td>\n",
       "      <td>37.000000</td>\n",
       "      <td>3148.000000</td>\n",
       "      <td>647.000000</td>\n",
       "      <td>1725.000000</td>\n",
       "      <td>605.000000</td>\n",
       "      <td>4.743250</td>\n",
       "      <td>264725.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>-114.310000</td>\n",
       "      <td>41.950000</td>\n",
       "      <td>52.000000</td>\n",
       "      <td>39320.000000</td>\n",
       "      <td>6445.000000</td>\n",
       "      <td>35682.000000</td>\n",
       "      <td>6082.000000</td>\n",
       "      <td>15.000100</td>\n",
       "      <td>500001.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          longitude      latitude  housing_median_age   total_rooms  \\\n",
       "count  20640.000000  20640.000000        20640.000000  20640.000000   \n",
       "mean    -119.569704     35.631861           28.639486   2635.763081   \n",
       "std        2.003532      2.135952           12.585558   2181.615252   \n",
       "min     -124.350000     32.540000            1.000000      2.000000   \n",
       "25%     -121.800000     33.930000           18.000000   1447.750000   \n",
       "50%     -118.490000     34.260000           29.000000   2127.000000   \n",
       "75%     -118.010000     37.710000           37.000000   3148.000000   \n",
       "max     -114.310000     41.950000           52.000000  39320.000000   \n",
       "\n",
       "       total_bedrooms    population    households  median_income  \\\n",
       "count    20433.000000  20640.000000  20640.000000   20640.000000   \n",
       "mean       537.870553   1425.476744    499.539680       3.870671   \n",
       "std        421.385070   1132.462122    382.329753       1.899822   \n",
       "min          1.000000      3.000000      1.000000       0.499900   \n",
       "25%        296.000000    787.000000    280.000000       2.563400   \n",
       "50%        435.000000   1166.000000    409.000000       3.534800   \n",
       "75%        647.000000   1725.000000    605.000000       4.743250   \n",
       "max       6445.000000  35682.000000   6082.000000      15.000100   \n",
       "\n",
       "       median_house_value  \n",
       "count        20640.000000  \n",
       "mean        206855.816909  \n",
       "std         115395.615874  \n",
       "min          14999.000000  \n",
       "25%         119600.000000  \n",
       "50%         179700.000000  \n",
       "75%         264725.000000  \n",
       "max         500001.000000  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>longitude</th>\n",
       "      <th>latitude</th>\n",
       "      <th>housing_median_age</th>\n",
       "      <th>total_rooms</th>\n",
       "      <th>total_bedrooms</th>\n",
       "      <th>population</th>\n",
       "      <th>households</th>\n",
       "      <th>median_income</th>\n",
       "      <th>median_house_value</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ocean_proximity</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>&lt;1H OCEAN</th>\n",
       "      <td>-118.275</td>\n",
       "      <td>34.03</td>\n",
       "      <td>30.0</td>\n",
       "      <td>2108.0</td>\n",
       "      <td>438.0</td>\n",
       "      <td>1247.0</td>\n",
       "      <td>421.0</td>\n",
       "      <td>3.87500</td>\n",
       "      <td>214850.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>INLAND</th>\n",
       "      <td>-120.000</td>\n",
       "      <td>36.97</td>\n",
       "      <td>23.0</td>\n",
       "      <td>2131.0</td>\n",
       "      <td>423.0</td>\n",
       "      <td>1124.0</td>\n",
       "      <td>385.0</td>\n",
       "      <td>2.98770</td>\n",
       "      <td>108500.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ISLAND</th>\n",
       "      <td>-118.320</td>\n",
       "      <td>33.34</td>\n",
       "      <td>52.0</td>\n",
       "      <td>1675.0</td>\n",
       "      <td>512.0</td>\n",
       "      <td>733.0</td>\n",
       "      <td>288.0</td>\n",
       "      <td>2.73610</td>\n",
       "      <td>414700.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>NEAR BAY</th>\n",
       "      <td>-122.250</td>\n",
       "      <td>37.79</td>\n",
       "      <td>39.0</td>\n",
       "      <td>2083.0</td>\n",
       "      <td>423.0</td>\n",
       "      <td>1033.5</td>\n",
       "      <td>406.0</td>\n",
       "      <td>3.81865</td>\n",
       "      <td>233800.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>NEAR OCEAN</th>\n",
       "      <td>-118.260</td>\n",
       "      <td>33.79</td>\n",
       "      <td>29.0</td>\n",
       "      <td>2195.0</td>\n",
       "      <td>464.0</td>\n",
       "      <td>1136.5</td>\n",
       "      <td>429.0</td>\n",
       "      <td>3.64705</td>\n",
       "      <td>229450.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 longitude  latitude  housing_median_age  total_rooms  \\\n",
       "ocean_proximity                                                         \n",
       "<1H OCEAN         -118.275     34.03                30.0       2108.0   \n",
       "INLAND            -120.000     36.97                23.0       2131.0   \n",
       "ISLAND            -118.320     33.34                52.0       1675.0   \n",
       "NEAR BAY          -122.250     37.79                39.0       2083.0   \n",
       "NEAR OCEAN        -118.260     33.79                29.0       2195.0   \n",
       "\n",
       "                 total_bedrooms  population  households  median_income  \\\n",
       "ocean_proximity                                                          \n",
       "<1H OCEAN                 438.0      1247.0       421.0        3.87500   \n",
       "INLAND                    423.0      1124.0       385.0        2.98770   \n",
       "ISLAND                    512.0       733.0       288.0        2.73610   \n",
       "NEAR BAY                  423.0      1033.5       406.0        3.81865   \n",
       "NEAR OCEAN                464.0      1136.5       429.0        3.64705   \n",
       "\n",
       "                 median_house_value  \n",
       "ocean_proximity                      \n",
       "<1H OCEAN                  214850.0  \n",
       "INLAND                     108500.0  \n",
       "ISLAND                     414700.0  \n",
       "NEAR BAY                   233800.0  \n",
       "NEAR OCEAN                 229450.0  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby(\"ocean_proximity\").median()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>longitude</th>\n",
       "      <th>latitude</th>\n",
       "      <th>total_rooms</th>\n",
       "      <th>median_income</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>-114.310000</td>\n",
       "      <td>41.950000</td>\n",
       "      <td>39320.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>-119.569704</td>\n",
       "      <td>35.631861</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>median</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2127.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>-124.350000</td>\n",
       "      <td>32.540000</td>\n",
       "      <td>2.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>skew</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.646657</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         longitude   latitude  total_rooms  median_income\n",
       "max    -114.310000  41.950000      39320.0            NaN\n",
       "mean   -119.569704  35.631861          NaN            NaN\n",
       "median         NaN        NaN       2127.0            NaN\n",
       "min    -124.350000  32.540000          2.0            NaN\n",
       "skew           NaN        NaN          NaN       1.646657"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.agg({\"longitude\": [\"min\", \"max\", \"mean\"],\n",
    "        \"latitude\": [\"min\", \"max\", \"mean\"],\n",
    "        \"total_rooms\": [\"min\", \"max\", \"median\"],\n",
    "        \"median_income\": [\"skew\"]})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<1H OCEAN     9136\n",
       "INLAND        6551\n",
       "NEAR OCEAN    2658\n",
       "NEAR BAY      2290\n",
       "ISLAND           5\n",
       "Name: ocean_proximity, dtype: int64"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[\"ocean_proximity\"].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>longitude</th>\n",
       "      <th>latitude</th>\n",
       "      <th>housing_median_age</th>\n",
       "      <th>total_rooms</th>\n",
       "      <th>total_bedrooms</th>\n",
       "      <th>population</th>\n",
       "      <th>households</th>\n",
       "      <th>median_income</th>\n",
       "      <th>median_house_value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>longitude</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.879203</td>\n",
       "      <td>-0.150752</td>\n",
       "      <td>0.040120</td>\n",
       "      <td>0.063879</td>\n",
       "      <td>0.123527</td>\n",
       "      <td>0.060020</td>\n",
       "      <td>-0.009928</td>\n",
       "      <td>-0.069667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>latitude</th>\n",
       "      <td>-0.879203</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.032440</td>\n",
       "      <td>-0.018435</td>\n",
       "      <td>-0.056636</td>\n",
       "      <td>-0.123626</td>\n",
       "      <td>-0.074299</td>\n",
       "      <td>-0.088029</td>\n",
       "      <td>-0.165739</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>housing_median_age</th>\n",
       "      <td>-0.150752</td>\n",
       "      <td>0.032440</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.357162</td>\n",
       "      <td>-0.306544</td>\n",
       "      <td>-0.283879</td>\n",
       "      <td>-0.281989</td>\n",
       "      <td>-0.147308</td>\n",
       "      <td>0.074855</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>total_rooms</th>\n",
       "      <td>0.040120</td>\n",
       "      <td>-0.018435</td>\n",
       "      <td>-0.357162</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.915021</td>\n",
       "      <td>0.816185</td>\n",
       "      <td>0.906734</td>\n",
       "      <td>0.271321</td>\n",
       "      <td>0.205952</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>total_bedrooms</th>\n",
       "      <td>0.063879</td>\n",
       "      <td>-0.056636</td>\n",
       "      <td>-0.306544</td>\n",
       "      <td>0.915021</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.870937</td>\n",
       "      <td>0.975627</td>\n",
       "      <td>-0.006196</td>\n",
       "      <td>0.086259</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>population</th>\n",
       "      <td>0.123527</td>\n",
       "      <td>-0.123626</td>\n",
       "      <td>-0.283879</td>\n",
       "      <td>0.816185</td>\n",
       "      <td>0.870937</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.903872</td>\n",
       "      <td>0.006268</td>\n",
       "      <td>0.003839</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>households</th>\n",
       "      <td>0.060020</td>\n",
       "      <td>-0.074299</td>\n",
       "      <td>-0.281989</td>\n",
       "      <td>0.906734</td>\n",
       "      <td>0.975627</td>\n",
       "      <td>0.903872</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.030305</td>\n",
       "      <td>0.112737</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>median_income</th>\n",
       "      <td>-0.009928</td>\n",
       "      <td>-0.088029</td>\n",
       "      <td>-0.147308</td>\n",
       "      <td>0.271321</td>\n",
       "      <td>-0.006196</td>\n",
       "      <td>0.006268</td>\n",
       "      <td>0.030305</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.676778</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>median_house_value</th>\n",
       "      <td>-0.069667</td>\n",
       "      <td>-0.165739</td>\n",
       "      <td>0.074855</td>\n",
       "      <td>0.205952</td>\n",
       "      <td>0.086259</td>\n",
       "      <td>0.003839</td>\n",
       "      <td>0.112737</td>\n",
       "      <td>0.676778</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                    longitude  latitude  housing_median_age  total_rooms  \\\n",
       "longitude            1.000000 -0.879203           -0.150752     0.040120   \n",
       "latitude            -0.879203  1.000000            0.032440    -0.018435   \n",
       "housing_median_age  -0.150752  0.032440            1.000000    -0.357162   \n",
       "total_rooms          0.040120 -0.018435           -0.357162     1.000000   \n",
       "total_bedrooms       0.063879 -0.056636           -0.306544     0.915021   \n",
       "population           0.123527 -0.123626           -0.283879     0.816185   \n",
       "households           0.060020 -0.074299           -0.281989     0.906734   \n",
       "median_income       -0.009928 -0.088029           -0.147308     0.271321   \n",
       "median_house_value  -0.069667 -0.165739            0.074855     0.205952   \n",
       "\n",
       "                    total_bedrooms  population  households  median_income  \\\n",
       "longitude                 0.063879    0.123527    0.060020      -0.009928   \n",
       "latitude                 -0.056636   -0.123626   -0.074299      -0.088029   \n",
       "housing_median_age       -0.306544   -0.283879   -0.281989      -0.147308   \n",
       "total_rooms               0.915021    0.816185    0.906734       0.271321   \n",
       "total_bedrooms            1.000000    0.870937    0.975627      -0.006196   \n",
       "population                0.870937    1.000000    0.903872       0.006268   \n",
       "households                0.975627    0.903872    1.000000       0.030305   \n",
       "median_income            -0.006196    0.006268    0.030305       1.000000   \n",
       "median_house_value        0.086259    0.003839    0.112737       0.676778   \n",
       "\n",
       "                    median_house_value  \n",
       "longitude                    -0.069667  \n",
       "latitude                     -0.165739  \n",
       "housing_median_age            0.074855  \n",
       "total_rooms                   0.205952  \n",
       "total_bedrooms                0.086259  \n",
       "population                    0.003839  \n",
       "households                    0.112737  \n",
       "median_income                 0.676778  \n",
       "median_house_value            1.000000  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.corr('spearman')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Additional Resources"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- [Pandas Documentation](https://pandas.pydata.org/docs/getting_started/intro_tutorials/06_calculate_statistics.html)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}