1. Basic Concepts.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "code_folding": []
   },
   "outputs": [],
   "source": [
    "# imports\n",
    "\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "\n",
    "import numpy as np\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.datasets.samples_generator import make_regression \n",
    "from scipy import stats \n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Basic Concepts\n",
    "\n",
    "## What is \"learning from data\"?\n",
    "\n",
    "> In general **Learning from Data** is a scientific discipline that is concerned with the design and development of algorithms that allow computers to infer (from data) a  model that allows *compact representation* (unsupervised learning) and/or *good generalization* (supervised learning).\n",
    "\n",
    "This is an important technology because it enables computational systems to adaptively improve their performance with experience accumulated from the observed data. \n",
    "\n",
    "Most of these algorithms are based on the *iterative solution* of a mathematical problem that involves data and model. If there was an analytical solution to the problem, this should be the adopted one, but this is not the case for most of the cases.\n",
    "\n",
    "So, the most common strategy for **learning from data** is based on solving a system of equations as a way to find a series of parameters of the model that minimizes a mathematical problem. This is called **optimization**.\n",
    "\n",
    "The most important technique for solving optimization problems is **gradient descend**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preliminary: Nelder-Mead method for function minimization.\n",
    "\n",
    "The most simple thing we can try to minimize a function $f(x)$ would be to sample two points relatively near each other, and just repeatedly take a step down away from the largest value. This simple algorithm has a severe limitation: it can't get closer to the true minima than the step size.  \n",
    "\n",
    "The Nelder-Mead method dynamically adjusts the step size based off the loss of the new point. If the new point is better than any previously seen value, it **expands** the step size to accelerate towards the bottom. Likewise if the new point is worse it **contracts** the step size to converge around the minima. The usual settings are to half the step size when contracting and double the step size when expanding. \n",
    "\n",
    "This method can be easily extended into higher dimensional examples, all that's required is taking one more point than there are dimensions. Then, the simplest approach is to replace the worst point with a point reflected through the centroid of the remaining n points. If this point is better than the best current point, then we can try stretching exponentially out along this line. On the other hand, if this new point isn't much better than the previous value, then we are stepping across a valley, so we shrink the step towards a better point.\n",
    "\n",
    "> See \"An Interactive Tutorial on Numerical Optimization\": http://www.benfrederickson.com/numerical-optimization/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Gradient descend (for *hackers*) for function minimization: 1-D\n",
    "\n",
    "Let's suppose that we have a function $f: \\mathbb{R} \\rightarrow \\mathbb{R}$. For example: \n",
    "\n",
    "$$f(x) = x^2$$\n",
    "\n",
    "Our objective is to find the argument  $x$ that minimizes this function (for maximization, consider $-f(x)$). To this end, the critical concept is the **derivative**.\n",
    "\n",
    "The derivative of $f$ of a variable $x$, $f'(x)$ or $\\frac{\\partial f}{\\partial x}$,  is a measure of the rate at which the value of the function changes with respect to the change of the variable. \n",
    "\n",
    "It is defined as the following limit:\n",
    "\n",
    "\n",
    "$$ f'(x) = \\lim_{h \\rightarrow 0} \\frac{f(x + h) - f(x)}{h} $$\n",
    "\n",
    "The derivative specifies how to scale a small change in the input in order to obtain the corresponding change in the output: \n",
    "\n",
    "$$ f(x + h) \\approx f(x) + h f'(x)$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4.0000\n"
     ]
    }
   ],
   "source": [
    "# numerical derivative at a point x\n",
    "\n",
    "def f(x):\n",
    "    return x**2\n",
    "\n",
    "def fin_dif(x, \n",
    "            f, \n",
    "            h = 0.00001):\n",
    "    '''\n",
    "    This method returns the derivative of f at x\n",
    "    by using the finite difference method\n",
    "    '''\n",
    "    return (f(x+h) - f(x))/h\n",
    "\n",
    "x = 2.0\n",
    "print(\"{:2.4f}\".format(fin_dif(x,f)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The limit as $h$ approaches zero, if it exists, should represent the **slope of the tangent line** to $(x, f(x))$. \n",
    "\n",
    "For values that are not zero it is only an approximation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> **NOTE**: It can be shown that the “centered difference formula\" is better when computing numerical derivatives:\n",
    "\n",
    "> $$ \\lim_{h \\rightarrow 0} \\frac{f(x + h) - f(x - h)}{2h} $$\n",
    "\n",
    "> The error in the \"finite difference\" approximation can be derived from Taylor's theorem and, assuming that $f$ is differentiable, is $O(h)$. In the case of “centered difference\" the error is $O(h^2)$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The derivative tells how to chage $x$ in order to make a small improvement in $f$. \n",
    "\n",
    "Then, we can follow these steps to decrease the value of the function:\n",
    "\n",
    "+ Start from a random $x$ value.\n",
    "+ Compute the derivative $f'(x) = \\lim_{h \\rightarrow 0} \\frac{f(x + h) - f(x - h)}{2h}$.\n",
    "+ Walk a small step (possibly weighted by the derivative module) in the **opposite** direction of the derivative, because we know that $f(x - h \\mbox{ sign}(f'(x)) < f(x)$ for  small enough $h$. \n",
    "\n",
    "The search for the minima ends when the derivative is zero because we have no more information about which direction to move. $x$ is a critical o stationary point if $f'(x)=0$. \n",
    "\n",
    " + A **minimum (maximum)** is a critical point where $f(x)$ is lower (higher) than at all neighboring points. \n",
    " + There is a third class of critical points: **saddle points**.\n",
    "\n",
    "If $f$ is a **convex function**, this should be the minimum (maximum) of our functions. In other cases it could be a local minimum (maximum) or a saddle point."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function matplotlib.pyplot.show>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlkAAADGCAYAAADhV5X4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xd81FXWx/HPJIFIrwESkKZgoiudQARSXCAUFVkQWMQIPCKuuCAooLDqoqvSLLDqKo8CgjQVUJGitCBFUIyIhaJ00whNDCqBkOePswR5BALJTH4zme/79ZpXkmEmc3KZTM7ce+65roMHj+cgIiIiIm4V4HQAIiIiIkWRkiwRERERD1CSJSIiIuIBSrJEREREPEBJloiIiIgHKMkSERER8YA8k6zk5B/p2rUzrVs3p02bSKZMeQWA8eOfoUGD64iLa0VcXCtWrPgo9z6TJj1HZGRDoqKasGrVCs9FLyIiIuKlXHn1yUpPTyM9PY0GDRqRmfkzbdtG8+abc3j//QWUKlWaQYMGn3f7HTu2M3Bgfz76aDVpaal0734bGzd+SWBgoEd/EBERERFvkudMVtWq1WjQoBEApUuXoX7960hNTbno7ZctW0zXrt0IDg6mVq3a1KlTl6Skze6LWERERMQHXFFN1v79+/j66600bdoMgKlTpxATE8WQIfdz7NhRAFJTUwgLq557n7Cw6qSlpboxZBERERHvF3S5N8zMzKR//7t46qmxlClTlr597+Ghh0bicrkYO/ZfPPHEaCZNeuWyH3jGjGnMnDkdgJ07t3P11VdfcfBX6syZMwQEqNbfXTSe7qcxdS+Np/tpTN1L4+l+hTGmhw4dYvv2vXne7rKSrFOnTtG/fx+6devBLbfcBkCVKlVy/71Pn7vp06cHAKGhYaSkJOf+W0pKMtWqhf7heyYk9CMhoR8AHTvGsXmz55cUExMTiY2N9fjj+AuNp/tpTN1L4+l+GlP30ni6X2GMaaNGTS7rdnmmejk5OTz44CDq17+Ov/3tgdzr09PTcj9fsmQR4eERAMTHd2LhwvmcPHmSffv2snv3bpo0aXal8YuIiIj4tDxnsjZt2sg778wlIuIG4uJaATB69OMsWPAu3377NeCiZs2aTJw4CYDw8Ai6dOlK69bNCQoKYty4idpZKCIiIn4nzySrZcsoDh48/ofr27aNv+h9hg4dztChwwsWmYiIiIgPU7WdiIiIiAcoyRIRERHxACVZIiIiIh6gJEtERETEA5RkiYiIiHiAkiwRERERD1CSJSIiIuIBSrJEREREPEBJloiIiIgH+EeSdeYM/Oc/uLKynI5EREREPGXBAoLT0vK+XSHxjyRr7Vq4/34inn3WEi4REREpWhYvhh49qDN1qtOR5PKPJCsmBiZMoEpiIgwZAjk5TkckIiIi7rJxI9xxBzRqxPcPPuh0NLn8I8kCePhh9vfsCS+9BM8843Q0IiIi4g7btkHnzlC9OixZQnbJkk5HlCvI6QAK0+5776Vm8eLwj39AlSowYIDTIYmIiEh+HTgA7dtD8eLw8cf2t/2775yOKpdfJVkEBMAbb8ChQ3DffRASArff7nRUIiIicqWOHIEOHeD4cVizBurUcTqiP/Cf5cKzihWDd96B5s2hVy/45BOnIxIREZErceIE3Hor7NoF778PjRo5HdEF+V+SBVCqlO1CqFvX/pO+/NLpiERERORyZGVB9+5W7D5rFsTGOh3RRflnkgVQqZKt35YvD/HxsHOn0xGJiIjIpWRnQ0ICLFsGr70G3bo5HdEl+W+SBVCjBixfbp+3awc//uhsPCIiInJhOTnwwAMwbx6MGwf33ON0RHny7yQLoH59y4iPHbMdCocOOR2RiIiI/H+PPQavvgojR8KIEU5Hc1mUZAE0aQKLFsGePdCxI/z8s9MRiYiIyFnPPw9PP22tl5591uloLpuSrLOio+Htt60IvksX+PVXpyMSERGRqVPhoYes2P0//wGXy+mILpuSrN+79VZ4801ITLT2/KdOOR2RiIiI/5o3z2qv4uPhrbcgMNDpiK5InklWcvKPdO3amdatm9OmTSRTprwCwNGjR+jevQstWjSie/cuHDt2FICcnBxGjRpOZGRDYmKi2Lp1i2d/Ane7807LlBcvhrvusp0MIiIiUrgWL4Y+faB1a1iwAIKDnY7oiuWZZAUFBTFmzNOsW/c5S5euZOrU/2XHju1MnvwC0dExbNq0hejoGCZPfgGAlSs/ZvfuXWzatIXnnpvEiBFDPf5DuN3AgTBhgmXQAwfCmTNORyQiIuI/Vq+29gwNG1rNtBedR3gl8kyyqlatRoMG1km1dOky1K9/HampKSxbtpiePXsD0LNnb5Yu/RCApUuX0KPHX3G5XDRrFslPP/1EenqaB38ED3n4YdvJ8MYbMGyYbR0VERERz9q40cp3rrnGdv+XK+d0RPl2RWcX7t+/j6+/3krTps3IyMigatVqAFSpUpWMjAwA0tJSCAurkXufsLDqpKam5N72rBkzpjFz5vTc+yQmJhbgx7g8mZmZV/Y4cXFc+9131Jg0ib1HjrC3f3+PxeaLrng8JU8aU/fSeLqfxtS9NJ7nK/XDDzQaOpTT5crx5ZgxZH3zzRV/D28a08tOsjIzM+nf/y6eemosZcqUPe/fXC4Xrius9k9I6EdCQj8AOnaMI7YQ2uInJiZe+ePExsKAAdR+4w1qh4fDqFGeCM0n5Ws85ZI0pu6l8XQ/jal7aTx/57vvbNNZ+fIUW7uWm2rXzte38aYxvawk69SpU/Tv34du3Xpwyy23ARASEkJ6ehpVq1YjPT2NypUrA1CtWhgpKec6p6ekJBMaGuaB0AuJy2Wt+3/7DUaPhhIlYKgP1pmJiIh4q++/hz//GYKCYOVKyGeC5W3yrMnKycnhwQcHUb/+dfztbw/kXh8f34l582YDMG/ebDp06AxAhw4defvtOeTk5LB582eULVv2D0uFPicwEKZPtx4dw4bBK684HZGIiEjRsGcP3HwznD5tCVb9+k5H5DZ5zmRt2rSRd96ZS0TEDcTFtQJg9OjHGTx4KAMG9GXWrBnUqFGT11+fDkDbtvGsWPExkZENKVmyJJMmFZGEJCjITvs+eRIGDYKrrgLVaImIiOTfgQM2g3XihO0ovP56pyNyqzyTrJYtozh48PgF/23+/EV/uM7lcjFu3PMFj8wbFS9uXeG7dLHmaMHB1ldLRERErkxqqiVYhw/bDFbDhk5H5Hbq+H6lrroKFi6EmBhISIC5c52OSERExLekpdkSYUoKLF0KzZo5HZFHKMnKj5Il4cMPoVUr60b7zjtORyQiIuIbDh60Gaz9+2HJErjpJqcj8hglWflVqpQ9OVq2hL/+FebPdzoiERER75aRYQnWnj12bE50tNMReZSSrIIoXdqmOSMjoVcveO89pyMSERHxTocOQdu28MMPthrkJb2sPElJVkGVKWNt/5s2hR494IMPnI5IRETEuxw5Au3awc6ddhbhzTc7HVGhUJLlDmXLwkcfQePG1ktLiZaIiIg5fNiWCLdtg/fft9ksP6Eky13KlTs/0dLSoYiI+LuMDJu12r7dEqz27Z2OqFApyXKn8uXh449t6fCOO1QMLyIi/uvgQUuwzi4Rxsc7HVGhU5LlbmdntCIjoWdPtXcQERH/k54OcXGwa5ftIvSjJcLfU5LlCWXLWjF8VJS1d5g3z+mIRERECkdamiVYe/daqyM/KXK/ECVZnlKmjLV3uOkm6N0b3nrL6YhEREQ8KznZWjPs329/A/2gTcOlKMnypLN9tGJj7Qie1193OiIRERHP2LfPmoumpNhqThFvNHo5lGR5WqlS1nStQwcYMABeftnpiERERNzrhx+gTRvrh7ViBbRu7XREXkFJVmEoUcIOle7SBR54ACZOdDoiERER99i2zWatfvkFVq2yjV8CKMkqPMHBttOwRw8YPhyeesrpiERERApm61aIiYEzZ2DNGusVKbmCnA7ArxQrBrNmWcL1+ONw4gQ8+yy4XE5HJiIicmU++8xKYUqWtBms+vWdjsjrKMkqbEFBMH261WqNGwfHj8NLL0GAJhVFRMRHJCbCrbdCSIjVYNWt63REXklJlhMCAuCVV6yf1vjx8PPPMG2aJWAiIiLebPFiOz6ubl075aR6dacj8lr6q+4UlwvGjrUO8aNHQ2YmzJ1rS4kiIiLeaN486NMHGja0Ng2VKzsdkVfTGpWTXC4YNQomT7YDpW+5xeq0REREvM0bb9gpJi1bwsqVSrAug5Isb/D3v9ty4apVdr7TkSNORyQiInLOhAlwzz3Qvr2dz1uunNMR+QQlWd6ib194911ISrJ+I8nJTkckIiL+LicHRo6EESOgZ0/44APbTSiXJc8ka8iQ+7n++rpER7fIvW78+Gdo0OA64uJaERfXihUrPsr9t0mTniMysiFRUU1YtWqFZ6Iuqrp2tTXu/futW+733zsdkYiI+KvTp+2kkvHj4b77rAVR8eJOR+VT8kyyevW6k7lzF/zh+oEDB7F69XpWr15P27bxAOzYsZ2FC+ezdu1nzJ27gJEjh5Gdne3+qIuyuDhYvdoK4Vu3hi+/dDoiERHxN7/9Zs2z33gDHnvMdsQHBjodlc/JM8mKimpF+fIVLuubLVu2mK5duxEcHEytWrWpU6cuSUmbCxyk32naFNats52GsbHWj0RERKQwHD8OnTvbcXAvvghPPqmm2fmU7xYOU6dO4e2359CoUWPGjHma8uUrkJqaQtOmzXNvExZWnbS01Avef8aMacycOR2AtLQUEgshkcjMzCyUx3GX4Oeeo8Hw4ZRo355to0eTERPjdEjn8bXx9AUaU/fSeLqfxtS9vG08ix8+zI2PPEKpPXvY8eijpDds6HNv9L1pTPOVZPXtew8PPTQSl8vF2LH/4oknRjNp0itX9D0SEvqRkNAPgI4d44iNjc1PKFckMTGxUB7Hrf78Z7j1Vm4YMwb+/W8YNMjpiHL55Hh6OY2pe2k83U9j6l5eNZ47d0K/fpCRAR9+SESHDkQ4HVM+eNOY5mt3YZUqVQgMDCQgIIA+fe7myy+/ACA0NIyUlHO74lJSkqlWLdQ9kfqrihXtyIJbb4UHHrDGpTk5TkclIiJFyWefQatWVg+8erWdSSgFlq8kKz09LffzJUsWER5uuW58fCcWLpzPyZMn2bdvL7t376ZJk2buidSflSgB8+fDvffCM8/A//wPnDrldFQiIlIULF1qm67KlIENG6B587zvI5clz+XCgQP7sX79Oo4cOUzDhuGMGDGK9evX8u23XwMuatasycSJkwAID4+gS5eutG7dnKCgIMaNm0igdiO4R1AQvPoqhIbCmDGQlgZvvw2lSzsdmYiI+KqpU+0NfIMGsGQJVKvmdERFSp5J1muvTfvDdXfemXDR2w8dOpyhQ4cXLCq5MJcL/vlPO4zzb3+DmBj48ENLvERERC5XTo79PXnySWjXzpphly3rdFRFjjq++6IBA6zr7o4dEBUF333ndEQiIuIrsrKswP3JJ+3j4sVKsDxESZav6tQJ1qyxhnGtWtnnIiIil3K2B9abb9pM1htvQLFiTkdVZCnJ8mVNm8LGjbaG3r49zJ7tdEQiIuKtfvwR2rSxvlfTpsETT6jJqIcpyfJ1tWvbbpCoKLjzTpv+VYsHERH5vS++gMhI2LPHlgf79nU6Ir+gJKsoqFABPvoI7rrL3pncfTecPOl0VCIi4g3efx+io21ZcP16W/mQQqEkq6gIDrY19qeegpkzbbfI4cNORyUiIk7JyYEXXoCuXeGGG2DTJrjxRqej8itKsooSlwv+8Q+rzfrsM2jZ0o5JEBER/3L6NNx/PwwbBn/5i9VhqQdWoVOSVRT99a+wciUcO2aJ1qpVTkckIiKF5ehR24H+6qswcqQ1ri5Z0umo/JKSrKKqVSubGg4NtfX3//zH6YhERMTTdu60N9eJidaeYexYCNCfeqdo5IuyunXh008hPt6mjR94wKaQRUSk6FmxAlq0gCNHbDWjf3+nI/J7SrKKurJlrTv8Qw/Byy9Dx442lSwiIkXHK69Ahw5Qo4bV5LZp43REgpIs/xAYCBMn2tTxmjX2TmfbNqejEhGRgsrKspWKQYPsTfT69VCnjtNRyX8pyfIn/ftbEfyxY5ZoLVrkdEQiIpJfBw9C27ZWcztiBLz3ns4g9DJKsvxN69bW+bdePejSBZ5+Wh3iRUR8zRdfQLNm8Pnn1rZn3DhbtRCvoiTLH119NaxbB717W1+tHj0gM9PpqERE5HLMnm1vmF0uWx7861+djkguQkmWvypRwjrDT5wICxbATTfBrl1ORyUiIhdz+jQMH27n1EZG2ixWkyZORyWXoCTLn7lctutw6VI7nb1ZM1iyxOmoRETk/zt40HoeTpxohe4rVkCVKk5HJXlQkiX2i/vFF1C7NtxyC4wZA2fOOB2ViIiANZZu2tT6Hr75prXjKVbM6ajkMijJElOnDmzYAHfdBf/8J9x2m/ppiYg4KScHpkyB6GgICrLX6IQEp6OSK6AkS84pUQKmT7d3SR99BM2bw5YtTkclIuJ/fv0V7rkHBg6EuDjYvBkaN3Y6KrlCSrLkfC6XrfevWWO/5C1bWhNTtXkQESkc338PUVEwdSqMHg2LF0OlSk5HJfmgJEsu7Kab4Msv7WiGe+6Bfv3gl1+cjkpEpGhbsMA2IR04YMnVv/6l/lc+LM8ka8iQ+7n++rpER7fIve7o0SN0796FFi0a0b17F44ds9qdnJwcRo0aTmRkQ2Jioti6VUtNPq1KFVi2DB5/HGbMsFmtnTudjkpEpOg5dQqGDYNu3SA8HJKSoFMnp6OSAsozyerV607mzl1w3nWTJ79AdHQMmzZtITo6hsmTXwBg5cqP2b17F5s2beG55yYxYsRQz0QthScw0HYbLl0KKSn2DmvuXKejEhEpOvbvh5gYeOEF+PvfYe1aqFXL6ajEDfJMsqKiWlG+fIXzrlu2bDE9e/YGoGfP3ixd+iEAS5cuoUePv+JyuWjWLJKffvqJ9PQ0D4QthS4+3pYP//Qn6y48cCABJ086HZWIiE+rtG4dNGoE33xjb2AnT4bixZ0OS9wkXzVZGRkZVK1aDYAqVaqSkZEBQFpaCmFhNXJvFxZWndTUFDeEKV7h6qutIP6RR2DKFJrcfz9s2+Z0VCIivufkSXjwQW587DFroZOUBD17Oh2VuFlQQb+By+XC5XJd8f1mzJjGzJnTAUvOEhMTCxpKnjIzMwvlcYq8+HgqVqjAdc88Q3aTJuwcMoT0Dh2cjqpI0HPUvTSe7qcxLbirkpO54cknKbNzJ3tuvZX9DzxAzo8/2skbUmDe9BzNV5IVEhJCenoaVatWIz09jcqVKwNQrVoYKSnnniQpKcmEhoZd8HskJPQjIaEfAB07xhEbG5ufUK5IYmJioTyOX4iNZUPdutz08stEjBtHRHKy9dcqW9bpyHyanqPupfF0P41pAc2eDX/7m9W7LlzIvvLlNZ5u5k3P0XwtF8bHd2LevNkAzJs3mw4dOgPQoUNH3n57Djk5OWze/Blly5bNXVaUoiercmU7P2vMGJgzx+oKNm50OiwREe9z/Lh1a7/zTmjQwBo9336701GJh+WZZA0c2I9Ondryww/f07BhOLNmzWDw4KGsWbOaFi0asWZNIoMH2y7Ctm3jqVWrNpGRDXnoocGMG/e8x38AcVhgoLV4+OQTa1jaujU88wxkZzsdmYiId/jsM+vWPmuWHVuWmKjdg34iz+XC116bdsHr589f9IfrXC6XEit/ddNN9s7svvusQ/HHH8PMmVYsLyLij7KzYfx4eyNavbq9GW3VyumopBCp47u4T7lyVm/w5pvwxRc2JT5njtNRiYgUvj17IDYWRo2yBqNbtijB8kNKssS9XC6rO9iyBa6/Hnr3tr5aR486HZmIiOfl5MD06dCwIWzdaqdlzJkD5cs7HZk4QEmWeMY111hPrX/9C959F2680YrkRUSKqowMm7Xq1w+aNLEk66677M2n+CUlWeI5QUFWn7VxI5QpA+3aweDBOmhaRIqeRYvszeTixTBhAqxcqeJ2UZIlhaBpU+tmPHgw/PvfNo2+fr3TUYmIFNyxY9C3L9x2G1SpAp9/Dg8/bDuvxe8pyZLCUaIETJoEq1fD6dPQpg0MHw6//eZ0ZCIi+bNsmZ3n+tZb8I9/wObNtuFH5L+UZEnhio21OoV774WJE613zGefOR2ViMjlO37cXsM6drRTLj79FJ56Sgc7yx8oyZLCV6YMvPoqfPQRZGZCVJTNaqlWS0S83ZIlcMMN8MYbMGKElUI0b+50VOKllGSJc9q3h2++gXvusVmthg1tR6KIiLc5fNja03TubLNX69fDuHFw1VVORyZeTEmWOKtcOXjtNVi1Cs6cseXE+++36XgREW/w7rvW92/OHHjsMZu9atnS6ajEByjJEu8QF2e1WkOH2lLiDTfABx84HZWI+LMff4SuXeGOO+yIsM2b4cknITjY6cjERyjJEu9RqhQ8/zxs2GDdkbt0scZ+yclORyYi/iQ729rNXH+91Y6OG2f9/ho2dDoy8TFKssT7tGxp0/HPPmtFphER8MortpwoIuJJX31lB94PHmwfv/nGCtyDgpyOTHyQkizxTsWKwSOP2AtcixYwaJAdrrpli9ORiUhRlJlpu5ybNrXDnWfPhqVLoW5dpyMTH6YkS7zbNdfAxx/DzJmwa5e9AA4ZAj/95HRkIlIU5ORYYXtEhO1y7tsXtm+3g+115qAUkJIs8X4uF/TpAzt2wH33Wa1EeLi908zJcTo6EfFV339vDUXvuAMqVbK2DK+/DhUrOh2ZFBFKssR3VKgAL79sHeKvvhruvBP+/GdbUhQRuVy//AKPP25H4nz6qR35tXmz1WCJuJGSLPE9zZrZC+Orr1qRaqNGVqR69KjTkYmIN8vJgXnzbCb8qadsBmv7dnv9UGG7eICSLPFNgYEwcCDs3GkfX34Z6tWDKVNs+7WIyO999ZU1O+7VCypXhrVr7WDn0FCnI5MiTEmW+LZKlSzBSkqyBqYDB9o5YjqeR0QAMjLsFIkmTeDbb+2Eic8/h9atnY5M/ICSLCkaGjaExERbCjh0yN6xdu1qha0i4n9++w3Gj4drr7UZ7kGD7PXg3nttJlykECjJkqLD5YIePWwX4tNPw4oV1rF56FA4csTp6ESkMJytu4qIgJEjITraNsdMnmybZ0QKkZIsKXpKlIBRo+xda79+9uJ67bXw3HP27lZEiqZ166xpca9eULasvdFatMgK3UUcUKAkq2nTPxET05K4uFa0axcDwNGjR+jevQstWjSie/cuHDumHV/ikGrVbJlgyxaIjISHH4brroMZM1QcL1KUfPst3HYbtGkDe/dar6ukJGvxIuKgAs9kLViwmNWr17N8uRUaT578AtHRMWzatIXo6BgmT36hwEGKFMiNN8KyZfauNiQE7r4bGje2cxHVzFTEdx04AP37Q4MGttnlmWfghx/gf/5HdVfiFdy+XLhs2WJ69uwNQM+evVm69EN3P4RI/vz5z9bIdN48a0bYuTPExMAnnzgdmYhciYMHYdgwa9syaxY8+CDs3g2PPgolSzodnUgu18GDx/P9Vr5ZsxspV648LpeLhIR+JCT049prr+aHHw4AkJOTQ716NXO//r0ZM6Yxc+Z0ANLSUpg7d25+w7hsmZmZlC5d2uOP4y98eTxdp04Rungxtd56i+DDhznSrBl7+vfn54gIR+Py5TH1RhpP93NyTIN+/pmr582jxvz5BGRlkd6uHXv79uW3atUciccd9Bx1v8IY0wcfHJa7gncpBUqyUlNTCA0NIyMjgzvu6MKzz07grrt6nZdU1atXk++/33/J79OxYxybN2/ObxiXLTExkdjYWI8/jr8oEuP5yy/wn//As8/C4cNW1/Hkk9YSwgFFYky9iMbT/RwZ0+PHbQPLxIl2OHzPnjBmjNVY+jg9R92vMMa0UaMml5VkFWi5MDQ0DICQkBA6dbqFpKQvCAkJIT09DYD09DQqV65ckIcQ8aySJeGhh2DPHjtmY80aO6ana1crnBUR5/z0k/1e1q4Njz1my/tffQVz5xaJBEuKvnwnWSdOnCAz8+fczxMTVxEREUF8fCfmzZsNwLx5s+nQobN7IhXxpDJl4B//sGTriSdg9Wpo2tRmtgphllVEfufoUfjnP6FWLTvIuXVr69L+/vtW5C7iI/KdZGVkHOSWW+KJjb2JDh3iaNcunptvbsfgwUNZs2Y1LVo0Ys2aRAYPHurOeEU8q0IFe3Hft8+WDdets2N6OnWys85ExHMOHrQed7Vr23JgXJzNKH/wgR0ML+Jj8n3seO3adUhM3PCH6ytWrMT8+YsKFJSI48qVs+WJIUPgpZfghResc3SrVraDqVMn6zAvIgW3d6/VW73xBpw8Cd262cyyQ7WRIu6iju8il1K2rL2z3rfPCm8PHIBbbrEX/9mz4dQppyMU8V3ffAMJCefOF+zdG7Ztg3feUYIlRYKSLJHLUbIk/P3v1uhw+nQ4fRruvBOuucaO6zl+3OkIRXxDTg4sXw7x8dYoeP58+93avdtmslTQLkWIkiyRK1GsmHWM/+YbOxOtbl07rqdGDfu4/9LtSkT8VlYWvPmm7d5t3x62brWD3Pfvt+X4GjWcjlDE7ZRkieRHQIAtGyYm2u7DW26BF1+0pKt7d+siryN7RCA11TaT1KwJffvCmTMwbZrVYY0aBZUqORygiOcoyRIpqKZNrT5r927rubV6tfXzadTIDqr95RenIxQpXDk58OmnVmNVs6bt1G3WDD76yGaw+vaF4GCnoxTxOCVZIu5SsyaMG2fF8f/7v3bdgAG2DDJsGOzY4Wx8XmznHbeROvm53K9zsrPZ1b8Pyc8+CUDm55s4svDdS36P00cOkzJxrEfjlDxkZtpzv3lzuOkmWLwYHngAdu6EDz+0ZULtyhU/oiRLxN1KloR77oEtW2w5sW1b+Pe/ITzc+v7Mm2f1KZLLFXwVWfv3c+bkSQB+2bqFoIrnlpFKN29Bxa7dL/k9gipWIuzhRzwap1zE1q0waBCEhcG991obhpdfhuRkq7e69lqnIxRxRL77ZIkWRMzNAAAOpklEQVRIHlwuWzaMiYG0NKtDmTIFevWCKlVs63q/fnD99U5H6hVKNWnKiaTNlIlqxfF1n1CmdTS/bvsWgJ9Wr+S3Xd9T9Z77SHvpRQJKluS3XT+Qfewolfv0pUxUK04dTCd57FPUfv4lflq9kszPN1IpNZXdb79FhVtvh9OnOf7JalzFilH90ScILFOGA0+MIiShH1ddU4/s48fZ98gw6r7yeu79c377jay01Ive368dP25vGKZOhY0bbfmvRw+47z6IitKMlQiayRIpHNWqWRPTXbtg6VJravrii3DDDdCypSVfP/3kdJSOKtOqDT+vX8uZrCyy9u3lqnr1L3rb00ePcPVTYwl75DEOzXrzgrfJ2r+PIx1vo+azz3F4zlu4goOpNWESV9UP5/gnq/KMJ2v/PsKGP5rv+xdJOTmU27LFdthWq2azVsePw/PPQ0oKzJhhy4RKsEQAzWSJFK6AAOjQwS4HD8KsWdYbaOBAePBBIqKirFC+fXsI8q9fz+BadTiVcZCf139CqSZNL3nb0pEtcQUEEHx1TU5fJDkt8acG5BQvTlC5cgSULEmpppH2ODVrcXLf3jzjKfGnBgSUKElACfJ1/yJl5057rr71Fo1377YmvQkJ0L+/1V8pqRK5IM1kiTilShUYOhS+/ho++wz69qXi5s3QubPVtgwebNf7USuI0s0iyZgxjTKtoi95O1dQsXNfXGR8XL9PUgMCCCj23/u4AsjJzrZPAwPJOWP3P3Mq64rvX6QdPGi1hC1aWIPQp56CunXZ9uij1pbh1VchMlIJlsglKMkScZrLZbMBr7zChnffhffeszquKVPsD1y9etZPaMuWIp9wlb25LZXu6EVwrdqF8nhBIVU4ufsHADI3ri+Ux/Rqhw9b25H27c8l+llZdq7ggQOwfDnp7dvb5g4RyZOSLBEvklOsGHTpYme3paXZH7xrroHx46FxY9uh+NhjRTbhKlapMhU63Vpoj1fxtq4c+3gp+4YPIfv4z4X2uF7l8GHblNGxo9VZDRhgPd9GjLBZ1i+/tP5v1as7HamIz3EdPHjc8Vfqjh3j2Lx5s8cfJzExkdjYWI8/jr/QeLrfRcc0IwMWLrTdXImJ1jW7Th24/Xa7tGoFgYGFHa7X03P0IvbvtxnThQth7VrIzobatW13YM+eltBfZBlQY+peGk/3K4wxbdSoCcuXr8nzdv5VWSviq0JCbCfXvfdCejp88IH9kXz5ZetDFBJiR/t07gzt2llhsshZZ87AF19Yc9BFiyApya6//noYORK6drWTC1RfJeJWSrJEfE3VqrakM2CAbZ9ftsxmJBYssGWfYsWgTRvo1Mku4eH64+mPjh2D5cstsVq61ArZXS5rGTJunM2A1r94mwwRKTglWSK+rGxZW+Lp0QNOnbLz4hYvtsvDD9ulRg2b3WrXzrrPh4Q4HbV4QlaWNQVdvtwun39uM1jly1vLkM6d7WPlyk5HKuI3lGSJFBXFikF0tF3GjYN9++xA3uXLbWlx2jS7XcOGEBtrlzZtoFKlS31X8VanTtmyX2IirFkDn3wCJ05YL7bISBg92nYJtmzpdz3XRLyFfvNEiqpatc7VcWVn2x/k5cth5Up47TWYNMlud+ON1jKiVSs7DqVmTS0veqPMTJud2rDBitXXrbOkCiAiwpqDtmtn52OWL+9srCICKMkS8Q+BgdaLq3lz67l18qT9wV6zxi7TpsFLL9ltw8Is2YqKshmRxo2hdGln4/c32dnWZf2LL2wJcMMG+OorW/4DO46pb19LjqOjrU5PRLyOkiwRfxQcDK1b22X0aDh92noibdhgdV0bNsD8+XZbl8uK55s1sx1ojRpBgwZQoYKzP0NRceoU7NhhSVRSEmzebB8zM+3fS5e2Jb/Ro+1cwBYtNPYiPkJJlohYzU7jxnYZNMiuS0+3mZTNm+2yYgXMnHnuPjVqWLLVoIHNrISH2/ErZco48zN4u+xs2LMHtm+Hbdssqd26Fb77zhItgKuusv+Dvn0tqW3WzMZVPdBEfJLHkqxVq5YzevRIsrOz6dPnbgYPHuaphxIRT6ha9VwbiLNSUy0x+Oor+7h1K3z8sc2EnVWjhiUG9epZt/q6dc99LOrLjqdPw48/wq5d1jV91y67bN9uy39ZvzsfsXp1S1A7dLCPN95otVUqUhcpMjzy25ydnc3IkQ/xzjvvExZWnfbtY4mP78R114V74uFEpLCEhtolPv7cdVlZ5xKJbdvOfZwzx3o1/V6lSnD11ecuNWrYpWrVc5eQkALP3Lz3ZTITPtpB8rFfqb5xFcPjr+P2xgU4FiYnx3qSpaefu6Sm2nl+Bw5YYnXgACQnn59wFitmndTDw+3YmoiIczN+FSsW6GcUEe/nkSQrKWkzderUpXbtOgB07dqNZcsWK8kSKYqKF7fkISLCOof/3tGj58/q7NtnycjevbZD7v8nYWA1YJUqWd1RxYr2sUIF2zFXpgyUKmUzYqVK2SU42JKZ4sWheHHeOwiPfnuSX7Pt2yUf+5VH39kC337L7ZXP2NJcVpZdfv3Vap9OnDj38aefLO4jR+zj0aN2vt9vv134Zz+bKLZubYnj72fvatTQUp+IH/NIkpWWlkr16jVyvw4NDSMpyfNnE4qIl6lQ4Vxt0YVkZkJKyvkzROnpdlbj2UTn0CH4/nv7+sQJ2xl5CRPum8qv5aqcd92vZ2DC+mRuf7X/xe8YEGDJW9my5xK8a6+1zytVOn+2rWpVO0w5JMTuJyJyAY4t/s+YMY2ZM6cDkJaWQmJioscfMzMzs1Aex19oPN3Pr8e0cmW73HDDJW/mys4m4NdfCfztNwJ//RXX6dMEnD6N69QpArKzSUm+cEf7lHIhJL30EmeCgsj57yU7OJjsEiU4U6IEZ4oVu/z+YGdnuLZtu9Kf0uf59XPUAzSe7udNY+qRJKtatVCSk3/M/To1NYXQ0LDzbpOQ0I+EhH4AdOwYVyinkOu0c/fSeLqfxrTgwsauIvnYr3+8vnxJmpzdOSn5pueoe2k83c+bxtQj89yNGzdl9+7d7Nu3l6ysLBYunE98fKe87ygiUkDD46+jRLHz66BKFAtkePx1DkUkIv7KIzNZQUFBjB07gZ49u5KdnU3v3ncRHh7hiYcSETnP2V2EubsLy5co+O5CEZF88FhNVtu28bRtG5/3DUVE3Oz2xtW5vXF1r1o2EBH/o20xIiIiIh7gFa2F9+zZQ6NGTTz+OIcPH6JSpcoefxx/ofF0P42pe2k83U9j6l4aT/crjDE9cGDfZd3OdfDg8RyPRuJF2rWLYfnyNU6HUWRoPN1PY+peGk/305i6l8bT/bxpTLVcKCIiIuIBSrJEREREPCBw+PBH/+l0EIWpYcPGTodQpGg83U9j6l4aT/fTmLqXxtP9vGVM/aomS0RERKSwaLlQRERExAOKfJL1wQcLadMmkqpVy7FlS1Lu9fv376NmzSrExbUiLq4VDz/8oINR+paLjSnApEnPERnZkKioJqxatcKhCH3X+PHP0KDBdbnPyxUrPnI6JJ+1atVyoqKaEBnZkMmTn3c6HJ/XtOmfiIlpSVxcK9q1i3E6HJ80ZMj9XH99XaKjW+Red/ToEbp370KLFo3o3r0Lx44ddTBC33OhMfWm19Ein2SFh1/PtGmziIpq9Yd/q127DqtXr2f16vVMnPiiA9H5pouN6Y4d21m4cD5r137G3LkLGDlyGNnZ2Q5F6bsGDhyU+7zUqQn5k52dzciRDzFnznzWrfucBQveZceO7U6H5fMWLFjM6tXrvWZ7vK/p1etO5s5dcN51kye/QHR0DJs2bSE6OobJk19wKDrfdKExBe95HS3ySVb9+tdx7bX1nA6jSLnYmC5btpiuXbsRHBxMrVq1qVOnLklJmx2IUPxdUtJm6tSpS+3adShevDhdu3Zj2bLFToclfi4qqhXly1c477plyxbTs2dvAHr27M3SpR86EZrPutCYepMin2Rdyv79+7j55tZ06dKRjRs3OB2Oz0tNTSEs7NwhvGFh1UlLS3UwIt80deoUYmKiGDLkfi0d5FNaWirVq9fI/To0NIzU1BQHI/J9LpeLHj1up23baGbMmOZ0OEVGRkYGVatWA6BKlapkZGQ4HFHR4C2vo15xrE5Bdet2GxkZ6X+4/tFHH6djx84XvE/VqtVISvqWihUr8dVXX3L33b1Zu3YTZcqU9XS4PiE/YyqX51Jj27fvPTz00EhcLhdjx/6LJ54YzaRJrzgQpcj5Fi36iNDQMDIyMrjjji7Uq1f/gmUYkn8ulwuXy+V0GD7Pm15Hi0SSNX/+B1d8n+DgYIKDgwHrp1G7dh127fqhUM5Q9AX5GdPQ0DBSUpJzv05JSaZatVB3hlUkXO7Y9ulzN3369PBwNEVTtWqhJCf/mPt1amoKoaFhDkbk+86OX0hICJ063UJS0hdKstwgJCSE9PQ0qlatRnp6GpUr6xzDgqpSpUru506/jvrtcuGhQ4dyi7L37t3D7t27qFWrtrNB+bj4+E4sXDifkydPsm/fXnbv3k2TJs2cDsunpKen5X6+ZMkiwsMjHIzGdzVu3JTdu3ezb99esrKyWLhwPvHxnZwOy2edOHGCzMyfcz9PTFxFRISem+4QH9+JefNmAzBv3mw6dNBKQUF50+tokZjJupTFixcxatRwDh8+RO/ed/CnP93I22+/x6efrmf8+KcJCipGQEAAEya8SIUKFZ0O1ydcbEzDwyPo0qUrrVs3JygoiHHjJhIYGOh0uD5lzJjH+PbbrwEXNWvWZOLESU6H5JOCgoIYO3YCPXt2JTs7m96971LCWgAZGQfp2/dOALKzT/OXv9zBzTe3czgq3zNwYD/Wr1/HkSOHadgwnBEjRjF48FAGDOjLrFkzqFGjJq+/Pt3pMH3KhcZ0/fq1XvM6qo7vIiIiIh7gt8uFIiIiIp6kJEtERETEA5RkiYiIiHiAkiwRERERD1CSJSIiIuIBSrJEREREPEBJloiIiIgHKMkSERER8YD/A3zHQieIihwEAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff8bceff208>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = np.linspace(-15,15,100)\n",
    "y = x**2\n",
    "\n",
    "fig, ax = plt.subplots(1, 1)\n",
    "fig.set_facecolor('#EAEAF2')\n",
    "plt.plot(x,y, 'r-')\n",
    "plt.plot([0],[0],'o')\n",
    "plt.ylim([-10,250])\n",
    "plt.gcf().set_size_inches((10,3))\n",
    "plt.grid(True)\n",
    "ax.text(0,\n",
    "        20,\n",
    "        'Minimum',\n",
    "        ha='center',\n",
    "        color=sns.xkcd_rgb['pale red'],\n",
    "       )\n",
    "plt.show"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function matplotlib.pyplot.show>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmIAAADFCAYAAAD30J6ZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3XlclWX6x/HPAQSzzF0ENzAXtNxAUdyQxHBJyd3M1JapadOs1CwzbdU0FadfM9Oipbm0IKVpboO4ZRqh6ViaiWKxiUtuKSDw++MecRx3OfCc5ft+vc4LeM52cXs45/K+7+e6bAcPHi9AREREREqch9UBiIiIiLgrJWIiIiIiFlEiJiIiImIRJWIiIiIiFlEiJiIiImIRJWIiIiIiFlEiJiIiImIRJWIiIiIiFlEiJiIiImIRL6sDuFYNGwYSEBBQrM9x6tQpbr755mJ9Dnei8bQ/jan9aUztS+NpfxpT+yqp8dy3bx+7du2/6u2cJhELCAggMTGxWJ8jISGBjh07FutzuBONp/1pTO1PY2pfGk/705jaV0mNZ7Nmwdd0Oy1NioiIiFjEskQsPn4VYWHBhIY2ZebMaVaFISJu6sutqbSdFM+w5adoOymeL7emWh2SiLghSxKxvLw8xox5lgULYtmw4XsWLfqC3bt3WRGKiLihL7emMnbRDlL/OA1A6h+nGbtoh5IxESlxluwRS0pKJDCwDgEBgQD06tWH5cuX0qBBkBXhiIizOXUKDh+Go0fN5cgR8/XECXPdyZPnv2ZnQ06OueTmQk4OUxo/yGmfchc85OncPKbMXcc9ExZCqVLg7Q0+PnDzzXDLLee/3nILVKhw/lKxormULQs2m0UDIiLOypJELCMjnerVaxT+7OfnT1LSxRvx58yZzdy5H/3nPmkkJCQUa1wnT54s9udwJxpP+3P1MfXIzsYnKwufgwfxOXiQ0llZeB86hPeRI+Zy9CjeR4/ieebMFR8n39OTvJtuIr90afJ8fCjw8qLAy4v8UqUo8PQkLeTWS94vrVRZTvz0Ex65udjy8vDIycHzzBk8T5/GIzf3ys9ZqhQ5FSuSU6GCuVSsSE7lypypUoXsqlXJ/s/XvDJlbnh8nIGrv0atoDG1L0cbT4c+a3LIkAcYMuQBALp2jSj2sxx0Zop9aTztzyXG9Ngx2LULfvkF9u6F5OTzXzMyLr59pUrg52cuzZqBr6+5VKp04axUhQpmVuqWW/Dw9r7ivgv/SfGFy5IXHK9QhrJ79lz6Trm5ZpbtxInzM3HnZuOOHMHj4EFKZ2ZSOjPT/B7JyXDwIBQUXPz71KkDt91mvtapA/XrQ1AQVKly7ePooFziNepgNKb25WjjaUkiVq2aH6mpvxf+nJ6ehp+fvxWhiEhxOXYMtm83l507TfL1888XJls2G9SoYZKSbt0gMBBq1YKaNc2lRg0oXdruoY2KasDYRTs4nZtXeOymUp6Mimpw+TuVKgXly5tLzZrX9kQ5OZCWBr/9Br//DgcOwL59Jkn7/nv4/HPIOx8DlSqZhCwoCG6/HZo0MRcXSNBE5NIsScSaNw8hOTmZlJT9+Pn5ExcXyz/+8aEVoYiIPaSlQWKiuWzbZpKvlJTz15crBw0bQpcu5mtQkJkFCgw0+7BK2D3NqwMwZcVuUv84TfXyNzEqqkHhcbvx9oaAAHO5lLNnTXL2yy/nE9Vdu2DxYvjwv94Tq1UzCVnTptCihbkEBmpPmogLsCQR8/LyYtKkKQwY0Iu8vDwGDbqfoKCGVoQiItfr+HHYsgW+/dbM6iQmnp/l8vAwSVabNvDXv56f0ale3eGShnuaV+ee5tWtXabw8jq/PNmly4XXHTwIO3acn1X88UeIiTGzbGCWYkNCoGVLCAszl8qVS/53EJEisWyPWGRkFJGRUVY9vYhcqwMHYO1ak3h9+61JDgoKTGIVFAR33WUSghYtzB4uF9+MXmKqVoVOnczlnJwc+Pe/z88+JibClClmZg2gXj2TBLdpAx06QIMGDpcAi8iFHHqzvohYYN8+SEgwydfatbB/vzl+663QujX06mU+6Fu1MkuOUnK8vSE42FweecQc+/NPk5Bt2mQS5WXL4OOPzXW+vhAebi4dO5plYSVmIg5FiZiIu/vjD4iPh5UrYdUqs5EczMbx8HAYOdJ8veMO8PS0Nla5WJkyZvarQwfzc0EB7NlzPpFOSIDPPjPX+flBZCR07my++vlZFraIGErERNxNfj5s3QpLl8I335j9Xvn5plBpRAQ8/bT52qiR2fMlzsVmMydC1K8Pf/mLScySk2HNGli92syYzZ1rbtu4sdmb1r27meUsVcra2EXckBIxEXdw8iSsWHE++crIMB/YLVvCiy+afV6tWumD2BXZbKY8yG23wcMPm6R72zYz+7liBUyfbvaZlSsHUVEmKevWTRv/RUqIEjERV5WVZcogfPml+dDNzjY1sM592HbpovpU7sjD4/w+szFjzFmwq1ebJH3ZMrOM6eFhljrvucdcate2OmoRl6VETMSVpKVBbCx88QVs2GBmP2rXhsceg+hoaNfOlEwQOefWW6F3b3PJz4ekJPjqK4iLM8vUTz9tkrbevaF/f3NmpojYjTaAiDi7jAz4v/8zG+pr1IDhw03bnXHjzF6wffvM8lPHjk6ZhP3SryfpM98u/LkgL4+9Dw4m9c1XbujxTn6/mSNxX9grPNfi4WHKkLz6qimT8csv8NZbpujuuHFm31lwMEyadP6kDhEpEiViIs7o+HH46CNz5pu/Pzz5JBw+DBMmwE8/mVpfEyeaul5OXq7A5lOanAMHyM/OBuDP7dvwqljphh/vlpatqNirr73Cc2316sGoUaYsxoEDMG2aKaExdqzZcxYaCjNnmuKzInJDnO+/xyLuKjeXSps2wT/+YZaOzpwxH4bjxsGAAaY3oYu6OTiEU0mJlA1ry/EN6yjbrgOnf94JwOk9v5A1+30KcnOweftQ7fHheFevwdGvvyL7wH6qPT6C7JT9pMdMpdabb3Pi2w2c2bsH34f/SsY7Myh36BAHVn5N3rFj+D4+nONr4znzy25K161PtSefBmDP4P7U+8SUgDixaSOnfvieak8+TcY7M7B5e5O9P/mK93cJNWuaUiYjR5racp9/DvPnw4gR8Mwz5oSPwYPxqFDB6khFnIpmxEQc3c6d8NxzUKMGjV94wWysfughU8Bzzx545RWXTsIAyrZtz4mN68nPySEnZT+l69UvvM67eg1qvjqJ2lNiqDRgEIcWmNIM5bv1IDcjnRObN5Hx7kx8H3kcj0v0tfTIzqbm61OoMuwh0ia/RoXu0dSe9g7ZB1I4s+/qy2/5p04V6f5OKSDAzJRt3WpmX0eNMkuZ991Hm759TdmMTZtM6QwRuSLNiIk4omPHYOFCmDXL1Pny8oKePdkREkLjUaPcrsyET+1AcrMOcmLjOm4ODrnguvw/T5HxzgxyM9IAGwV5pt2PzcMD3yeeJuXZ4ZTvHMVNQY0u+dhnAupgs9nwrhWAZ7ny+NQOAMC7Zi3OZh2EwDpXjO3mFi2LdH+nd8cd8Oab8PrrsH49WZMm4bdgAXzwgWmB9eCDcP/9pnG5iFxEM2IijqKgwDTRfughs+/rr3817WumTy88G/KwGxfdvKVFKFlzZlO2bYcLjh9eOI8ydzQmYNo7VH9+HAW5uYXX5aan4VG6NGePHrns4xb8p1uAzWbD9l9ja7PZKMjLO/fD+dvn5lxwf5tXqavf3x14eEB4OLvHjIH0dPjwQ9OdYfRos6zZt6+Zzc3PtzpSEYeiREzEaidOwHvvmcbZoaHw6adw331mJmz7dlM+QPW+uPXOSCr1G1g443RO3p9/Fm7eP5bwr/PHT53i4Kz3qfnKm+SdOMGJTRtv+Lm9ypUj+/ffKMjP5+SW7274cdxG2bJmJmzDBti1y7yGExJMa6X69U0B2awsq6MUcQhKxESssmeP2ehcvTo8+iicPQvvvmtmv957z1S9d/IzHu2pVKXKVOjW46LjFaN7c2j+HFJGjYC887MtWR9/QPku3fD2r47vY09xaN7HnD32xw09d+X7hpI26VV+Gzcaz/LajH5dGjQwidfvv8O8eWa2d/RoU2pl6FDTsFzEjdkOHjzuFLspu3aNILGY/2ATEhLo2LFjsT6HO9F4XkJ+vmkr87e/mVZDpUqZIpmPPw5hYVdNvDSm9qcxta9rGs+ffjL/6fj4Y9N+KywMnnoK+vQx5THkAnqN2ldJjWezZsGsWrX2qrfTjJhISfjzT/PBExRk+vht3Wpqfh04AJ98Yhoua/ZL3EWjRvDOO2aWLCbGLFMOGmTOxnzjDVOQWMRNKBETKU4ZGabOV82a8MQTptfj/PmQkgIvv6wzycS9lStnOkHs3m36XDZubJrQ16xpihT/+qvVEYoUOyViIsVh1y5z9mPt2uZ/+B06wPr1sHkz3Huvll9E/puHB3Ttapbtt283y/XvvWc29vfuDd/pBAlxXUrEROwpMdHsc2nUCBYsMMnY7t2mgXK7dlp+FLmaxo1h9mwzazx2rDnbMiwMIiJg5UoViRWXo0RMpKgKCiA+3vR9bNkS/vUveOEF80Hy7rumX5+IXB8/P1Mk9sABePtt04A8Ksr8jX3xBbhTjTZxaUrERG5UQQEsXw5t20KnTqYV0VtvmQ+O115T7S8Re7jlFtPLMjkZ3n/fNLzv18/MnM2fr4RMnJ4SMZHrVVAAS5dC69ZmX8vvv5uZr337TM+9W2+1OkIR1+PjAw8/DD//bNp/eXqawseNGsHcuaYOn4gTKrZE7K233qBJkwZERLQlIqItq1evKLwuJuZtQkObEhYWTHz86uIKQcS+ziVgoaFw992QmQn//Kc5s+uxx6B0aasjFHF9np4wYAD8+KNZoixdGoYMgYYNYc4czZCJ0ynWGbFHH32CNWs2smbNRiIjowDYvXsXcXGxrF+/hYULFzFmzDPk6Q9HHF18vFmCvPtuOHzYNDTeswceeURnQIpYwcPDnBizdas5GeaWW0yl/jvugM8+U09LcRolvjS5fPlSevXqg4+PD7VrBxAYWIekJLW4EAe1aZPZ/9Wpk9n79Y9/mLMgH3rIbZtvizgUDw+45x744QczQ+bhYWbMgoNhyRKdZSkOz6s4H3zWrPf47LMFNGvWnIkTX6d8+Qqkp6cREtKy8Db+/tXJyEi/5P3nzJnN3LkfAZCRkUZCQkJxhsvJkyeL/TnciTOP58379hH4wQdU/vZbcipUIOWJJ0jv2ZN8b2/YeOPNo4vKmcfUUWlM7cvS8axUCWbOxDc+noCPPuKmnj051qgRyY8+yrEmTayJyQ70GrUvRxvPIvWa7NOnJ1lZmRcdHzt2PCEhLalUqRI2m41Jk14jMzODmJh3ef75ZwkJaUm/fgMBePrpJ+jUqTM9etxzxedSr0nn45Tj+dtvpuL9xx9D2bKmOfHw4WbZwwE45Zg6OI2pfTnMeObmmnpkEydCWprZVvDmm2bp0sk4zJi6CEfrNVmkGbHY2MXXdLvBg4cyeHB/APz8/ElLSy28Li0tlWrV/IoShkjRHTkCkybBzJlmKWPkSFNMslIlqyMTkRtRqpTZwzl4sPm7njQJmjQx+8heecW0URJxAMW2RywzM6Pw+2XLlhAU1BCAqKhuxMXFkp2dTUrKfpKTkwkOblFcYYhcWU4OzJgBdevC1KkwcKApHDl1qpIwEVdQpgw8/zzs3Wvqkc2fb1onvfginDhhdXQixbdHbOLEl9i5cwdgo1atWkydGgNAUFBDoqN70a5dS7y8vJg8eSqenp7FFYbIpRUUwFdfmbpfv/4KnTub5MuJ95GIyBVUqmT+xp96yiRhb7wBH34Ir74KDz5oymKIWKDYErF3333/steNHDmKkSNHFddTi1xZUpL5n/Hatab20LJl0KWL+kCKuIPateGTT8zez2eeMcuXf/ubaaPUubPV0YkbUmV9cR9ZWeZNt0UL+OknUw1/+3ZTHV9JmIh7CQ2F9evh88/h5Em46y5TBiM52erIxM0oERPXl5sLMTGm+fbs2WYj/p49phq+V7FWcBERR2azQd++pm3SpEmwerVpmTRuHJw6ZXV04iaUiIlr+9e/oFkzePppaNXKzIC9/TaUK2d1ZCLiKHx8YMwYc6JOv37w+uvQoAEsWKCCsFLslIiJa0pNNWdARkbCmTPw5ZewfLnZEyYicin+/qaB+IYN4OsLgwaZrho//WR1ZOLClIiJa8nNhWnTICjIJF8TJ8LOnRAdrX1gInJt2raFLVtMS7Nt26BpUzNjdvKk1ZGJC1IiJq5jwwYICYFnn4UOHUwCNn48lC5tdWQi4mw8PeHRR01v2fvvh7feMvvHYmO1XCl2pURMnN+RI/Dww9C+PRw7BnFx8PXXcNttVkcmIs6uShWYNcv8R69CBbO5v2dPOHDA6sjERSgRE+dVUGA20zZsCB99ZIqz/vSTOQVdy5AiYk9t28IPP5iTfeLjzezY9Olw9qzVkYmTUyImzmnfPlP/a9AgU6AxMdEsHdx8s9WRiYir8vIyRWB37oTwcPN9q1amSLTIDVIiJs4lL89sxr/9dti40TTz3bTJlKgQESkJAQFm+8Onn5oztFu2hNGj4fRpqyMTJ6RETJzHzp1meeDZZ8+fUv7UU+oRJyIlz2aD/v1h1y546CGYMsX0ql23zurIxMkoERPHl5trGvM2bw5798L8+bB4MdSsaXVkIuLuypeH994zxaPz8syS5RNPwIkTVkcmTkKJmDi2rVtNb8jx46FPHzMLdu+92owvIo7lzjthxw7TxePvfzfbJ1autDoqcQJKxMQx5eaaYqyhoaZZ91dfmTMkq1SxOjIRkUu7+WZzJuXGjeb7qChTi0yzY3IFSsTE8ezYYc5EmjDBtCn6979N3R4REWcQFmZm8597Dt5/Hxo3NiUvRC5BiZg4jrNn4Y03THX81FRYtMj0fatY0erIRESuT+nSZgP/hg3g7W1OMHrySTh1yurIxMEoERPHsGcPtGsHL75oCrL++9/Qq5fVUYmIFE2bNqZf5YgR8H//Z0rtbN5sdVTiQJSIibUKCswZR82amZ5uCxbAZ59pL5iIuI4yZWDGDFizBnJyTBmeCRPMXlhxe0rExDqZmWbv16OPmv817thh9oSJiLiijh1h+3bTEWTiRJOQ/fKL1VGJxZSIiTWWLDEbWFevhpgYWLECatSwOioRkeJVrhzMmWNm/vfuNasB//iHWR0Qt6RETErW6dOm2GHPnibx+uEHGD4cPPRSFBE30q+fWQVo3x4ee8zsiT10yOqoxAL69JOSs2OH6cn27rvmtO5Nm6BRI6ujEhGxhr8/fPONqT32zTfQtKnKXLghJWJS/AoK4J13TBJ2+LCpNj1lCvj4WB2ZiIi1PDxMNf7vvoNbb4XISHj+ebOpX9xCkRKxxYvjaN8+FF/fcmzblnTBdTExbxMa2pSwsGDi41cXHo+PX0VYWDChoU2ZOXNaUZ5enMHhw2YZ8qmnzBvM9u3QubPVUYmIOJbmzSExEf7yF5g82ZTzSU62OiopAUVKxIKCGjF79jzCwtpecHz37l3ExcWyfv0WFi5cxJgxz5CXl0deXh5jxjzLggWxbNjwPYsWfcHu3buK9AuI4yq3Y4fZiLpypdmQv2SJylKIiFzOzTfDP/8JX3xhais2bw6ff251VFLMvIpy5/r1G1zy+PLlS+nVqw8+Pj7Urh1AYGAdkpISAQgMrENAQCAAvXr1YfnypTRoEFSUMMTR5OfDpEk0e+klqFPH7AULDrY6KhER59Cnj+kwMnAg9O9PvZ49oXVrU61fXE6RErHLSU9PIySkZeHP/v7VychIB6B69fMlCvz8/AsTtEuZM2c2c+d+BEBGRhoJCQnFEW6hkydPFvtzuLpSR47Q8M03qZiYSFqHDiSPHk3e8eOgcbULvUbtT2NqXxpP+7G99hqBH3xArU8/5eTtt7Pz5Zc5XauW1WE5PUd7jV41EevTpydZWZkXHR87djxdu3YvlqDOGTLkAYYMeQCArl0j6NixY7E+X0JCQrE/h0tbu9aUpvjjD3j/ffbcdhsdIyKsjsql6DVqfxpT+9J42llkJNubNaPJ1Km0evxx+OADFb4uIkd7jV41EYuNXXzdD+rn509aWmrhz2lpqVSr5gdAaurvhcfT09Pw8/O/7scXB5Ofb86CfOEFqFvX7Alr3FizYCIidnCkdWvTr3LgQLj3XtNI/O23dea5iyiW8hVRUd2Ii4slOzublJT9JCcnExzcgubNQ0hOTiYlZT85OTnExcUSFdWtOEKQknL0KERHm9Ot+/aF7783SZiIiNhPjRqmV+Wzz5rm4e3bw/79VkcldlCkRGzp0iU0bRpEYuIWBg3qR//+9wAQFNSQ6OhetGvXkoEDezN58lQ8PT3x8vJi0qQpDBjQi7ZtWxAd3YugoIZ2+UXEAomJZhP+ihUwcyYsXGjq4IiIiP2VKgVTp0JsLOzebd5/ly61OiopoiJt1u/evQfdu/e45HUjR45i5MhRFx2PjIwiMjKqKE8rjuCDD8x+MF9fWL8eWrWyOiIREffQuzc0aWJWIe6+G156CV5+GTw9rY5MboAq68v1yc6GRx4xRQfDwyEpSUmYiEhJq1vXlAYaNgxefRV69IAjR6yOSm6AEjG5dr/9ZvYlvP++2Zj/zTdQubLVUYmIuKebboJZs+Dvf4fVq6FFC7OpX5yKEjG5NvHxZj/Crl0QFwevv65pcBERq9ls8Ne/wrp1ZsUiLAzmzrU6KrkOSsTkygoKYPp00x+yalVzVuQ991gdlYiI/LfWrc9vFRkyBEaOhLNnrY5KroESMbm806fN/oNnnjElKr77Dhpcuq2ViIhYzNcXVq2CESNgxgzo0gUOH7Y6KrkKJWJyab//Dh06wJw58Morpglt2bJWRyUiIldSqpRJwmbPNme0t2wJ27dbHZVcgRIxudjGjWbT565d8OWX5tRoD71UREScxrBhF+4b++ILqyOSy9Cnq1xo1iyIiDCzX999Z5YkRUTE+bRqZQpvN2kC/fqZWmP5+VZHJf9DiZgYeXmmdcZDD5n6YFu2wO23Wx2ViIgUhZ+f6fs7bJjZZjJgAPz5p9VRyX9RIiZw/Dj07AnTpsGTT5r6YBUqWB2ViIjYg4+PWe041x6pfXuzD1gcghIxd5ecbPYPrFhhigL+7W/gVaTOVyIi4mhsNrPqsWQJ7NkDoaFm5UMsp0TMna1fb/4Y09Nh5UpTFFBERFxX9+6mNVLp0mYbyqefWh2R21Mi5q7mzoVOnaBSJdi8Ge680+qIRESkJNx+u3nfb9kSBg6E114zxbvFEkrE3E1BgTlzZsgQaNvWnBlZr57VUYmISEmqUsUUfx082JQoGjbMlLqQEqfNQO7kzBl48EFYsMD80f3zn+DtbXVUIiJiBR8fU7S7bl2YMAH274dFi8xKiZQYzYi5i0OHIDLSJGFvvGHOoFESJiLi3mw2s0ryySdmhSQszGzmlxKjRMwd/Pqr+eNKTDQbM8eONX98IiIiAPfdB//6Fxw5Yj4vNm2yOiK3oUTM1Z37H87RoxAfD/37Wx2RiIg4onbtTAJWvrw5gSsuzuqI3IISMVcWF2faFZUrZ/642rSxOiIREXFk9eqZz4umTaFPH5g50+qIXJ4SMVf1t7+ZP6KmTeHbb3VmpIiIXJsqVcwKSnQ0jBgBzzyjHpXFSImYq8nPh+eeg+HDTdui+HioWtXqqERExJmUKQNffAFPPQXTp5selWfOWB2VS1L5CleSk2PKU8ybB088ATEx4OlpdVQiIuKMPD3N50hAgGmPlJUFX35p9pCJ3RRpRmzx4jjatw/F17cc27YlFR4/cCCFWrWqEhHRloiItjz33NOF1/3441bCw1sTGtqUF14YRYGq+drHiRNw990mCXvjDbM0qSRMRESKwmYzS5Pz5pltLh06QFqa1VG5lCIlYkFBjZg9ex5hYW0vui4gIJA1azayZs1Gpk6dUXh89OiRvP32TDZv3kZy8l7i41cVJQQByMw0m/Lj42H2bJWnEBER+xo0CJYtg337zJn4u3ZZHZHLKFIiVr9+A+rWvfZN4JmZGZw4cYIWLUKx2Wz0738vy5YtLUoIsnevaVX088+weLGpmC8iImJvkZGwdq3ZK9a2rWqN2Umx7RE7cCCFO+9sR9myZRk79iVat25Denoafn7VC2/j71+djIzLT3HOmTObuXM/AiAjI42EhITiCheAkydPFvtz2NMtv/5Kk9GjseXlsX3KFE6UKQMOFL+zjacz0Jjan8bUvjSe9udoY1p62jSajBmDT0QEOydO5EirVlaHdF0cbTyvmoj16dOTrKzMi46PHTuerl27X/I+vr7VSEraScWKlfjxx60MHTqI9es3X3dwQ4Y8wJAhDwDQtWsEHTt2vO7HuB4JCQnF/hx2s26d2TxZrhysXElIUJDVEV3EqcbTSWhM7U9jal8aT/tzyDHt3Bm6dKHJuHGmX+W991od0TVztPG8aiIWG7v4uh/Ux8cHHx8fAJo2bU5AQCB79/6Kn58/6emphbdLS0ulWjX/6358t7dkiamQHxAAK1dCzZpWRyQiIu6kalVYs8aUSbrvPtMa6YknrI7KKRVLHbFDhw6Rl5cHwP79+0hO3kvt2gH4+lajbNmyJCZuoaCggM8+W0DXrt2KIwTXNWcO9OoFjRvD+vVKwkRExBrlysHy5dCjBzz5JEycCKqEcN2KlIgtXbqEpk2DSEzcwqBB/ejf/x4ANm3aSMeOYUREtOWhh4YwZcoMKlSoCMDkydN45pmnCA1tSkBAIJ063VX038JdzJgBQ4dCx46mOWvlylZHJCIi7uymmyA21nw2TZhgiomrCv91KdJm/e7de9C9e4+LjvfoEU2PHtGXvE+zZsGsW3f9+8XcWkEBvPKKeZH37g3z58N/ln5FREQs5eUFs2ZBpUowbRocPw4ffmiOy1VplBxdQYFpWTRtmilN8f77enGLiIhj8fCAqVNN1f3x4+HkSU0aXCP1mnRkeXnwyCMmCRs+XP/DEBERx2WzwUsvmW00ixaZjfynTlkdlcNTIuaocnPNmSgffADjxpkXtof+uURExMGNGGEmDlavhqgoOHbkPKSeAAAPFUlEQVTM6ogcmj7ZHdGZM2Yv2KefwuTJ8OqralkkIiLO48EHYcEC2LwZ7rwTDh2yOiKHpUTM0fz5p5nO/fprePddGD3a6ohERESuX//+8NVXsHOn6YeceXFxeFEi5lhOnICuXU1pitmz4bHHrI5IRETkxnXrBkuXQnIyhIdDaurV7+NmlIg5ij/+gLvugo0bYd48Ne8WERHX0KmTKfyalgYdOkBKitURORQlYo7g8GHzQv3hB/j8cxg40OqIRERE7Kd9e1i1yrRC6tABfv3V6ogchhIxq2VlmbXznTvhyy9N+yIRERFX06oVxMebkhbh4bB7t9UROQQlYlbKzDRJ2K+/ms353dR3U0REXFjz5pCQAGfPmnZ9P/9sdUSWUyJmlYwMk4Tt22c2MkZGWh2RiIhI8bvjDlizxnSOiYiAn36yOiJLKRGzQnq6efGlpMCyZeZ7ERERd9GokZkZ8/AwM2P//rfVEVlGiVhJS0szL7rffjNnkYSHWx2RiIhIyQsKMslYqVJmQmLHDqsjsoQSsZKUmmqSsLQ0WLHCnEUiIiLirurXN8mYj49JxrZvtzqiEqdErKSkp5s2D+npJglr29bqiERERKxXrx6sXQs33WRKObnZMqUSsZJwbmN+aqpZjmzTxuqIREREHMdtt5kN/D4+ZtJi506rIyoxSsSKW2ameVH9/jt8841mwkRERC6lbl2TjHl5mc9NNyltoUSsOB08aKZZU1JMiQrtCRMREbm8evVMMubhYVaSdu2yOqJip0SsuBw6ZJKw5GRTrFVnR4qIiFxdgwamAj+YmbFffrE2nmKmRKw4HD0KnTubivlLlqhOmIiIyPVo2NAkY2fPmmQsOdnqiIqNEjF7O34cunQxlYLj4sysmIiIiFyfRo1g9Wo4fdp8lv72m9URFQslYvZ08qTpF5mUBJ99ZhIyERERuTFNmsDKlXDkyPkSUC6mSInYhAnjaNMmhPDwMIYOHcSxY38UXhcT8zahoU0JCwsmPn514fH4+FWEhQUTGtqUmTOnFeXpHcvp09CzJ2zaBPPnQ3S01RGJiIg4v5AQU/opPd3MjB08aHVEdlWkRCw8PIJ16zazdu0mbrutLjExJrHavXsXcXGxrF+/hYULFzFmzDPk5eWRl5fHmDHPsmBBLBs2fM+iRV+we7cLnBGRnQ29e5vqwB9/DP36WR2RiIiI6wgLM9UH9u+HyEg4fNjqiOymSIlYREQnvLy8AAgJaUlaWioAy5cvpVevPvj4+FC7dgCBgXVISkokKSmRwMA6BAQE4u3tTa9efVi+fGnRfwsrnT0LAweabP2992DwYKsjEhERcT3h4fDVV+Ysyi5dzJ5sF+BlrwdasGAu0dG9AUhPTyMkpGXhdf7+1cnIMOu61avXKDzu5+dPUlLiZR9zzpzZzJ37EQAZGWkkJCTYK9xLOnny5PU9R34+QZMmUW3VKvY89RSpdeuaWTEBbmA85ao0pvanMbUvjaf9aUz/S6lSVBo/ntvHj+d4+/ZsnzyZ/NKlr+shHG08r5qI9enTk6yszIuOjx07nq5duwMwffoUPD296Nt3gF2DGzLkAYYMeQCArl0j6Nixo10f/38lJCRc+3MUFMATT8CqVfD669R74QXqFWt0zue6xlOuicbU/jSm9qXxtD+N6f/o2BHq1KH8oEF0iIkxs2Te3td8d0cbz6smYrGxi694/cKF81i5cjmxsUuw2WyAmek6t0wJkJaWSrVqfgCkpv5eeDw9PQ0/P/8bCtxSBQXw/PPw97/DmDEwdqzVEYmIiLiPgQNNpYK//AUGDYKFC01rJCdUpD1i8fGreOedGcyd+yllypQpPB4V1Y24uFiys7NJSdlPcnIywcEtaN48hOTkZFJS9pOTk0NcXCxRUd2K/EuUuDffhLfegsceM9//JwEVERGREvLwwzBtGsTGmoQsP9/qiG5IkdLH559/jpycHPr1M6UaQkJaMnXqDIKCGhId3Yt27Vri5eXF5MlT8fT0BGDSpCkMGNCLvLw8Bg26n6CghkX/LUrSO+/Aiy/C/feb75WEiYiIWGPkSLNpf8IEKFsWYmKc7nO5SInYli0/Xva6kSNHMXLkqIuOR0ZGERkZVZSntc4nn8BTT8E998CsWaYpqYiIiFhn/HiTjE2bBhUrmqTMiTjngqoVvv4ahg0zlX0XLHDatWgRERGXYrPB1Kmmz/PEiSYZGz7c6qiumbKJa7FunSnSGhwMX34J13mqrIiIiBQjm83U8jx6FEaMMMmYk9T11Nra1WzdCj16QGAgLFtm1qBFRETEsXh5mRWrO+80K1hLllgd0TVRInYlv/wCUVFQvrxpOlq5stURiYiIyOWULm1WroKDoX9/s6Ll4JSIXU5aGtx1l/l+1SqoUePKtxcRERHrlS1rVrACA82K1o+XP7HQESgRu5Q//jB9rA4fhm++gfr1rY5IRERErlXlyrBiBdx6q/k837fP6oguS4nY/zp9Gnr2hF27IC4OQkKsjkhERESuV82aJhnLyTErXAcPWh3RJSkR+y+2vDy4917YsMHUDIuMtDokERERuVGNGpnyU6mp0K0bnDhhdUQXUSJ2TkEB9aZPN81DZ840m/xERETEuYWFweefw7Zt0Ls3ttxcqyO6gOqInfPSS/gvXQrjxsGTT1odjYiIiNhL9+7w4YcwbBhB+fnQqZPDdMdxjCisdvYsJCaS1r07vPKK1dGIiIiIvQ0dCpMnc3NKChw7ZnU0hTQjBqYI3OLF7Fm/Hn8naxYqIiIi12jUKJKaNqVDhQpWR1JIM2LneHtT4OlpdRQiIiJSXGw28n18rI7iAkrERERERCyiRExERETEIkrERERERCyiRExERETEIkrERERERCyiRExERETEIkrERERERCyiRExERETEIkrERERERCyiRExERETEIkVKxCZMGEebNiGEh4cxdOggjh37A4ADB1KoVasqERFtiYhoy3PPPV14nx9/3Ep4eGtCQ5vywgujKCgoKNpvICIiIuKkipSIhYdHsG7dZtau3cRtt9UlJmZa4XUBAYGsWbORNWs2MnXqjMLjo0eP5O23Z7J58zaSk/cSH7+qKCGIiIiIOK0iJWIREZ3w8vICICSkJWlpqVe8fWZmBidOnKBFi1BsNhv9+9/LsmVLixKCiIiIiNPystcDLVgwl+jo3oU/HziQwp13tqNs2bKMHfsSrVu3IT09DT+/6oW38fevTkZG2mUfc86c2cyd+xEAe/b8QrNmwfYK95IOHz5EpUqVi/U53InG0/40pvanMbUvjaf9aUztq6TG87ffUq7pdldNxPr06UlWVuZFx8eOHU/Xrt0BmD59Cp6eXvTtOwAAX99qJCXtpGLFSvz441aGDh3E+vWbryd+AIYMeYAhQx647vvdqM6dw1m1am2JPZ+r03jan8bU/jSm9qXxtD+NqX052nheNRGLjV18xesXLpzHypXLiY1dgs1mA8DHxwcfHx8AmjZtTkBAIHv3/oqfnz/p6eeXL9PSUqlWzb8o8YuIiIg4rSLtEYuPX8U778xg7txPKVOmTOHxQ4cOkZeXB8D+/ftITt5L7doB+PpWo2zZsiQmbqGgoIDPPltA167divYbiIiIiDipIu0Re/7558jJyaFfv2jAbNifOnUGmzZt5K23XsfLqxQeHh5MmTKDChUqAjB58jSGD3+M06dP06lTZzp1uqvov4Wd3H//MKtDcCkaT/vTmNqfxtS+NJ72pzG1L0cbT9vBg8dVyEtERETEAqqsLyIiImIRJWIiIiIiFnH7RGzx4jjatw/F17cc27YlFR6/UpsmubLLjSlATMzbhIY2JSwsmPj41RZF6NzeeusNmjRpUPjaXL16hdUhOaX4+FWEhQUTGtqUmTOnXf0OclUhIXcQHt6aiIi2dO4cbnU4TmfEiMdp1KgOHTq0Kjx29OgR+vaNplWrZvTtG80ffxy1MELnc6kxdbT3ULdPxIKCGjF79jzCwtpedN3l2jTJlV1uTHfv3kVcXCzr129h4cJFjBnzTOHZtXJ9Hn30icLXZmRklNXhOJ28vDzGjHmWBQti2bDhexYt+oLdu3dZHZZLWLRoKWvWbHSoOk3OYuDA+1i4cNEFx2bOnE6HDuFs3ryNDh3CmTlzukXROadLjSk41nuo2ydi9es3oG7delaH4VIuN6bLly+lV68++Pj4ULt2AIGBdUhKSrQgQnF3SUmJBAbWISAgEG9vb3r16sPy5Wq3JtYKC2tL+fIVLji2fPlSBgwYBMCAAYP45puvrQjNaV1qTB2N2ydiV3KuTVN0dFe+++5bq8Nxeunpafj7/2+Lq3QLI3Jes2a9R3h4GCNGPK6lihuQkZFO9eo1Cn82xaYv325Nro3pIXwPkZEdmDNnttXhuISsrCx8fasBULWqL1lZWRZH5Boc6T3Ubr0mHdm1tGn6X5dr01S27K3FHa5TuJExlWt3pfEdNuxhnn12DDabjUmTXuPll18kJuZdC6IUudCSJSvw8/MnKyuLfv2iqVev/iW3fciNsdlshR1s5MY52nuoWyRiV2vTdCmXa9NU3I3HncWNjKmfnz9paf/b4srPnmG5jGsd38GDhzJ4cP9ijsb1VKvmR2rq74U/p6en4eendmtFdW4Mq1SpQrdud5OU9IMSsSKqUqUKmZkZ+PpWIzMzg8qV1fy7qKpWrVr4vSO8h2pp8jIu16ZJblxUVDfi4mLJzs4mJWU/ycnJBAe3sDosp5OZmVH4/bJlSwgKamhhNM6pefMQkpOTSUnZT05ODnFxsURFqd1aUZw6dYqTJ08Ufp+QEE/DhnptFlVUVDc+/XQ+AJ9+Op8uXbTiUFSO9h7qFjNiV7J06RJeeGEUhw8fYtCgftxxR2M+++zLK7Zpkiu73JgGBTUkOroX7dq1xMvLi8mTp+Lp6Wl1uE5n4sSX2LlzB2CjVq1aTJ0aY3VITsfLy4tJk6YwYEAv8vLyGDTofsvfjJ1dVtZBhg27D4C8vLP07t2PO+/sbHFUzuXRRx9g48YNHDlymKZNgxg9+gWGDx/JX/4yjHnz5lCjRi0++OAjq8N0Kpca040b1zvUe6haHImIiIhYREuTIiIiIhZRIiYiIiJiESViIiIiIhZRIiYiIiJiESViIiIiIhZRIiYiIiJiESViIiIiIhb5fzqi56vTj9ljAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff8bceff550>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = np.linspace(-15,15,100)\n",
    "y = -x**2\n",
    "\n",
    "fig, ax = plt.subplots(1, 1)\n",
    "fig.set_facecolor('#EAEAF2')\n",
    "plt.plot(x,y, 'r-')\n",
    "plt.plot([0],[0],'o')\n",
    "plt.ylim([-250,10])\n",
    "plt.gcf().set_size_inches((10,3))\n",
    "plt.grid(True)\n",
    "ax.text(0,\n",
    "        -30,\n",
    "        'Maximum',\n",
    "        ha='center',\n",
    "        color=sns.xkcd_rgb['pale red'],\n",
    "       )\n",
    "plt.show"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function matplotlib.pyplot.show>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmgAAADGCAYAAABmZnz+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3XtYlHX6x/H3AIKHPGAqJ1N0zbAsPIWxqEiaoGZmmiQmqb/MrXZt1U3LdjusWyt5tt12aytdXRMttFITzRBPlUpk2kFLSTROYoqKIsgwvz++iZrnZmBm4PO6ruea4ZmHee65L4Tb79Fy8OAxGyIiIiLiMjycHYCIiIiInE8FmoiIiIiLUYEmIiIi4mJUoImIiIi4GBVoIiIiIi5GBZqIiIiIi7G7QDt16hTR0d3p3v23dO0aRkLCiwBkZu4jJiaKsLBQRo0aTklJCQDFxcWMGjWcsLBQYmKi2L8/s/y9Zs+eTlhYKOHhHUhJWWtvaCIiIiJuye4CzcfHh6SkFaSmfkJKymbWrVtLWtpWJk9+jtGjH2fr1i+pX78BCxfOB2DhwvnUr9+ArVu/ZPTox5k8+TkAdu/exbJlSWzcuJXExKVMnDgOq9Vqb3giIiIibsfuAs1isXDdddcBcPr0aU6fLsVisbBp03r69bsXgNjYIaxatQKA5OSVxMYOAaBfv3vZuDEVm81GcvJKBgwYiI+PD82bB9OiRUvS09PsDU9ERETE7ThkDJrVaiUqKoKbb/4NkZFRBAe3pF69+nh5eQEQGBhEbm4OALm5OQQFNQXAy8uLunXrcfjwYXJysgkMDCp/z3O/R0RERKQ68XLEm3h6erJu3WaOHi1g+PChfP/9d45420uaP38uCxbMA+C773Zxww03VOj9ysrK8PDQfApHUk4dS/l0POXUsZRPx1NOL8+rsJBa2dlYa9akqGlTbFfIVWXl89ChQ+zate+K1zmkQDujfv0GRER0JS1tK8eOHaW0tBQvLy+ys7Pw9w8AwN8/gKysHwkMDKK0tJTjx4/RsGFDAgICyc7OKn+vc7/nl+LjRxAfPwKA3r2jSEur2K7Q1NRUunfvXqH3qG6UU8dSPh1POXUs5dPxlNPLWLIE4uIgPBxWrYL69a/4LZWVz3btOlzVdXaXiocOHeLo0QIAioqKWL9+Ha1btyYiohvLl78HwOLFi4iJ6QtAdHQfFi9eBMDy5e/RpUskFouF6Og+LFuWRHFxMZmZ+8jIyKBDh072hiciIiLVydtvw5AhpjhbvfqqijNXZHcLWl5eLn/4w++wWq3YbGXcc88AevXqTevWIYwePYK//30yt94aytCh8QAMHRrP448/QlhYKL6+vrz22lwAQkLa0L//ALp0uR0vLy8SEqbh6elpb3giIiJSXfz3vzByJHTrBsuXw8+TGN2R3QXaLbe0JSVl0wXng4NbsHp16gXna9asyZtvzr/oe40d+yRjxz5pb0giIiJS3bzxBjzyCPToAe+/D7VrOzsiu2h0oYiIiLi3mTNh1CiIjoYPPnD74gxUoImIiIi7stng2Wdh3DgYNMi0nNWq5eyoHMKhszhFREREKkVZGfzxj/DKK2bc2euvQxUau64WNBEREXEvpaUwYoQpzsaONePPqlBxBirQRERExJ0UFcH998P8+fDXv8L06WCxODsqh1MXp4iIiLiHI0fgnntg82aYMwf+8AdnR1RhVKCJiIiI6ztwAGJiYM8eSEyEwYOdHVGFUoEmIiIiru2rr0xxdvw4JCdDVJSzI6pwGoMmIiIirmvDBuja1cza3LChWhRnoAJNREREXFViIvTqBX5+8OmnEBrq7IgqjQo0ERERcS02G7z4otn0PCzMTApo3tzZUVUqjUETERER11FSAqNHw7x5MHQovPkm+Pg4O6pKpxY0ERERcQ1HjpjJAPPmwfPPw4IF1bI4A7WgiYiIiCv4/nvo1w9++MEUZg8+6OyInEoFmoiIiDjXRx+Zdc08Pc3zbt2cHZHTqYtTREREnMNmg1mzTLfmDTfAtm0qzn6mAk1EREQqX3ExjBxpNjvv3x8++QRatHB2VC5DBZqIiIhUrpwcs+DsvHnw3HPw7rtw3XXOjsql2F2gZWX9yIABfenS5Xa6dg3j9ddfBeDIkcMMGtSfzp3bMWhQfwoKjgBgs9mYNOlJwsJCiYwMZ8eO7eXvlZi4kM6d29G5czsSExfaG5qIiIi4mo0boUMH+PJLeOcdM1vTQ+1Fv2R3Rry8vHjhhRfZtGkbq1Z9zFtv/Yfdu3cxZ85MunWLZMuW7XTrFsmcOTMB+PjjNWRk7GXLlu1Mnz6bCRPGAqagmzYtgeTkFFavXse0aQnlRZ2IiIi4uTPjzaKioG5d2LIFBg1ydlQuy+4Czc/Pn9tuawfAddfVpXXrm8jJySY5eSWxsXEAxMbGsWrVCgBWrfqQwYOHYLFY6NQpjKNHj5KXl8u6dR8TGRmFr29DGjTwJTIyipSUtfaGJyIiIs5WWGh2BRg7Fu6+20wGaNvW2VG5NIe2Ke7fn8nOnTvo2LET+fn5+Pn5A9CkiR/5+fkA5OZmExjYtPx7AgODyMnJJicnh6CgoHPOB5KTk+PI8ERERKSy7doFd9xhujP//ndYuhTq13d2VC7PYeugFRYWMnLkMCZPnkLduvXOe81isWCxWBx1K+bPn8uCBfMAU/ClpqY67L0vprCwsMLvUd0op46lfDqecupYyqfjuUNO/dasofXMmVh9fPj25Zc50rEjbNjg7LAuytXy6ZAC7fTp04wc+SADBw7m7rvvAaBx48bk5eXi5+dPXl4ujRo1AsDfP5Ds7B/Lvzc7O4uAgEACAgLYvHnTOeeziYjoctH7xcePID5+BAC9e0fRvXt3R3yMS0pNTa3we1Q3yqljKZ+Op5w6lvLpeC6d0xMn4Pe/N7M0u3bFc9EiQs/pJXNFrpZPu7s4bTYbf/zj47RufROPPvr78vPR0X1YvPhtABYvfpuYmL4AxMT0ZsmSRdhsNtLStlKvXj38/PyJiurB+vUpFBQcoaDgCOvXpxAV1cPe8ERERKQyff01hIXBf/8Lf/4zpKSAixdnrsjuFrQtWz7jnXcSadPmFqKiIgB45plnGTNmLKNGDWfhwvk0bdqMN96YB0DPntGsXbuGsLBQateuzezZZlkOX9+GjBs3gV69ugMwfvxEfH0b2hueiIiIVAabDd54A554AurVgzVroGdPZ0fltuwu0O64I5yDB49d9LWkpOUXnLNYLCQkzLjo9XFxw4iLG2ZvSCIiIlKZDh2Chx+G9983RdmCBeDv7+yo3JpWhhMREZFfb80auPVWWLUKpk+H1atVnDmACjQRERG5dqdOmXXNoqOhYUPYuhXGjdOuAA6iLIqIiMi1SUuDTp3MzgB/+IP5OjTU2VFVKSrQRERE5OqUlMBf/mIWni0oMN2ac+ZArVrOjqzKcdhCtSIiIlKFbd8ODz0EO3aYx1mzoEEDZ0dVZakFTURERC6tpAReeAFuvx0OHjQzNefNU3FWwdSCJiIiIhf36acwapRZfDYuznRnXn+9s6OqFtSCJiIiIuc7ftwM/o+IgGPHYMUKWLhQxVklUoEmIiIiZ61YATffDP/8pynSvv4a+vZ1dlTVjgo0ERERgcxMuPde6NcP6teHTz6B2bOhbl1nR1YtqUATERGpzoqL4aWXoE0b+OgjmDIF0tPNUhriNJokICIiUl2tXQuPPw7ffQf33QczZ0KzZs6OSlALmoiISPWzZw8MGAB33QVWq1lwNilJxZkLUYEmIiJSXRw7BhMnwi23mO7MF1+Er76CmBhnRya/oC5OERGRqs5qNYvLPvMM5OWZnQBeegkCA50dmVyCCjQREZGqymaDDz+Ep54yLWUREWYZjU6dnB2ZXIG6OEVERKqiLVsgKgruvhtOnYIlS2DjRhVnbkIFmoiISFWyaxfcf79ZJuPbb82Cs998Y85ZLM6OTq6SujhFRESqgj174K9/NVsy1aoFzz0H48droVk35ZAWtCeeeIybb25Jt26dy88dOXKYQYP607lzOwYN6k9BwREAbDYbkyY9SVhYKJGR4ezYsb38exITF9K5czs6d25HYuJCR4QmIiJStWVmwsMPQ0gIvPsujBsHP/wAzz+v4syNOaRAe+CBoSQmLj3v3Jw5M+nWLZItW7bTrVskc+bMBODjj9eQkbGXLVu2M336bCZMGAuYgm7atASSk1NYvXod06YllBd1IiIicr6a2dnwyCNw442wYIFZcHbvXpg6FRo3dnZ4YieHFGjh4RE0aOB73rnk5JXExsYBEBsbx6pVKwBYtepDBg8egsVioVOnMI4ePUpeXi7r1n1MZGQUvr4NadDAl8jIKFJS1joiPBERkarjm29g2DA6DxsG8+eb1rM9e8y+mQEBzo5OHKTCxqDl5+fj5+cPQJMmfuTn5wOQm5tNYGDT8usCA4PIyckmJyeHoKCgc84HkpOTc9H3nj9/LgsWzCt/v9TU1Ir5ED8rLCys8HtUN8qpYymfjqecOpbyab/rdu+m+dtv02jjRsp8fNh3zz3kDR1KSaNGpuVs715nh+jWXO1ntFImCVgsFiwOnDkSHz+C+PgRAPTuHUX37t0d9t4Xk5qaWuH3qG6UU8dSPh1POXUs5fNXKisz2zBNmwapqVCvHkyahOcf/8iBr75STh3I1X5GK2yZjcaNG5OXlwtAXl4ujRo1AsDfP5Ds7B/Lr8vOziIgIJCAgACysrLOOZ9NgJpqRUSkOiouhjffhLZtzTpme/aYIm3/fvjb3+Dnv6lSdVVYgRYd3YfFi98GYPHit4mJ6QtATExvlixZhM1mIy1tK/Xq1cPPz5+oqB6sX59CQcERCgqOsH59ClFRPSoqPBEREdfz44/w5z/DDTeYsWXe3vC//0FGhlkyo359Z0colcQhXZyjR49g8+ZNHD78E6GhIUyYMIkxY8YyatRwFi6cT9OmzXjjjXkA9OwZzdq1awgLC6V27drMnv0qAL6+DRk3bgK9enUHYPz4ifj6NnREeCIiIq7LZoNNm+CVV2DpUtOteffdMGYM9OihxWWrKYcUaK+9Nvei55OSll9wzmKxkJAw46LXx8UNIy5umCNCEhERcW0FBWZR2ddfhx07oEEDGDsWHn0UWrZ0dnTiZNpJQEREpLLYbPDZZ6YoW7wYioqgQwfz9dChULu2syMUF6ECTUREpKLl5JjWsnnz4Ouv4brrID4eRo2Cjh2dHZ24IBVoIiIiFeHUKfjgA1OUrV5txpaFh5vWsiFDTJEmcgkq0ETEpf2UtITjm9aDhwcWiwdNRj9GrRtvuqrvPX0wj6wpkwme8Y8LXjvw3CQax4+g5m9uJOOxh2k+ZQae9epd1fseeG4SpUeOYPGugUfNmvg/OgbvoKaXvD73X6/ge3d/fG5odslrCrd+Ro2AwMteI26gtBTWrYNFi8yA/6NHoWlTeOop02J209X97IqoQBMRl1W0excnPt9Gs5dn4VGjBtZjx7CVnnZ2WAAEPDGOmr+5kYKPkslfMI+gp/58yWv9H/3DFd+vcOtn1Ol4uwo0d1RWBp9+ComJsGQJHDxoFpQdMAAefBCiosDT09lRiptRgSYiLqu04DCe9erhUaMGwHktXD+9k0jh51uxlZRQq3UITUY/jsVi4dTePeT+aw4AdW5rX359WXExea/OoTjzB2oENsVWUnLRex7bsI4jH66gcUEBebu/psnDv8NymT+utW9uS8FKM2P95M4vyZ//FjZrGTVbtaLJqMfwqFHjvNa67x8cjG+ffhSmb8PD25vACc9wOi+XwrStnPzmKw4nLSHgT0/h7a+Ful1aaSls2GBayZYtg+xsqFnTLI8xZAj06WO+FvmVKmyhWhERe9W5rT2lhw7xw5jfkfeff3Hy66/KX2sQ05fmU2YQPOMflJWUcOLzbQDkvjqbJiMfIXjanPPe6+iaVVh8vAme9SqNYodwKmPPBfcr/vEAxz/ZRLO/JZD/wDDw8DDdq5dRmLYV72bNKSspIfefswkYO4HgGa9gs5ZxdM2HF1xvKz5FzdY3ETxtDrXa3MLRj9dQ66Y2XNcpjMbDRtB82mwVZ67qxAkzpuz//g/8/c0aZW+9BXfcYRaTzcuDd96B++5TcSZ2UwuaiLgsj1q1aJYwg6Jd33Dyq53kzHyZRkMfon5UD05+vZMj7y+lrLgYa+FxfG5ohrXNzZSdOEHtm9sCUC+yOye2fw7AyW+/xrf33QD4NG+BT/PgC+53cueXnMrYy/6nxtO4sJCT3t54XmLl9pzZM/Dw9sarSROajHyE09lZ1GjSBO/AIADqd7+TguSV+Pbtf973Wby8qNPxdgBqtmzFiR3bHZIrqSA//ggrVsDy5ZCSYgb+16tnWsruuw9iYqBOHWdHKVWQCjQRcWkWT09q33IrtW+5FZ9mzTm2PoW6EV05+Ma/aDZlBjUaNebQkrcpO33xLstrVS8yisZDH7rixslnxqCdUVxYeHU38PTCcmZleA8PsFrtiFYc7tQp2LjRzLpcvRq++rnVtmVLGD0a+vWDrl3NFkwiFUhdnCLiskqyfqQkJ7v86+J9P+DVqDG2n4sxz7r1KCsqovCzT8zXda7Do04dir79BoBjG892T9ZucwvHNm0w77M/k+LMfRfcr3bb2yj87BNKjxYAYD1+nNP5B68q1hqBQZw+eLA83mPr11Hr55a8q+FRqxZlp4qu+npxEKsVPv/cbETeuzc0bAi9epltl5o0gYQEs27Znj0wa5bp1lRxJpVALWgi4rLKTp3i4FuvU3biBHh64O0fgN/o3+NZ5zrq94gmc/wf8GzQ4LyWLP/HniD3X3OwYKF2aLvy8/V79Sbv1Tns++NjeAc1pWbLVhfcz+eGZjR64EGyJj9H48Lj/Pjh+zR5eDQ1Gje5Yqwe3t74P/4EOTMSyicJ1O/V+6o/a92IruT9+58UfLicgPGaJFBhrFbYuRPWrzfLYaxfb7ZcArMExqhREB0NkZHquhSnshw8eMzm7CDs0bt3FGlpaRV6jyt1dci1U04dS/l0POXUsZyWz5MnYds2sxn5xo1mOYxjx8xrLVuaJTDOHIGBlR+fHfQz6liVlc927Trw0UeXn3wEakETEZGqoqwMdu2CLVvMsXWr2YT8zDi/tm0hLg66dDFH8+bOjVfkMlSgiYiI+ykthd27IT3djCFLT4cvvoAzkzXq14fbbzcr+IeHw29/C76+zo1Z5BqoQBMREddls5n1xXbuPHvs2AHffGNmXALUrg3t2sHw4dCpE3TuDK1bm1myIm5KBZqIiDjf6dOwbx989x18++3ZY9cuOHLk7HX+/nDbbfD449C+PXToYIoxbaUkVYwKNBERqRxHj5oi7Icfzh7ff2+OffvOXxPOzw9CQiA2Ftq0gVtvNUejRs6KXqRSqUATERH7HT9u9qPMyYGsLDhwAPbvN48HDhCRkWGuOVfdunDjjaZbcsgQaNXKfB0SYtYjE6nGXK5AS0n5iGeemYjVauXBBx9izJhxzg5JRKqR977IYurq3WQVFBH0WQpPRt/Eve2DnB1W5Ssuhp9+gkOHzONPP0F+Phw8aMaEHTxojtxcU5idOHHhezRsCDfcADfcwMHmzQnq0gVatDBHcLB5/cyuCiJyHpcq0KxWKxMnjuedd94nMDCIXr26Ex3dh5tuCnF2aCJSDbz3RRZPL91J0WnT1ZZVUMTTS3cCuH6RVlZmBs0XFZnj5ElTNJ04YWY2nnl+7NiFx9GjZpxXQYE5jhwx338p119vVtn38zNjwO6+GwICzDpiAQEQFARNm5630Ov3qakEac0ukavmUgVaenoaLVq0JDi4BQADBgwkOXmlCjQRd3emeCguNse5z0tKzADxM4+nT5slFC52WK1nj7Ky8w+b7fzn5x5w4eNFTD15C0U2n/POFZ22MnXJFu5d/tXZkxd7r1/e89w4fhnjuZ/j3OPcz3omF2eOM/k5k7df5vLMjMarVaOG2fS7bl2z/ESDBmYl/QYNzNGwoSnEGjUyj9dfD40bm69r1Li2e4nINXOpAi03N4egoKblXwcEBJKefuEuAfPnz2XBgnk/f082qampFRpXYWFhhd+julFOHcvh+bRa8TpxAq/jx/EqLKRGYSFehYV4njiB18mTeP58eJ04gWdREZ6nTuFZVITHqVPm+alTeBQX41FcjGdxMR6nTzsutgqUPeEDuEiPW3ZZDXjuuat6D5vFAhZL+SMWCzYPj/PO2Tw8wMMDm6cnNg8Pc3h6Xvzw8qKsRg1sXl7YatSgrFYtyry9zfMzh7c3ZTVrYvXxOfvc2xtrrVqU1aqFtWbNs0edOpTWro3tWveTPHzYHLt3X9v3/Uz/5h1POXUsV8unSxVoVys+fgTx8SMAs9VTRW/NoO00HE85dawr5rO09Ox4oXPHD505zh1ndOjQ2b0JL6d2bdP6Ureu6cqqU8e0rtSpY16rVevsY61aULOmOXx8zj+8vU2LzJnHM4eX19nD0/Ps45nDw+Pso8Vy4eMvD7jw8RcCp6aSVXBhS1Sgb+3zZxhe5r0sv3gUQ//mHU85dSxXy6dLFWj+/gFkZf1Y/nVOTjYBAe61N5pIpSsuplZWltn4+ecZcxw4YGbSnZlVl5dnutZ+qWZNM5aocWPThdWy5dnuLF/fs11fDRqYldkbNDjbLeblUr8+HOLJ6JDzxqAB1KrhyZPRIVr0VEQqlUv9hm3fviMZGRlkZu4jICCQZcuS+Pe/33R2WCLOZbOZ2XN79pj1ovbuPX8tqexsOv9yXFWjRmaQdmCgGcQdEHD28PMzR5MmprVLs+jKnZkIUD6Ls0Gt6juLU0ScyqUKNC8vL6ZMmUps7ACsVitxccMICWnj7LBEKkdxsSnCzl1F/bvvTFF27NjZ6zw8TPHVogX07AktWrCrqIiQXr3MkgZNm5ouRflV7m0fxL3tg1yuu0NEqheXKtAAevaMpmfPaGeHIVJxyspMy9eOHefvLbh37/njnIKDzay68PCzC3jeeCM0b27Ga50jNzWVEBUTIiJVhssVaCJVSmmpaQlLTzfH55/D9u1nF/W0WOA3vzFb2Nx/P9x8s9nW5qabzAB7ERGpllSgiTiKzWbGhm3ZYo6tW01RdmZ9qjp1zObOI0dCaKgpym655bzFPEVEREAFmsivV1ICX3wBmzaZ45NPzJIVYGZHduwIjz5q9hns0MF0T3p6OjdmERFxCyrQRK5WcTF89plZzmL9etNKVlRkXmvVCnr3hjvugM6doW1brbYuIiK/mgo0kUuxWmHbNli71hRln3xiuis9PKBdOxg9Grp0gYgI8Pd3drQiIlKFqEATOdePP8Lq1eZYu9ZsGg1w222mILvzTujWzSzYKiIiUkFUoEn1ZrWawfzLl8OKFWbJCzALuvbvD716mbXGGjd2bpwiIlKtqECT6ufECdNC9v778OGHZu9JT0/o2hWmToXoaDOGTCvsi4iIk6hAk+rh6FHTQpaUBMnJZnC/ry/06QN33w0xMeq2FBERl6ECTaqu48fhvfcgMRE++ghOnzZ7U44cCQMHmhazKrjht4iIuD/9dZKq5dQp0225aJFpMTt1Cpo1gzFjTFHWubOZhSkiIuLCVKCJ+7PZzBIY//0vLFliujObNIH/+z8YMsTsZamiTERE3IgKNHFf+/fD/Pnm+P57s3floEEwdKhZDkPdlyIi4qb0F0zcy+nTZkmM11+HNWtM61n37jBpkunCrFvX2RGKiIjYTQWauIeMDHjjDXjrLcjLg6ZN4dlnYfhwCA52dnQiIiIOpQJNXFdZmVmv7JVXYNUqM46sb1945BGzLIa6MEVEpIrSXzhxPUePwrx58M9/mrFl/v6mtWzUKNNyJiIiUsWpQBPX8cMPMGsWvPmmWe3/jjvg+efNwH9vb2dHJyIiUmnsWnvggw+W0bVrGH5+9dm+Pf2812bPnk5YWCjh4R1ISVlbfj4l5SPCwzsQFhbKnDkzys9nZu4jJiaKsLBQRo0aTklJiT2hiTvZtg1iY6FVK3j1VbjvPnPu008hLk7FmYiIVDt2FWghITczd+5CwsMjzju/e/culi1LYuPGrSQmLmXixHFYrVasVisTJ45n0aIkNm3axtKl77J79y4AJk9+jtGjH2fr1i+pX78BCxfOtyc0cXU2G6xcCZGREBZmtl/6059MK9r8+dCpk7MjFBERcRq7CrTWrW+iVasbLzifnLySAQMG4uPjQ/PmwbRo0ZL09DTS09No0aIlwcEt8Pb2ZsCAgSQnr8Rms7Fp03r69bsXgNjYIaxatcKe0MRVlZVBUhIdR482e2D+8APMmAEHDkBCgsaYiYiIUEFj0HJysunY8fbyrwMDg8jNzQEgKOjsH+CAgEDS09M4fPgw9erVx+vnWXnnXn8x8+fPZcGCeQDk5maTmprq+A9xjsLCwgq/R1VnsVppkpJCs4ULqZOZiSUwkF0TJpB3113YvLwgPf3KbyKXpJ9Rx1NOHUv5dDzl1LFcLZ9XLNAGDryH/Py8C84//fSz9O7dt0KCupL4+BHEx48AoHfvKLp3716h90tNTa3we1RZVqvZF/OFF2DPHmjbFhYtIq1xY7r36EGIs+OrIvQz6njKqWMpn46nnDqWq+XzigVaUtIH1/ymAQGBZGdnlX+dnZ2Fv38AAFlZP5afz8nJJiAgkIYNG3Ls2FFKS0vx8vI673pxU2Vl8O67Zhbmt9/CbbfBsmVwzz1mPTMX+l+KiIiIq6mQHaSjo/uwbFkSxcXFZGbuIyMjgw4dOtG+fUcyMjLIzNxHSUkJy5YlER3dB4vFQkREN5Yvfw+AxYsXERPjnNY5sZPNBu+/D+3bm5mZFgu88w588QXce682LRcREbkKdv21XLlyOaGhIaSlbSUu7n4GDzaD/ENC2tC//wC6dLmdBx64j4SEaXh6euLl5cWUKVOJjR1AREQn+vcfQEhIGwD+8pcX+Pe//0FYWChHjhxm6NB4+z+dVK5PPoEuXUwhVlQECxfCjh1mHTMVZiIiIlfNrkkCffv2o2/ffhd9bezYJxk79skLzvfsGU3PntEXnA8ObsHq1an2hCPOsmsXPP00vPeeWfX/tddg5EhtxSQiIvIrqVlDfr2DB+E4uVSlAAAO/UlEQVR3vzMD/z/+GCZPNhMBHnlExZmIiIgd9FdUrl1JCcyZYwqykyfh0UfNXpmNGzs7MhERkSpBBZpcPZsNli+H8eNNS1mfPjB9OoRosQwRERFHUhenXJ1du6BXL+jf33RfrlpltmpScSYiIuJwKtDk8k6cgEmTzDpmaWkwe7aZmRkT4+zIREREqix1ccrFnVnP7IknYP9+eOghePllaNLE2ZGJiIhUeSrQ5EL79sHvf2+6MNu2hQ0boGtXZ0clIiJSbaiLU86yWmHWLLjlFrMV0/TpZhNzFWciIiKVSi1oYuzYAQ8/DNu2mdmZ//oXNGvm7KhERESqJbWgVXenTsEzz0DHjqZrc9EiWLFCxZmIiIgTqQWtOtu2zQz+//ZbGD4cpk2D6693dlQiIiLVnlrQqqPiYvjznyE8HI4dg+RkmDtXxZmIiIiLUAtadfPFF6bVbOdO02o2cyY0aODsqEREROQcakGrLqxW+NvfICwM8vPNlk1z56o4ExERcUFqQasO9u2DBx+EzZshNhZefRUaNnR2VCIiInIJKtCquoUL4bHHzPMFC2DoULBYnBuTiIiIXJa6OKuqggKIizMtZ7feCl9+aZ6rOBMREXF5KtCqoq1boX17WLIEJk82uwIEBzs7KhEREblKdhVozz//Z377245ERobz0ENxHD1aUP7a7NnTCQsLJTy8Aykpa8vPp6R8RHh4B8LCQpkzZ0b5+czMfcTERBEWFsqoUcMpKSmxJ7TqyWaDGTMgIgLKymDjRrOchpd6skVERNyJXQVaZGQUGzZsYf36T/nNb1oxe7YpuHbv3sWyZUls3LiVxMSlTJw4DqvVitVqZeLE8SxalMSmTdtYuvRddu/eBcDkyc8xevTjbN36JfXrN2Dhwvn2f7rq5PBh6N8fxo+Hvn1h+3azzpmIiIi4HbsKtKioHnj93DrTsePtZGdnAZCcvJIBAwbi4+ND8+bBtGjRkvT0NNLT02jRoiXBwS3w9vZmwICBJCevxGazsWnTevr1uxeA2NghrFq1ws6PVo18+im0a2cWnJ01C5YtA19fZ0clIiIiv5LDxqAtWrSAHj3uAiAnJ5vAwKDy1wIDg8jNzSE3N4egoKbl5wMCAsnJyebw4cPUq1e/vNg7c71cgc0Gr7wC3bqZbsxPPoEnntBEABERETd3xcFJAwfeQ35+3gXnn376WXr37gvAzJlT8fT0YtCgWMdHeBHz589lwYJ5AOTmZpOamlqh9yssLKzwe1wrj6IibpoxA7+1azn029+y6+mnKS0sNBMC3IAr5tSdKZ+Op5w6lvLpeMqpY7laPq9YoCUlfXDZ1xMTF7JmTTJJScux/NxyExAQWN7dCZCdnYW/fwAAWVk/lp/PyckmICCQhg0bcuzYUUpLS/Hy8jrv+ouJjx9BfPwIAHr3jqJ79+5X+hh2SU1NrfB7XJM9e+C+++Crr+Bvf6PR00/TxcO9JuS6XE7dnPLpeMqpYymfjqecOpar5dOuv+opKR/xj3/MYsGCxdSuXbv8fHR0H5YtS6K4uJjMzH1kZGTQoUMn2rfvSEZGBpmZ+ygpKWHZsiSio/tgsViIiOjG8uXvAbB48SJiYvra98mqquXLoVMnyMqCVavgmWfAzYozERERuTy71l946qk/UVJSwv339wfMRIFp02YREtKG/v0H0KXL7Xh5eZGQMA1PT08ApkyZSmzsAKxWK3FxwwgJaQPAX/7yAqNHj+Dvf5/MrbeGMnRovJ0frYqx2eCll8yyGR07wrvvam0zERGRKsquAm3r1i8v+drYsU8yduyTF5zv2TOanj2jLzgfHNyC1atT7Qmn6jp5EkaOhMWLzVZN//kP1Krl7KhERESkgmgFU1e3fz/ce69Z1ywhAZ58UrM0RUREqjgVaK5s82YzGeDUKVixAvr0cXZEIiIiUgk0utxV/e9/EBUF9evDli0qzkRERKoRFWiuxmaD55+HYcPMnpqffQYhIc6OSkRERCqRujhdSXExPPywaT0bPhxeew28vZ0dlYiIiFQyFWiu4qefYMAA2LgRXnwRnn5akwFERESqKRVormDvXoiJgQMHYNEieOABZ0ckIiIiTqQCzdk+/9xMACgthY8/NuPOREREpFrTJAFnWrMGuneHmjXNkhoqzkRERAQVaM7zv/9B377QsiV8+qlmaoqIiEg5FWiVzWaDqVPNMhpdu8KGDRAY6OyoRERExIWoQKtMNhs89RRMmACDB8OqVWYhWhEREZFzqECrLFYr/O538PLL8NhjZramj4+zoxIREREXpAKtMpw+DQ8+CK+/DpMmwT/+AR5KvYiIiFycltmoaCdPwv33w4cfQkKC6d4UERERuQwVaBXp2DHo18/sDvDaa/DII86OSERERNyACrSKUlBgdgf4/HN4+23tDiAiIiJXTQVaRTh8GHr1gh074J134N57nR2RiIiIuBG7RqpPmTKZyMhwoqIiuP/+/uTm5gBgs9mYNOlJwsJCiYwMZ8eO7eXfk5i4kM6d29G5czsSExeWn//yyy+IjLyDsLBQJk16EpvNZk9oznPoEPToATt3wrJlKs5ERETkmtlVoD3++BOsX/8p69ZtplevGKZNSwDg44/XkJGxly1btjN9+mwmTBgLwJEjh5k2LYHk5BRWr17HtGkJFBQcAWDChLFMnz6HLVu2k5Gxl5SUj+z8aE5w8CBERcGuXfDBB2anABEREZFrZFeBVrduvfLnJ0+exGKxALBq1YcMHjwEi8VCp05hHD16lLy8XNat+5jIyCh8fRvSoIEvkZFRpKSsJS8vl+PHj9OpUxgWi4XBg4fw4Ycr7ftklS031+yrmZEBK1dCdLSzIxIRERE3ZfcYtJde+itLliyiXr16LF1qiqrc3GwCA5uWXxMYGEROTjY5OTkEBQWdcz6QnJwccnKyCQgIOu/63Nxse0OrPHl5cOedsH+/2R2gWzdnRyQiIiJu7IoF2sCB95Cfn3fB+aeffpbevfsyadKzTJr0LLNnT+fNN19j4sRnKiTQc82fP5cFC+YB8P3339GuXYcKvd9PPx3i+usbXf4i75rQqjWM+WOFxlJVXFVO5aopn46nnDqW8ul4yqljVVY+DxzIvKrrrligJSV9cFVvNHDgYOLiBjFx4jP4+weSnf1j+WvZ2VkEBAQSEBDA5s2bzjmfTUREFwICAsnJyTrven//S28gHh8/gvj4EVcVlyPcdVckH320vtLuVx0op46lfDqecupYyqfjKaeO5Wr5tGsMWkbGnvLnyckradWqNQAxMb1ZsmQRNpuNtLSt1KtXDz8/f6KierB+fQoFBUcoKDjC+vUpREX1wM/Pn7p165KWthWbzcaSJYvo3buPfZ9MRERExE3ZNQZt8uTn2bv3eywWD2644QamTp0FQM+e0axdu4awsFBq167N7NmvAuDr25Bx4ybQq1d3AMaPn4ivb0MAEhJmMGbMoxQVFdGjx1306NHLntBERERE3JZdBdrcuf+76HmLxUJCwoyLvhYXN4y4uGEXnG/XrgMbNmyxJ5wKM2zYcGeHUOUop46lfDqecupYyqfjKaeO5Wr5tBw8eMxNV4QVERERqZrsGoMmIiIiIo6nAu0yPvhgGV27huHnV5/t29PLz+/fn0mzZk2IioogKiqCP/1JS2tcjUvlE2D27OmEhYUSHt6BlJS1TorQvb388kvcdttN5T+Xa9eudnZIbikl5SPCwzsQFhbKnDkXH6oh16Zjx7ZERt5BVFQEd90V6exw3NITTzzGzTe3pFu3zuXnjhw5zKBB/encuR2DBvUv35lHruxi+XS136Eq0C4jJORm5s5dSHh4xAWvBQe3YN26zaxbt5lp02Y5ITr3c6l87t69i2XLkti4cSuJiUuZOHEcVqvVSVG6t9GjHy//uezZU7tZXCur1crEieNZtCiJTZu2sXTpu+zevcvZYVUJS5euZN26zS61jIE7eeCBoSQmLj3v3Jw5M+nWLZItW7bTrVskc+bMdFJ07udi+QTX+h2qAu0yWre+iVatbnR2GFXGpfKZnLySAQMG4uPjQ/PmwbRo0ZL09DQnRCjVXXp6Gi1atCQ4uAXe3t4MGDCQ5GQ323ZOqqTw8AgaNPA971xy8kpiY+MAiI2NY9WqFc4IzS1dLJ+uRgXar7R/fyZ33tmF/v1789lnnzg7HLeWk5NNYOAvt/rKcWJE7uutt14nMjKcJ554TN0dv0Jubg5BQWe3qTOLaLvRtnMuyuyxfC89e3Zj/vy5zg6nysjPz8fPzx+AJk38yM/Pd3JE7s+VfofavRenu7vSVlYX4+fnT3r61zRseD1ffvkFDz0Ux8aNW87bPL66+jX5lKt3ufwOH/4w48dPxGKxMGXK33juuWfK1yAUcably1cTEBBIfn4+99/fnxtvbH3RoSPy61ksFiwWi7PDcGuu9ju02hdoV7uV1bl8fHzw8fEBIDS0PcHBLdi7d0+F7wnqDn5NPgMCAsnO/uVWXwGODKvKuNr8PvjgQzz44OAKjqbq8fcPICvr7DZ1OTnZBARcets5uTpncti4cWP69Lmb9PTPVaA5QOPGjcnLy8XPz5+8vFwaNdK+nPZo0qRJ+XNX+B2qLs5f4dChQ+WD2Pft+4GMjL00bx7s3KDcWHR0H5YtS6K4uJjMzH1kZGTQoUMnZ4fldvLycsuff/jhckJC2jgxGvfUvn1HMjIyyMzcR0lJCcuWJREdrW3n7HHixAkKC4+XP09NTaFNG/1sOkJ0dB8WL34bgMWL3yYmRr0U9nC136HVvgXtclauXM6kSU/y00+HiIu7n7Ztb2XJkvf49NPNvPzyi3h51cDDw4OpU2eVb1kll3apfIaEtKF//wF06XI7Xl5eJCRMw9PT09nhup0XXvgLX3+9E7DQrFkzpk2b7eyQ3I6XlxdTpkwlNnYAVquVuLhhTv8l7e7y8w8yfPhQAKzWUu67737uvPMuJ0flfkaPHsHmzZs4fPgnQkNDmDBhEmPGjGXUqOEsXDifpk2b8cYb85wdptu4WD43b97oUr9DtZOAiIiIiItRF6eIiIiIi1GBJiIiIuJiVKCJiIiIuBgVaCIiIiIuRgWaiIiIiItRgSYiIiLiYlSgiYiIiLgYFWgiIiIiLub/AZtOm6T32V1vAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff8bcdcee10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = np.linspace(-15,15,100)\n",
    "y = x**3\n",
    "\n",
    "fig, ax = plt.subplots(1, 1)\n",
    "fig.set_facecolor('#EAEAF2')\n",
    "plt.plot(x,y, 'r-')\n",
    "plt.plot([0],[0],'o')\n",
    "plt.ylim([-3000,3000])\n",
    "plt.gcf().set_size_inches((10,3))\n",
    "plt.grid(True)\n",
    "ax.text(0,\n",
    "        400,\n",
    "        'Saddle Point',\n",
    "        ha='center',\n",
    "        color=sns.xkcd_rgb['pale red'],\n",
    "       )\n",
    "plt.show"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are two problems with numerical derivatives:\n",
    "+ It is approximate.\n",
    "+ It is very slow to evaluate (two function evaluations: $f(x + h) , f(x - h)$ ).\n",
    "\n",
    "Our knowledge from Calculus could help!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We know that we can get an **analytical expression** of the derivative for **some** functions. \n",
    "\n",
    "For example, let's suppose we have a simple quadratic function, $f(x)=x^2−6x+5$, and we want to find the minimum of this function. \n",
    "\n",
    "#### First approach\n",
    "\n",
    "We can solve this analytically using Calculus, by finding the derivate $f'(x)  =  2x-6$ and setting it to zero:\n",
    "\n",
    "\\begin{equation}\n",
    "\\begin{split}\n",
    "2x-6  & = & 0 \\\\\n",
    "2x  & = & 6 \\\\\n",
    "x & = & 3 \\\\\n",
    "\\end{split}\n",
    "\\end{equation} "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function matplotlib.pyplot.show>"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlkAAADGCAYAAADhV5X4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xt8zvX/x/HHtY05E4bNWTmHMcaisaI5hJZTX4eFksq5ol/pm0r1rb6Vpn4qFaIcQ5KGHOZUlK+U+FLCsJM5m7CZ6/fHO5Ofw07Xtc91eN5vt+vWdrm26+XdtXle78/7/Xrbjhw5bUdEREREHMrH6gJEREREPJFCloiIiIgTKGSJiIiIOIFCloiIiIgTKGSJiIiIOIFCloiIiIgTZBuyEhIOExXVhTZtWnDnnaFMnToFgDfeeJXGjesSEdGaiIjWrFq1IutrYmLeIjS0CWFhzVizZpXzqhcRERFxUbbs+mSlpCSTkpJM48bBpKWdoX37cD79dA5LliyiePESDBs28qrH79mzm6FDB7NixVqSk5Po2bMbmzf/hK+vr1P/IiIiIiKuJNuZrIoVK9G4cTAAJUqUpE6duiQlJd7w8cuXLyMqqgf+/v5Ur16DmjVrsW3bVsdVLCIiIuIGcrUm6+DBeHbs+IWQkOYATJs2lbZtwxg16nFOnjwBQFJSIkFBlbO+JiioMsnJSQ4sWURERMT1+eX0gWlpaQwePICJE1+jZMlSDBz4ME8++TQ2m43XXnuZCRPGExMzJcdPPHPmdGbNmgHAb7/tpmrVqrkuPrcuXbqEj4/W+juKxtPxNKaOpfF0PI2pY3n9eNrtFDt8GN/z5/mzalUyixTJ97csiDE9evQou3cfyPZxOQpZGRkZDB7cnx49enPvvd0AqFChQtaf9+//IP379wYgMDCIxMSErD9LTEygUqXAa75ndPQgoqMHAdCpUwRbtzr/kmJcXBzt2rVz+vN4C42n42lMHUvj6XgaU8fy6vG02yE6Gn7/HebNg969HfJtC2JMg4Ob5ehx2UY9u93O6NHDqFOnLo89Njzr/pSU5KyPv/lmKfXq1QcgMrIzixcv5MKFC8THH2Dfvn00a9Y8t/WLiIiIJ3vlFfjsM3jpJYcFLFeT7UzWli2bWbBgLvXrNyQiojUA48c/z6JFX7Bz5w7ARrVq1XjzzRgA6tWrT/fuUbRp0wI/Pz9ef/1N7SwUERGRK2bPhn/+E/r3h+ees7oap8k2ZLVqFcaRI6evub99+8gbfs2YMWMZM2Zs/ioTERERz7NhAwwaBG3bwscfg81mdUVO48Wr7URERKRA/fYb3Hcf1KwJixaBv7/VFTmVQpaIiIg439Gj0KUL+PjAsmVQtqzVFTldjls4iIiIiOTJ+fPQvTscOgRr18Ktt1pdUYFQyBIRERHnuXQJBg6E776D+fMhLMzqigqMLheKiIiI8zz9tOmD9cYb0KuX1dUUKIUsERERcY733oM334Rhw+Cpp6yupsApZImIiIjjffkljBxp1mLFxHh0q4YbUcgSERERx9q8Gf7xDwgNNY1HvbQpuUKWiIiIOM7evdC1K1SuDEuXQrFiVldkGYUsERERcYyUFIj860SY2FgICLC2HouphYOIiIjkX1qaaTaanAxr1kDt2lZXZDmFLBEREcmf9HTo0QO2b4clS6BlS6srcgkKWSIiIpJ3djs8/DCsXAnTppnZLAG0JktERETy45lnYNYsePllGDTI6mpcikKWiIiI5M3kyfD66/D44/Dss1ZX43IUskRERCT3Zs+GUaPg/vtN2PLCZqPZUcgSERGR3Fm+HB58ENq1g88/99pmo9lRyBIREZGc27zZ7CRs1MjsJCxSxOqKXJZCloiIiOTMrl1m92BgoGk2WqqU1RW5NIUsERERyd7Bg3DPPVC4sGnXULGi1RW5PPXJEhERkZtLTTUBKy0N1q+HWrWsrsgtKGSJiIjIjZ06Zc4jPHjQzGA1bmx1RW5DIUtERESu788/oWtX2LEDvvoK2rSxuiK3ku2arISEw0RFdaFNmxbceWcoU6dOAeDEieP07Nmdli2D6dmzOydPngDAbrfz7LNjCQ1tQtu2Yfzyy3bn/g1ERETE8dLToWdP2LgRPvsMOnWyuiK3k23I8vPz48UXX2Hjxh+JjV3NtGkfsWfPbiZPnkR4eFu2bNlOeHhbJk+eBMDq1SvZt+8PtmzZzltvxTBu3Bin/yVERETEgTIzITra7CD88EPo08fqitxStiGrYsVKNG4cDECJEiWpU6cuSUmJLF++jD59+gLQp09fYmO/BiA29ht69/4HNpuN5s1DOXXqFCkpyU78K4iIiIjD2O0wbBjMmwdvvAFDhlhdkdvK1Zqsgwfj2bHjF0JCmpOamkrFipUAqFChIqmpqQAkJycSFFQl62uCgiqTlJSY9djLZs6czqxZM7K+Ji4uLh9/jZxJS0srkOfxFhpPx9OYOpbG0/E0po7lcuNpt3Pr++9TdcEC4vv2ZX+LFuBK9eWAK41pjkNWWloagwcPYOLE1yhZ8urmYzabDVsuzyyKjh5EdLQ5rbtTpwjatWuXq6/Pi7i4uAJ5Hm+h8XQ8jaljaTwdT2PqWC43ni+8AAsWwPDhVJ88mepueB6hK41pjpqRZmRkMHhwf3r06M2993YDICAgIOsyYEpKMuXLlwegUqUgEhMPZ31tYmICgYFBjq5bREREHOnf/4YXX4RBgyAmRgc+O0C2IctutzN69DDq1KnLY48Nz7o/MrIz8+bNBmDevNl07NgFgI4dOzF//hzsdjtbt/5AqVKlrrlUKCIiIi5kyhQYN84scP/oI/DRgTCOkO3lwi1bNrNgwVzq129IRERrAMaPf56RI8cwZMhAPv98JlWqVOPjj2cA0L59JKtWrSQ0tAnFihUjJmaKU/8CIiIikg+ffmoWunftCrNmga+v1RV5jGxDVqtWYRw5cvq6f7Zw4dJr7rPZbLz++tv5r0xERESca948GDwY2reH+fOhUCGrK/Iomg8UERHxRosWQb9+0Lo1fPklFClidUUeRyFLRETE2yxdatZfhYbCsmVQvLjVFXkkhSwRERFvEhtrjstp2tR8XLKk1RV5LIUsERERb7FqFURFQcOGsGIFlC5tdUUeTSFLRETEG6xdC926QZ06sHIl3HKL1RV5PIUsERERTxcXB126QM2aZjbrrwbi4lwKWSIiIp5s3borAWvNGqhQweqKvIZCloiIiKdatw46d4YaNUzAqljR6oq8ikKWiIiIJ7ocsKpXV8CyiEKWiIiIp1HAcgkKWSIiIp5kzRro1OlKwKpUyeqKvJZCloiIiKdYudIscr/1VrOjUAHLUgpZIiIinuCbb6BrV6hb1/TE0i5CyylkiYiIuLuvvoL77oPbbzeXCNUHyyV4T8iy262uQERExPEWLYIePSA42DQaLVvW6orkL94RshISoHVriv/xh9WViIiIOM7nn0Pv3tCiBXz7rY7KcTHeEbLOnoVDhwgeMwZ++MHqakRERPLvo49gwAAIDzcL3nXYs8vxjpBVpw5s2MDFkiXh7rtN/xARERF3FRMDjzxiWjUsWwYlSlhdkVyHd4QsgBo1+CkmBqpWhY4dYflyqysSERHJvVdfhdGj4f77YfFiKFrU6orkBrwnZAHp5cubWax69aBbN/PiFBERcQd2O4wfb279+sG8eVC4sNVVyU14VcgCICDAbG8NCYFeveCzz6yuSERE5OYuXYLhw80s1pAh8Omn4OdndVWSDe8LWWB2X6xcaRYLDhgA771ndUUiIiLXl5EB0dEwZQqMHQsffgi+vlZXJTmQbcgaNepxGjSoRXh4y6z73njjVRo3rktERGsiIlqzatWKrD+LiXmL0NAmhIU1Y82aVc6p2hFKljTdcbt3hxEjYOJE9dISERHXcv686YH1+edmFuv118Fms7oqyaFs5xofeKAfDz30CMOHD73q/qFDhzFs2Mir7tuzZzeLFy9kw4YfSE5OomfPbmze/BO+rpq4ixSBL76Ahx6C55+H48fhrbfAxzsn+ERExIWcOWPWD69bZ2axHnvM6ookl7JNE2FhrSlTJmfNzZYvX0ZUVA/8/f2pXr0GNWvWYtu2rfku0qn8/GD6dBg1Ct55xwSuixetrkpERLxZaircdRds2GDWDitguaU8r5qbNm0q8+fPITi4KS+++AplytxCUlIiISEtsh4TFFSZ5OSk6379zJnTmTVrBgDJyYnExcXltZQcS0tLu/HzdO9O9dOnqTl9Okd//51d//wnl/z9nV6TO7vpeEqeaEwdS+PpeBpTx7reeBZJTqbxuHH4HznCrokTORYUBBrzHHOl12ieQtbAgQ/z5JNPY7PZeO21l5kwYTwxMVNy9T2iowcRHT0IgE6dImjXrl1eSsmVuLi4mz9PRASEhFB+xAjCX37ZHLipIwpuKNvxlFzTmDqWxtPxNKaOdc14/vqrac/w55+wejWNWre2rDZ35Uqv0TwtPqpQoQK+vr74+PjQv/+D/PTTfwAIDAwiMTEh63GJiQlUqhTomEoLyrBhMHcubNlidh8mJlpdkYiIeINNm+DOO83HGzaAApbby1PISklJzvr4m2+WUq9efQAiIzuzePFCLly4QHz8Afbt20ezZs0dU2lB6t0bYmPhwAG44w747TerKxIREU+2bBl06AAVKpiwdfvtVlckDpDt5cKhQwexadNGjh8/RpMm9Rg37lk2bdrAzp07ABvVqlXjzTdjAKhXrz7du0fRpk0L/Pz8eP31N113Z2F27r7bXAPv1Mm8m1i2DEJDra5KREQ8zSefwNChEBxs3uAHBFhdkThItiHrww+nX3Nfv37RN3z8mDFjGTNmbP6qchUhIeYdRWSkWa81fz506WJ1VSIi4gnsdqp/+inMmGH+nVmwwPRwFI+hhlDZqV0bvvsO6tc3jUs//tjqikRExN1dvAhDh1Jzxgx48EFYulQBywMpZOVEpUrm0mGHDubMqAkT1B1eRETy5uxZiIqCjz4ivn9/06uxUCGrqxIn0OmSOVWihGnpMHQovPQSHDpkzo/SD4aIiOTUkSOmi/uPP8KUKeyvX5/qOibHY2kmKzcKFTILFCdMMO88unaF06etrkpERNzBnj3QqhX88gssXKgu7l5AISu3bDZ44QWzNmvVKtPT5PBhq6sSERFXtn49hIVBWhqsXQv33Wd1RVIAFLLy6qGH4JtvYP9+aNkSfvrJ6opERMQVzZ59pQfW5s3m3wzxCgpZ+XHPPabFg6+vmdH65hurKxIREVdht8Orr5pjcsLCzE71WrWsrkoKkEJWfjVqZN6Z1K1r1mhNyd0ZjiIi4oHS02HwYBg/3oSsFSugbFmrq5ICppDlCEFBsG6daVQ6bBiMGmV6oIiIiPc5etRcHpwxw6zhnTUL/P2trkosoJDlKCVKwOLF8MQTMHmymdU6dcrqqkREpCDt3m12EG7ZYtZiTZhgNkyJV1LIciRfX3jrLZg61ew8vOMOszBeREQ83+rVZu3VmTNmB+E//mF1RWIxhSxnGDIEVq6EpCRzqPTGjVZXJCIizvTBB9CxI1SpYmaxwsKsrkhcgEKWs0REmB+0smXhrrtg2jSrKxIREUfLyIDHHzeNRS/vOK9Rw+qqxEUoZDlT7dpm52G7dqav1ujRWhAvIuIpjh41wer992HcOHP0WqlSVlclLkQhy9luucX0zxozBmJioFMnOH7c6qpERCQ/fv3VLAf5/nuYORNef92syxX5G4WsguDnB2+/bS4Zrl9vfjB37bK6KhERyYslS8yaq/PnTfueAQOsrkhclEJWQRo0yOw4SUszW3y//NLqikREJKcuXTItGe67D+rVgx9/1BE5clMKWQXtjjvMD2a9ehAVBc8/b35wRUTEdZ06ZcLVSy/BwIGwYQNUrmx1VeLiFLKsULWquWw4aBBMnGgal548aXVVIiJyPbt3mxmr2Fh4912z9KNIEaurEjegkGWVIkXgk0/MWYcrV0KLFrBzp9VViYjI3y1ZYtbRHj9umo0OH64O7pJjCllWstlMb5XL67RatoT5862uSkRELl6E//kfc4mwbl34z38gPNzqqsTNKGS5gjZtzA9wkybQp4/pp5WebnVVIiLe6cgRiIw0bRmGDjWndlStanVV4oayDVmjRj1Ogwa1CA+/soPixInj9OzZnZYtg+nZszsnT54AwG638+yzYwkNbULbtmH88st251XuaYKCIC7OBKyYGNMxPiHB6qpERLzL5s3QrBl89x1Mn26Oy/H3t7oqcVPZhqwHHujH3LmLrrpv8uRJhIe3ZcuW7YSHt2Xy5EkArF69kn37/mDLlu289VYM48aNcU7VnqpQIZg0CebNg59/Nj/oa9ZYXZWIiOez2+G998wlwcKFTZPRgQOtrkrcXLYhKyysNWXK3HLVfcuXL6NPn74A9OnTl9jYrwGIjf2G3r3/gc1mo3nzUE6dOkVKSrITyvZwvXubNg/lykGHDvDqq2rzICLiLKdPm6UaI0aYy4T/+Q8EB1tdlXiAPK3JSk1NpWLFSgBUqFCR1NRUAJKTEwkKqpL1uKCgyiQlJTqgTC9Uvz788IP5wR8/Hjp3hr/GWUREHGT7dggJgUWLzBqsJUvMcWgiDuCX329gs9mw5WE768yZ05k1awZgwllcXFx+S8lWWlpagTyPQw0ZQmBgILXffZeMBg3Y9dxznGrSxOqqADcdTxenMXUsjafjecyY2u0Efv21+d1aujS7Jk3iVKNGpodhAfKY8XQhrjSmeQpZAQEBpKQkU7FiJVJSkilfvjwAlSoFkZh4OOtxiYkJBAYGXfd7REcPIjp6EACdOkXQrl27vJSSK3FxcQXyPA4XEQEDBuDfuzdNn3gCXn4Znn4afKzdHOq24+nCNKaOpfF0PI8Y09OnTfuc2bPhnnvw/+wzmgYEWFKKR4yni3GlMc3Tv9KRkZ2ZN282APPmzaZjxy4AdOzYifnz52C329m69QdKlSqVdVlR8ik4GLZuhV694NlnoVMnSNZ6NxGRXNm61WwqmjvXnLgRGwsWBSzxfNmGrKFDB9G5c3v27v2dJk3q8fnnMxk5cgzr1q2lZctg1q2LY+RIs4uwfftIqlevQWhoE558ciSvv/620/8CXqVUKZgzx2wpXr/e9NVascLqqkREXN+lS/D22+b82PR0WLcOnnvO8isC4tmyvVz44YfTr3v/woVLr7nPZrMpWDmbzWaa47VpYxbFd+wITz0Fr7xith2LiMjVjhwx7RhiY00H908+gbJlra5KvIAivLtq2NC0eXjsMXjzTWjdGvbutboqERHXsnKlmfVfs8b0wVq0SAFLCoxCljsrWtQcML1woQlYwcHmdHi73erKRESsdf48jBlj+l6VLQtbtsCwYTrcWQqUQpYnuP9++OUXc1L8Qw9Bz55w7JjVVYmIWOPXX83vw3fegeHDzWJ3F2l9I95FIctTVK0Kq1bBG2/A0qXQqJGZJhcR8RaXLsG770Lz5pCSAsuWmc+LFrW6MvFSClmexMcHxo41neLLlDHT5KNGwblzVlcmIuJchw6Z33kjR0L79rBjhzkpQ8RCClmeKDjYnL01YgRMngxNm5rgJSLiaex2+OwzM3v//ffw4YdmNr9CBasrE1HI8lhFi5qA9e23cPas6Q3zz3+a/jAiIp4gNdU0aB4wAG6/HX7+GR55RIvbxWUoZHm6y9Pm/fub43hatTKLQkVE3NmXX5rZq6VLzcHO69bBrbdaXZXIVRSyvEGZMjBjBixeDAkJ5kiJV16BjAyrKxMRyZ1jx6BvX4iKgsBA0y9w3Djw9bW6MpFrKGR5k/vuM7NYUVHmOImWLc30uoiIO1i8GBo0gAUL4MUXzVrTxo2trkrkhhSyvE1AAMybZxqYJiSYrc4vvKC1WiLiuo4eNbNX998PlSubvlfPPw+FClldmchNKWR5q/vvh127zPmHL74ILVqYaXcREVdht8Ps2VC/PnzxBbz0kuncrsai4iYUsrxZuXJm6/OSJeadYqtW5hiKtDSrKxMRbxcfD126QL9+ZkH7tm1mh7Rmr8SNKGQJdOtmZrWGDjXHUNx+OyxfbnVVIuKNMjNNl/aGDWH9eoiJgU2bzO8lETejkCVG6dLmsOkNG0yPrU6dTNuHI0esrkxEvMX27aan38iR0KaN2agzcqR2DorbUsiSq7VpY37RPf88zJ8PdevC1KnmTDAREWdIS4MnnzQbcQ4cMMsYYmOhRg2rKxPJF4UsuZa/v1kM//PPZoHp0KHQurXaPYiI4y1ZYtoyvP02PPQQ7N5t1mGpa7t4AIUsubH69WHtWpg5E/74A0JCzLvNM2esrkxE3N2+fWY96H33meUKmzaZcwdvucXqykQcRiFLbs5mM+eC7d4NDz8MkyaZS4iff262V4uI5Ma5czBhgpm9WrMG3njD7By84w6rKxNxOIUsyZmyZeGDD2DzZqhSBfr3J3j0aF1CFJGcsduvXBp86SVz8sSePTB2rNoyiMdSyJLcCQ01QevjjykWH2/OQRw+HI4ft7oyEXFV//0vdO5sLg0WK2ZmsObMMd3bRTyYQpbkno8PPPQQP8yaBY8/Du+/D7Vrw3vv6dBpEbnixAkYPdqcL/j99/DWW2b3ckSE1ZWJFAiFLMmziyVLmqaBP/0EwcEwYoTZjbhihdWliYiVLl688ubr3XfNrsHff4cnntClQfEq+QpZISG307ZtKyIiWtOhQ1sATpw4Ts+e3WnZMpiePbtz8uQJhxQqLqxxY1i1Cr780hw03bGjOQ7jv/+1ujIRKUh2u+lv1bSpmeVu1Mgsav/gA3M4vYiXyfdM1qJFy1i7dhPffrsOgMmTJxEe3pYtW7YTHt6WyZMn5btIcQM2G3TvDjt3wptvwsaN5hfso49CcrLV1YmIs23fDvfcY9ZenTtnDnRes0aHOYtXc/jlwuXLl9GnT18A+vTpS2zs145+CnFl/v6ml9bevTBsGHzyCdx2m2luqoOnRTzP4cMwcKDZBLNtmzlrcNcu6NFDDUXF69mOHDmd52ZHzZs3onTpMthsNqKjBxEdPYjbbqvK3r2HALDb7dSuXS3r87+bOXM6s2bNACA5OZG5c+fmtYwcS0tLo0SJEk5/Hm+Rk/EsmpBAzY8+osK6daTfcgsHBg4kqXNn7H5+BVSle9Fr1LE0no53eUz9Tp2i2pw5VF68GJvdzuEePTjYrx8XNd65oteo4xXEmI4e/UTWFbybyVfISkpKJDAwiNTUVHr16s6//vVvBgx44KpQVbt2NX7//eBNv0+nThFs3bo1r2XkWFxcHO3atXP683iLXI3n5s2mH87GjVCrlumT88ADOvj1/9Fr1LE0no63ITaWO7dtM01Ez5yB6Gh44QWdM5hHeo06XkGMaXBwsxyFrHxdLgwMDAIgICCAzp3vZdu2/xAQEEBKilmDk5KSTPny5fPzFOIpWrWC9eth2TIoWRL69zc7Er/6Sp3jRdzBhQvw3nu07NcPnnvOtGH45ReYMUMBS+QG8hyyzp49S1ramayP4+LWUL9+fSIjOzNv3mwA5s2bTceOXRxTqbg/m80sit22zTQiPH/eLJYPCzNtHxS2RFxPero5U7B2bRgxgj+rVoXvvjO7iW+/3erqRFxankNWauoR7r03knbt7qBjxwg6dIjkrrs6MHLkGNatW0vLlsGsWxfHyJFjHFmveAIfH3OpcNcumDoVkpJM24fWrWHlSoUtEVeQkQHTppmzSh991HRn//Zbtr/zjnljJCLZyvPq4xo1ahIX990195ctW46FC5fmqyjxEoUKwZAhZk3H9OnwyisQGWnC1gsvwN13a3eSSEFLT4dZs+Bf/4I//oDmzU1j0chI8/MYF2d1hSJuQx3fxXr+/uad8t698L//CwcOQIcO5t3y0qWa2RIpCOfPm5+/226Dhx+G0qXNmskffjAzzXrDI5JrClniOvz9TZfovXvNO+eUFOjWzSyQnzcPMjOtrlDE86SlwdtvQ82a5rD3qlVN1/atW6FrV4UrkXxQyBLXU6SImdn67Tf49FNz+eKBB6BBA/joI/OOW0Ty58gR+Oc/oVo100C4QQPToX3jRs1ciTiIQpa4rkKFzHqtX3+FBQtM64dHHjHbxV99FU7oXEyRXPvjDzNjXL26WQfZrh18/z2sXm3aMihciTiMQpa4Pl9f6NkTfvzRvNNu2hTGjzeXNUaPhn37rK5QxLXZ7bBpk/k5qlPHHHfVv785xH3RItPHTkQcTiFL3IfNZt5px8bCzz/D/fdfWagbFQXr1mmRvMjfZWTA7NkQGgpt2pg3KWPHwv795tJ73bpWVyji0RSyxD01bgwzZ5qdiM88Axs2mMsezZqZDtRatyXe7MgRcymwZk3o1w9On4YpU+DQIXjtNQgKsrpCEa+gkCXurXJl84/JoUPmnXlGBgwaBFWqwLhxupQo3sNuN2ur+vUzr//nnoN69eDrr81lwcceg+LFra5SxKsoZIlnKFrU9PbZsePKAt633zaXEjt3Nv/QqAWEeKIzZ8wbjJAQuOMO81p/9FETrFatgi5dzCkLIlLg9JMnnsVmg7vuMrsR4+NhwgSzfqtrV7MrccIEc4lRxJ3Z7WYjyCOPmEt/jzxiZnHffx8SEmDyZDOLJSKWUsgSz1W58pVQtXChOcx24kSoVcscEbJgAVy4YHWVIjl37Bi8955ZexgaCp9/Dr16mQObf/nFzGCVKGF1lSLyF4Us8XyFCpmdiLGxJnBNmGAupfTubWYBhg2DLVu0M1FcU0aGOV6qRw8IDIQRI8yM7fvvQ2KiOcQ5LEz9rURckEKWeJdq1UzI2r/fhK7ISPOPVKtWUL++aXIaH291lU73W69uJE1+K+tze2YmfwzuT8K/XgIg7cctHF/8hVXlXePc778R/9Qo4p8axYGnRnJmy/dWl+Rcly8HjhljFrF362Y6sQ8fDtu3w7ZtZtaqdGmrKxWRm/CzugARS/j6mqNDOnaEU6fgiy9MS4jx483tjjvMUT69e0PFilZX63A2/yKkHzzIpQsX8PH3589ftuNXtlzWn5do0RJatLSwwqv5V6tOtdffxubry8UTx4l/ahQlmodi8/W1ujTH2r0b5swxva327oXCheHhBtznAAAOJUlEQVTee2HgQPNaLVTI6gpFJBc0kyVSujQ89JBpZrpvn5nNOnMGRo40lxM7dICPP4bUVKsrdajizUI4u20rAKc3rqdkm/CsPzu1djUpH38AQPJ773Bk2lQOjh/H/mFDOPP9ppt+38yzZ9k/8jHSEw4DkPTOvzm5akW+avXx988KVPb0dPCkK2O//WZec02bmtnUl182R9588ok5JH3hQrNxQwFLxO1oJkvk72rWNM1Nn3kGdu6EuXPNzMKQITB0KLRta9Z3RUWZhfVurGTrOzm2YB7FQ1qQHn+A0ne159x/d173sRdPHKfqxNdITzhM4usvUzKsNQDxT42i+psxVz3Wt3hxKjz0CMn/G0OZLl3JTEujTPvIa77nkRkfc+7XHdetq2xUz2vuP/f7HlKmTCYjNZVKI8a47yyW3Q67dpnw9MUXpu0ImEvW77xjZk8DA62tUUQcQiFL5EYaNjS7EV96ybSBWLjQ3EaMMLdWrcwMQ9euZueimy089q9ek4zUI5zZtJ7izUJu+tgSoa2w+fjgX7UaF0+dyrr//wesy4o3aUra95s48vGHN3xMhYEP56reorXrUmPS/3Lh8CGS33uH4k1D8ClcOFffwzIXL5qzA7/6ytz27jWvlzZtICbGBPcqVayuUkQcTCFLJDs2GwQHm9vEiWZn4sKF8OWXV9Zw1ahhwta990J4OBQpYnXVOVKieSipM6dT9YVXyEw7c8PH2fz+dqkqB7sw7ZcukZ5wGB9/fy6lpUG58tc8JrczWZf5V6mKT5EipB+Kp8ittbOtxTLHjsG338I338CyZXD8uFljdddd8MQTcN99mrES8XAKWSK5Vb++ObLkuefMFvqvvzZb7D/6CN5913Sfb9fO7Fzs2BHq1HHZWa5Sd7XHp3hx/KvX4M+d1waevDqxbAmFK1ehXN8BJE+ZTLVX3sDmd/Wvm9zMZGWkJONXPgCbry8ZqUdIT0ygUICLbUjIzIStW2H5crNz9YcfTCAtV86E727d4J57oGRJqysVkQKikCWSH5e7bT/yCPz5J6xdCytWmNvo0eYx1aub2Yu77jLH/bjQWq5C5cpzS+euef76663JSk84zOnV31LtX2/iU7QYRes35NjC+ZTv0zfPz3Nu9385/uVEbL5+4GOj4sOP4luqVJ6/n0PY7WZWc/Vqc4uLMztVbTbTKPT556FTJ2je3OxmFRGvo5Al4ijFiplz4rp0MZ/v32/C1sqV5tLi9Onm/jp1TNgKDzdrcqpVK/BSa382/5r7ijVsRLGGjQAoHXE3pSPuBqDS8NE3/NrrrbcqXLkKNd6ZkvV5hYEP5bveUm0jKNU2It/fJ18uXYJff4UNG8xt/XpISjJ/VqMG9OwJd99tdqOWv/byqIh4H4UsEWepWdM0jHz0UfMP9M8/m5muNWtMH6QPPzSPq1rVhK3WrSnh52d6dLnLgm5PduaMaQi6ZYs5tmbTJjhxwvxZ5crmkvBdd5lgVbOmpaWKiGtyWshas+Zbxo9/mszMTPr3f5CRI59w1lOJuD4fH9MHqWlTs+g5M9Ns3d+40dzWrYM5c2gOMGrUlbPpWrSAkBCoXVuXnJzp3Dnz/+Onn0yw2rzZtFm4vMi/Th2zAzA8HO6808xcueg6OxFxHU4JWZmZmTz99JMsWLCEoKDK3HNPOyIjO1O3rk6FFwFMYLq8Y3H4cPOPeXw8O6dPp+HZs2bR9Ecfme39YC5FNm5sHt+0KTRqBA0a6FiV3LLbITnZXPb79VdzRM1PP5lAlZlpHlO2LLRsaQ5ebtnShN2yZa2tW0TcklNC1rZtW6lZsxY1apgp9KioHixfvkwhS+RGbDaoUYPUiAhzGQpMb6WdO00I2L7d3ObMgQ8+uPJ1VaqYsNWwIdSrZ2a8atc2C/J9vPhAh4wMOHCAst9/b8Zvzx4TpH799colP4BKlcysYbdu5r/NmpmNCpqlEhEHcErISk5OonLlK431AgOD2PbX8R0ikkN+ftCkibldZrfDgQMmLOzcaYLDzp0meJ07d+VxRYvCbbfBrbea0PD3W7Vqpq2AE0LYlz8l8O8Ve0g8eY6gMkUZG1mX+5o6YTdlRoaZkTp82BzovX+/GZcDB8zH+/fDxYs0vvz4MmVMEO3VyzSOvf1283mFCo6vTUTkL5YtfJ85czqzZs0AIDk5kbi4OKc/Z1paWoE8j7fQeDpejse0ZEnTcb5VK/P5pUv4HzlC0YQEiiUkUPTwYYomJFB0+3b8V6zA7+8BDLjk60t6uXKkly3LhfLlSS9blozSpckoVYqM0qW5+NfHF4sXJ7N4cS4WK8Ylf/+bzvB8l5jBjF/TSb9kPk84eY5xC7az67+7uCPoJufu2e34njuH39mz+Kal4Xf2LH5nzlD41CkKnTpFoZMnKXTyJIVPnqTwsWP4Hz1KoZMnsf2/pqjppUtzvlIlzgcFca5FC85VrcqxcuWgTh0ySpW6tvZdu8xNckU/946l8XQ8VxpTp4SsSpUCSfjrcFiApKREAgODrnpMdPQgoqMHAdCpUwTtLl8icaK4uLgCeR5vofF0PKeMqd1uLpHFx5vbwYP4JCVRJCmJIomJpg3B7t2mI/mlSzf+Pj4+JtwVK2Y62v/95ufH+JBHSfe/eo1Y+iVY9kMqz26dYtY8paebfmLnzl25nT178+ctXNi0RKhQwVwKbdvW7O4LCjL/rVEDqlencIkSFAb+3j1Lr1HH05g6lsbT8VxpTJ0Sspo2DWHfvn3Exx8gMDCIxYsX8sEHnzjjqUQkOzabWbhdtqxZNH8jly7ByZNw9Ki5HTsGp0+b25kzVz4+dw7On7/6lplJYuHrNwdN9C9lgpivrwlMRYuaoFa0qLkVL24W8P/9VqYMBASYcFWypNZIiYhbckrI8vPz47XX/k2fPlFkZmbSt+8A6tWr74ynEhFH8fG5Esbq1Mn1lwe9toaEk+euvf+WYrBqlSMqFBFxK05bk9W+fSTt20c669uLiIsZG1mXZxbt4FxGZtZ9RQv5MjayroVViYhYRx3fRcQhLu8iLJDdhSIibsB25Mhpe/YPc6569WpQtWp1pz/PsWNHKVdOZ4o5isbT8TSmjqXxdDyNqWNpPB2vIMb00KF4du8+kO3jXCJkFZQOHdry7bfrrC7DY2g8HU9j6lgaT8fTmDqWxtPxXGlMvbgltIiIiIjzKGSJiIiIOIHv2LHPvGB1EQWpSZOb9AmSXNN4Op7G1LE0no6nMXUsjafjucqYetWaLBEREZGCosuFIiIiIk7g8SHrq68Wc+edoVSsWJrt27dd9WcxMW8RGtqEsLBmrFmjjtR58cYbr9K4cV0iIloTEdGaVatWWF2SW1qz5lvCwpoRGtqEyZPftrocjxAScjtt27YiIqI1HTq0tboctzRq1OM0aFCL8PCWWfedOHGcnj2707JlMD17dufkyRMWVuherjee+h2adwkJh4mK6kKbNi24885Qpk6dArjWa9TjQ1a9eg2YPv1zwsJaX3X/nj27Wbx4IRs2/MDcuYt4+uknyMzMvMF3kZsZOnQYa9duYu3aTerynweZmZk8/fSTzJmzkI0bf2TRoi/Ys2e31WV5hEWLlrF27SaX2c7tbh54oB9z5y666r7JkycRHt6WLVu2Ex7elsmTJ1lUnfu53niCfofmlZ+fHy+++AobN/5IbOxqpk37iD17drvUa9TjQ1adOnW57bba19y/fPkyoqJ64O/vT/XqNahZsxbbtm21oELxdtu2baVmzVrUqFGTwoULExXVg+XLl1ldlghhYa0pU+aWq+5bvnwZffr0BaBPn77Exn5tRWlu6XrjKXlXsWIlGjcOBqBEiZLUqVOXpKREl3qNenzIupGkpESCgq4c9xEUVJnk5CQLK3Jf06ZNpW3bMEaNelyXDvIgOTmJypWrZH0eGBhEUlKihRV5BpvNRu/e99G+fTgzZ063uhyPkZqaSsWKlQCoUKEiqampFlfk/vQ7NP8OHoxnx45fCAlp7lKvUY84u7BHj26kpqZcc/8zzzxPp05dLKjIs9xsfAcOfJgnn3wam83Ga6+9zIQJ44mJmWJBlSJXW7p0BYGBQaSmptKrV3dq165zzbIByR+bzYbNZrO6DLem36H5l5aWxuDBA5g48TVKlix11Z9Z/Rr1iJC1cOFXuf6awMAgEhMTsj5PTEygUqVAR5blMXI6vv37P0j//r2dXI3nqVQpkISEw1mfJyUlEhgYZGFFnuHyGAYEBNC5871s2/YfhSwHCAgIICUlmYoVK5GSkkz58jp3Lz8qVKiQ9bF+h+ZeRkYGgwf3p0eP3tx7bzfAtV6jXnu5MDKyM4sXL+TChQvExx9g3759NGvW3Oqy3E5KSnLWx998s5R69epbWI17ato0hH379hEff4D09HQWL15IZGRnq8tya2fPniUt7UzWx3Fxa6hfX69NR4iM7My8ebMBmDdvNh076mpBfuh3aN7Z7XZGjx5GnTp1eeyx4Vn3u9Jr1OObkS5btpRnnx3LsWNHKVWqNLff3oj5878EYNKkfzN79iz8/Px4+eXXuPvueyyu1v08/vgQdu7cAdioVq0ab74Zk3UtXHJu1aoVPPfc/5CZmUnfvgMYM2as1SW5tQMH9jNwYD8AMjMvcv/9vTSmeTB06CA2bdrI8ePHCAiowLhxz9KpUxeGDBnI4cOHqFKlGh9/PINbbilrdalu4XrjuWnTBv0OzaPNm7+nW7dI6tdviI+PmTMaP/55mjVr7jKvUY8PWSIiIiJW8NrLhSIiIiLOpJAlIiIi4gQKWSIiIiJOoJAlIiIi4gQKWSIiIiJOoJAlIiIi4gQKWSIiIiJOoJAlIiIi4gT/BzeyF1yGVHTvAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff8bcdf1828>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = np.linspace(-10,20,100)\n",
    "y = x**2 - 6*x + 5\n",
    "\n",
    "fig, ax = plt.subplots(1, 1)\n",
    "fig.set_facecolor('#EAEAF2')\n",
    "plt.plot(x,y, 'r-')\n",
    "plt.plot([3],[3**2 - 6*3 + 5],'o')\n",
    "plt.ylim([-10,250])\n",
    "plt.gcf().set_size_inches((10,3))\n",
    "plt.grid(True)\n",
    "ax.text(3,\n",
    "        10,\n",
    "        'Min: x = 3',\n",
    "        ha='center',\n",
    "        color=sns.xkcd_rgb['pale red'],\n",
    "       )\n",
    "plt.show"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Second approach\n",
    "\n",
    "To find the local minimum using **gradient descend**: you start at a random point, and move into the direction of steepest **descent** relative to the derivative:\n",
    "\n",
    "+ Start from a random $x$ value.\n",
    "+ Compute the derivative $f'(x)$ analitically.\n",
    "+ Walk a small step in the **opposite** direction of the derivative. \n",
    "\n",
    "In this example, let's suppose we start at $x=15$. The derivative at this point is $2×15−6=24$. \n",
    "\n",
    "Because we're using gradient descent, we need to subtract the gradient from our $x$-coordinate: $f(x - f'(x))$. However, notice that $15−24$ gives us $−9$, clearly overshooting over target of $3$. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function matplotlib.pyplot.show>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlkAAADGCAYAAADhV5X4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xlc1NX+x/HXAIq4p6Is7iUuuaAoShpKaYiWimvXhdQyb2kudbVf2W51q5sZ1s/KSktzy+uWGWoumFpaZpbp1RaXlE3cxTQR5/fHuaH+TNlm+M7yfj4e8wiGgflwGvDN+Z7zObbDh0/ZERERERGH8rG6ABERERFPpJAlIiIi4gQKWSIiIiJOoJAlIiIi4gQKWSIiIiJOoJAlIiIi4gR5hqyUlEPEx3elXbtW3HprJNOmTQXglVdepGnT+sTEtCUmpi2rV6/M/ZzExElERjYjKqoFa9eudl71IiIiIi7KllefrIyMdDIy0mnaNJysrNN07BjNhx/OZenSRZQpU5YRI0Zd8fg9e3YzfPhQVq5cR3p6Gr17d2Pz5u/w9fV16jciIiIi4krynMmqVi2Ipk3DAShbthxhYfVJS0u95uNXrFhOfHwv/P39qVWrNnXq1GXbtq2Oq1hERETEDRRoTdZvvx1gx44fiIhoCcD06dNo3z6K0aMf5MSJ4wCkpaUSEhKa+zkhIaGkp6c5sGQRERER1+eX3wdmZWUxdOggJk58iXLlyjN48H088sij2Gw2XnrpeZ5+egKJiVPz/cQzZ85g1qwPAPjpp93UqFGjwMUX1MWLF/Hx0Vp/R9F4Op7G1LE0no6nMXUsrx9Pu53Shw7he+4cv9eoQU6pUkX+ksUxpkeOHGH37v15Pi5fISs7O5uhQwfSq1df7ryzGwBVq1bN/fjAgfcwcGBfAIKDQ0hNTcn9WGpqCkFBwVd9zYSEISQkDAEgLi6GrVudf0kxOTmZDh06OP15vIXG0/E0po6l8XQ8jaljefV42u2QkAA//wzz50Pfvg75ssUxpuHhLfL1uDyjnt1uZ8yYEYSF1eeBB0bm3p+RkZ779mefLaNBg4YAxMZ2YfHihfzxxx8cOLCfvXv30qJFy4LWLyIiIp7shRfgo4/gueccFrBcTZ4zWVu2bGbBgnk0bHgzMTFtAZgw4SkWLfo3O3fuAGzUrFmTV19NBKBBg4Z07x5Pu3at8PPz4+WXX9XOQhEREblkzhx48kkYOBCeeMLqapwmz5DVpk0Uhw+fuur+jh1jr/k5Y8eOY+zYcUWrTERERDzPhg0wZAi0bw/vvQc2m9UVOY0Xr7YTERGRYvXTT9CjB9SpA4sWgb+/1RU5lUKWiIiION+RI9C1K/j4wPLlUKmS1RU5Xb5bOIiIiIgUyrlz0L07HDwI69bBjTdaXVGxUMgSERER57l4EQYPhi+/hI8/hqgoqysqNrpcKCIiIs7z6KOmD9Yrr0CfPlZXU6wUskRERMQ53nwTXn0VRoyAf/zD6mqKnUKWiIiION6SJTBqlFmLlZjo0a0arkUhS0RERBxr82b4298gMtI0HvXSpuQKWSIiIuI4v/wCd90FoaGwbBmULm11RZZRyBIRERHHyMiA2P+eCJOUBIGB1tZjMbVwEBERkaLLyjLNRtPTYe1aqFfP6oosp5AlIiIiRXP+PPTqBdu3w9Kl0Lq11RW5BIUsERERKTy7He67D1atgunTzWyWAFqTJSIiIkXx2GMwaxY8/zwMGWJ1NS5FIUtEREQKZ8oUePllePBBePxxq6txOQpZIiIiUnBz5sDo0dCzpwlbXthsNC8KWSIiIlIwK1bAPfdAhw4we7bXNhvNi0KWiIiI5N/mzWYnYZMmZidhqVJWV+SyFLJEREQkf3btMrsHg4NNs9Hy5a2uyKUpZImIiEjefvsN7rgDSpY07RqqVbO6IpenPlkiIiJyfZmZJmBlZcEXX0DdulZX5BYUskREROTaTp405xH+9puZwWra1OqK3IZCloiIiPy133+Hu+6CHTvgk0+gXTurK3Irea7JSkk5RHx8V9q1a8Wtt0YybdpUAI4fP0bv3t1p3Tqc3r27c+LEcQDsdjuPPz6OyMhmtG8fxQ8/bHfudyAiIiKOd/489O4NGzfCRx9BXJzVFbmdPEOWn58fzz77Ahs3fkNS0hqmT3+XPXt2M2XKZKKj27Nly3aio9szZcpkANasWcXevb+yZct2Jk1KZPz4sU7/JkRERMSBcnIgIcHsIHznHejXz+qK3FKeIatatSCaNg0HoGzZcoSF1SctLZUVK5bTr19/APr1609S0qcAJCV9Rt++f8Nms9GyZSQnT54kIyPdid+CiIiIOIzdDiNGwPz58MorMGyY1RW5rQKtyfrttwPs2PEDEREtyczMpFq1IACqVq1GZmYmAOnpqYSEVM/9nJCQUNLSUnMf+6eZM2cwa9YHuZ+TnJxchG8jf7KysorlebyFxtPxNKaOpfF0PI2pY7nceNrt3PjWW9RYsIAD/fuzr1UrcKX68sGVxjTfISsrK4uhQwcxceJLlCt3ZfMxm82GrYBnFiUkDCEhwZzWHRcXQ4cOHQr0+YWRnJxcLM/jLTSejqcxdSyNp+NpTB3L5cbzmWdgwQIYOZJaU6ZQyw3PI3SlMc1XM9Ls7GyGDh1Ir159ufPObgAEBgbmXgbMyEinSpUqAAQFhZCaeij3c1NTUwgODnF03SIiIuJI//oXPPssDBkCiYk68NkB8gxZdrudMWNGEBZWnwceGJl7f2xsF+bPnwPA/Plz6Ny5KwCdO8fx8cdzsdvtbN36NeXLl7/qUqGIiIi4kKlTYfx4s8D93XfBRwfCOEKelwu3bNnMggXzaNjwZmJi2gIwYcJTjBo1lmHDBjN79kyqV6/Je+99AEDHjrGsXr2KyMhmlC5dmsTEqU79BkRERKQIPvzQLHS/6y6YNQt8fa2uyGPkGbLatIni8OFTf/mxhQuXXXWfzWbj5ZdfK3plIiIi4lzz58PQodCxI3z8MZQoYXVFHkXzgSIiIt5o0SIYMADatoUlS6BUKasr8jgKWSIiIt5m2TKz/ioyEpYvhzJlrK7IIylkiYiIeJOkJHNcTvPm5u1y5ayuyGMpZImIiHiL1ashPh5uvhlWroQKFayuyKMpZImIiHiDdeugWzcIC4NVq+CGG6yuyOMV6FgdTxe0tAYNyzfIfT8+tDujwkZe5zOuFLGqNavaJ1HZv5IzyhMRETfx4uwTzFt7Bl8f03Jq6pgqbN51jmFdy1G6VMHmNz5ceZpOEQGEVCnCP9nJydC1K9SpY2az/ttAXJxLIesypXxLsS7mc6vLEBERN/bVrnMs33yWb94Kwb+kjSMnczifbaf/olMM6FiW0gXYxJeTY+fDlVncXLtk4UPW+vWXAtbatVC1auG+jhSYQlY+RKxqTd+afViV/jkXLl7gvVbvUK/cTRw7f4zhW0eQfi6dljdEYMdudakiImKx9KM5VKngg39JcyxNlQq+vLH4FKlHc+j4SDqVK/iyZlIQI14/yrpvg/GblULP6NI8c4+5fHfjgIP06VCGNd+eY0zv8nz703kS/plJgL+NjVOCCfAvwEzY+vXQpQvUrm0CVrVqTviO5Vq0Jusy53LOEbOuU+5tScrS3I9VLlmJNR1Wck+dQUz95W0AXt09mdaVI9lw2zq6BHfm0NkUq0oXEREX0allAIcyL9DwnkOMTDzK+u/P8VB8eUIq+7J6UhBrJpmj5iYOrcjUe9P4bloIX3x/jh/2ns/9GpXL+/LN2yEM6FiWiLCSzHwskG/fCS1cwKpVSwHLIprJusz1Lhd2DY4DoFmFpixPTQLgq6ObmRH5HgCdgjpSsUTF4ilURERcVtkAH76eGsKGHedI/v4c/Z8/zIv3Xb3IfMH6M0yeH0zA7FTSjuXwnwPnaVq3JAB9OxSxb5UClktQyMqnkj7+APjafMmx51hcjYiIuDJfXxsdwgPoEB5Akzolmbkq64qP70vL5rUFp3h1QAZ3xdVn6CuZnDt/aclJmVK2wj/52rVw552XLhEGBRX+a0mR6HJhEURVbsOiQ4sBWJOxlhPZJyyuSERErLbnYDY/H8rOfX/7L+epVc2PcgE+nP79IgCnfrdTppSNMqUuknE8hxVfn73m1ytX2ofTZy/m78lXrTKL3G+80ewoVMCylGayLvPnmqw/3VY1hidvfvyaj/9Hg7EM3zqCW9fG0KpSS6oHhBZHmSIi4sKyzl5k9JvHOJl1ET9fuDG0BG+Prcy8tWfo+lgGwZX9WDMpiPCbSjLkrVDCamVyS+Nrbzm8546yjHj9aN4L3z/7zDQabdhQbRpchELWZdK7H/zL+7+9Y0vu2+E3NGNJu38DUKlkJRbcMrdYahMREfcQEebPxinBV90/Mr48I+PL574/fXwgyck76dChwxWP+3V2jSve7xldhp7ReazR+uQTc1ROkybw+edQSf0aXYH3XC60q72CiIh4oEWLoFcvCA83M1gKWC7DO0JWSgq0bUuZX3+1uhIRERHHmT0b+vaFVq3MDJaOynEp3hGyzpyBgwcJHzsWvv7a6mpERESK7t13YdAgiI42C9512LPL8Y6QFRYGGzZwoVw5uP120z9ERETEXSUmwv33Q1wcLF8OZctaXZH8Be8IWQC1a/NdYiLUqAGdO8OKFVZXJCIiUnAvvghjxkDPnrB4MQQEWF2RXIP3hCzgfJUqZharQQPo1s28OEVERNyB3Q4TJpjbgAEwfz6ULGl1VXIdXhWyAAgMNB1wIyKgTx/46COrKxIREbm+ixdh5EgzizVsGHz4IfipC5Or876QBWb3xapVZrHgoEHw5ptWVyQiIvLXsrMhIQGmToVx4+Cdd8DX1+qqJB/yDFmjRz9Io0Z1iY5unXvfK6+8SNOm9YmJaUtMTFtWr16Z+7HExElERjYjKqoFa9eudk7VjlCunOmO2707PPQQTJyoXloiIuJazp0zPbBmzzazWC+/DLYinGsoxSrPuca77x7Avffez8iRw6+4f/jwEYwYMeqK+/bs2c3ixQvZsOFr0tPT6N27G5s3f4evqybuUqXg3/+Ge++Fp56CY8dg0iTw8c4JPhERcSGnT5v1w+vXm1msBx6wuiIpoDzTRFRUWypWzF9zsxUrlhMf3wt/f39q1apNnTp12bZta5GLdCo/P5gxA0aPhtdfN4HrwgWrqxIREW+WmQm33QYbNpi1wwpYbqnQq+amT5/Gxx/PJTy8Oc8++wIVK95AWloqERGtch8TEhJKenraX37+zJkzmDXrAwDS01NJTk4ubCn5lpWVde3n6d6dWqdOUWfGDI78/DO7nnySi/7+Tq/JnV13PKVQNKaOpfF0PI2pY/3VeJZKT6fp+PH4Hz7MrokTORoSAhrzfHOl12ihQtbgwffxyCOPYrPZeOml53n66QkkJk4t0NdISBhCQsIQAOLiYq46INMZkpOTr/88MTEQEUGVhx4i+vnnzYGbOqLgmvIcTykwjaljaTwdT2PqWFeN548/mvYMv/8Oa9bQpG1by2pzV670Gi3U4qOqVavi6+uLj48PAwfew3fffQtAcHAIqakpuY9LTU0hKOjqk8hd2ogRMG8ebNlidh+mplpdkYiIeINNm+DWW83bGzaAApbbK1TIyshIz337s8+W0aBBQwBiY7uwePFC/vjjDw4c2M/evXtp0aKlYyotTn37QlIS7N8Pt9wCP/1kdUUiIuLJli+HTp2galUTtho3troicYA8LxcOHz6ETZs2cuzYUZo1a8D48Y+zadMGdu7cAdioWbMmr76aCECDBg3p3j2edu1a4efnx8svv+q6Owvzcvvt5hp4XBwMbgPdKsO5TKhQHW5/Cpr2tbpCERFxU0u+S+FfK/eQcuIsoauXMm5JIj0aNTJ/4AcGWl2eOEieIeudd2Zcdd+AAQnXfPzYseMYO3Zc0apyFRER8NET8MWTcO6wue/kQVj239YVCloiIlJAS75L4bFFOzibnQNAygU/HusyCuIb00MBy6OoIVRedk67Oopmn4U1z1lSjoiIuLd/rdyTG7D+dNanBP9a/5tFFYmzKGTl5eShgt0vIiJyHaknzhbofnFfCll5qVD9r++/UMqcJyUiIpJfhw8Tcu7EX34opGJAMRcjzqaQlZfbn4IS//+F7wdLj8Jdd8GpU5aUJSIibmbPHmjThnHJHxLgc+VZuQElfBkXW9+iwsRZFLLy0rQv3DUFKtQAbOa/Pd+CMW/D6tWmp8khXToUEZHr+OILiIqCrCx6vDORf/ZpTuh/Z65CKwbwz55N6NE81OIixdEKfayOV2na9+qdhE2BGjWgd29o3Ro+/RSaN7ekPBERcWFz5sCQIVCnDnz2GdStSw+gR/NQl+pOLo6nmayiuOMO0zTO19fMaH32mdUViYiIq7Db4cUXzTE5UVHw5ZdQt67VVUkxUsgqqiZNYPNmqF/frNGaWrAzHEVExAOdPw9Dh8KECSZkrVwJlSpZXZUUM4UsRwgJgfXroWtXc/bh6NFw4YLVVYmIiBWOHDFH5HzwATzzDMyaBf7+VlclFlDIcpSyZWHxYnj4YZgyxcxqnTxpdVUiIlKcdu+GNm1gyxazFuvpp8Fms7oqsYhCliP5+sKkSTBtmtl5eMstsG+f1VWJiEhxWLPGrL06fRrWrYO//c3qisRiClnOMGwYrFoFaWkQGQkbN1pdkYiIONPbb0PnzlC9upnFioqyuiJxAQpZzhITY37QKlWC226D6dOtrkhERBwtOxsefBAeeODSjvPata2uSlyEQpYz1atndh526AD33gtjxmhBvIiIpzhyxASrt96C8ePhk0+gfHmrqxIXopDlbDfcYPpnjR0LiYkQFwfHjlldlYiIFMWPP5rlIF99BTNnwssvm3W5IpdRyCoOfn7w2mvmkuEXX5gfzF27rK5KREQKY+lSs+bq3DnTvmfQIKsrEhelkFWchgwxO06ysswW3yVLrK5IRETy6+JF05KhRw9o0AC++cYcqyZyDQpZxe2WW8wPZoMGEB8PTz1lfnBFRMR1nTxpwtVzz8HgwbBhA4TqQGe5PoUsK9SoYS4bDhkCEyeaxqUnTlhdlYiI/JXdu82MVVISvPGGWfpRqpTVVYkbUMiySqlS8P775qzDVaugVSvYudPqqkRE5HJLl5p1tMeOmWajI0eqg7vkm0KWlWw201vlz3VarVvDxx9bXZWIiFy4AP/zP+YSYf368O23EB1tdVXiZhSyXEG7duYHuFkz6NfP9NM6f97qqkREvNPhwxAba9oyDB9uTu2oUcPqqsQN5RmyRo9+kEaN6hIdfWkHxfHjx+jduzutW4fTu3d3Tpw4DoDdbufxx8cRGdmM9u2j+OGH7c6r3NOEhEBysglYiYmmY3xKitVViYh4l82boUUL+PJLmDHDHJfj7291VeKm8gxZd989gHnzFl1x35Qpk4mObs+WLduJjm7PlCmTAVizZhV79/7Kli3bmTQpkfHjxzqnak9VogRMngzz58P335sf9LVrra5KRMTz2e3w5pvmkmDJkqbJ6ODBVlclbi7PkBUV1ZaKFW+44r4VK5bTr19/APr1609S0qcAJCV9Rt++f8Nms9GyZSQnT54kIyPdCWV7uL59TZuHypWhUyd48UW1eRARcZZTp8xSjYceMpcJv/0WwsOtrko8QKHWZGVmZlKtWhAAVatWIzMzE4D09FRCQqrnPi4kJJS0tFQHlOmFGjaEr782P/gTJkCXLvDfcRYREQfZvh0iImDRIrMGa+lScxyaiAP4FfUL2Gw2bIXYzjpz5gxmzfoAMOEsOTm5qKXkKSsrq1iex6GGDSM4OJh6b7xBdqNG7HriCU42a2Z1VYCbjqeL05g6lsbT8TxmTO12gj/91PxurVCBXZMnc7JJE9PDsBh5zHi6EFca00KFrMDAQDIy0qlWLYiMjHSqVKkCQFBQCKmph3Ifl5qaQnBwyF9+jYSEISQkDAEgLi6GDh06FKaUAklOTi6W53G4mBgYNAj/vn1p/vDD8Pzz8Oij4GPt5lC3HU8XpjF1LI2n43nEmJ46ZdrnzJkDd9yB/0cf0Tww0JJSPGI8XYwrjWmh/pWOje3C/PlzAJg/fw6dO3cFoHPnOD7+eC52u52tW7+mfPnyuZcVpYjCw2HrVujTBx5/HOLiIF3r3URECmTrVrOpaN48c+JGUhJYFLDE8+UZsoYPH0KXLh355ZefadasAbNnz2TUqLGsX7+O1q3DWb8+mVGjzC7Cjh1jqVWrNpGRzXjkkVG8/PJrTv8GvEr58jB3rtlS/MUXpq/WypVWVyUi4vouXoTXXjPnx54/D+vXwxNPWH5FQDxbnpcL33lnxl/ev3Dhsqvus9lsClbOZrOZ5njt2plF8Z07wz/+AS+8YLYdi4jIlQ4fNu0YkpJMB/f334dKlayuSryAIry7uvlm0+bhgQfg1VehbVv45RerqxIRcS2rVplZ/7VrTR+sRYsUsKTYKGS5s4AAc8D0woUmYIWHm9Ph7XarKxMRsda5czB2rOl7VakSbNkCI0bocGcpVgpZnqBnT/jhB3NS/L33Qu/ecPSo1VWJiFjjxx/N78PXX4eRI81idxdpfSPeRSHLU9SoAatXwyuvwLJl0KSJmSYXEfEWFy/CG29Ay5aQkQHLl5v3AwKsrky8lEKWJ/HxgXHjTKf4ihXNNPno0XD2rNWViYg418GD5nfeqFHQsSPs2GFOyhCxkEKWJwoPN2dvPfQQTJkCzZub4CUi4mnsdvjoIzN7/9VX8M47Zja/alWrKxNRyPJYAQEmYH3+OZw5Y3rDPPmk6Q8jIuIJMjNNg+ZBg6BxY/j+e7j/fi1uF5ehkOXp/pw2HzjQHMfTpo1ZFCoi4s6WLDGzV8uWmYOd16+HG2+0uiqRKyhkeYOKFeGDD2DxYkhJMUdKvPACZGdbXZmISMEcPQr9+0N8PAQHm36B48eDr6/VlYlcRSHLm/ToYWax4uPNcRKtW5vpdRERd7B4MTRqBAsWwLPPmrWmTZtaXZXINSlkeZvAQJg/3zQwTUkxW52feUZrtUTEdR05YmavevaE0FDT9+qpp6BECasrE7kuhSxv1bMn7Nplzj989llo1cpMu4uIuAq7HebMgYYN4d//hueeM53b1VhU3IRCljerXNlsfV661Pyl2KaNOYYiK8vqykTE2x04AF27woABZkH7tm1mh7Rmr8SNKGQJdOtmZrWGDzfHUDRuDCtWWF2ViHijnBzTpf3mm+GLLyAxETZtMr+XRNyMQpYYFSqYw6Y3bDA9tuLiTNuHw4etrkxEvMX27aan36hR0K6d2agzapR2DorbUsiSK7VrZ37RPfUUfPwx1K8P06aZM8FERJwhKwseecRsxNm/3yxjSEqC2rWtrkykSBSy5Gr+/mYx/PffmwWmw4dD27Zq9yAijrd0qWnL8NprcO+9sHu3WYelru3iARSy5NoaNoR162DmTPj1V4iIMH9tnj5tdWUi4u727jXrQXv0MMsVNm0y5w7ecIPVlYk4jEKWXJ/NZs4F270b7rsPJk82lxBnzzbbq0VECuLsWXj6aTN7tXYtvPKK2Tl4yy1WVybicApZkj+VKsHbb8PmzVC9OgwcSPiYMbqEKCL5Y7dfujT43HPm5Ik9e2DcOLVlEI+lkCUFExlpgtZ771H6wAFzDuLIkXDsmNWViYir+s9/oEsXc2mwdGkzgzV3runeLuLBFLKk4Hx84N57+XrWLHjwQXjrLahXD958U4dOi8glx4/DmDHmfMGvvoJJk8zu5ZgYqysTKRYKWVJoF8qVM00Dv/sOwsPhoYfMbsSVK60uTUSsdOHCpT++3njD7Br8+Wd4+GFdGhSvUqSQFRHRmPbt2xAT05ZOndoDcPz4MXr37k7r1uH07t2dEyeOO6RQcWFNm8Lq1bBkiTlounNncxzGf/5jdWUiUpzsdtPfqnlzM8vdpIlZ1P722+ZwehEvU+SZrEWLlrNu3SY+/3w9AFOmTCY6uj1btmwnOro9U6ZMLnKR4gZsNujeHXbuhFdfhY0bzS/Yv/8d0tOtrk5EnG37drjjDrP26uxZc6Dz2rU6zFm8msMvF65YsZx+/foD0K9ff5KSPnX0U4gr8/c3vbR++QVGjID334ebbjLNTXXwtIjnOXQIBg82m2C2bTNnDe7aBb16qaGoeD3b4cOnCt3sqGXLJlSoUBGbzUZCwhASEoZw0001+OWXgwDY7Xbq1auZ+/7lZs6cwaxZHwCQnp7KvHnzCltGvmVlZVG2bFmnP4+3yM94BqSkUOfdd6m6fj3nb7iB/YMHk9alC3Y/v2Kq0r3oNepYGk/H+3NM/U6epObcuYQuXozNbudQr178NmAAFzTeBaLXqOMVx5iOGfNw7hW86ylSyEpLSyU4OITMzEz69OnOP//5LwYNuvuKUFWvXk1+/vm3636duLgYtm7dWtgy8i05OZkOHTo4/Xm8RYHGc/Nm0w9n40aoW9f0ybn7bh38+v/oNepYGk/H25CUxK3btpkmoqdPQ0ICPPOMzhksJL1GHa84xjQ8vEW+QlaRLhcGB4cAEBgYSJcud7Jt27cEBgaSkWHW4GRkpFOlSpWiPIV4ijZt4IsvYPlyKFcOBg40OxI/+USd40XcwR9/wJtv0nrAAHjiCdOG4Ycf4IMPFLBErqHQIevMmTNkZZ3OfTs5eS0NGzYkNrYL8+fPAWD+/Dl07tzVMZWK+7PZzKLYbdtMI8Jz58xi+ago0/ZBYUvE9Zw/b84UrFcPHnqI32vUgC+/NLuJGze2ujoRl1bokJWZeZg774ylQ4db6Nw5hk6dYrnttk6MGjWW9evX0bp1OOvXJzNq1FhH1iuewMfHXCrctQumTYO0NNP2oW1bWLVKYUvEFWRnw/Tp5qzSv//ddGf//HO2v/66+cNIRPJU6NXHtWvXITn5y6vur1SpMgsXLitSUeIlSpSAYcPMmo4ZM+CFFyA21oStZ56B22/X7iSR4nb+PMyaBf/8J/z6K7RsaRqLxsaan8fkZKsrFHEb6vgu1vP3N38p//IL/O//wv790KmT+Wt52TLNbIkUh3PnzM/fTTfBffdBhQpmzeTXX5uZZv3BI1JgClniOvz9TZfoX34xfzlnZEC3bmaB/PzGjno0AAAOmUlEQVT5kJNjdYUinicrC157DerUMYe916hhurZv3Qp33aVwJVIEClniekqVMjNbP/0EH35oLl/cfTc0agTvvmv+4haRojl8GJ58EmrWNA2EGzUyHdo3btTMlYiDKGSJ6ypRwqzX+vFHWLDAtH64/36zXfzFF+G4zsUUKbBffzUzxrVqmXWQHTrAV1/BmjWmLYPClYjDKGSJ6/P1hd694ZtvzF/azZvDhAnmssaYMbB3r9UVirg2ux02bTI/R2Fh5rirgQPNIe6LFpk+diLicApZ4j5sNvOXdlISfP899Ox5aaFufDysX69F8iKXy86GOXMgMhLatTN/pIwbB/v2mUvv9etbXaGIR1PIEvfUtCnMnGl2Ij72GGzYYC57tGhhOlBr3ZZ4s8OHzaXAOnVgwAA4dQqmToWDB+GllyAkxOoKRbyCQpa4t9BQ84/JwYPmL/PsbBgyBKpXh/HjdSlRvIfdbtZWDRhgXv9PPAENGsCnn5rLgg88AGXKWF2liFdRyBLPEBBgevvs2HFpAe9rr5lLiV26mH9o1AJCPNHp0+YPjIgIuOUW81r/+99NsFq9Grp2NacsiEix00+eeBabDW67zexGPHAAnn7arN+66y6zK/Hpp80lRhF3ZrebjSD3328u/d1/v5nFfestSEmBKVPMLJaIWEohSzxXaOilULVwoTnMduJEqFvXHBGyYAH88YfVVYrk39Gj8OabZu1hZCTMng19+pgDm3/4wcxglS1rdZUi8l8KWeL5SpQwOxGTkkzgevppcymlb18zCzBiBGzZop2J4pqys83xUr16QXAwPPSQmbF96y1ITTWHOEdFqb+ViAtSyBLvUrOmCVn79pnQFRtr/pFq0wYaNjRNTg8csLpK8XZ/Xg4cO9YsYu/WzXRiHzkStm+HbdvMrFWFClZXKiLXoZAl3snX1xwdMmcOpKfDe+9BtWqmyWnt2tC2Lbzxhjk/UaS47N5t/ggICzOXA6dONf2tPvkEDh0ymzmaNbO6ShHJJ4UskQoV4N57TTPTvXvNbNbp0zBqlLmc2KmTCWGZmVZXKp7op5/Ma655czOb+vzz5sib9983IX/hQrNxo0QJqysVkQJSyBK5XJ06prnpDz+YMxMff9xcWhw2DIKCzM7FN980O7hECsNuh5074bnnTFPd+vXNDGqpUvD662bGavVqGDoUKla0uloRKQKFLJFruflmsxvx55/hu+9M4EpPNwuPq1c3i41ffNH05tKiebmeCxfMTOkjj5hLgY0bwzPPmBCVmGia6X71FYwebRa3i4hH8LO6ABGXZ7NBeLi5TZxodiYuXAhLlpgZiD/Xcd11F9x5J0RHm1kJ8W5Hj8Lnn8Nnn8Hy5XDsGJQsaWZDH34YevRQoBLxcApZIgXVsKE5suSJJ8wW+k8/NVvs333XLJYPCDDnKMbGmsX1YWHaXu8NcnJg61ZYscLsXP36azPDWbmyCd/dusEdd0C5clZXKiLFRCFLpCj+7LZ9//3w+++wbh2sXGluY8aYx9SqZWYvbrvNHPcTGmptzeIYdruZ1VyzxtySk+HkSROoIyPhqacgLg5atjS7WUXE6yhkiThK6dLmnLiuXc37+/aZsLVqlbm0OGOGuT8szISt6GizPb9mTetqlvy7eNFshtiwwdy++ALS0szHateG3r3h9tvNbtQqVSwtVURcg0KWiLPUqWMaRv797+Yf6O+/NzNda9ea/lzvvGMeV6OGCVtt21LWz88c8luypLW1i2nj8c035jSAL7+ETZvg+HHzsdBQc0n4tttMsKpTx9JSRcQ1OS1krV37ORMmPEpOTg4DB97DqFEPO+upRFyfj4/pg9S8uVn0nJNjdiVu3Ghu69fD3Lm0BLPD7M+z6Vq1gogIqFdPl5yc6exZ8//ju+9MsNq8GXbturRrNCzMHM0UHQ233mpmrrTOTkTy4JSQlZOTw6OPPsKCBUsJCQnljjs6EBvbhfr1dSq8CGAC0587FkeONP+YHzjAzhkzuPnMGbNo+t13zfZ+MJcimzY1j2/eHJo0gUaNdKxKQdntpg3Hjz+a2/btJljt2mWCL0ClStC6tTl4uXVrE3YrVbK2bhFxS04JWdu2baVOnbrUrm2m0OPje7FixXKFLJFrsdmgdm0yY2LMZSgwvZV27jQhYPt2c5s7F95++9LnVa9uwtbNN0ODBmbGq149syDfx4vb4GVnw/79VPrqKzN+e/aYIPXjj5cu+YFpMNuihdn516KFudWqpVkqEXEIp4Ss9PQ0QkOr574fHBzCtm1bnfFUIp7Lz8+cU3f5WXV2O+zfb8LCzp0mOOzcaYLX2bOXHhcQADfdBDfeaELD5beaNU1bAXcOYdnZZkbq0CFzoPe+fWZc9u83b+/bBxcu0PTPx1esaIJonz6mEWjjxub9qlWt+x5ExONZtvB95swZzJr1AQDp6akkJyc7/TmzsrKK5Xm8hcbT8fI9puXKQZs25gZw8SL+hw8TkJJC6ZQUAg4dIiAlhYDt2/FfuRK/ywMYcNHXl/OVK3O+UiX+qFKF85UqkV2hAtnly5NdoQIX/vv2hTJlyClThgulS3PR3985Mzx2O75nz+J35gy+WVn4nTmD3+nTlDx5khInT1LixAlKnDhByRMnKHn0KP5HjlDixAls/6/L/vkKFTgXFMS5kBDOtmrF2Ro1OFq5MoSFkV2+/NW179plblIg+rl3LI2n47nSmDolZAUFBZOScij3/bS0VIKDQ654TELCEBIShgAQFxdDhz8vkThRcnJysTyPt9B4Op5TxtRuN5fIDhwwt99+wyctjVJpaZRKTTVtCHbvNh3JL1689tfx8THhrnRp09H+8pufn/m4r6/5r4+Ped6cHHPZMyfH3M6fN/3Ezp69dDtz5vrPW7KkaYlQtaq5FNq+vdndFxJi/lu7NtSqRcmyZSkJlL/sU/UadTyNqWNpPB3PlcbUKSGrefMI9u7dy4ED+wkODmHx4oW8/fb7zngqEcmLzWYWbleqZBbNX8vFi3DiBBw5Ym5Hj8KpU+Z2+vSlt8+ehXPnrrxdHqIuXjRv22wmfPn6miDm62sCU0CACWoBAeZWpoxZwH/5rWJFCAw04apcOa2REhG35JSQ5efnx0sv/Yt+/eLJycmhf/9BNGjQ0BlPJSKO4uNzKYyFhVldjYiI23PamqyOHWPp2DHWWV9eRERExKW58fYiEREREdflEsfq7Nu3j/DwFk5/nqNHj1C5ss4UcxSNp+NpTB1L4+l4GlPH0ng6XnGM6cGDB/L1ONvhw6fseT/MM3Tq1J7PP19vdRkeQ+PpeBpTx9J4Op7G1LE0no7nSmOqy4UiIiIiTqCQJSIiIuIEvuPGPfaM1UUUp2bNrtMnSApM4+l4GlPH0ng6nsbUsTSejucqY+pVa7JEREREiosuF4qIiIg4gceHrE8+Wcytt0ZSrVoFtm/fdsXHEhMnERnZjKioFqxdu9qiCt3bK6+8SNOm9YmJaUtMTFtWr15pdUluae3az4mKakFkZDOmTHnN6nI8QkREY9q3b0NMTFs6dWpvdTluafToB2nUqC7R0a1z7zt+/Bi9e3endetwevfuzokTxy2s0L381Xjqd2jhpaQcIj6+K+3ateLWWyOZNm0q4FqvUY8PWQ0aNGLGjNlERbW94v49e3azePFCNmz4mnnzFvHoow+Tk5NjUZXubfjwEaxbt4l16zapy38h5OTk8OijjzB37kI2bvyGRYv+zZ49u60uyyMsWrScdes2ucx2bndz990DmDdv0RX3TZkymejo9mzZsp3o6PZMmTLZourcz1+NJ+h3aGH5+fnx7LMvsHHjNyQlrWH69HfZs2e3S71GPT5khYXV56ab6l11/4oVy4mP74W/vz+1atWmTp26bNu21YIKxdtt27aVOnXqUrt2HUqWLEl8fC9WrFhudVkiREW1pWLFG664b8WK5fTr1x+Afv36k5T0qRWluaW/Gk8pvGrVgmjaNByAsmXLERZWn7S0VJd6jXp8yLqWtLRUQkJCc98PCQklPT3Nworc1/Tp02jfPorRox/UpYNCSE9PIzS0eu77wcEhpKWlWliRZ7DZbPTt24OOHaOZOXOG1eV4jMzMTKpVCwKgatVqZGZmWlyR+9Pv0KL77bcD7NjxAxERLV3qNeoSx+oUVa9e3cjMzLjq/scee4q4uK4WVORZrje+gwffxyOPPIrNZuOll57n6acnkJg41YIqRa60bNlKgoNDyMzMpE+f7tSrF3bVsgEpGpvNhs1ms7oMt6bfoUWXlZXF0KGDmDjxJcqVK3/Fx6x+jXpEyFq48JMCf05wcAipqSm576emphAUFOzIsjxGfsd34MB7GDiwr5Or8TxBQcGkpBzKfT8tLZXg4BALK/IMf45hYGAgXbrcybZt3ypkOUBgYCAZGelUqxZERkY6Varo3L2iqFq1au7b+h1acNnZ2QwdOpBevfpy553dANd6jXrt5cLY2C4sXryQP/74gwMH9rN3715atGhpdVluJyMjPfftzz5bRoMGDS2sxj01bx7B3r17OXBgP+fPn2fx4oXExnaxuiy3dubMGbKyTue+nZy8loYN9dp0hNjYLsyfPweA+fPn0LmzrhYUhX6HFp7dbmfMmBGEhdXngQdG5t7vSq9Rj29Gunz5Mh5/fBxHjx6hfPkKNG7chI8/XgLA5Mn/Ys6cWfj5+fH88y9x++13WFyt+3nwwWHs3LkDsFGzZk1efTUx91q45N/q1St54on/IScnh/79BzF27DirS3Jr+/fvY/DgAQDk5FygZ88+GtNCGD58CJs2beTYsaMEBlZl/PjHiYvryrBhgzl06CDVq9fkvfc+4IYbKlldqlv4q/HctGmDfocW0ubNX9GtWywNG96Mj4+ZM5ow4SlatGjpMq9Rjw9ZIiIiIlbw2suFIiIiIs6kkCUiIiLiBApZIiIiIk6gkCUiIiLiBApZIiIiIk6gkCUiIiLiBApZIiIiIk6gkCUiIiLiBP8HQQw44QpPXOkAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff8bcd64e80>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = np.linspace(-10,20,100)\n",
    "y = x**2 - 6*x + 5\n",
    "start = 15\n",
    "\n",
    "fig, ax = plt.subplots(1, 1)\n",
    "fig.set_facecolor('#EAEAF2')\n",
    "plt.plot(x,y, 'r-')\n",
    "plt.plot([start],[start**2 - 6*start + 5],'o')\n",
    "ax.text(start,\n",
    "        start**2 - 6*start + 35,\n",
    "        'Start',\n",
    "        ha='center',\n",
    "        color=sns.xkcd_rgb['blue'],\n",
    "       )\n",
    "\n",
    "d = 2 * start - 6\n",
    "end = start - d\n",
    "\n",
    "plt.plot([end],[end**2 - 6*end + 5],'o')\n",
    "plt.ylim([-10,250])\n",
    "plt.gcf().set_size_inches((10,3))\n",
    "plt.grid(True)\n",
    "ax.text(end,\n",
    "        start**2 - 6*start + 35,\n",
    "        'End',\n",
    "        ha='center',\n",
    "        color=sns.xkcd_rgb['green'],\n",
    "       )\n",
    "plt.show"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To fix this, we multiply the gradient by a step size. This step size (often called **alpha**) has to be chosen carefully, as a value too small will result in a long computation time, while a value too large will not give you the right result (by overshooting) or even fail to converge. \n",
    "\n",
    "In this example, we'll set the step size to 0.01, which means we'll subtract $24×0.01$ from $15$, which is $14.76$. \n",
    "\n",
    "This is now our new temporary local minimum: We continue this method until we either don't see a change after we subtracted the derivative step size (or until we've completed a pre-set number of iterations)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Local minimum occurs at 3.000000.\n"
     ]
    }
   ],
   "source": [
    "old_min = 0\n",
    "temp_min = 15\n",
    "step_size = 0.01\n",
    "precision = 0.0001\n",
    "\n",
    "def f(x):\n",
    "    return x**2 - 6*x + 5\n",
    "    \n",
    "def f_derivative(x):\n",
    "    import math\n",
    "    return 2*x -6\n",
    "\n",
    "mins = []\n",
    "cost = []\n",
    "\n",
    "while abs(temp_min - old_min) > precision:\n",
    "    old_min = temp_min \n",
    "    gradient = f_derivative(old_min) \n",
    "    move = gradient * step_size\n",
    "    temp_min = old_min - move\n",
    "    cost.append((3-temp_min)**2)\n",
    "    mins.append(temp_min)\n",
    "\n",
    "# rounding the result to 2 digits because of the step size\n",
    "print(\"Local minimum occurs at {:3.6f}.\".format(round(temp_min,2)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "An important feature of gradient descent is that **there should be a visible improvement over time**: In this example, we simply plotted the squared distance from the local minima calculated by gradient descent and the true local minimum,  ``cost``, against the iteration during which it was calculated. As we can see, the distance gets smaller over time, but barely changes in later iterations. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function matplotlib.pyplot.show>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlkAAADFCAYAAABnw+dWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xl0VGWexvFvQQECYdNANhoCStgJEAgdAlkQJYA0okiUaJrYg4w6jdsIPeLa9Ki0K9hjK9pih0FAh2UGMQgIxaYmAgJiC0qzZwFkD0tCkpo/XhOCJgRqyU2qns85dSp1q+reXz3nhvy476332g4fPuVERERERDyqjtUFiIiIiPgiNVkiIiIiXqAmS0RERMQL1GSJiIiIeIGaLBEREREvUJMlIiIi4gVqskRERES8QE2WiIiIiBeoyRIRERHxArvVBQB07tyO8PBwr2/nzJkzNG7c2OvbqcmUgaEcDOWgDEopB0M5GMrh8hns2bOHHTv2VrmOGtFkhYeHs3HjRq9vx+FwkJCQ4PXt1GTKwFAOhnJQBqWUg6EcDOVw+Qx69ux9RevQcKGIiIiIF6jJEhEREfECNVkiIiIiXqAmS0RERMQL1GSJiIiIeEGVTdZDDz1Aly7tiYvr94vn3nzzDVq1asrRo0cBcDqdPPHE40RHRxIfH8O2bVs8X7GIiIhILVBlk3XnnSnMm7fwF8uzsw/icHxG69a/Klv22WfL2b37n2RmbuGVV6YzadIjnq1WREREpJaossmKiYmlefMWv1j+1FP/wdNPT8Vms5Uty8j4hDFj7sJms9GnTzQnT57k0KE8z1YsIiIiUgu4NBlpRsZSQkJC6Nat+yXL8/JyCA1tXfY4NDSM3NwcgoKCf7GO9PRZzJ79ftn7HA6HK6VcsXonT3K2pMTr26np8vPz/T4DUA6llIMyKKUcDOVgKAfPZHDVTdbZs2eZPv1lPvxwsVsbTk1NIzU1DYChQxO9O7Psd9/BK6+wafhwokaN8t52agHN4msoB0M5KINSysFQDoZy8EwGV91k7d27h/3795GYGAtATk42gwcPZNmy1QQHh5KTc7DstTk52YSEhLpVoEdcfz3Y7TTes8fqSkRERMRPXPUUDl26dOUf/9jNpk3b2bRpO6GhYaxcuY6goCCSkoby4YdzcTqdbNyYRdOmTSscKqx29etDhw403rvX6kpERETET1TZZE2YkMawYYPZtesHIiM7MWdOeqWvHTx4CG3bhhMdHcljj01k2rRXPVqsW7p1o2F2Npw/b3UlIiIi4geqHC58++1Zl31+06btZT/bbLaa1ViV17UrlJTA999Djx5WVyMiIiI+zn9mfO/UCWw22L696teKiIiIuMl/mqzGjTkfHAzffmt1JSIiIuIH/KfJAs60awc7dkBRkdWliIiIiI/zryYrPBwKC+Gf/7S6FBEREfFxftVknQ0PNz9oyFBERES8zK+arKKAAAgL08nvIiIi4nV+1WQB0K0b/OMfZjoHERERES/xvyarRw84c0bnZYmIiIhX+V+T1b27ud+2zdo6RERExKf5X5PVogW0aQNbt1pdiYiIiPgw/2uyACIjzTcML1ywuhIRERHxUVU2WQ899ABdurQnLq5f2bJnn32S/v2jiI+P4be/HcvJkyfKnps+/RWioyOJienNqlUrvVO1uyIjzXxZO3daXYmIiIj4qCqbrDvvTGHevIWXLIuPT2Tt2kzWrPmC66+/genTzUWhd+7cwaJFC1i3Lot58xYyefKjFBcXe6dyd3TrZq5jqCFDERER8ZIqm6yYmFiaN29xybLExBux2+0AREX1JScnG4Bly5YyatTtNGjQgLZtw2nXrj2bN2/0QtluatwYOnRQkyUiIiJeY3d3BXPnzmbkyNsAyM3NISqqb9lzoaFh5OXlVvi+9PRZzJ79PgB5eTk4HA53S6lSfn5+2XaC6tWj5fr1/GP5ckrq1/f6tmuK8hn4M+VgKAdlUEo5GMrBUA6eycCtJuu1116ibl07o0cnX/V7U1PTSE1NA2Do0EQSEhLcKeWKOByOi9tp3hy+/Za4666DqCivb7umuCQDP6YcDOWgDEopB0M5GMrBMxm4/O3CefPmsHz5Mv7613ex2WwAhISElg0dAuTkZBMcHOJWgV7TpQvUqwdbtlhdiYiIiPggl5qsVatW8Je/vM7s2fNp1KhR2fIhQ4axaNECCgoK2LdvL7t376Z37z4eK9aj6teHrl1h82arKxEREREfVOVw4YQJaWzYsJ5jx44SGdmJSZOeYPr0VygsLOSOO0YC5uT3l19+nU6dOjNy5CgGDOiL3W5n2rSXqVu3rtc/hMuiouBvf4Mff4TAQKurERERER9SZZP19tuzfrEsJSW10tc/8sjjPPLI4+5VVV1Km6zNm+Hmm62uRkRERHyIf874Xqp1a3MEa9MmqysRERERH+PfTZbNZo5mbdkCRUVWVyMiIiI+xL+bLDBN1tmzusSOiIiIeJSarB49oG5dDRmKiIiIR6nJatwYOndWkyUiIiIepSYLoHdv2L0bjh+3uhIRERHxEWqyAPr+dL3FjTXwYtYiIiJSK6nJAmjbFlq1gsxMqysRERERH6EmC8xUDv36wddfQ0GB1dWIiIiID1CTVSo6GgoLdcFoERER8Qg1WaW6dTPfNNSQoYiIiHiAmqxSdruZmDQrC0pKrK5GREREarkqm6yHHnqALl3aExfXr2zZ8ePHGD16JP369WT06JGcOGGmPnA6nTzxxONER0cSHx/Dtm21bOitXz84eVKzv4uIiIjbqmyy7rwzhXnzFl6ybMaM14iLiyczcwtxcfHMmPEaAJ99tpzdu/9JZuYWXnllOpMmPeKdqr0lKsrM/p6VZXUlIiIiUstV2WTFxMTSvHmLS5YtW7aU5OSxACQnjyUj42MAMjI+YcyYu7DZbPTpE83Jkyc5dCjPC2V7SePG5tysL74Ap9PqakRERKQWs7vypiNHjhAUFAxAq1ZBHDlyBIC8vBxCQ1uXvS40NIzc3Jyy15aXnj6L2bPfL3ufw+FwpZSrkp+fX+V2rg0IIHTVKn6YP5+C4F/WXdtdSQb+QDkYykEZlFIOhnIwlINnMnCpySrPZrNhs9mu+n2pqWmkpqYBMHRoIgkJCe6WUiWHw1H1dnr2hM8/J7C4GKqhpup2RRn4AeVgKAdlUEo5GMrBUA6eycClbxe2bNmybBjw0KE8AgMDAQgODiUn52DZ63JysgkJCXWrwGrXvLkZMtywwepKREREpBZzqckaMmQY8+d/AMD8+R+QlDQcgKSkoXz44VycTicbN2bRtGnTCocKa7zYWDhwAPbvt7oSERERqaWqbLImTEhj2LDB7Nr1A5GRnZgzJ52JEx9hzZrV9OvXkzVrHEycaL5FOHjwENq2DSc6OpLHHpvItGmvev0DeEX//uZSO+vXW12JiIiI1FJVnpP19tuzKly+YMGSXyyz2Wy1t7Eqr0UL6NrVNFljx1pdjYiIiNRCmvG9MqVDhgcOWF2JiIiI1EJqsipTOmS4dq3VlYiIiEgtpCarMtdeCz16wJo1mphURERErpqarMtJTITcXPj+e6srERERkVpGTdblxMRA/fqwerXVlYiIiEgtoybrcho1gn79zHlZRUVWVyMiIiK1iJqsqiQmwunT8PXXVlciIiIitYiarKr06gVNm2rIUERERK6Kmqyq2O0QFweZmXDmjNXViIiISC2hJutKDBoEhYWaM0tERESumJqsK3HDDdCuHSxfbnUlIiIiUku41WS99dZfGDgwmri4fkyYkMb58+fZt28vSUmJREdHMn78OAoLCz1Vq3VsNrj5Zti1C3bvtroaERERqQVcbrJyc3N49923Wb58DWvXZlJcXMLixQuYOvUZJkx4kKysrTRr1pw5c9I9Wa91EhKgXj0dzRIREZEr4taRrKKiIs6fP0dRURHnzp2lVasg1q9fw4gRtwKQnHwXGRkfe6RQywUEmOsZOhzm/CwRERGRy7C7+saQkFAeeOD39OrVlYYNryE+fhCRkb1o2rQZdrtZbWhoGHl5uRW+Pz19FrNnvw9AXl4ODofD1VKuWH5+vlvbadyiBe327+fgm29yomdPzxVWjdzNwFcoB0M5KINSysFQDoZy8EwGLjdZJ04cZ9myT9i48RuaNWvG736XyqpVK674/ampaaSmpgEwdGgiCQkJrpZyxRwOh3vbiY+HzEwC8/LM8GEt5HYGPkI5GMpBGZRSDoZyMJSDZzJwebhw7VoHbdq0JTAwkHr16jF8+AiysjI5deokRT9dgiYnJ5vg4BC3CqxRbDYYOhS+/Rb27rW6GhEREanBXG6ywsJas2nTV5w9exan08m6dWvo2LEjsbFxLFmyGID58+eSlDTcY8XWCIMHm4tGf+wj55qJiIiIV7jcZEVF9eWWW0YyePBA4uN/TUlJCffck8ZTTz3HW2/9hejoSI4fP0ZKSqon67VekyZmqHD1asjPt7oaERERqaFcPicLYPLkKUyePOWSZeHh7fj0U4c7q635hg83UzmsWAGjRlldjYiIiNRAmvHdFe3bQ5cu8MknUFJidTUiIiJSA6nJctUtt0BeHmzcaHUlIiIiUgOpyXJVTAwEBsKiRVZXIiIiIjWQmixX2e1w662wfTvs3Gl1NSIiIlLDqMlyx803Q+PGsHCh1ZWIiIhIDaMmyx0NG8KwYfDFF5Bb8eWDRERExD+pyXLXiBFQty4sXmx1JSIiIlKDqMlyV4sWMGgQrFwJx45ZXY2IiIjUEGqyPGH0aCgq0rlZIiIiUkZNlieEhEBiImRkwIkTVlcjIiIiNYCaLE8ZMwYuXNDRLBEREQHcbLJOnjzBvffeQ//+UcTG9uGrrzI5fvwYo0ePpF+/nowePZITJ457qtaaLTTUXDh66VIdzRIRERH3mqwpUyYzaNBgPv98E6tXf05EREdmzHiNuLh4MjO3EBcXz4wZr3mq1povOdkczVqwwOpKRERExGIuN1mnTp3kyy8/JyUlFYD69evTrFlzli1bSnLyWACSk8eSkfGxZyqtDcLCzDcNly6FI0esrkZEREQsZDt8+JTTlTd+8802/v3fJxIR0Ylvv91OZGRP/vSnaURGdmLXrgMAOJ1OOnRoU/a4vPT0Wcye/T4AeXk5zJs3z/VPcYXy8/MJCAjw6jbqnThBxKuvcrJHDw6OHu3VbbmiOjKoDZSDoRyUQSnlYCgHQzlcPoOHH36UFSvWVLkOu6sbLy4uYtu2rTz//EtERfVlypRJvPHGq5e8xmazYbPZKnx/amoaqalpAAwdmkhCQoKrpVwxh8NRLdvh2DFaLl7MDeHhEB7u/e1dhWrLoIZTDoZyUAallIOhHAzl4JkMXB4uDAkJIzQ0jKiovgCMGHEr27ZtpWXLlhw6lAfAoUN5BAYGulVgrXTHHdCoEaSnW12JiIiIWMTlJisoKIjQ0DB27foBgLVrHUREdGLIkGHMn/8BAPPnf0BS0nDPVFqbNGliJij96ivYts3qakRERMQCLg8XAjz//Evcf/+/UFhYSNu24cyY8SYlJSWMHz+OOXPSad26De+++76HSq1lfvMbMznpO+/A66+b6xuKiIiI33CryerevUeFJ34tWLDEndX6hvr14Xe/gxdegGXLYLgfHtETERHxY5rx3ZtiYiAyEv77v+H0aaurERERkWqkJsubbDa47z44exZmz7a6GhEREalGarK8rU0bGDHCDBnu3Gl1NSIiIlJN1GRVh5QUuO46eOMNKCqyuhoRERGpBmqyqkPDhnD//bBvHyxaZHU1IiIiUg3UZFWX6GiIjYW5cyEnx+pqRERExMvUZFWn++4zUzu8/jqUlFhdjYiIiHiRmqzqdO21MGECfPedhg1FRER8nJqs6paQAP37m7mz9u61uhoRERHxEjVZ1c1mgwcegMaN4dVX4cIFqysSERERL1CTZYVmzWDiRNizB2bNsroaERER8QI1WVaJjjYXkV6yBL74wupqRERExMPcbrKKi4sZNGgAKSl3ALBv316SkhKJjo5k/PhxFBYWul2kz0pLgw4dYPp0OHzY6mpERETEg9xusmbO/CsRERFlj6dOfYYJEx4kK2srzZo1Z86cdHc34bvsdpg0CZxOePFFUEMqIiLiM9xqsnJyslm58lNSUn4LgNPpZP36NYwYcSsAycl3kZHxsftV+rLgYHj0UfjhB/iv/zINl4iIiNR6dnfe/OSTf+Dpp/9Ifn4+AMeOHaNp02bY7Wa1oaFh5OXlVvje9PRZzJ79PgB5eTk4HA53Srki+fn51bIdV7Tq2ZNWH31E7tmzHI2N9dp2anIG1Uk5GMpBGZRSDoZyMJSDZzJwuclavjyDwMBAIiN7sWHDuqt+f2pqGqmpaQAMHZpIQkKCq6VcMYfDUS3bcUl8PFxzDYFZWTB8OPTs6ZXN1OgMqpFyMJSDMiilHAzlYCgHz2Tg8nBhVlYmn36aQVRUN+67L43169cyZcokTp06SVFREWCGE4ODQ9wq0G/YbGbYsE0beP55TVQqIiJSy7ncZD355LNs3bqDTZu2M3PmLAYMiOOtt/5GbGwcS5YsBmD+/LkkJQ33WLE+r2FDeOYZaNQInn0WfvzR6opERETERR6fJ+upp57jrbf+QnR0JMePHyMlJdXTm/BtgYGm0Tp7Fp57Dn46301ERERqF7dOfC8VGzuQ2NiBAISHt+PTTx2eWK3/atcOnnjCNFnPPgtTp5qjXCIiIlJraMb3mqpnT5g82UztMHWq5tASERGpZdRk1WS//jU88ghs3w5/+hMUFFhdkYiIiFwhNVk1XUKCuZj0li0Xz9USERGRGk9NVm0weDA8/jjs2AFPPQWnT1tdkYiIiFRBTVZtMXCgORl+925zf+KE1RWJiIjIZajJqk2io82QYW4uPPYY7N9vdUUiIiJSCTVZtU3PnvDCC3DhghlC3LTJ6opERESkAmqyaqMOHeDVVyEoyMyl9fHHVlckIiIiP6Mmq7YKDIQ//9kMIb79NsyYobm0REREahA1WbXZNdeYk+DHjIEVK8x5WgcPWl2ViIiIoCar9qtTB+65xwwbHjtmJi9dvdrqqkRERPyey01WdvZBRo0azoABfRk4MJqZM98E4PjxY4wePZJ+/XoyevRITpw47rFi5TJ69zZDhjfcYM7XevllzaclIiJiIZebLLvdznPP/Sfr139FRsZnvPfeO+zcuYMZM14jLi6ezMwtxMXFM2PGa56sVy7nuuvM5XdSUmD9erj/fvj8c6urEhER8UsuN1lBQcH06NETgICAJkREdCQ3N4dly5aSnDwWgOTksWRk6Jtv1apuXbjzTnjtNdN0vfACTJumyUtFRESqme3w4VNOd1eyf/8+Ro4cytq1X9KrV1d27ToAgNPppEOHNmWPy0tPn8Xs2e8DkJeXw7x589wto0r5+fkEBAR4fTs1ha24mMA1a2i1ejVOu51DN97I/u7dadysmdWlWc7f9oXKKAdlUEo5GMrBUA6Xz+Dhhx9lxYo1Va7D7oki7r33HqZOfZEmTZpe8pzNZsNms1X4vtTUNFJT0wAYOjSRhIQEd0upksPhqJbt1Cg33gjZ2fDOO7TKzOTar74i7LnnzKSmfswv94UKKAdlUEo5GMrBUA6eycCtbxdeuHCBe++9m9tvH8Mtt/wGgJYtW3LoUB4Ahw7lERgY6FaB4gFhYeZyPE8/ja242Fxk+pln4IcfrK5MRETEZ7ncZDmdTh5++EEiIjpy//3/VrZ8yJBhzJ//AQDz539AUtJw96sU99ls0LcvPzz8MNx7r2mwHn0Unn9e10AUERHxApeHCzMzv+Sjj+bRuXNXEhNjAZgy5WkmTnyE8ePHMWdOOq1bt+Hdd9/3VK3iAU67HUaNgiFD4H//FxYtgi+/hJgYuO026NjR6hJFRER8gstN1q9/HcPhw6cqfG7BgiUuFyTVpFEjuOsuuOUWWLwYli410z1062aarT59zNEvERERcYnbJ75LLdekiZkxfvRoWL7cNFx//KO5+HRSEtx0E+jbiCIiIldNTZYYDRvCyJEwfDh88QV88gn8/e8wZw707w+DBkFkJNi1y4iIiFwJ/cWUS9ntMHCguR04AMuWwapVsHatOaI1YAAkJJhztzScKCIiUik1WVK5X/0Kxo+HceNg82ZwOGDFCnP+VlCQOcIVHQ2dO5uZ5kVERKSMmiypWr160K+fuZ09a4YT166FJUvMtxMDAiAqyjzfq5d5LCIi4ufUZMnVadTIzCJ/441w7hx8/TVkZcFXX8GaNWYIsX176NHDnMPVpYs530tERMTPqMkS1zVsaIYM+/eHkhLYuRO2bjW30qNcderADTdAp07mPK5OnaBlS53PJSIiPk9NlnhGnTrm3KzOneHOO6GgAL77Dr75BrZvNyfQ/9//mdc2b24arnbtLt6Cg9V4iYiIT1GTJd7RoIG5CHXphaiLimDfPnO0a+dO+P57M8zodJrnr7kG2rY1DVebNtC6tbnmYmCgaeBERERqGTVZUj3sdrj+enMbNswsKygw103cuxf27DG3devgzJmL76tXD0JCTMMVFgatWplby5bm/pprLPk4IiIiVVGTJdZp0AA6dDC3Uk4nnDgB2dmQk2Pus7Ph4EFzcn1R0aXrCAi42HS1bAktWphb8+bmVvqzJlEVEZFqpr88UrPYbBcbpW7dLn2upASOH4fDh+HIkYv3R45AXp4596v8UbDyAgIuNlxNmhCWm2uOoAUEXHpr0uTiz40a6TwxERFxmdearFWrVjBlymSKi4u5++7fMnHio97alPiLOnXguuvMrXPnil9TWGiOhB0/ful9+Z8PHKDJrl3m6NiFC5Vvz2YzR9saNjS3a6659P7nPzdoYIY3S+/r1794+/nj8st0zpmIiE/ySpNVXFzM5MmP8dFH/0toaBg335zAkCHD6Nixkzc2J3JR/foXz9u6jB0OB8EJCaYpO30a8vPN/Zkzl/58/ryZD+zcuYs/nzoFhw5dXHb27MUT+F1Rt6652e2eubfZTONW/la6rPzzNhutdu40RwQv956KloH5ufRW/nFlP1f3666EzUbD7GzYtesXyyt7/dWs2+11XO163Fh3gyNHzH88XFmPD6l/5Ig5RaG61NCs6//4ozllozZq3NhcBq4G8EqTtXnzRtq1a094eDsARo26nWXLlqrJkpqnfv2LR8dc5XSaI2IFBaZpK3+7cOHifUHBxcc/f76oCIqLr/y+sLDi5SUl5lZcbOoqfVx6K7/M6aTV0aNmQlk/dv3RozBvntVlWK7D0aOQnm51GZaLUA7ATzn8/e9Wl+GaYcPg/vutrgLwUpOVl5dLWFjrsschIaFs3rzxktekp89i9uz3f3p9Dg6HwxulXCI/P79atlOTKQPDshxKhyAbNKj+bf+c00n+6dM0adwYSkqwOZ1lTZit/L3T+YvHZTfA9tO6SteJ04mt3DbKP1fRa11aR/nlFa3jCj8/wLlz52hY/qoEV3lUssLjEJWt42qPeFby+qs69nGFtZw7d44DlV2dwZ0jtbXMZXPwsJp5DMs4d+4cB2vp1ToKAgI454F/3z3xd8KyE99TU9NITU0DYOjQRBISEry+TYfDUS3bqcmUgaEcDIfDQbyf5+BwOIjy8wzA5NBfOSiHnzgcDmL8PAdP/J3wyhm3wcEhZGcfLHucm5tDSEioNzYlIiIiUiN5pcnq1SuK3bt3s2/fXgoLC1m0aAFDhgzzxqZEREREaiSvDBfa7XZefPElkpNHUVxczNix99CpUyVfuRcRERHxQV47J2vw4CEMHjzEW6sXERERqdE0C6KIiIiIF6jJEhEREfECNVkiIiIiXqAmS0RERMQLLJuMtLw9e/bQs2dvr2/n6NEfue66QK9vpyZTBoZyMJSDMiilHAzlYCiHy2dw4MC+K1qH7fDhU35zvYSbbopnxYo1VpdhKWVgKAdDOSiDUsrBUA6GcvBMBhouFBEREfECNVkiIiIiXlD38cf/41mri6hOkZG9rC7BcsrAUA6GclAGpZSDoRwM5eB+Bn51TpaIiIhIddFwoYiIiIgXqMkSERER8QK/aLJWrVpBTExvoqMjmTHjVavLqVZRUd2Ij/81iYmx3HRTPADHjx9j9OiR9OvXk9GjR3LixHGLq/S8hx56gC5d2hMX169sWWWf2+l08sQTjxMdHUl8fAzbtm2xqmyPqiiDP//5eXr06EhiYiyJibGsXPlp2XPTp79CdHQkMTG9WbVqpRUle0V29kFGjRrOgAF9GTgwmpkz3wT8a3+oLAN/2x/Onz/PkCEJJCT0Z+DAaKZN+08A9u3bS1JSItHRkYwfP47CwkIACgoKGD9+HNHRkSQlJbJ//5XNjVTTVZbD73//r/Tp071sf/jmm22Ab/5OlCouLmbQoAGkpNwBeH5f8Pkmq7i4mMmTH2Pu3AWsX/8VCxf+Dzt37rC6rGq1cOFSVq/eUDbfx4wZrxEXF09m5hbi4uKZMeM1iyv0vDvvTGHevIWXLKvsc3/22XJ27/4nmZlbeOWV6Uya9IgVJXtcRRkATJjwIKtXb2D16g0MHjwEgJ07d7Bo0QLWrcti3ryFTJ78KMXFxdVdslfY7Xaee+4/Wb/+KzIyPuO9995h584dfrU/VJYB+Nf+0KBBAxYs+BiH43NWrdrA6tUr2bgxi6lTn2HChAfJytpKs2bNmTMnHYA5c9Jp1qw5WVlbmTDhQaZOfcbiT+AZleUA8MwzU8v2h+7dewC++TtRaubMvxIREVH22NP7gs83WZs3b6Rdu/aEh7ejfv36jBp1O8uWLbW6LEstW7aU5OSxACQnjyUj42OLK/K8mJhYmjdvccmyyj53RsYnjBlzFzabjT59ojl58iSHDuVVe82eVlEGlVm2bCmjRt1OgwYNaNs2nHbt2rN580YvV1g9goKC6dGjJwABAU2IiOhIbm6OX+0PlWVQGV/dH2w2GwEBAQBcuHCBCxeKsNlsrF+/hhEjbgUgOfmusn3B7CN3ATBixK2sW+fA6az93xWrLIfK+OLvBEBOTjYrV35KSspvAXPEztP7gs83WXl5uYSFtS57HBISetl/XHyNzWZjzJhbGTw4jvT0WQAcOXKEoKBgAFojJc3JAAAEc0lEQVS1CuLIkSNWllhtKvvceXk5hIZe3EdCQ8N8eh95772ZxMfH8NBDD5QNkeXm5hAaGlb2mtDQMPLycq0q0Wv279/HN99sIyqqj9/uD+UzAP/bH4qLi0lMjKVLl+uJj08kPLw9TZs2w243V5kr/1nL//2w2+00adKUY8eOWVa7J/08h6iovgA8//wfiY+P4amn/kBBQQHgu78TTz75B55++o/UqWNaoWPHjnl8X/D5JsvfLVnyKZ99to65cxfw3nvv8MUXGy553mazXfZ/ML7KXz/3uHH/QlbWVlav3kBQUDDPPDPF6pKqTX5+Pvfeew9Tp75IkyZNL3nOX/aHn2fgj/tD3bp1Wb16A1u3fsfXX2/ihx++t7okS/w8h++++wdPPvksn3++ieXLHRw/fpw33vC9U0lKLV+eQWBgoNfnAvP5Jis4OITs7INlj3NzcwgJCbWwoupV+llbtmzJsGG3sHnzJlq2bFl2qPfQoTwCA/3jIqCVfe7g4FByci7uIzk52T67j7Rq1Yq6detSp04d7r77t3z99SbA7Cc5Odllr8vJySY4OMSqMj3uwoUL3Hvv3dx++xhuueU3gP/tDxVl4K/7A0CzZs2JjR3Ixo1ZnDp1kqKiIuDSz1r+70dRURGnT5/i2muvtaxmbyjNYdWqlQQFBWOz2WjQoAF33XV32f7gi78TWVmZfPppBlFR3bjvvjTWr1/LlCmTPL4v+HyT1atXFLt372bfvr0UFhayaNEChgwZZnVZ1eLMmTPk558u+9nhWEXnzp0ZMmQY8+d/AMD8+R+QlDTcyjKrTWWfOylpKB9+OBen08nGjVk0bdq0bBjJ15Q/j+KTT5bQqVNnwGSzaNECCgoK2LdvL7t376Z37z5WlelRTqeThx9+kIiIjtx//7+VLfen/aGyDPxtf/jxxx85efIEAOfOnWPNmtVEREQQGxvHkiWLAZg/f27ZvmD2kbkALFmymAED4n3iiGdFOXTo0KFsf3A6nWRkfEynTl0A3/ydePLJZ9m6dQebNm1n5sxZDBgQx1tv/c3j+4Ldex+hZrDb7bz44kskJ4+iuLiYsWPvKfuHxNcdOXKYceNSACguLuK22+5g0KCb6NmzN+PHj2POnHRat27Du+++b22hXjBhQhobNqzn2LGjREZ2YtKkJ5g48ZEKP/fgwUNYuXI50dGRNGrUiOnT37S2eA+pKIMNG9bx7bffADbatGnDyy9PB6BTp86MHDmKAQP6YrfbmTbtZerWrWvtB/CQzMwv+eijeXTu3JXExFgApkx52q/2h8oyWLjwf/xqfzh0KI/f//5fKS4uxuks4Te/GcXNNw8lIqITEyak8cILU+nePZKUlFQAUlJSefDB+4iOjqRFixa8/fYsiz+BZ1SWw2233cLRoz/idDrp2rU7L730OuCbvxOVeeqp5zy6L+iyOiIiIiJe4PPDhSIiIiJWUJMlIiIi4gVqskRERES8QE2WiIiIiBeoyRIRERHxAjVZIiIiIl6gJktERETEC/4f0siW9GyofIcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff8bcd54208>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = np.linspace(-10,20,100)\n",
    "y = x**2 - 6*x + 5\n",
    "\n",
    "x, y = (zip(*enumerate(cost)))\n",
    "\n",
    "fig, ax = plt.subplots(1, 1)\n",
    "fig.set_facecolor('#EAEAF2')\n",
    "plt.plot(x,y, 'r-', alpha=0.7)\n",
    "plt.ylim([-10,150])\n",
    "plt.gcf().set_size_inches((10,3))\n",
    "plt.grid(True)\n",
    "plt.show"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function matplotlib.pyplot.show>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlkAAADGCAYAAADhV5X4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xd4VNXWx/HvmclkMumkkEoJSCC0hBZAEIiCNBUpghcQKXIRQQERvPYCcpVrC94XFRUQRAEFVESKCEHRC4iIICoWehrpPZl23j+OBJBOZphksj7PMw/JZCazsh3Dj733WVs5ebJQRQghhBBCOJTO1QUIIYQQQrgjCVlCCCGEEE4gIUsIIYQQwgkkZAkhhBBCOIGELCGEEEIIJ5CQJYQQQgjhBJcMWampJxg4sD9du3bghhsSWbBgPgBz586hdeumJCV1ISmpC5s3b6x8TnLySyQmxtO5c1u2bNnsvOqFEEIIIaop5VJ9sjIzM8jMzKB16wSKi4vo2bMb7777AZ98shofH18mTXrgrMcfPPgrEyaMZePGrWRkpDNkyG3s2PEDer3eqT+IEEIIIUR1csmZrLCwcFq3TgDA19eP2NimpKenXfDxGzasY+DAwRiNRho0aEhMTCP27NntuIqFEEIIIWqAK9qTdezYUfbv30e7du0BWLhwAd27d2bKlPvIz88DID09jcjIqMrnREZGkZGR7sCShRBCCCGqP4/LfWBxcTFjx97FrFnP4+fnz+jR9zB9+sMoisLzz8/mqaceIzl5/mW/8JIli1i6dDEAv/32K/Xq1bvi4q+U3W5Hp5O9/o4i4+l4MqaOJePpeDKmjlXrx1NV8T5xAn15OaX16mHz8qryt7wWY5qdnc2vvx655OMuK2RZLBbGjh3J4MFDueWW2wCoW7du5ddHjrybkSOHAhAREUlaWmrl19LSUgkPjzjne44aNYZRo8YA0LdvErt3O39JMSUlhR49ejj9dWoLGU/HkzF1LBlPx5MxdaxaPZ6qCqNGwe+/w4oVMHSoQ77ttRjThIS2l/W4S0Y9VVWZOnUSsbFNmThxcuX9mZkZlR9//vlamjWLA6B3736sWbOKiooKjh49wqFDh2jbtv2V1i+EEEIId/bcc/Dee/Dssw4LWNXNJWeydu7cwYcfLicurgVJSV0AeOyxJ1m9+iMOHNgPKNSvX58XX0wGoFmzOAYMGEjXrh3w8PDghRdelCsLhRBCCHHa++/DE0/AyJHw+OOursZpLhmyOnXqzMmThefc37Nn7ws+Z9q0GUybNqNqlQkhhBDC/Xz9NYwZA927w9tvg6K4uiKnqcW77YQQQghxTf32G9x+O8TEwOrVYDS6uiKnkpAlhBBCCOfLzob+/UGng3XrICjI1RU53WW3cBBCCCGEuCrl5TBgABw/Dlu3QuPGrq7ompCQJYQQQgjnsdth9Gj49ltYuRI6d3Z1RdeMLBcKIYQQwnkefljrgzV3Ltxxh6uruaYkZAkhhBDCOf77X3jxRZg0CR56yNXVXHMSsoQQQgjheB9/DA88oO3FSk5261YNFyIhSwghhBCOtWMH/OMfkJioNR6tpU3JJWQJIYQQwnH++ANuvRWiomDtWvD2dnVFLiMhSwghhBCOkZkJvf86EWb9eggNdW09LiYtHIQQQghRdcXFWrPRjAzYsgWaNHF1RS4nIUsIIYQQVWM2w+DBsHcvfPIJdOzo6oqqBQlZQgghhLh6qgr33AObNsHChdpslgBkT5YQQgghquKRR2DpUpg9G8aMcXU11YqELCGEEEJcnXnz4IUX4L774NFHXV1NtSMhSwghhBBX7v33YcoUGDRIC1u1sNnopUjIEkIIIcSV2bAB7r4bevSAZctqbbPRS5GQJYQQQojLt2OHdiVhq1balYReXq6uqNqSkCWEEEKIy/Pzz9rVgxERWrNRf39XV1StScgSQgghxKUdOwY33wyenlq7hrAwV1dU7UmfLCGEEEJcXFaWFrCKi+Grr6BRI1dXVCNIyBJCCCHEhRUUaOcRHjumzWC1bu3qimoMCVlCCCGEOL/SUrj1Vti/Hz79FLp2dXVFNcol92Slpp5g4MD+dO3agRtuSGTBgvkA5OXlMmTIADp2TGDIkAHk5+cBoKoqjz46g8TEeLp378y+fXud+xMIIYQQwvHMZhgyBLZvh/feg759XV1RjXPJkOXh4cEzzzzH9u3fsX79lyxc+BYHD/7KvHmv0K1bd3bu3Eu3bt2ZN+8VAL78chOHDv3Jzp17eemlZGbOnOb0H0IIIYQQDmSzwahR2hWEb74Jw4a5uqIa6ZIhKywsnNatEwDw9fUjNrYp6elpbNiwjmHDhgMwbNhw1q//DID16z9n6NB/oCgK7dsnUlBQQGZmhhN/BCGEEEI4jKrCpEmwYgXMnQvjx7u6ohrrivZkHTt2lP3799GuXXuysrIICwsHoG7dMLKysgDIyEgjMjK68jmRkVGkp6dVPvaUJUsWsXTp4srnpKSkVOHHuDzFxcXX5HVqCxlPx5MxdSwZT8eTMXWsajeeqkrj11+n3ocfcnT4cA536ADVqb7LUJ3G9LJDVnFxMWPH3sWsWc/j53d28zFFUVCu8MyiUaPGMGqUdlp3375J9OjR44qefzVSUlKuyevUFjKejidj6lgyno4nY+pY1W48n34aPvwQJk+mwbx5NKiB5xFWpzG9rGakFouFsWNHMnjwUG655TYAQkNDK5cBMzMzCAkJASA8PJK0tBOVz01LSyUiItLRdQshhBDCkf7zH3jmGRgzBpKT5cBnB7hkyFJVlalTJxEb25SJEydX3t+7dz9WrHgfgBUr3qdPn/4A9OnTl5UrP0BVVXbv3oW/v/85S4VCCCGEqEbmz4eZM7UN7m+9BTo5EMYRLrlcuHPnDj78cDlxcS1ISuoCwGOPPckDD0xj/PjRLFu2hOjo+rz99mIAevbszebNm0hMjMfb25vk5PlO/QGEEEIIUQXvvqttdL/1Vli6FPR6V1fkNi4Zsjp16szJk4Xn/dqqVWvPuU9RFF544eWqVyaEEEII51qxAsaOhZ49YeVKMBhcXZFbkflAIYQQojZavRpGjIAuXeDjj8HLy9UVuR0JWUIIIURts3attv8qMRHWrQMfH1dX5JYkZAkhhBC1yfr12nE5bdpoH/v5uboityUhSwghhKgtNm+GgQOhRQvYuBECAlxdkVuTkCWEEELUBlu3wm23QWwsbNoEdeq4uiK3JyFLCCGEcHcpKdC/P8TEaLNZfzUQF84lIUsIIYRwZ9u2nQ5YW7ZA3bqurqjWkJAlhBBCuKtt26BfP2jYUAtYYWGurqhWkZAlhBBCuKNTAatBAwlYLiIhSwghhHA3ErCqBQlZQgghhDvZsgX69j0dsMLDXV1RrSUhSwghhHAXmzZpm9wbN9auKJSA5VISsoQQQggHm7Msn9bjUmkzPpV2E1LZ+UsFyasKKC23X/H3endjEWnZ1ks/8PPP4dZboWlTrSeWXEXoch6uLkAIIYRwJ//7uZx1O8r47vVIjJ4K2QU2zBaV4asLGdHTF+8rOIfZZlN5d2MxLRp6Ehlykb+yP/1UOyqnVSv44gsICqr6DyKqrPbMZKmqqysQQghRC2Tk2AgJ0GH0VAAICdCz6utS0nJs9JyewU3TMwCY9GoO970TQetxqTz9bl7l8xuPOM6/3sqlw71pLN9awve/mRn17yzaTUilrOI8M2GrV8PgwZCQoDUalYBVbdSOkJWaCl264PPnn66uRAghhJvr1d7EiSwrcXefYHJyDtt+LOf+gf5EBuvZ/FI4X76k7ZOaNTaQ+ePS+WFBJF/9WM6+Q+bK7xHsr+e7NyIZ0dOXdrGeLHkklO/fjMJk/Ntf28uWwdCh0KGDNoMlR+VUK7UjZJWUwPHjJEybBrt2uboaIYQQbszXpGPX/EhenxZMSKCO4bNP8u7GonMe9+G2Eu59O4L296bx81ELvxw9HbKG9vC59Au99RbcdRd066ZteJfDnqud2hGyYmPh66+x+vnBTTdp/UOEEEIIJ9HrFXokmHj67jrMuz+Y1V+XnvX1w+kWXv6wkP+MyOSHt6Lo19FEufn0thYfL+XiL5CcDP/8p9aqYd068PV1xo8hqqh2hCyAhg35ITkZ6tWDPn1gwwZXVySEEMINHTxu4fcTlsrP9/5hpkGYB34mHUWl2p6qwlIVHy8FHy87mXk2Nuwqu+D38/PWUVR2xl6sOXNg6lQYNAjWrAGTyWk/i6iaWnV1oTkkRJvFuvlmuO02WLECBg50dVlCCCHcSHGZnSn/zaWg2I6HHhpHGXhjWjDLt5TQ/5FMIoI9+PKlcBKu82TM61HENsji+pYXvuTw7pt9mfRqDiZPhe3Bb2J6fhaMGAGLF4NHrfprvMapff91QkO1Drj9+sEdd2hv0pEjXV2VEEIIN9Eu1sj2eRHn3D95oD+TB/pXfr5wZigpKQfo0aPHWY/7c1m9sz4f1M2HQV1NcP/98Px8GD8eXn8d9Hqn1C8cp/YsF56pTh1tk2C3btqmwf/+19UVCSGEEOdnscCoUTB/PsyYAW++KQGrhrhkyJoy5T6aN29Et24dK++bO3cOrVs3JSmpC0lJXdi8eWPl15KTXyIxMZ7OnduyZctm51TtCH5+WnfcAQO0fx3MmiW9tIQQQlQv5eVaD6xly7S9WC+8AMolNsWLauOSy4V33jmCceP+yeTJE866f8KESUya9MBZ9x08+Ctr1qzi6693kZGRzpAht7Fjxw/oq2vi9vKCjz6CcePgySchNxdeegl0tXOCTwghRDVSVKTtH962TZvFmjjR1RWJK3TJNNG5cxcCAy+vudmGDesYOHAwRqORBg0aEhPTiD17dle5SKfy8IBFi2DKFHj1VS1wWS/jjCghhBDCWbKy4MYb4euv4b33JGDVUFe98X3hwgWsXPkBCQlteOaZ5wgMrEN6ehrt2nWofExkZBQZGennff6SJYtYunQxABkZaaSkpFxtKZetuLj4wq8zYAANCguJWbSI7N9/5+cnnsBuNDq9pprsouMproqMqWPJeDqejKljnW88vTIyaD1zJsaTJ/l51ixyIiNBxvyyVaf36FWFrNGj72H69IdRFIXnn5/NU089RnLy/Cv6HqNGjWHUqDEA9O2bdM7VFc6QkpJy8ddJSoJ27Qi5/366zZ6tHbgpRxRc0CXHU1wxGVPHkvF0PBlTxzpnPH/6SWvPUFoKX35Jqy5dXFZbTVWd3qNXtfmobt266PV6dDodI0fezQ8/fA9AREQkaWmplY9LS0slPPzcy1irtUmTYPly2LlTu/owLc3VFQkhhKgNvvkGbrhB+/jrr0ECVo13VSErMzOj8uPPP19Ls2ZxAPTu3Y81a1ZRUVHB0aNHOHToEG3btndMpdfS0KGwfj0cOQLXXw+//ebqioQQQrizdeugVy+oW1cLWy1buroi4QCXXC6cMGEM33yzndzcHOLjmzFz5qN8883XHDiwH1CoX78+L76YDECzZnEMGDCQrl074OHhwQsvvFh9ryy8lJtu0tbA+/bV/jWxbh0kJrq6KiGEEO7mnXdgwgRISND+gR8a6uqKhINcMmS9+eaic+4bMWLUBR8/bdoMpk2bUbWqqot27bR/UfTure3XWrkS+vd3dVVCCCFqmPT8Mn48kU9uiZkgH0/iowOJCPCiwbvvaieP9O4NH36o9XAUbkMaQl1Kkybw7bcQF6c1Ln37bVdXJIQQogZJzy/ji58zKTPbCPE1Uma28cVP6aTfO4WYxYvh7rth7VoJWG5IQtblCA/Xlg579dLOjHrqKekOL4QQ4rL8eCIfPy8P/LwM6BQFP8WG3+uv8ePW7zg6cqTWq9FgcHWZwglq3wHRV8vXV2vpMGECPPssHD+unR8l/2MIIYT4mzOXBw+kFdAqKgA/DFBQALNn4fP772Tfcx+HE+vTQI7JcVsSsq6EwaBtUKxfH555RmvvsHIl+Ptf+rlCCCFqhVPLg35eHoT4GvHU6/juSB4dTWaCnp8N+XmUzHiEoM6JkH3Q1eUKJ5LlwiulKPD009rerM2btZ4mJ064uiohhBDVxN+XB+MiAlAzMvhl/hLs5WUUPT2LohbxxEcHurpU4WQSsq7WuHHw+edw+DB07Ag//ODqioQQQlQDuSVmfIynF4qCvv8fiQtfpcLPn+xnn8cU15RezcOICDS5sEpxLchyYVXcfLPW4qF/f21Ga+VK6NfP1VUJIYS4xs7cg3U0p4Ryi436Qd7w4UewdCnG+LZ0e3AMfTo1cXWp4hqSmayqatUKduyApk3h1lth/pWd4SiEEKJm+3uLhnB/E98fzuHYa29hX7qUoqSbKJo6nfhm0a4uVVxjMpPlCJGRsG0bDB+unX148CC89BJ4yPAKIYS7O3MPFkA9gxXWvkdGZj5ed44i6M5BdIoOlOXBWkhSgKP4+sKaNTBzJrz8snbe4fLlEBDg6sqEEEI4waklwg0/pRPub6JRqA9B+Vnw7LNEZedgnDSV4ZOHuLpM4UKyXOhIer02g7VggXbl4fXXaxvjhRBCuJUzlwgjArwoqrCwd+cBch9/BsrKKHlmNkE9uri6TOFiErKcYfx42LQJ0tO1Q6W3b3d1RUIIIRzozCXCmBA/1AO/oKz9jEPRTSiaM5ei6IbSokFIyHKapCTYuROCguDGG2HhQldXJIQQogrS88vY8FM67+88yte/Z1FhtYHNStDSd0hY8Rb+kWFk3D4MU2SYtGgQgOzJcq4mTbQrD4cN0/pq7dsHL74oG+KFEKKGOW8X91/T6bh2KUE/7CZo8CAMdwwn3suDPi0jXF2uqCZkJsvZ6tTRmpZOmwbJydC3L+TmuroqIYQQV+CcLu72ItTPPuOXfCv2adMoGjaCIrNNlgjFWWRK5Vrw8NCuOGzVCu69V9un9emn0Ly5qysTQghxARc85HnnToJefonEgBD23TWR7IQmBHnq6dQoWJYIxVkkZF1LY8ZoTUsHDYJOnWDJErj9dldXJYQQ4m/Ouzx4OJeOP24naOV70OQ6jDP+Rbe6IbI8KC5Ilguvteuvh+++g2bNYOBAePJJsNtdXZUQQghOb25/bcvvHM4uxmKza8uDgQbUL7fwy44fsd90E0XPzKbIy0+WB8VFSchyhXr14KuvtJmtWbO043jy811dlRBC1Gpn9r7SKaCgsPd4Abl/HiHoyUdI3LWRih43kj1mAiZvL7mCUFySLBe6ipcXvPMOdOgADzyg/fnxx9CihasrE0KIWunMze3+JgMVVjumY4c58tmnBBUVYXz4X3Rr1VyWB8Vlk5ksV1IUmDgRtm6F4mLo2BFWrnR1VUIIUWuct/cV0LCOibLt36J+8gmFYZEUvfASRQ0ay/KguCISsqqDrl3h++8hPl7rqTV1KpjNrq5KCCHc2pnLg5Wb24/kkZueTdALz5GwbgVq69aoAwdiCg+V5UFxxS4ZsqZMuY/mzRvRrVvHyvvy8nIZMmQAHTsmMGTIAPLz8wBQVZVHH51BYmI83bt3Zt++vc6r3N1ERkJKihawkpO1jvGpqa6uSggh3NY5va8iAlCzsvgl+S3sv/6CYdw4YoYNYHLPZvRpGSEBS1yxS4asO+8cwfLlq8+6b968V+jWrTs7d+6lW7fuzJv3CgBffrmJQ4f+ZOfOvbz0UjIzZ05zTtXuymCAV16BFSvgxx+hbVvYssXVVQkhhFs5tUS44ad0DmYUkVtSAagEpWwi8a0XqTAYyX72BUw9e8jslaiSS4aszp27EBhY56z7NmxYx7BhwwEYNmw469d/BsD69Z8zdOg/UBSF9u0TKSgoIDMzwwllu7mhQ7U2D8HB0KsXzJkjbR6EEMIBzlwijAjwoqjCwt5DWeS+mAxvLsDYojndZtzD8CFdZPZKVNlVXV2YlZVFWFg4AHXrhpGVlQVARkYakZHRlY+LjIwiPT2t8rHiCsTFwa5d8M9/wmOPaS0fli6F0FBXVyaEEDXKmZ3bj+aUEObvhZ+XgZgQP/b+8AfKpo0cykrHcPcYim7qTadm8neWcIwqt3BQFAVFUa74eUuWLGLp0sWAFs5SUlKqWsolFRcXX5PXcajx44mIiKDJa69had6cnx9/nIL4eFdXBdTQ8azmZEwdS8bT8WramOaW2diTZcPbQ8FLDwcybRzUqTTy19Fg3/fEb97Gn2ENOXBjHwxNQokpOMTBvUc5eI3qq2njWRNUpzG9qpAVGhpKZmYGYWHhZGZmEBISAkB4eCRpaScqH5eWlkpEROR5v8eoUWMYNWoMAH37JtGjR4+rKeWKpKSkXJPXcbikJLjrLoxDh9LmwQdh9mx4+GHQufbi0Bo7ntWYjKljyXg6Xk0b0w0/pdMuzIaflwEAa0AuBQUleKV8SdQXn0CbNnhOnkLb4ECX9L+qaeNZE1SnMb2qv6V79+7HihXvA7Bixfv06dMfgD59+rJy5Qeoqsru3bvw9/eXpUJHSUiA3bvhjjvg0Uehb1/IkP1uQghxPuff3A4Ni7Kwf7SKrKOp2EeMoOhfj1FkMEn/K+EUl5zJmjBhDN98s53c3Bzi45sxc+ajPPDANMaPH82yZUuIjq7P228vBqBnz95s3ryJxMR4vL29SU6e7+z6axd/f/jgA21ma+pUra/WkiXQu7erKxNCiGrjzMOdIwK8KCy3sPdYHgm/7CLovUXERjUmY9hdZMc3JcjoQafGIbLBXTjFJUPWm28uOu/9q1atPec+RVF44YWXq16VuDBFgQkTtAamw4ZBnz7w0EPw3HPg6enq6oQQwmVObXD/6rcsjB464iL8tc3tv6ehfLmFQ7//giHxevSjxjK6fYwEK+F00vG9pmrRQmvzMHEivPgidOkCf/zh6qqEEMIlLnS4Mwf2kzB/Lv6HfiPjpn6YZk6nlwQscY3IAdE1mckE8+dDz54wbpy2b2vePBgzRpvxEkIIN3ah1gz+JgMVFRZM337NkV07aeupYJgynvjGMXK4s7imZCbLHQwaBPv2QWKiFraGDIGcHFdXJYQQTvP3cwdzS8z8nllMbkkFDcvzKVv+Ier3eyjs0Imif/+HotBI2dwurjkJWe6iXj3YvBnmzoW1a6FVK9i0ydVVCSGEQ526avC1Lb9zOLsYi82OTlEI9TOiU+DIhq8IemQGCcd+Qr19EGrXrph8vOR4HOESErLciU4HM2ZoneIDA7WrDqdMgbIyV1cmhBBVdqF9V7klFTRUKrB/9hlZ3+zCnhCP4bk5xHRoweSkJnI8jnAZ2ZPljhIS4PvvtYal8+bBxo1aq4fERFdXJoQQV+x8Vw36mwxUWO2YDDqObNtF26X/R6x3HTJuu4PspOsJ8vWkU3SghCvhUhKy3JXJpAWs227TNsJffz088gg88YS0ehBC1Bhn9rw6c/aqYbA3R1Kz8dqWQuGhIxQ1aoJ+7HhG39BSgpWoNiRkubuePWH/fq156ezZsG6dNqvVsqWrKxNCiAu6+OyVnvwfD5CwYjG/eIeidu6KaciNdKofJAFLVCuyJ6s2CAyExYthzRpITYW2bbXmpRaLqysTQohzXGjvVaDJk7LCUtT16yhcux6Dvx8xE0Yx+cEh9GkdJQFLVDsSsmqT22+Hn36CgQPh8cehY0f48UdXVyWEEMD5rxz0NxlQdGizV3t/IuH/5qD+9jtqh0RMzz9Hr55tJVyJaktCVm0TGgorVsCqVdqsVvv28PTTYDa7ujIhRC128dmrEm326rMNGAICiJlwN5Nn3kmf+GgJWKJak5BVWw0aBD//rJ1/+Mwz0KGDdkyPEEJcQxefvdKRv/sHEl6bg/rbH6iJiZj+PZtevWT2StQMErJqs+BgeO89+OQTyM6GTp1g2jQoLnZ1ZUKIWuCis1e5Bahr1lC44UsMocHETLybyTNk9krULBKyhNbm4eefYcIEePVV7crDDRtcXZUQwo2l55ex+NvD7E8t4PfMYhQFbfbKQyH/q/+RMG82amqq1rF99rP0uqmNhCtR40gLB6EJCNAOmx4+HMaPh759YcQIePllqFvX1dUJIdzAqbYMh7NKOJpbSlGZmQbBPlRY7RSV2VBzswjevoXC7FwM111HzMjR9OoaJ+FK1FgykyXO1rUr7N0LTz4JK1dC06awYAHY7a6uTAhRg525NFhYbkavg8JyKzklFZgUG8F7/kfAxytRi4pQe/bE9PgjErBEjSczWeJcRqO2Gf7OO2HiRG0ZcdEieOMNiI93dXVCiBrkfE1FiytsBJo8sQVC6u9H8P3fFox5ORS260yrW3vQq12MhCvhFiRkiQuLi4OtW7XN8dOnQ7t22oHTTz8Nfn6urk4IUU39fVkwNsz3rI3tep1KeU4uwVs2Y07PxWjSk3XnXQTFREvAEm5FQpa4OEWBu+6C/v3h0UfhlVfggw/gP/+ByEhXVyeEqGbOPGvw1LLgHydL0Ov+2tiuU7Hs/I6yvXspR6FuYlua3JJEuEWlV/MwCVjCrUjIEpcnKEhbLhw7FiZPhpEjSWjdWjsHUZYQhRCcvmIwt8RCqK+RzMJyIgK8KbfYsdhslP3+J17btmIvKadJw7ocvKEP/vVCMJk86dQkUAKWcDsSssSVSUyEHTtg0SK8p0/XzkGcOBGefVYLYkKIWuViVwxmFZkx6HWEVBRj3vI1Cbu+5JembVEH3ExEjwT6REuwEu5NQpa4cjodjBvHrrp16bppk9b64YMPtM3yEyaAweDqCoUQ18D5lgZPXTEY6mciyltH6o69eOz/Dj+7GcOdw4jpdiO9WkVKuBK1grRwEFfN6ucHr70GP/wACQlw//3a0uHGja4uTQjhZH9vJppZWE6gyZOoQG/S8soo272HoHffxv+3n7HFNcf/gUmYBtwiAUvUKlWayWrXriW+vr7odHo8PDz44ott5OXlMn78GI4fP0q9eg14++3FBAbWcVS9ojpq3Ro2b4ZPP9WuQuzTB/r1gxdf1K5QFEK4hfT8Mr7PsPLL1j8uvDSYmYr5m90YT/xJVuNmRHe/ntG3d5RgJWqlKs9krV69jq1bv+GLL7YBMG/eK3Tr1p2dO/fSrVt35s17pcpFihpAUWDAADhwQAtX27dDq1Zw772QkeHq6oQQVZCeX8YHO48y5/Nf+DXPSmp+ydmdWLm3AAAbm0lEQVTNRD31RNnLSNu8nfyPP6NuSQ5NRg6k1fh/SMAStZrDlws3bFjHsGHDARg2bDjr13/m6JcQ1ZnRqM1m/fEHTJoE77wD112n7deSg6eFqFHODFebDmRgMigowMHMYvSKoi0NZuZTtvYzgt5biH92OrauN+A/eSKm6zvRq3m4BCxRqyknTxaqV/vk9u1bERAQiKIojBo1hlGjxnDddfX444/jAKiqSpMm9Ss/P9OSJYtYunQxABkZaSxfvvxqy7hsxcXF+Pr6Ov11aovLGU9Taioxb71F3W3bMNepw5HRo0nv1w/VQ665OB95jzqWjOfVyy2zsSfLxsliOyhwosiOTlEIMtgotOlQzFZaHjlAcWoW4cV5HGkRj67pddzUxESQSe/q8msMeY863rUY06lTH6xcwbuYKv1Nt3btRiIiIsnKyuKOOwbQpEnsWV9XFAVFUc773FOhDKBv3yR69OhRlVIuS0pKyjV5ndrissdzxAjYsQPPGTOIfeUVYj/5RGv5cOedoJdfxmeS96hjyXheuVMtGXb/loUxUIev0UxEgDce2cWUVNgozcsmPi+Tn49lY888RlhsI2JvG0WEb6A0E70K8h51vOo0plVaLoyI0Dp+h4aG0q/fLezZ8z2hoaFkZmp7cDIzMwgJCal6laLm69QJvvoK1q3TjuQZOVK7IvHTT0G96slUIYSDnLk0uPNQDmVmKwoKWUVmsovLqetjwH70GB4/7EP/7XZivBVsQ4fhP/h2TJFhErCEOI+rDlklJSUUFxdVfpySsoW4uDh69+7HihXvA7Bixfv06dPfMZWKmk9RtKsO9+zR+mqVl2ub5Tt31to+SNgS4po7374rnaIju9hMidlClL8naT/9gX7RIiJ2bsNg8CB70DDajB3ClCEduS+pCX1aRkjAEuI8rnq5MCvrJKNHjwDAZrMyaNAd3HhjLxIS2jJ+/GiWLVtCdHR93n57saNqFe5Cp9OWCgcPhsWLYfZsre1D587a4dO9emmBTAjhNOn5ZaQcPMm3f+ZQXG4h1M+TnBI76QUVmDwNRAUYST3wB01++IYQs4q9ji/lPa6nUbCeCQNukFAlxGW46pDVsGEMKSnfnnN/UFAwq1atrVJRopYwGGD8eBg1ChYtgueeg969oUsXLWzddJOELSEc7GLhyqhXsNtUTu77lUbfbcNs1aMG+ODd43o69mxLfHQgB/fulIAlxGWSS7yE6xmNWj+tMWO0lg9z5mizWR07wmOPwS23SNgSogr+fr6gzWb7K1xVnBWuOHoc+6HDFFSYKTP5EdC5MzGJLc9qxXDQxT+LEDWJhCxRfRiNcN99MHastoz4wgtw221aR/lHH4UhQ+RqRCEuw5mhKjW/lPSCcuoHeWOx2dDrFP7MKiUu3J9Ak4GSsgo4fBz74cNYzFaivSCrXVuy60XQuXEQPZrKhnYhrpaELFH9eHlpM1vjxmkb5P/9b20PV2wsPPQQ3HWX9hghxFnOXAo0GfSUW2zklZqx2OyUVtg4lFNMXLg/AV4GUjMLqHfkF/48ehKL1U60j46s+HaUR4Vzs4QrIRxCQpaovgwGbb/WiBGwZg08/zz885/wxBPwwAMwcSLUkXMxhQD48VgeH3x3nKM5JZgMOnKLy7GjoKp2fI2eFFZYtHCVmk30Hz/x88lS9Om/EREbR1bTVpSHh3JzIwlXQjiShCxR/en12lLh4MGQkgJz52p7tebMgXvu0QJXo0aurlIIlzg1e7Vqzwm8PfXYVTs6xYPM4jLCfI2UWVUwqpRl5tDgt338XGTHmnuCpk2bYL95NOWePhKuhHASCVmi5lAUSErSbvv2aQdR/9//wbx5Wr+tqVOhW7datUne8+YjtIoxVH4+tIcPD/8j8LKf33jEcXbOjyQk4Oy9bqqq0mtGJqufqYunAXpMy8BsUbHaYFA3b56+W5tB/HJPGf9akIddVfHx0rFwZgjXRRl4cH4u234sA6C0XOVkvo2cTxpcVk3JqwoY398Pby/HHa26/5CZVz4qYOHMUId9T1c5334ru13Fbrfjqfcku9iMQWfFx6CnqMyMT0EeJXuP4p2dhb4sj5iENtj6dKd+vRAahfoSHx0o4UoIJ5GQJWqm1q1hyRJtv9b8+fDmm/Dxx1oX+SlTtD1ctWDflslT4fs3oxz+fT/fWUbrRp74++hQVZXNL4bja9Jhsap0m5pOnw4mOjX3YnJyDqufrUtcA09e/6SQOcvyWTgzlJfvC6r8Xv9dU8jeP8yX/drzVhcyoqcv3g78z9eqkScnsmwcy7RSP6xm/tq72H4rs81OuVWl3GIl2NtIdmExwdmZpOaVEJidSqABjPEtyW7UiM5N68qslRDXSM38bSPEKVFRWn+txx+HZcvg1Ve1VhAPPaRdpXjvvbVyKbHxiOPcdbMv6/5XhsWqsvzJUJrV9ySnwMaIOVmkZdvo1Nx4wSb7H3xZwj39tQNWFUXB16TNDlqsKlYrlWeSKgoUlmrfpKDETkTwub9Slm8t4am7z51dKymzc+esLFKzrdjs8NiIQDLzbaTl2Og5PYPgAD1fvhTOpt1lPPNuPmaLSqNID96ZEYKvSUfjEccZ0t2HjbvK8DIqvPdoKNdFGfhoWwmzluaj14FqCeeHHtrr3dLZmxUpJcwYFlDF0b12Ts1a7Tuez48nCigzWwn0Npyz3yq3pAJ/kwc5WflEZRwhICsPxWLGPzyKel06EBXfhEZ1/WTWSohrTEKWcA8mk7Y/a9w42LoVXn8dXn5ZW1Ls00drDdG3r9u1gCgzq7SbkFr5+cN3BjI0yQeAEH89370RyeufFPLyh4UsmB7CrKX5dGnpxRN3BbJuRykL1xef9/t+e6Cc16cFV35us6kk3pfGH6lWJg7wo2OcEYA3p4dw66OZmIwK/t46vnkt4qzvczTTypEMKzcmnDsttfG7MiKD9aydEwZAQbGdAF8dr35UwOaXwgkJ0JNdYGPOsnw2zQ3Dx6Rj7vICXvmokCfu0kJbgI+OvW9HsXRTMQ/Oz+XT58KYvTSfz58PIyrEg882fAU0A6BdrCdzlxdU65B1vqXAYB8jh7KL8PTQU1hhxWTQk1lsPr3fSm/BeDITjxPH8CmrQLVWoNRvQFRcIx7q1Yr4+nJxiBCuIiFLuBdFgRtv1G6pqfD227BgAdx6K0RHa7NbY8ZAw4aurtQhLrZcOPAGbwDaxhpZs70UgK/3l/PhU3UB6N/Jmzp+59/3lFtkx8/79Nf0eu118ottDH4qi58Om2kZ40nyqkLWzgmjY5yRF1cU8NAbuSyYfvpQ+BVbSxh8gzd6/bn75Fo28mTGm7n8661c+nfy5oZW5waxHb9U8MtRC92mpgNgtkCn5sbKr9/5V6C880Yfpr+eC8D1LY2MnZvNHd19CD3jZesG6knPsZ33560OTl0dWFRmoaDMQoVV62lVbraSU2yhYbABH4Oe4oq/9lvl5OOTlUFJZgbepSUEGyGraTw+jRvQU5YEhagWJGQJ9xUVBU89pTUyXbsW3noLZs3Sbr16aTNft92mNUF1Q0aDljD0OrDaruzwbQ+9gt2uotOdHY4CffX0SPBi43dlhNXRs+9Pc+Ws1tAePvR/JPOsx6/cWsK8B4I4n9hoA9+9Hsn6XWU8uSiPG9uYKmeoTlFV6NnOxLLHzr9h/cxrHE59PH9qCDt/qeDznaU8tTaSrl1sBAfoKTereHlWr4siTu2z2nk4h/0nCgj29cTboEOv05FXZiHM10h+uQU/Lw9yS8wEeyocP3aSsIwjpGIisDiHwPAgjLFtKa8TKv2thKhmJGQJ92cwwKBB2u3YMe2cxHfegaFDIShI2yQ/ahQkJrr9lYk3tPLigy0lPDYykPW7Sskrsp/3cbHRHhxKt3JdlIGsfBsGDy1glVXY2fx9GTPuDKCOn46CEju/nbAQG21g854ymtU/faXjr8fM5BXb6Nz8/CE2LdtKkL+OET19CfDVsfDzIgD8TDqKSu2EBOjpFGfkgXk5/JFq4booAyVldlJzbMRGa6+zMqWEh/8RyMqUksoZrj/TLHSMM9IxzsiHX57keJaV4AA9v5+w0PKMKzGvtb8vBZ4sKiejoAId4O2pQ1GgoNRKptVKoxBfbbaq3IqiqvgXF5B5Mp/A1ENE2m3YAwLxjwqlXov2RIUFylWCQlRTErJE7VK/vja79fjj8MUX2hWKCxdqVyg2bXq6+WmDy2s34Gp/35N1c3sT/x5//pkjgCfuCmTEnCxaj0ulc3Mj9euef49av07ebPuxnOuiDKTn2hj7QjY2u4pdhSHdfbilk7YU+eaDwQx9+iQ6HQT66nj7obOXCof28KncJP93+w9b+NeCXHQ6MOgV/jtF2wN2T38/+j+SSUSwB1++FM47M0MY+VwWFRZtNu7ZMXUqQ1ZesZ0241MxGhTe+2u26+EFefxxwoIKtIiuIL6xJwApP5bTt6P35QyrQ6Xnl/HJDyfY/OtJ7HYVq82OClRY7djtNlD05JWZ8TUaUIByC9qslaWUw+n5RKYfxmaxEmQ0Ym/YEP+G0fiFh/BQh/qy30qIak45ebLwytYRnKBv3yR2797t9NdJSUmhR48eTn+d2sJtxrOgAD76SAtcX32l3Xf99doM19ChEBZ2zUqpLmOanmNl9AvZbJwb7upSLuhCPb7OdGo8K8wqSQ+m81VyBB7n2R/maGcuAx7KKqGo3EyonxflFhsniyowGfQoiorFBkHenuSXmvE2GlAqyqnIy0eXX0BobgYWDwMN/TzIj2lK3euiiQr2dfmsVXV5j7oLGU/HuxZjmpDQli++2HbJx8lMlhABAdpViePGweHDsHy5dmbiAw9oDU5vvBGGDdManobW/GaWlyMi2IN7+vlRWGLH38dxTUFd5dhJK3PuqeO0gHUqVO1PLSCn2Ex6filWO+h0UG62UFxhQ6+zoNOp6HW6yv1udtUO5eV45eVjLSwkOCeDEqMXnj4+KM2b0651DAmxEbIUKEQNJSFLiDPFxMAjj2i3AwdOB67x42HCBOjeXdvbNXCgtrHejd3Rw8fVJVzUn8vqXfZjm0QbaBLtmP1YZwaq0gobZWYLR/NKKTerBPsayCmpoNRsw8egR9FpS7oGvZ4Kiw2Dhw4PBWzlZgxlxXgXFFJcXoan3UKwUYc9tglh0eHclFBfNrAL4QYkZAlxIS1aaFciPvss/PgjrFql3e6/X7t16qS1hrj1VmjZ0u03zddmp4JVysGT/JpRiNUG4QFGyi02MgorAJUgk4GMggrKrVY8dDrsKujR4aGzY1dVLGUVGEuLUIqKKVP0eJeXEKuUkBbVCEudIOpG1aFjjFwdKIQ7kZAlxKUoinZcT0KCFrp++UULWx9/rB1U/dhjWt+tW2+FW27Rzk+sBUf6uKsz91Ol5ZdRVK71rVLtYLXbAQUPvcKJ3DJUQAdYVQUbCigqCjpsdhWb1YaptBh9URF5ZpU6xQV4qWZUv0B8AvwIq9+YgLqBdIn0l2AlhJuSkCXElYqL065OfPxxSEuDzz473Yfrtde07vM9ekDv3lq3+dhYmeWqpi4WqHR6BVWFonILdlVFh4IdFZ1Oh7+HgXK71tjUoFNQ7HbMVjtGSwW2kjKs5RVYysrwLM3HhBVraDReDevTuFEYHZuGSagSopaQkCVEVURGwj//qd1KS7UjfTZu1G5Tp2qPadDgdBf6pCS338tVHf14LI8Pvz/O/tQCSiqsqKqdcoudogorOhSMfzUAPTNQAeh0yl/5WMGOtvynqipmqw29AnaLFSxmDOXl2MvL0JcVoRp98fHUEVDHG49GEdj9AhgaF8rtbepJsBKilpGQJYSjeHtD//7aDbQrFTduhE2btKXFRYu0+2NjtbDVrRt07ar17hJVdua+qT+ziykutVBSbkGfsoFyiw29Hox6PRa7SrnF/tdsFShAqdmGp4fyV4d7LVAB2FUVPQqnTtLWqTbsFisVJRa8S4uxqmBXFMLMhRj9fMkLaUgdfx+aRQcSG+7v8nYLQgjXkpAlhLPExMC992o3u13bPL91K2zZAu+/D2++qT2uXj0tbHXpgq+Hh9ajy9PTtbVXM6caem44kMGx3FJKzVbsfyUhnaKtxtrsnHWfTQUVsFttKIDOBhbFRmVmUqhslKqgYrGrGBVdZaBCVcGmotgteJitqDYLqs2GyWJGNRjw8DISaDLSICoQrzqB+Hp50FL2VwkhzuC0kLVlyxc89tjD2Gw2Ro68mwceeNBZLyVE9afTQZs22u3BB8Fmg/37Yft27bZtG3zwAe0BpkyBtm21Y346dIB27aBJE9BfuOlmdfXQ8t18tDfz0g90tL9yknLGDQXsfwWsU59zRitm1QaKtQIPmwXVYgO7Dftf4c3fZkZnMmL18cPT20izekEkNZNDmIUQF+eUkGWz2Xj44el8+OEnREZGcfPNPejdux9NmzZzxssJUfPo9aevWJw8Wfub/+hRDixaRIuSEti1S9tIn5ysPd7bG1q31h7fpg20agXNm2uNVKuRhCfWkW9xdRWnnbr6D/7KVOrpPz1UG3qbFbtNWzpUFDs2qxV/Sxk6oxGrtxeKp4lAfyO+Piai6pikxYIQ4oo4JWTt2bObmJhGNGwYA8DAgYPZsGGdhCwhLkRRoGFDspKStCsTAaxWrSHqDz/A3r3a7YMP4I03Tj8vOloLWy1aQLNm2oxXkybahnzdte3UXt0CFoCCDT3axKGCDcWmoqh2rDo9WCsw2q2oBgM2L0+8FE9CAoz4BvhKoBJCOIRTQlZGRjpRUdGVn0dERLJnj/PPJhTCrXh4QHy8djtFVeHIEfjpJy2A/fyz9ucbb0BZ2enHmUxw3XXQuLF2deOZt/r1ITjY4SHMdQHLdtYfoAUpVVG0TVqqHS+rBdXDgKIHT70ef09PFC8fVL2e8AAvkprK1X9CCMdz2cb3JUsWsXTpYgAyMtJISUlx+msWFxdfk9epLWQ8He+yx9TPT+s436mT9rndjvHkSUypqXinpmI6cQJTaiqmvXsxbtyIx5kBDLDr9ZiDgzEHBVEREoI5KAhLQAAWf38sAQFY//rY6uODzccHq7c3dqPRyf2+bJd117lObVS3o7fbUVBR9Xr0ehWDHgx6T0wGE+G+EF/XQOtQPUGmM/e3WYA0Du5N42DVfwi3J//fO5aMp+NVpzF1SsgKD48gNfVE5efp6WlERESe9ZhRo8YwatQYAPr2Tbomp5DLaeeOJePpeE4ZU1WFvDw4elS7HTuGLj0dr/R0vNLSID0dfv0VcnNPX553PjqdFu68vbWO9mfePDyg40OOrfti9CoeKOjQo0N7eQ8PHSZPD+oFeVfOTB3cu1Peow4m/987loyn41WnMXVKyGrTph2HDh3i6NEjREREsmbNKt544x1nvJQQ4lIUBYKCtFubNhd+nN0O+fmQna3dcnKgsFC7FRWd/risDMrLz77ZbASWF5DvdRUb8c+5aPL8V1F6Ktp5gf1bRzDq+kaXtbQnM1NCCFdySsjy8PDg+ef/w7BhA7HZbAwffhfNmsU546WEEI6i050OY7GxV/z0vVx68/v93RsyvW+Lq69RCCFqEKftyerZszc9e/Z21rcXQlRDe2f1d3UJQghRbVzba7yFEEIIIWqJanGszuHDh0lIaOv018nJySY4OMTpr1NbyHg6noypY8l4Op6MqWPJeDretRjT48ePXtbjlJMnC9VLP8w99OrVnS++2ObqMtyGjKfjyZg6loyn48mYOpaMp+NVpzGV5UIhhBBCCCeQkCWEEEII4QT6GTMeedrVRVxL8fEX6RMkrpiMp+PJmDqWjKfjyZg6loyn41WXMa1Ve7KEEEIIIa4VWS4UQgghhHACtw9Zn366hhtuSCQsLIC9e/ec9bXk5JdITIync+e2bNmy2UUV1mxz586hdeumJCV1ISmpC5s3b3R1STXSli1f0LlzWxIT45k372VXl+MW2rVrSffunUhK6kKvXt1dXU6NNGXKfTRv3ohu3TpW3peXl8uQIQPo2DGBIUMGkJ+f58IKa5bzjaf8Dr16qaknGDiwP127duCGGxJZsGA+UL3eo24fspo1a86iRcvo3LnLWfcfPPgra9as4uuvd7F8+WoefvhBbDabi6qs2SZMmMTWrd+wdes30uX/KthsNh5+eDoffLCK7du/Y/Xqjzh48FdXl+UWVq9ex9at31Sby7lrmjvvHMHy5avPum/evFfo1q07O3fupVu37syb94qLqqt5zjeeIL9Dr5aHhwfPPPMc27d/x/r1X7Jw4VscPPhrtXqPun3Iio1tynXXNTnn/g0b1jFw4GCMRiMNGjQkJqYRe/bsdkGForbbs2c3MTGNaNgwBk9PTwYOHMyGDetcXZYQdO7chcDAOmfdt2HDOoYNGw7AsGHDWb/+M1eUViOdbzzF1QsLC6d16wQAfH39iI1tSnp6WrV6j7p9yLqQ9PQ0IiOjKj+PjIwiIyPdhRXVXAsXLqB7985MmXKfLB1chYyMdKKiois/j4iIJD09zYUVuQdFURg69HZ69uzGkiWLXF2O28jKyiIsLByAunXDyMrKcnFFNZ/8Dq26Y8eOsn//Ptq1a1+t3qPV4lidqho8+DaysjLPuf+RR56kb185sLaqLja+o0ffw/TpD6MoCs8/P5unnnqM5OT5LqhSiLOtXbuRiIhIsrKyuOOOATRpEnvOtgFRNYqioCiKq8uo0eR3aNUVFxczduxdzJr1PH5+/md9zdXvUbcIWatWfXrFz4mIiCQtLbXy87S0VMLDIxxZltu43PEdOfJuRo4c6uRq3E94eASpqScqP09PTyMiItKFFbmHU2MYGhpKv363sGfP9xKyHCA0NJTMzAzCwsLJzMwgJETO3auKunXrVn4sv0OvnMViYezYkQwePJRbbrkNqF7v0Vq7XNi7dz/WrFlFRUUFR48e4dChQ7Rt297VZdU4mZkZlR9//vlamjWLc2E1NVObNu04dOgQR48ewWw2s2bNKnr37ufqsmq0kpISiouLKj9OSdlCXJy8Nx2hd+9+rFjxPgArVrxPnz6yWlAV8jv06qmqytSpk4iNbcrEiZMr769O71G3b0a6bt1aHn10Bjk52fj7B9CyZStWrvwYgFde+Q/vv78UDw8PZs9+nptuutnF1dY89903ngMH9gMK9evX58UXkyvXwsXl27x5I48//i9sNhvDh9/FtGkzXF1SjXbkyGFGjx4BgM1mZdCgO2RMr8KECWP45pvt5ObmEBpal5kzH6Vv3/6MHz+aEyeOEx1dn7ffXkydOkGuLrVGON94fvPN1/I79Crt2PE/brutN3FxLdDptDmjxx57krZt21eb96jbhywhhBBCCFeotcuFQgghhBDOJCFLCCGEEMIJJGQJIYQQQjiBhCwhhBBCCCeQkCWEEEII4QQSsoQQQgghnEBClhBCCCGEE0jIEkIIIYRwgv8Hw4zLJi0LWbsAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff8bcce54e0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = np.linspace(-10,20,100)\n",
    "y = x**2 - 6*x + 5\n",
    "\n",
    "fig, ax = plt.subplots(1, 1)\n",
    "fig.set_facecolor('#EAEAF2')\n",
    "plt.plot(x,y, 'r-')\n",
    "plt.ylim([-10,250])\n",
    "plt.gcf().set_size_inches((10,3))\n",
    "plt.grid(True)\n",
    "plt.plot(mins,cost,'o', alpha=0.3)\n",
    "ax.text(start,\n",
    "        start**2 - 6*start + 25,\n",
    "        'Start',\n",
    "        ha='center',\n",
    "        color=sns.xkcd_rgb['blue'],\n",
    "       )\n",
    "ax.text(mins[-1],\n",
    "        cost[-1]+20,\n",
    "        'End (%s steps)' % len(mins),\n",
    "        ha='center',\n",
    "        color=sns.xkcd_rgb['blue'],\n",
    "       )\n",
    "plt.show"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## From derivatives to gradient: $n$-dimensional function minimization.\n",
    "\n",
    "Let's consider a $n$-dimensional function $f: \\Re^n \\rightarrow \\Re$. For example: \n",
    "\n",
    "$$f(\\mathbf{x}) = \\sum_{n} x_n^2$$\n",
    "\n",
    "Our objective is to find the argument  $\\mathbf{x}$ that minimizes this function.\n",
    "\n",
    "The **gradient** of $f$ is the vector whose components are the $n$ partial derivatives of $f$. It is thus a vector-valued function. \n",
    "\n",
    "The gradient points in the direction of the greatest rate of **increase** of the function.\n",
    "\n",
    "$$\\nabla {f} = (\\frac{\\partial f}{\\partial x_1}, \\dots, \\frac{\\partial f}{\\partial x_n})$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3.000000 [2.0000000001, 2.0000000001, 2.0000000001]\n",
      "3.000000 [2.0000009999, 2.0000009999, 2.0000009999]\n"
     ]
    }
   ],
   "source": [
    "def f(x):\n",
    "    return sum(x_i**2 for x_i in x)\n",
    "\n",
    "def fin_dif_partial_centered(x, \n",
    "                             f, \n",
    "                             i, \n",
    "                             h=1e-6):\n",
    "    '''\n",
    "    This method returns the partial derivative of the i-th \n",
    "    component of f at x\n",
    "    by using the centered finite difference method\n",
    "    '''\n",
    "    w1 = [x_j + (h if j==i else 0) for j, x_j in enumerate(x)]\n",
    "    w2 = [x_j - (h if j==i else 0) for j, x_j in enumerate(x)]\n",
    "    return (f(w1) - f(w2))/(2*h)\n",
    "\n",
    "def fin_dif_partial_old(x, \n",
    "                        f, \n",
    "                        i, \n",
    "                        h=1e-6):\n",
    "    '''\n",
    "    This method returns the partial derivative of the i-th \n",
    "    component of f at x\n",
    "    by using the (non-centered) finite difference method\n",
    "    '''\n",
    "    w1 = [x_j + (h if j==i else 0) for j, x_j in enumerate(x)]\n",
    "    return (f(w1) - f(x))/h\n",
    "\n",
    "def gradient_centered(x, \n",
    "                      f, \n",
    "                      h=1e-6):\n",
    "    '''\n",
    "    This method returns the gradient vector of f at x\n",
    "    by using the centered finite difference method\n",
    "    '''\n",
    "    return[round(fin_dif_partial_centered(x,f,i,h), 10) for i,_ in enumerate(x)]\n",
    "\n",
    "def gradient_old(x, \n",
    "                 f, \n",
    "                 h=1e-6):\n",
    "    '''\n",
    "    This method returns the the gradient vector of f at x\n",
    "    by using the (non-centered)ç finite difference method\n",
    "    '''\n",
    "    return[round(fin_dif_partial_old(x,f,i,h), 10) for i,_ in enumerate(x)]\n",
    "\n",
    "x = [1.0,1.0,1.0]\n",
    "\n",
    "print('{:.6f}'.format(f(x)), gradient_centered(x,f))\n",
    "print('{:.6f}'.format(f(x)), gradient_old(x,f))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The function we have evaluated, $f({\\mathbf x}) = x_1^2+x_2^2+x_3^2$, is $3$ at $(1,1,1)$ and the gradient vector at this point is $(2,2,2)$. \n",
    "\n",
    "Then, we can follow this steps to maximize (or minimize) the function:\n",
    "\n",
    "+ Start from a random $\\mathbf{x}$ vector.\n",
    "+ Compute the gradient vector.\n",
    "+ Walk a small step in the opposite direction of the gradient vector."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> It is important to be aware that gradient computation is very expensive: if $\\mathbf{x}$ has dimension $n$, we have to evaluate $f$ at $2*n$ points.\n",
    "\n",
    "\n",
    "### How to use the gradient.\n",
    "\n",
    "$f(x) = \\sum_i x_i^2$, takes its mimimum value when all $x$ are 0. \n",
    "\n",
    "Let's check it for $n=3$: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "def euc_dist(v1,v2):\n",
    "    import numpy as np\n",
    "    import math\n",
    "    v = np.array(v1)-np.array(v2)\n",
    "    return math.sqrt(sum(v_i ** 2 for v_i in v))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's start by choosing a random vector and then walking a step in the opposite direction of the gradient vector. We will stop when the difference between the new solution and the old solution is less than a tolerance value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[-9, -2, -9]"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# choosing a random vector\n",
    "\n",
    "import random\n",
    "import numpy as np\n",
    "\n",
    "x = [random.randint(-10,10) for i in range(3)]\n",
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[-0.0, -0.0, -0.0]\n"
     ]
    }
   ],
   "source": [
    "def step(x,\n",
    "         grad,\n",
    "         alpha):\n",
    "    '''\n",
    "    This function makes a step in the opposite direction of \n",
    "    the gradient vector \n",
    "    in order to compute a new value for the target function.\n",
    "    '''\n",
    "    return [x_i - alpha * grad_i for x_i, grad_i in zip(x,grad)]\n",
    "\n",
    "tol = 1e-15\n",
    "alpha = 0.01\n",
    "while True:\n",
    "    grad = gradient_centered(x,f)\n",
    "    next_x = step(x,grad,alpha)\n",
    "    if euc_dist(next_x,x) < tol:\n",
    "        break\n",
    "    x = next_x\n",
    "print([round(i,10) for i in x])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Alpha\n",
    "\n",
    "The step size, **alpha**, is a slippy concept: if it is too small we will slowly converge to the solution, if it is too large we can diverge from the solution. \n",
    "\n",
    "There are several policies to follow when selecting the step size:\n",
    "\n",
    "+ Constant size steps. In this case, the size step determines the precision of the solution.\n",
    "+ Decreasing step sizes.\n",
    "+ At each step, select the optimal step.\n",
    "\n",
    "The last policy is good, but too expensive. In this case we would consider a fixed set of values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "step_size = [100, 10, 1, 0.1, 0.01, 0.001, 0.0001, 0.00001]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Learning from data\n",
    "\n",
    "In general, we have:\n",
    "\n",
    "+ A dataset $(\\mathbf{x},y)$ of $n$ examples. \n",
    "+ A target function $f_\\mathbf{w}$, that we want to minimize, representing the **discrepancy between our data and the model** we want to fit. The model is represented by a set of parameters $\\mathbf{w}$. \n",
    "+ The gradient of the target function, $g_f$. \n",
    "\n",
    "\n",
    "In the most common case $f$ represents the errors from a data representation model $M$. To fit the model is to find the optimal parameters $\\mathbf{w}$ that minimize the following expression:\n",
    "\n",
    "$$ f_\\mathbf{w} = \\frac{1}{n} \\sum_{i} (y_i - M(\\mathbf{x}_i,\\mathbf{w}))^2 $$\n",
    "\n",
    "For example, $(\\mathbf{x},y)$ can represent:\n",
    "\n",
    "+ $\\mathbf{x}$: the behavior of a \"Candy Crush\" player; $y$: monthly payments. \n",
    "+ $\\mathbf{x}$: sensor data about your car engine; $y$: probability of engine error.\n",
    "+ $\\mathbf{x}$: finantial data of a bank customer; $y$: customer rating.\n",
    "\n",
    "> If $y$ is a real value, it is called a *regression* problem.\n",
    "\n",
    "> If $y$ is binary/categorical, it is called a *classification* problem. \n",
    "\n",
    "Let's suppose that our model is a one-dimensional linear model $M(\\mathbf{x},\\mathbf{w}) = w \\cdot x $. \n",
    "\n",
    "### Batch gradient descend\n",
    "\n",
    "We can implement **gradient descend** in the following way (*batch gradient descend*):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import random\n",
    "\n",
    "# f = 2x\n",
    "x = np.arange(10)\n",
    "y = np.array([2*i for i in x])\n",
    "\n",
    "# f_target = 1/n Sum (y - wx)**2\n",
    "def target_f(x,y,w):\n",
    "    return np.sum((y - x * w)**2.0) / x.size\n",
    "\n",
    "# gradient_f = 2/n Sum 2wx**2 - 2xy\n",
    "def gradient_f(x,y,w):\n",
    "    return 2 * np.sum(2*w*(x**2) - 2*x*y) / x.size\n",
    "\n",
    "def step(w,grad,alpha):\n",
    "    return w - alpha * grad\n",
    "\n",
    "def BGD_multi_step(target_f, \n",
    "                   gradient_f, \n",
    "                   x, \n",
    "                   y, \n",
    "                   toler = 1e-6):\n",
    "    '''\n",
    "    Batch gradient descend by using a multi-step approach\n",
    "    '''\n",
    "    alphas = [100, 10, 1, 0.1, 0.001, 0.00001]\n",
    "    w = random.random()\n",
    "    val = target_f(x,y,w)\n",
    "    i = 0\n",
    "    while True:\n",
    "        i += 1\n",
    "        gradient = gradient_f(x,y,w)\n",
    "        next_ws = [step(w, gradient, alpha) for alpha in alphas]\n",
    "        next_vals = [target_f(x,y,w) for w in next_ws]\n",
    "        min_val = min(next_vals)\n",
    "        next_w = next_ws[next_vals.index(min_val)]   \n",
    "        next_val = target_f(x,y,next_w)    \n",
    "        if (abs(val - next_val) < toler):\n",
    "            return w\n",
    "        else:\n",
    "            w, val = next_w, next_val"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.999618\n"
     ]
    }
   ],
   "source": [
    "print('{:.6f}'.format(BGD_multi_step(target_f, gradient_f, x, y)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4.88 ms ± 353 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "BGD_multi_step(target_f, gradient_f, x, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "def BGD(target_f, \n",
    "        gradient_f, \n",
    "        x, \n",
    "        y, \n",
    "        toler = 1e-6, \n",
    "        alpha=0.01):\n",
    "    '''\n",
    "    Batch gradient descend by using a given step\n",
    "    '''\n",
    "    w = random.random()\n",
    "    val = target_f(x,y,w)\n",
    "    i = 0\n",
    "    while True:\n",
    "        i += 1\n",
    "        gradient = gradient_f(x,y,w)\n",
    "        next_w = step(w, gradient, alpha)\n",
    "        next_val = target_f(x,y,next_w)    \n",
    "        if (abs(val - next_val) < toler):\n",
    "            return w\n",
    "        else:\n",
    "            w, val = next_w, next_val"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.000065\n"
     ]
    }
   ],
   "source": [
    "print('{:.6f}'.format(BGD(target_f, gradient_f, x, y)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "127 µs ± 9.75 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "BGD(target_f, gradient_f, x, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Stochastic Gradient Descend\n",
    "\n",
    "The last function evals the whole dataset $(\\mathbf{x}_i,y_i)$ at every step. \n",
    "\n",
    "If the dataset is large, this strategy is too costly. In this case we will use a strategy called **SGD** (*Stochastic Gradient Descend*).\n",
    "\n",
    "When learning from data, the cost function is additive: it is computed by adding sample reconstruction errors. \n",
    "\n",
    "Then, we can compute the estimate the gradient (and move towards the minimum) by using only **one data sample** (or a small data sample).\n",
    "\n",
    "Thus, we will find the minimum by iterating this gradient estimation over the dataset.\n",
    "\n",
    "A full iteration over the dataset is called **epoch**. During an epoch, data must be used in a random order.\n",
    "\n",
    "If we apply this method we have some theoretical guarantees to find a good minimum:\n",
    "+ SGD essentially uses the inaccurate gradient per iteration. Since there is no free food, what is the cost by using approximate gradient? The answer is that the convergence rate is slower than the gradient descent algorithm.\n",
    "+ The convergence of SGD has been analyzed using the theories of convex minimization and of stochastic approximation: it converges almost surely to a global minimum when the objective function is convex or pseudoconvex, and otherwise converges almost surely to a local minimum."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  0   0\n",
      "  1   2\n",
      "  2   4\n",
      "  3   6\n",
      "  4   8\n",
      "  5  10\n",
      "  6  12\n",
      "  7  14\n",
      "  8  16\n",
      "  9  18\n",
      "\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "x = np.arange(10)\n",
    "y = np.array([2*i for i in x])\n",
    "data = zip(x,y)\n",
    "\n",
    "for (x_i,y_i) in data:\n",
    "    print('{:3d} {:3d}'.format(x_i,y_i))\n",
    "print()\n",
    "\n",
    "def in_random_order(data):\n",
    "    '''\n",
    "    Random data generator\n",
    "    '''\n",
    "    import random\n",
    "    indexes = [i for i,_ in enumerate(data)]\n",
    "    random.shuffle(indexes)\n",
    "    for i in indexes:\n",
    "        yield data[i]\n",
    "        \n",
    "for (x_i,y_i) in in_random_order(data):\n",
    "    print('{:3d} {:3d}'.format(x_i,y_i))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import random\n",
    "\n",
    "def SGD(target_f, \n",
    "        gradient_f, \n",
    "        x, \n",
    "        y, \n",
    "        toler = 1e-6, \n",
    "        epochs=100, \n",
    "        alpha_0=0.01):\n",
    "    '''\n",
    "    Stochastic gradient descend with automatic step adaptation (by\n",
    "    reducing the step to its 95% when there are iterations with \n",
    "    no increase)\n",
    "    '''\n",
    "    data = list(zip(x,y))\n",
    "    w = random.random()\n",
    "    alpha = alpha_0\n",
    "    min_w, min_val = float('inf'), float('inf')\n",
    "    epoch = 0\n",
    "    iteration_no_increase = 0\n",
    "    while epoch < epochs and iteration_no_increase < 100:\n",
    "        val = target_f(x, y, w)\n",
    "        if min_val - val > toler:\n",
    "            min_w, min_val = w, val\n",
    "            alpha = alpha_0\n",
    "            iteration_no_increase = 0\n",
    "        else:\n",
    "            iteration_no_increase += 1\n",
    "            alpha *= 0.95\n",
    "        for x_i, y_i in in_random_order(data):\n",
    "            gradient_i = gradient_f(x_i, y_i, w)\n",
    "            w = w - (alpha *  gradient_i)\n",
    "        epoch += 1\n",
    "    return min_w"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "w: 2.000000\n"
     ]
    }
   ],
   "source": [
    "print('w: {:.6f}'.format(SGD(target_f, gradient_f, x, y)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise: Stochastic Gradient Descent and Linear Regression\n",
    "\n",
    "The linear regression model assumes a linear relationship between data:\n",
    "\n",
    "$$ y_i = w_1 x_i + w_0 $$\n",
    "\n",
    "Let's generate a more realistic dataset (with noise), where $w_1 = 2$ and $w_0 = 0$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Once deleted, variables cannot be recovered. Proceed (y/[n])? y\n"
     ]
    }
   ],
   "source": [
    "%reset\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "\n",
    "import numpy as np\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.datasets.samples_generator import make_regression \n",
    "from scipy import stats \n",
    "import random\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function matplotlib.pyplot.show>"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnMAAADZCAYAAABLqLavAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3XlcVXXi//HXZRFRU8CNe7FyKcVdUzHSBBRFTVOwxdEkHb/lTGM5aS7ZOtV8s7Gm5VfT9u1ro18nG+NCmbkjbpWlaLtakZosLriCG8L9/XHmsiQqKPeeA7yfj0eP5Nxz7/ncx2fI95zl/bEdOHDchYiIiIhUSz5mD0BERERELp/CnIiIiEg1pjAnIiIiUo0pzImIiIhUYwpzIiIiItWYwpyIiIhINWZamMvM3Ed8/C307duLm2+O4K23/nHePi6Xi9mzpxMR0ZWoqEi+/np78WuLFi2kd+9u9O7djUWLFnpz6CIiIiKW4Wfagf38+Mtf/kqXLt3IyztBbGw/oqL6065dePE+a9asJCPjZzZv3s7WrV8yY8aDLF++liNHDvP888+xalUaNpuN2NgoBg8eSlBQsFlfR0RERMQUpp2Za948lC5dugHQoMFVtG3bjuzsrDL7LFv2CXfc8TtsNhs9e0Zw7Ngx9u/PYe3aNURFxRAcHEJQUDBRUTGkpq4242uIiIiImMoS98zt3buHb775mh49epbZnpOThcPRovhnhyOM7OwssrOzCQsLK7XdQXZ2ttfGKyIiImIVpl1mdcvLy+P3vx/H00/P4aqrGlb558+fP48FC94FYNeuHVx99dVVfozSioqK8PGxREaW/9CcWJPmxXo0J9akebm0vAIXR0+7OOcCPxsE1bXRwN92xZ976pQveXl+nDjhz7lzxufVq1dI/fpnCQoqxGbz7Iqohw4dYseO3Zfcz9QwV1BQwO9/fxejRt3BsGG3nvd6aKiDrKx9xT9nZWVitzuw2+1s2rSx1PYs+vTpW+4xEhMnkJg4AYAhQ2LYsmVLFX+LstLS0oiOjvboMaRyNCfWpHmxHs2JNWleLi5lWyYPO7/Bv6CweFugvy/PJnRmZPewi7zzfAUFkJYGTickJ8P+/VCnDgweDKNGwfDh0Lix9+akW7cbKrSfaVHf5XLx5z//ibZt2/HHP04ud5/Bg4fw73+/h8vlYsuWL2jYsCHNm4cSEzOAdetSOXr0CEePHmHdulRiYgZ4+RuIiIiI2eau2MmpUkEO4FRBIXNX7KzQ+0+dgo8+grvvhubNYdAgWLAA+vWDRYvg4EFYsgTGjzeCnBWZdmZu8+bPWbx4Ee3bdyQmpg8AjzzyOPv2GWfixo+fSGxsHKtXryQioiv16tXj5ZeN+pLg4BCmTp3BoEHRAEybNpPg4BBTvoeIiIiYJ+voqUptBzhxAj75BJKSjH/n50NwMNx6KyQkwMCBEBjoqRFXPdPC3I03RnLgwPGL7mOz2Xjuub+X+9qYMeMYM2acJ4YmIiIi1YQjKJDMcoKbI6hsGsvNNc7AOZ2wahWcOWOciRs3zghw0dHg7++lQVcx0x+AEBEREblc0+Pa8bDzmzKXWgP9fZke147sbOPeN6fTuBeusBCuvRbuu88IcJGR4Otr3tirisKciIiIVFvuhxzmrthJ1tFThBQF0yW/I3MnN+Kzz8DlgnbtYOZMI8DdcAPYrvxBV0tRmBMREZFqrW1AGEPOheFcCenbIB3o3h2eesoIcB06mD1Cz1KYExERkWrF5YL0dOPyqdMJO3YY22+6CZ5/HuLjoXVrc8foTQpzIiIiYnlFRfDppyUBbs8e43636Gi4/34YORIcDrNHaQ6FOREREbGk0iW+KSmQk2OU+A4aBE88YVSJWLX7zZsU5kRERMQyTp0yqkOcTqNK5MgRqF8fhg417n8bOhQaVv3qn9WawpyIiIiYyl3i63TCR0uKOH3KB5+AAhp3OMSsx+rw+B8aV6sSX29TmBMRERGvy801lslyOmHlSqPEt1FIIXXbZ9LwumzqXpOLzddFymFfeu+o/DqrtYnCnIiIiHhFdrZx71tSUkmJ7zXXwB//aCxkP2P9OrKOl13Nwb3OqsLchSnMiYiIiMf88ouxCkNSEsUlvm3bwowZRoArXeKb/XHl11kVhTkRERGPS9mWWbxCgSMokOlx7Wr0maYffjAunyYlwbZtxrZu3UpKfNu3L38VhoqusyplmRbmpky5j1WrltOkSVPWr9983uuvvvoySUn/BqCw8By7du3khx8yCA4OoUePTjRo0AAfH1/8/PxYtWqdt4cvIiJSISnbMsusHZp59BQPO78BqDGBzuUyQltSUtkS38jIypX4XmydVbkw08Lc6NFjmTjxXiZPnlTu65MnT2Hy5CkArFixjDfffI3g4JDi153OpTRWuYyIiFjc3BU7y4QTqBn3gRUVGZdN3QGuKkp8f7vOam04i1kVTAtzkZF92Lt3T4X2TU5eTHz8bR4ekYiISNW70P1e1fE+sIICWLfOCHDllfgOHw5NmlzZMUZ2D1N4qyTbgQPHXWYdfO/ePdx11x3lXmZ1O3nyJN26hbN58/biM3M9e3amUaMgbDYbiYkTSEyccMH3z58/jwUL3gUgJyeLRYsWVel3+K28vDwaNGjg0WNI5WhOrEnzYj2aE8+YlnaS3NPn/1XbuK6NF6LrXfL9Zs/L2bM+fPllMBs2NOXTTxtz4oQ/desW0rt3Lv36HaJ371zq1y+89AfVIN6akz//eWqFbiWz/AMQK1cuIyLixjKXWJcsWYHd7uDgwYPcfvsIrr++LZGRfcp9f+mwN2RIDNHR0R4db1pamsePIZWjObEmzYv1aE4847FGmeXeB/bYiM5EV+AMlBnzUrrE95NPIC8PgoKMe98SEmDQIF8CA5sBzbw6Lquw2u+K5cNccnLSeZdY7XbjInzTpk0ZOnQY6elbLxjmREREzFRd7gM7fNhYPqt0iW+zZjB2rBHgoqONS6piPZYOc8ePH+Ozzzbyj3+8XbwtPz8fl6uIBg2uIj8/n7S0VB56aKaJoxQREbk4q94H5i7xdTph7dqyJb4JCXDTTcZDDWJtpoW5SZMmsGnTRg4fzqVr13BmzJhNQUEBAOPHTwTgk08+Jjq6P/Xr1y9+38GDBxg/fixgVJYkJNxO//4Dvf8FREREqqHdu43w5nTCp5+WLfFNSIAePcrvgBPrMi3MvfnmvEvuM3r0WEaPHltmW8uWrUhL+9RTwxIREalx3CW+TiekpxvbunWDv/zFCHAdOijAVWeWvswqIiIilecu8XUHuB9+MLZHRsLcuUaAq0iJr1QPCnMiIiI1gLvE1x3gdu827neLioI//cko8Q2z3m17UgUU5kRERKopd4mv02ksZu8u8R04EB57DG699cpLfMX6FOZERESqkdOnYdUqI8B99JFRKVKvHgwdCqNGGf9u2NDsUYo3KcyJiIhY3IkTsGyZsYyWu8S3USPjzNuoUcZyWoGBZo9SzKIwJyIiYkGHD8OSJfDWW53YurWkxHfMGCPAqcRX3BTmRERELCInxyjxTUoqKfFt3ryBSnzlohTmRERETLR7t/HwQlJS+SW+J058TkxMtNnDFAtTmBMREfGyHTuM8Fa6xLdr1/JLfNPSTBumVBMKcyIiIh7mcsH27SUB7rclvvHx0KaNuWOU6kthTkRExAOKiuDzz0sC3O7d4ONjPLigEl+pSj5mHXjKlPvo0KE1/fr1Lvf1TZs20KZNC2Ji+hAT04fnn59T/Fpq6ioiI28gIqIrr7zyd28NWURE5KIKCmD1arjvPiOo9ekDr74KHTvCO+/A/v2wZo0R5hTkpKqYdmZu9OixTJx4L5MnT7rgPjfeGMnChYvLbCssLGTmzGksXvwhDkcYgwZFExc3lHbtwj09ZBERkfNcrMQ3IcH4d6NGZo9SajLTwlxkZB/27t1T6felp2+hVavWtGzZCoD4+FEsX75UYU5ERLzGXeLrdMLSpWVLfBMSIC5OJb7iPZa+Z27Lli+Ijr6J0NBQnnzyr4SHtycnJ5uwsBbF+9jtDtLTt1zwM+bPn8eCBe8CkJOTRZqHHwvKy8vz+DGkcjQn1qR5sR7NycUdP+7Hp582ZsOGpnz5ZQgFBT4EB58lOvoQN998kO7dj+Lv7wJg8+aqO67mxXqsNieWDXNdunRl69bvaNCgAatXr+Duu3/H5s3bK/05iYkTSEycAMCQITFER0dX8UjLSktL8/gxpHI0J9akebEezcn53CW+TqdR4nvuHFx9tXFPXEIC9OlTB19fB+Dw2Bg0L9ZjtTmxbJi76qqSVYJjY+OYOXMaubm5hIbayczcV/xadnYWdrvnfolERKR2cZf4Op2waZNRK3L99fDQQ8YyWj16lHTAiViBZcPc/v37adasGTabjfT0LRQVFRESEkKjRj3IyMhgz57d2O0OkpOTeOONd8weroiIVGM7dhjhLSmpbInvk08aAa50ia+I1ZgW5iZNmsCmTRs5fDiXrl3DmTFjNgUFBQCMHz+Rjz9O4d1338HX14/AwLq8+eY8bDYbfn5+zJkzlzvvjKewsJAxY8YRHt7erK8hIiLVkLvE1x3g3CW+N94If/ubcQlVJb5SXZgW5t58c95FX584cRITJ5ZfWxIbG0dsbJwnhiUiIjWUu8TX6TT++eUXo8Q3Ksq4By4+Xt1vUj1Z9jKriIjIlSoogPXrjfCWnAzZ2eDvDwMHwqOPGlUiTZqYPUqRK6MwJyIiNcrp08YqDElJZUt8hwwx7n9Tia/UNApzIiJS7eXlGSW+SUnll/gOGmQEOpGaSGFORESqpSNHYMkSI8CtWAFnzkCzZjBmjBHgYmKgTh2zRynieQpzIiJSLGVbJnNX7CTr6CkcQYFMj2vHyO7WeSrgQiW+f/iDu8QXfH3NHqWIdynMiYgIAJ9mFbBgzTecKigEIPPoKR52fgNgaqDbs6fkCdTflvgmJEDPnuqAk9pNYU5ERABI2lXAqQJXmW2nCgqZu2Kn18Pczp3G5VOnE7ZuNba5S3wTEqBjRwU4ETeFORERASD3tKvc7VlHT3n82KVLfJ1O+P57Y7u7xDc+Hq67zuPDEKmWFOZERASAxnVt5QY6R1CgR453sRLfP/4RRo6EFi08cmiRGkVhTkREABjV1p8FPxQW3zMHEOjvy/S4dmX2u5KHJM6dg3Xryi/xfeQRo0qkadMq/VoiNZ7CnIiIAHCTw58O7TtcNKilbMvkYWflHpJwl/g6nfDhh2VLfBMS4JZbVOIrciVMC3NTptzHqlXLadKkKevXbz7v9Q8+eJ//9/9eAlzUr9+Av/3tRTp16gxAjx6daNCgAT4+vvj5+bFq1Tovj15EpGYa2T3somfZ5q7YWebMHZT/kIS7xNfpNEp8T5wwAtvw4UaAi4tTia9IVTEtzI0ePZaJE+9l8uRJ5b5+zTUt+fDDTwgKCmbNmpU89NADLF++tvh1p3MpjRs39tZwRUSECz8MkXX0VHGJr9NplPiePm1cMh092ghw/furxFfEE0wLc5GRfdi7d88FX4+I6F385x49epGVleWNYYmIyEU4ggLJLBXoCvMCOPlTc4oyHDR7wbgnrkULuPdeYx1UlfiKeF61uGdu4cIFDBgwsPhnm83GHXeMxGazkZg4gcTECSaOTkSk9pge146H/vdHDn/flJO77JzZFwzYsF99jknTjACnEl8R77J8mNu4cT3/+td8lixZUbxtyZIV2O0ODh48yO23j+D669sSGdmn3PfPnz+PBQveBSAnJ4u0tDSPjjcvL8/jx5DK0ZxYk+bFei42J3v3BrJhQ1PWr2/Kz7uiAfBvehx79I/cPvgQIyPOYrNBfr7xtKpUHf2uWI/V5sTSYe67777lwQcns2hREiEhJffH2e0OAJo2bcrQocNIT996wTBX+szdkCExREdHe3TMaWlpHj+GVI7mxJo0L9ZTek5cLvjqK+P+t6SkkhLf3r1Ll/g2BBoCbc0acq2g3xXrsdqcWDbM7dv3KxMmjOW1196mTZvri7fn5+fjchXRoMFV5Ofnk5aWykMPzTRxpCIiNUNREXz2WckyWu4S3379VOIrYmWmhblJkyawadNGDh/OpWvXcGbMmE1BQQEA48dP5IUXnuPIkSPMnDnVGOh/KkgOHjzA+PFjASgsPEdCwu307z/wgscREZELO3cO1q83Atz770eSm6sSX5HqxrQw9+ab8y76+osvvsqLL7563vaWLVuRlvapp4YlIuJxV7KCQlU4c8Yo8U1Kgo8+gtxco/OtZ8/jTJrUVCW+ItWMZS+ziojURJezgkJVyMuD5cuNAHehEt8vvvjOUvcBiUjFKMyJiHhRRVdQqApmlviaffZRpDZRmBMR8aKLraBQFfbvh5QUI8ClppYt8U1IgL59PV/ia9bZR5HaSmFORMSLfruCQuntl2vPHkhONgLcxo1Grch118G0aUaA69XLuyW+3jz7KCKXEea+//47tm3byoED+zlz5jRBQcG0aXMdvXr1Jigo2BNjFBGpMabHtStz1gog0N+X6XHtKvU5O3ca4c3phC1bjG1dusATTxgBrlMn81Zh8PTZRxEpq0JhbvfuX3j33XdISvo3Bw8ewMfHh0aNGlGnTgDHjh3j1KmT+Pj4cNNNfRk7NpGRI0fh4+Pj6bGLiFQ77jNTlb2frHSJr9MJ331nbO/dG557zijxvf76i36E13ji7KOIXNglw9yDD04mKenfREREMm3aTHr16k14eHt8S910kZuby/btW1m7dg1PPfU4c+c+y4svvsaNN0Z6dPAiItXRyO5hFbrcWFQEmzeXBLiMjJIS31deMQKcFUt8q+rso4hUzCXDXN26ddm0aQtXX33NBfdp3LgxAwYMYsCAQTz11LN89FEyOTlZVTpQEZHawF3i63Qa98FlZRklvrGx8PDDMGKE9Ut8L/fso4hcnkuGuWeffb5SH+jj48PIkaMue0AiIrWNu8TX6YQPPzRKfAMDYcgQ4/63YcOqX4lvRc8+isiVq9QDEI8+OpOnn56Dzay7akVEagh3ia/TCR9/bJT4NmxolPiOGmWU+NarZ/YoRaQ6qFSYczo/YO/ePbzxxv9Sr5z/yqxZs5IBAwZV2eBERGqSI0eM4JaUVFLi26QJ3HmnEeA8WeIrIjVXpR45XbZsDb/8ksGttw5m//6c4u2pqasZMqQ/Y8feUeUDFBGpzvbvh7feMs60NWsGiYmwdatR4puWBtnZ8PbbMHiwgpyIXJ5Khblrr23J0qWrCAkJIS4uhn/9awG33DKQ3/1uFA0aXEVKyieVOviUKffRoUNr+vXrXe7rLpeL2bOnExHRlaioSL7+envxa4sWLaR372707t2NRYsWVuq4IiKVlbItkz5zUmk1ayl95qSSsi3zgvvu3Qsvv2w8dWq3w6RJxpOo06YZT6fu2WO8HhUFfqpuF5ErVOn/jDRs2Ij77nuA8ePHMnXq/XTs2JlPPllNjx69Kn3w0aPHMnHivUyePKnc19esWUlGxs9s3rydrVu/ZMaMB1m+fC1Hjhzm+eefY9WqNGw2G7GxUQwePFSlxSLiERVZnmrXLuP+t6SkkhLfzp2tUeIrIjVbpcJcauoqXnzxeb78cjM33xxFq1atee+9/+OXXzIuK8xFRvZh7949F3x92bJPuOOO32Gz2ejZM4Jjx46xf38OmzZtICoqhuDgEACiomJITV1NQsLtlR6DiMillLc81cmzhTz5bibpyWFlSnwjIqxX4isiNVulwtzvfncbffv2IyVlWXEhcKdOXZgy5T5+/vknZs58pEoHl5OThcNR0ojpcISRnZ1FdnY2YWFhpbY7yM7OLvcz5s+fx4IF7xZ/XlpaWpWO8bfy8vI8fgypHM2JNVWneXGvZuBywdmsIE7uCuXkrlD2Hq3PNz4uunQ5yv33H6Jv30M0a3bGeE+m8U91Up3mpDbRvFiP1eakUmEuJeUTIiP7lNmWmDiBa665lnvuGU9Gxk+8+ea8Kh3glUpMnEBi4gQAhgyJITo62qPHS0tL8/gxpHI0J9ZUXeYl6ctMzu79lbydRoArzKsLPkXUbXmIa/vv5dPX29OsWTAQDFTvU3HVZU5qG82L9VhtTioV5n4b5Nyio/uzZMkK7rrrzioZlFtoqIOsrH3FP2dlZWK3O7Db7WzatLHU9iz69OlbpccWkdrLXeL74tv5pK1sSuGpMGx+hQS2PkBg2xzqtTlA/atcPJvQmWbNzB6tiNR2VfYcVXh4e5YvT62qjwNg8OAhvPPOW8TH38bWrV/SsGFDmjcPJSZmAP/9309x9OgRANatS+XRR5+o0mOLiDWlbMv0yDJR+fmwbFnZEl/fgADqttlPvbY51G11EJ86xn1zvjYbzyZ01QoHImIJlwxz//73e4wadQe+vr6X/LAmTZoAkJHxMwcO7OfGG2+66P6TJk1g06aNHD6cS9eu4cyYMZuCggIAxo+fSGxsHKtXryQioiv16tXj5Zf/AUBwcAhTp85g0KBoAKZNm1n8MISI1FwVeaq0Mtwlvk6nsRpD6RLfhAT4Y+pK8HWd974il0tBTkQs45Jh7o03XuO55/7K7bffybBhI+nUqXO5+x0+nEtq6mpSUpLYuHEDL7/82iUPfqn762w2G8899/dyXxszZhxjxoy75DFEpOYo76nSUwWFzF2xs8Lhav9+Y/1TpxPWrDEWtg8Lg3vuMQJc374l3W9hX9UtfvihNEdQ4BV/FxGRqnLJMLd48Yds3LiOt99+gxdffJ769RvQtm1bQkIaU6dOAMePH2Pv3j3s2/crQUFB3Hbbncyd+xJ2u8Mb4xeRWiSrnGB1se1ue/dCcrIR4DZsMJ5KbdMGpk41AlyvXuBTToX69Lh2Zc4EAgT6+zI9rt0VfQ8Rkap0yTDXsWMbli9P5eOPV/L555+ya9dOvvpqOwcP7ufkyXyaNm3KjTfeRETEjfTpczP+/v7eGLeI1EKOoMAKnylzl/g6nfDll8a2zp3h8ceNdVArUuLrPtvniXv0RESqyiXDXN26gZw9a9zHNnLkUJYtW1Nc9SEi4k0XO1PmcsHXX5cEuG+/NV6PiIA5c4wzcJdT4juye5jCm4hY2iXDXHh4OPPn/y+tW7fB5Tr/RmAREW/57Zkye8NARoR14tP3mvHQ7fDzz8bl0ptvhldegZEj4eqrTR60iIiHXTLMPfHEM0ycOI4PPngfm83Gk08+Sq9evenQoSMdOnTi+uvbVuhJVxGRqjCscxiNjhpLaCX/H8zKBH9/GDAAZs2CW29F3W8iUqtcMsxFRvbhq692snHjekaPTiAgIIDU1NW8+eZrFBQU4O/vz3XXtaVDh4507NiZ++673xvjFpFa5MwZ48lTp9N4EvXQIQgMhMGDjfvfbrkFgoLMHqWIiDkqVBrs7+9PTMwAIiJu5IknnqFjx04UFhby44+7+P77b/nuu2/5/vtveeutfyjMiUiVyM83ut+SkkpKfBs2hGHDjAAXFwf165s9ShER81VqBYglS1YU/9nX15fw8PaEh7cnIeH2Kh+YiNQ+R48awS0pqfwS3/79ISDA7FGKiFhLlS3nJSJyOQ4cMC6dJiVdusRXRETOp/9EiojX/forfPBBGE88ARs3QlFRxUp8RUTkfApzIuIVP/5onH0rKfG9ns6d4bHHjADXufOlS3xFROR8poa51NRVPPLITAoLC7nrrrt54IGpZV5/7LFZbNy4AYBTp05y6NAhfvrpVwBCQ4No374jAC1atGDBgve9O3gRuSiXC775piTA/bbE1+HYzLhxvav0mCnbMrVag4jUOqaFucLCQmbOnMbixR/icIQxaFA0cXFDadcuvHifp5+eU/zn//mfN/jmm6+Lf65bN5C1azd5dcwicnFFRfDFFyWrMJQu8X35ZYiPLynxTUu7+HqqlZWyLbPM6hCZR0/xsPMbAAU6EanRTAtz6elbaNWqNS1btgIgPn4Uy5cvLRPmSnM6P2DGjNneHKKIVMC5c8bi9U6nsZh9ZqkS35kzYcQI75T4zl2xs8wyXwCnCgqZu2KnwpyI1GimhbmcnGzCwloU/2y3O0hP31Luvr/+upe9e/dw881RxdvOnDnNwIFR+Pn5cv/9Uxk6dJjHxywihouV+CYkGF1w3i7xzTpa/pm+C20XEakpqsUDEMnJSQwfPqLMsmHp6d9htzvYvfsXRo0aTvv2HWjVqvV5750/fx4LFrwLQE5OFmlpaR4da15ensePIZWjOakap0758MUXIWzY0JTPP29Mfr4f9euf48Ybc7n55oNERBwmMLAIgO3bL/15VT0vIXVt5J4+f/3okLo2zX8F6XfFmjQv1mO1OTEtzIWG2snM3Ff8c3Z2Fna7o9x9U1KSmDPnhTLb3Pu2bNmKm27qy7fffl1umEtMnEBi4gQAhgyJITo6uoq+QfnS0tI8fgypHM3J5XOX+DqdRonvqVPQuDGMHm2cgRswwI+AgOZA80p/dlXPy2ONyt4zBxDo78tjIzoTrcusFaLfFWvSvFiP1ebEtDDXvXsPMjIy2LNnN3a7g+TkJN54453z9vvxx10cO3aUXr0iircdPXqEwMB6BAQEkJubyxdffM7kyX/25vBFaix3ia/TaVxKLSgwSnwnTjSW0bJqia/7vjg9zSoitY1p/0n28/Njzpy53HlnPIWFhYwZM47w8PbMmfMM3brdwODBQwFITv6AkSNHYStVQLVr1y6mT5+CzeaDy1XEAw9MveCDEyJyab/+ajy84HQaDzMUFUHr1vDnPxsBrqIlvinbMnnyo+84eqoAgOB6/jwxvKPXAtXI7mEKbyJS65j6/69jY+OIjY0rs23WrEfL/FzeE6wREb1Zt+5zj45NpKb78UcjvCUluUt8oVMnePRRI8BVtsQ3ZVsm0xd/RUFRyX1rR04WMP2DrwDVg4iIeIoFL5aIiCe4S3zdAc5d4turl1HiGx8Pbdte/ufPXbGzTJBzKyh0qR5ERMSDFOZELKYqVzEoKjLOurlLfH/6yTjb5i7xHTkSrrmmasZ9sQoQ1YOIiHiOwpyIhVTFKgbnzhmL17sDXGam8cDCgAEwYwbceis0r/zDp5fkCAok8wKhzREUWPUHFBERQGFOxFIudxWDM2fVjmpDAAAUgUlEQVQgNdW4fGpWie/0uHbn3TMH4O9rY3pcO88eXESkFlOYE7GQyqxikJ8PK1YYAe7jj+H4cbjqKhg+3AhwgwdD/fqeHnEJd9g082lWEZHaSGFOxEIudKnSfZnyQiW+t9/uLvGFgABvj7qEqkFERLxPYU7EQqbHtTtvFQP/s4F0ze/OkCElJb4Oh1Him5BgPMxgxRJfERHxDv0VIGIh7rNaz7y/m1+2BHEuw0HeniBeL7IVl/gmJEBERMVKfEVEpOZTmBOxCHeJr9MZxtYvjFDXqRMkPGoEuC5dKlfiKyIitYPCnIhJSpf4Op3Gn8Eo8X32WSPAXUmJr4iI1A4KcyJedLES35deMlZhqKoSXxERqR0U5kQ8rHSJb3Iy7NtXUuI7fTqMGOGZEl8REakdTL2FOjV1FZGRNxAR0ZVXXvn7ea8vWrSQ9u1bERPTh5iYPvzf//2zzGu9e3ejd+9uLFq00JvDFotK2ZZJnzmptJq1lD5zUknZlmnaMc6cgWXL4J57wG6HmBh4+23o2RPmz4cDB4xqkXvvVZATEZErY9qZucLCQmbOnMbixR/icIQxaFA0cXFDadcuvMx+I0YkMGfOC2W2HTlymOeff45Vq9Kw2WzExkYxePBQgoKCvfkVxEIutgxWVS18cKmlttwlvk4nLFlSUuI7bBiMGuX9El8REakdTDszl56+hVatWtOyZSvq1KlDfPwoli9fWqH3rl27hqioGIKDQwgKCiYqKobU1NUeHrFY2cWWwfLkMfLzbMyYc5RRo6BpUyO0LV8Ot90GS5fCwYPwr38Z2xXkRETEE0w7M5eTk01YWIvin+12B+npW87b7+OPP+Kzzz6lTZvrePrpZwkLa0F2djZhYSUt8w6Hg+zs7HKPM3/+PBYsePc/x8wiLS2tSr/Hb+Xl5Xn8GHK+Cy3wnnn0FHl5riqZE/cxCk/W4eSPzTm5K5TTu5tAkQ9HmpwhLu4Q/fodpEuXY/j6GuuTfvbZFR+2xtLvivVoTqxJ82I9VpsTSz8AMWjQYOLjbyMgIIB//vN/uf/+P+B0flypz0hMnEBi4gQAhgyJITo62gMjLZGWlubxY8j5wj5PLTfQhQUF0qCBzxXPyb594PP9LrK2N+bMvhBw2fALyqdhz1+4tscRtr/aEx+fMEBLWVWUflesR3NiTZoX67HanJh2mTU01E5m5r7in7Ozs7DbHWX2CQlpTMB/Fpq86667+eqr7QDY7XYyM0tuPM/KysJut3th1GJV0+PaEejvW2ZboL8v0+PaFf9c2QckfvoJ/vY36N0brr4aflnSFtfpOjSK/BH7hPU47k3DMehHnrrHrtUYRETENKb9FdS9ew8yMjLYs2c3Z8+eJTk5ibi4oWX22b8/p/jPy5d/Qtv/NKjGxAxg3bpUjh49wtGjR1i3LpWYmAFeHb9Yy8juYTyb0JmwoEBsGGfknk3oXLw8lvvhhcyjp3BR8vBC6UDnLvH9y1+M1Rauvx5mzjS64Z59FnbsgPdXHKfj8H0ENDtBi+CyxxARETGDaZdZ/fz8mDNnLnfeGU9hYSFjxowjPLw9c+Y8Q7duNzB48FDefvsNVqz4BF9fP4KDg3nllTcACA4OYerUGQwaFA3AtGkzCQ4OMeuriEWM7B52wWB1oQck/rZ8J46CMJKSKlbi244LH0NERMQMpt4zFxsbR2xsXJlts2Y9WvznRx99kkcffbLc944ZM44xY8Z5cnhSg2SVup/OVQRn9oVwclco+3aF0nu2SnxFRKT6svQDECJVJbRBPTK+qs/JXaGc/Kk5RScDsPkVEtz2MC/NCmTYMAhWTaGIiFRDCnNSY508aXS+vf56e77+vCkn83yw1SkgsM0B6rXNIaTdYZ4b3YGR3c0eqYiIyOVTmJMa5dgx+Phj4/63Zcvg1Clo2DCE0Xf4YO96iDXHviUnPx9HUCDT4zro/jcREan2FObEclK2ZTJ3xU6yjp76T+hqd9HQdfAgfPihEeBWr4aCAnA44Pe/h4QEcLk+ZcCAKKAJzxDtte8hIiLiDQpzYimXWv/Ubd8+SE42Atz69UZ9SKtWMGWKEeB696a4+y0tzeX17yEiIuItCnNiKRdbY7XTVWE4nUaA27zZeK1jR3jkEWPt0y5djFoRERGR2kRhTiylTIWICwoOXcXJnaFk7Qrl+oeN7T17wn//t3EGrl27C3yQiIhILaEwJ5ZibxTILz/UMSpEdoVy7kgDwEXDlsf422yjxPfaa80epYiIiHUozInpCgth40bj8umu9/uRu98PfIqoe00uDSMyCGl/iLmJ7RjZPcjsoYqIiFiOwpyY4uxZSE01AlxKivFEat26EBfnR8seh/ns3LccOHOiQk+zioiI1GYKczVUZes9vOHkSVixwghwS5YYnXANGsCwYcYDDIMHGz9DCNDP1LGKiIhUF6aGudTUVTzyyEwKCwu56667eeCBqWVef/31V1m48J/4+vrRpEkTXnrpNa6+2lj1PDQ0iPbtOwLQokULFix43+vjt6qK1nt4w7FjsHQpJCWVlPiGhBgPL4waZayHWreuV4ckIiJSo5gW5goLC5k5cxqLF3+IwxHGoEHRxMUNpV278OJ9OnfuwsqV66hXrx7z5v0PTz31OG+//S4AdesGsnbtJpNGb20Xq/fwRpg7eBA++sgIcO4SX7u9pMS3Xz9jYXsRERG5cqb9lZqevoVWrVrTsmUrAOLjR7F8+dIyYa5v35JLbT179uKDD3T2rSJK13tUZHtV2LfPuPctKenSJb4iIiJSdUwLczk52YSFtSj+2W53kJ6+5YL7L1w4nwEDBhb/fObMaQYOjMLPz5f775/K0KHDyn3f/PnzWLDg3f8cM4u0tLQqGf+F5OXlefwYlxJS10bu6fNXPQipa6vSsWVmBrJ+fRM2bGjKDz80BODaa/MZO/Yg/fodok2bPGw2OHPGCHhmscKcyPk0L9ajObEmzYv1WG1OqsXFrsWLF/HVV9tISVlWvC09/Tvsdge7d//CqFHDad++A61atT7vvYmJE0hMnADAkCExREdHe3SsaWlpHj/GpTzWqOw9cwCB/r48NqIz0VdwmdXlgu++M86+OZ3w9dfGdneJb3w8hIfXB+oDLa/oO1QlK8yJnE/zYj2aE2vSvFiP1ebEtDAXGmonM3Nf8c/Z2VnY7Y7z9lu3bi0vvfQ8KSnLCAgIKN7u3rdly1bcdFNfvv3263LDXG3kvi+uKp5mdblgy5aSAPfjj8aSWX37wosvqsRXRETEbKaFue7de5CRkcGePbux2x0kJyfxxhvvlNnnm2++4qGHpvD++06aNm1avP3o0SMEBtYjICCA3NxcvvjicyZP/rO3v4KljewedtkPO5Qu8U1Ohl9/NR5Y6N8fpk2DESMgNLSKBywiIiKXxbQw5+fnx5w5c7nzzngKCwsZM2Yc4eHtmTPnGbp1u4HBg4fy5JOPkZ+fz8SJdwMlFSS7du1i+vQp2Gw+uFxFPPDA1DIPTkjlXbjEF555BoYPh+Bgs0cpIiIiv2XqPXOxsXHExsaV2TZr1qPFf05K+qjc90VE9Gbdus89Orba4GIlvgkJMGSIu8RXRERErKpaPAAhVcdd4ut0GiW+J0+WlPgmJEBsrEp8RUREqhOFuVrAXeLrdBolvmfPGiW+48cbqzCoxFdERKT60l/hNVRmpvHwgtMJ69aVlPjef78R4FTiKyIiUjMozNUgP/9shDenEz7/zy2FHTrA7NlGgOva1agVERERkZpDYa4ac5f4Op1GD5y7xLdHD/jrX4174ML1kK+IiEiNpjBXzbhLfN0Bzl3i26ePSnxFRERqI4W5aqCwEDZtKrmE6i7xjYlRia+IiEhtpzBnUWfPwtq1xtk3d4lvQEBJie+wYUaliIiIiNRuCnMWcvIkrFxpBDiV+IqIiEhFKMyZ7Phxo8Q3KUklviIiIlJ5CnMmOHQIPvyw/BLfhASjxNff3+xRioiISHVgam1sauoqIiNvICKiK6+88vfzXj9z5gz33DOeiIiuDB4cw969e4pfe/nlF4iI6Epk5A2kpq725rAvS2YmvPoq9O8PzZvDf/0XfP+9UeK7aRPs2wevvQYDBijIiYiISMWZdmausLCQmTOnsXjxhzgcYQwaFE1c3FDatSspRlu4cD6NGgXxxRdfkZz8AU8//QRvv/0uO3fuIDk5iQ0bviAnJ5vbbruVzz/fhq+vr1lfp1wXK/FNSIBu3VTiKyIiIlfGtDNz6elbaNWqNS1btqJOnTrEx49i+fKlZfZZvnwpd975OwCGDx/Jhg1puFwuli9fSnz8KAICArj22pa0atWa9PQtZnyNMlwu+OWX+jz1lBHUrrsOZsyAggKjxPeHH4yS36efhu7dFeRERETkypl2Zi4nJ5uwsBbFP9vtjvMCWel9/Pz8uOqqhhw+fJjs7Cx69OhVvJ/DEUZOTrZ3Bn4BLhfcdBN8/nmv4hLfv//dKPFt2dLUoYmIiEgNVuMfgJg/fx4LFrwLwI8/7qJbtxs8erwWLQ7RuHETTpyAf/7T+EfMlZtrzIlYi+bFejQn1qR5sR5vzcmvv+659E6YGOZCQ+1kZu4r/jk7Owu73VHuPg5HGOfOnePEieOEhIRgtzvIysos3i8rK5PQUHu5x0lMnEBi4gTPfIlyDBwYxapV67x2PLk0zYk1aV6sR3NiTZoX67HanJh2z1z37j3IyMhgz57dnD17luTkJOLihpbZJy5uKO+//x4AS5ak0LdvFDabjbi4oSQnJ3HmzBn27NlNRkYGN9zQ04yvISIiImIq087M+fn5MWfOXO68M57CwkLGjBlHeHh75sx5hm7dbmDw4KGMHZvIn/50LxERXQkODubNN+cBEB7enhEj4unbtxd+fn4899zzlnuSVURERMQbTL1nLjY2jtjYuDLbZs16tPjPdevW5Z135pf73gcfnM6DD0736Pgux7hx480egvyG5sSaNC/WozmxJs2L9VhtTmwHDhx3mT0IEREREbk8pq4AISIiIiJXRmHuMl3JUmTiGZeak9dff5W+fXsRFRXJqFHD+fXXvSaMsva51Ly4LVnyIc2aNWT79nQvjq52qsicfPihk759e3HzzRH84Q+/9/IIa6dLzcu+fb8SH38L/fv3JSoqktWrV5gwytpjypT76NChNf369S73dZfLxezZ04mI6EpUVCRff73dyyMsoTB3GdxLkb33XhIbN36J0/kBO3fuKLNP6aXIJk36E08//YRJo60dKjInnTt3YeXKdaxb9xnDho3gqaceN2m0tUdF5gUgL+8Eb7/9Oj166Kl0T6vInGRk/MTLL/+djz9eyYYNX/D008+ZNNraoyLz8uKLc7n11nhSUzfy1lvzmDlzmkmjrR1Gjx7LokXOC76+Zs1KMjJ+ZvPm7bzwwsvMmPGgF0dXlsLcZbiSpcjEMyoyJ3379qNevXoA9OzZq0xXoXhGReYFYM6cZ7j//j8TEFDXhFHWLhWZkwUL/snvf38PQUHBADRt2tSModYqFftdsXHixAkAjh8/RvPmod4faC0SGdmn+HegPMuWfcIdd/wOm81Gz54RHDt2jP37c7w4whIKc5ehvKXIsrOzLrhP6aXIxDMqMielLVw4nwEDBnpjaLVaRebl66+3k5mZycCBg709vFqpInPy888/8fPPP3HLLQMZMqQ/qamrvD3MWqci8zJjxsMkJb1P167hjBlzO88+O9fbw5RScnKycDhK5szhCLvo3zueVOOX8xL5rcWLF/HVV9tISVlm9lBqvaKiIh5/fDavvPK62UORUgoLz5GR8TMpKZ+QlZXJiBFDWLfuMxo1CjJ7aLWa0/kBd945lvvuu58vv9zMn/50L+vXb8bHR+dlajv9L+AyVGYpMqDMUmTiGRWZE4B169by0kvPM3/++wQEBHhziLXSpeYlL+8EO3Z8T3z8LfTo0YmtW79k3LjRegjCgyryu2K3hzF48FD8/f259tqWtGlzHRkZP3t7qLVKReblX/+az4gR8QD06tWb06fPkJub69VxSonQUAdZWSVzlpWVWe7fO96gMHcZrmQpMvGMiszJN998xUMPTWHBgkW6B8hLLjUvDRs2YseO3Wzd+i1bt35Ljx69WLBgEd263WDiqGu2ivyuDB16C5s2bQAgNzeXn3/+iWuvbWnCaGuPisxLWFgLNmww1gPdtWsnZ86cpkkTzy/2LuUbPHgI//73e7hcLrZs+YKGDRuadh+jLrNehitZikw8oyJz8uSTj5Gfn8/EiXcD0KJFCxYseN/kkddsFZkX8a6KzElMTCxr16bSt28vfH19eeKJpwkJaWz20Gu0iszLX/7y30ydej9vvPEaNpuNV155XScJPGjSpAls2rSRw4dz6do1nBkzZlNQUADA+PETiY2NY/XqlUREdKVevXq8/PI/TBurVoAQERERqcZ0mVVERESkGlOYExEREanGFOZEREREqjGFOREREZFqTGFOREREpBpTmBMRERGpxhTmRERERKoxhTkRERGRakxhTkSkkj79dCPNmjVk9eoVxdv27NlNhw6tmT17uokjE5HaSCtAiIhchoSEYZw5c4alS1dx/PgxbrllINdccy3z5y/C19fX7OGJSC2iM3MiIpdhxozZfPnlZtauXcN//dfd+Pn58+ab8xTkRMTrdGZOROQy3XbbCLZs2UzDho1YvjwVhyPM7CGJSC2kM3MiIpepVavWnDx5khkzZivIiYhpFOZERC7D/PnzeO+9BXTs2JmFC+ebPRwRqcUU5kREKiktLZVZs6bxwguvMHfui2zd+iVr1qw0e1giUkvpnjkRkUrYseMHhg0bxMSJ9/Dww48Dxr1zeXnHWb58rcmjE5HaSGfmREQq6ODBg9x11x1ER/dn1qzHirdPmzaD9PStrFq13MTRiUhtpTNzIiIiItWYzsyJiIiIVGMKcyIiIiLVmMKciIiISDWmMCciIiJSjSnMiYiIiFRjCnMiIiIi1ZjCnIiIiEg1pjAnIiIiUo0pzImIiIhUY/8fe9wujNISy9MAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff8bcc8ff28>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# x: input data\n",
    "# y: noisy output data\n",
    "\n",
    "x = np.random.uniform(0,1,20)\n",
    "\n",
    "# f = 2x + 0\n",
    "def f(x): return 2*x + 0\n",
    "\n",
    "noise_variance =0.1\n",
    "noise = np.random.randn(x.shape[0])*noise_variance\n",
    "y = f(x) + noise\n",
    "\n",
    "fig, ax = plt.subplots(1, 1)\n",
    "fig.set_facecolor('#EAEAF2')\n",
    "plt.xlabel('$x$', fontsize=15)\n",
    "plt.ylabel('$f(x)$', fontsize=15)\n",
    "plt.plot(x, y, 'o', label='y')\n",
    "plt.plot([0, 1], [f(0), f(1)], 'b-', label='f(x)')\n",
    "plt.ylim([0,2])\n",
    "plt.gcf().set_size_inches((10,3))\n",
    "plt.grid(True)\n",
    "plt.show"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Complete the following code in order to:\n",
    "+ Compute the value of $w$ by using a estimator based on minimizing the squared error.\n",
    "+ Get from SGD function a list, `target_value`, representing the value of the target function at each iteration.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Write your target function as f_target 1/n Sum (y - wx)**2\n",
    "def target_f(x,y,w):\n",
    "    # your code here\n",
    "    return\n",
    "\n",
    "# Write your gradient function\n",
    "def gradient_f(x,y,w):\n",
    "    # your code here\n",
    "    return\n",
    "\n",
    "def in_random_order(data):\n",
    "    '''\n",
    "    Random data generator\n",
    "    '''\n",
    "    import random\n",
    "    indexes = [i for i,_ in enumerate(data)]\n",
    "    random.shuffle(indexes)\n",
    "    for i in indexes:\n",
    "        yield data[i]\n",
    "\n",
    "# Modify the SGD function to return a 'target_value' vector\n",
    "def SGD(target_f, \n",
    "        gradient_f, \n",
    "        x, \n",
    "        y, \n",
    "        toler = 1e-6, \n",
    "        epochs=100, \n",
    "        alpha_0=0.01):\n",
    "    \n",
    "    # Insert your code among the following lines\n",
    "    \n",
    "    data = zip(x,y)\n",
    "    w = random.random()\n",
    "    alpha = alpha_0\n",
    "    min_w, min_val = float('inf'), float('inf')\n",
    "    iteration_no_increase = 0\n",
    "    epoch = 0\n",
    "    while epoch < epochs and iteration_no_increase < 100:\n",
    "        val = target_f(x, y, w)\n",
    "        if min_val - val > toler:\n",
    "            min_w, min_val = w, val\n",
    "            alpha = alpha_0\n",
    "            iteration_no_increase = 0\n",
    "        else:\n",
    "            iteration_no_increase += 1\n",
    "            alpha *= 0.95\n",
    "        for x_i, y_i in in_random_order(data):\n",
    "            gradient_i = gradient_f(x_i, y_i, w)\n",
    "            w = w - (alpha *  gradient_i)\n",
    "        epoch += 1\n",
    "    return min_w"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "unsupported operand type(s) for -: 'float' and 'NoneType'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-34-2c273941b2f4>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;31m# Print the value of the solution\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      2\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 3\u001b[0;31m \u001b[0mw\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtarget_value\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mSGD\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtarget_f\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mgradient_f\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0my\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      4\u001b[0m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'w: {:.6f}'\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mformat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mw\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m<ipython-input-33-a1422d448255>\u001b[0m in \u001b[0;36mSGD\u001b[0;34m(target_f, gradient_f, x, y, toler, epochs, alpha_0)\u001b[0m\n\u001b[1;32m     38\u001b[0m     \u001b[0;32mwhile\u001b[0m \u001b[0mepoch\u001b[0m \u001b[0;34m<\u001b[0m \u001b[0mepochs\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0miteration_no_increase\u001b[0m \u001b[0;34m<\u001b[0m \u001b[0;36m100\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     39\u001b[0m         \u001b[0mval\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtarget_f\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0my\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mw\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 40\u001b[0;31m         \u001b[0;32mif\u001b[0m \u001b[0mmin_val\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0mval\u001b[0m \u001b[0;34m>\u001b[0m \u001b[0mtoler\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     41\u001b[0m             \u001b[0mmin_w\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmin_val\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mw\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mval\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     42\u001b[0m             \u001b[0malpha\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0malpha_0\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mTypeError\u001b[0m: unsupported operand type(s) for -: 'float' and 'NoneType'"
     ]
    }
   ],
   "source": [
    "# Print the value of the solution\n",
    "\n",
    "w, target_value = SGD(target_f, gradient_f, x, y)\n",
    "print('w: {:.6f}'.format(w))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'w' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-5-c43f4e8f79c8>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      5\u001b[0m \u001b[0mplt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mplot\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0my\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'o'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlabel\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m't'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      6\u001b[0m \u001b[0mplt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mplot\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mf\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mf\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'b-'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlabel\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'f(x)'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0malpha\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0.5\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 7\u001b[0;31m \u001b[0mplt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mplot\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0mw\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0mw\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'r-'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlabel\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'fitted line'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0malpha\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0.5\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlinestyle\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'--'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      8\u001b[0m \u001b[0mplt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mxlabel\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'input x'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      9\u001b[0m \u001b[0mplt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mylabel\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'target t'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mNameError\u001b[0m: name 'w' is not defined"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhsAAAFoCAYAAAD+eTlPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzt3Xt0VOWh///PngmBEQyZ3EAwIJckQhLgQLgJUi71tHwV\ny6+aqkeX9Va1tLbVpVU5WvV4LF5RC6LSckCtCCKt1rraY0UsUOEIoXITAlSUqyQkM4RgQpjM/v0x\nEAy5zSSz99zer7W6Kjt7Zp48xsyHvef5PEZZWZUpAAAAizgiPQAAABDfCBsAAMBShA0AAGApwgYA\nALAUYQMAAFiKsAEAACxF2AAAAJYibAAAAEsRNgAAgKUIGwAAwFJJVj75a68t1KpVK/Xll1+qc+fO\nKigYoh//+A716dO31cd9+OEHWrDgJR06dEjZ2X10++0/1dix46wcKgAAsIilVzY2bfqnrrjiKs2f\nv0jPPTdP9fU+3XXXT3XiRG2Lj9m6dbMeeeQ/NW3adC1a9LomTJiomTPv1p49n1s5VAAAYBHDzo3Y\nvF6vpk27RHPn/lZDhw5r9pyHHrpftbW1euKJZxuO3XbbjcrJydPdd99n11ABAECY2PqZjerqYzIM\nQykpKS2es3XrFhUVjW50bNSoMdq2bbPVwwMAABawLWyYpqnf/OYZDRkyTP369W/xvMrKCqWlpTU6\nlpaWrsrKCquHCAAALGDpB0S/6ZlnHteePXv04osLQn6saZqSjKDPz8joJsMI/nwAABJVZaX04YfS\n1q2BP//kJ8fC/hq2hI3Zs5/Q2rX/0Asv/E4ZGRmtnhu4ilHZ6JjHU9nkakdrDMNQVVWN6uv97Rov\nQuN0OpSS4mLObcSc2485tx9zbq3qaunjj5369FOH/Kemt3Nna/6ibnnYmD37Ca1Zs0pz5rysnj17\ntnl+QUGhSko+UXHx1Q3HNmz4P+XnDwnpdevr/fL5+OG0E3NuP+bcfsy5/ZjzwBX+nfu88lbXKbVb\nsnKzU9t9Bb+2Vlq/3qkNG5w6eVKSTDmd0rBh9Ro3zpo1I5aGjaefflwffPC/euKJ2XK5XA2fu+ja\ntZs6d+4sSfrv/35ImZlZuu22n0iSiouv0U9/equWLPm9LrpovP72t/9VaekO/fKXD1g5VAAAolJJ\nabmWrdytMm9Nw7GsVJeKJw3UiLzMoJ/H55M2bnRo3bok1Z5qoDAMadAgv8aP9yk1VUpKcsjnC/d3\nYHHYeOed5TIMQ3fccVuj4/ff/ytNnXqZJKms7LAcjjOfUy0oGKKHH35M8+fP0/z5Lyo7O1uzZj3T\n6odKAQCIRyWl5Zr39haZZ11wKPPWaN7bWzRjemGbgcPvl7Ztc2jNGqeOHTtzNaR/f78uvrhePXpY\n34Bha8+GXTIzz5XHczzhL7vZJSnJIbe7K3NuI+bcfsy5/RJ9zk3T1P0vr2t0ReNsWW6XZt06ptlb\nKqYp7drl0OrVTlVUnPl6796mJkzwKTu76dt/4MpG+OfattUoAAAgeDv3eVsNGpJU5qnRrv1HlZud\n2uj43r2GVq1K0sGDZ0JGRkYgZAwYYMruBZuEDQAAopC3ui7I8040/PPhw4ZWrXJqz54zH09ISTE1\nbly98vP9ckRo+1XCBgAAUSi1W3KQ53WWxyOtWZOk7dvPpAmXSxo71qdhw/xKivC7PWEDABD3wrl0\n1C652anKSnW1eivFfU43fbE9XX/a7GzoykhOloqK6jVyZL1OLfyMOMIGACCuhWvpqN0Mw1DxpIHN\nrkap9znkPZSmtNRB+rTWKUkNXRmjR9erW7cIDLgVEbp7AwCA9U4vHT376sDppaMlpeURGllwRuRl\nasb0QmW5XZIkv99Q5UG3yrfnqX+3wcrq3lWGIQ0e7NfNN9dpypToCxoSVzYAAHHKNE0tW7m7yVWB\nM1+Xln20W8NzM6L2loppmurmStLlF/XTZ5859a/t56pHSrLSz+8iyd6ujI4gbAAA4lJHlo5Gg5LS\ncr354W59/rlTFfsydeLrZHXtUqX8fmmtdmVEI8IGACAutWfpaLQoKS3X06/sVvmXmao55mo47nNU\n66C+UN6IfsrOjt7Pm5yNsAEAiEuhLB2NJl99Jc167rgOHejTcKxT55NKzz6ilMwqGYb01t93a0Re\n9N7+ORthAwAQl4JZOprldinn/O42jqplp7sy1qyr06EDnSRJzk71Su9doe49vXI4ztwyiebbP80h\nbAAA4lJrS0cDX5eKJw6M+NWB6mpp7VqnNm0KdGXU1tXL4fTLfZ5H7l6VciY1v1dJNN7+aQlhAwAQ\nt04vHV320W6Veb7Rs+F2qXhiZHs2amul9eud2rDBqZMnA8ecTmn4cJ88536upOT6Vh8fbbd/WkPY\nAADEtRF5mRqem6Gd+7w6erxOqd06K+f87hG7ouHzSRs3OrRuXZJqawPHDEMaNMiv8eN96t69i7a/\nnBwzt3+CQdgAAMQ9wzCU18cd0TH4/dK2bQ6tWePUsWNngk7TrozYuP0TCsIGAAAWMU1TpXu92vqZ\nqV3bzpXT37Xha611ZUTz7Z/2IGwAAGCBktJy/c8f9qt067mnujJq1LVLJ40b3l3XXtlFAwaYau3i\nRLTd/ukIwgYAAGH2t39U6un5HlV7shqOdep8Uueef0hfqkpH6wtlGG1fnYiG2z/hQNgAACBMPB5p\n9Wqn5v7ep+O1gVsmzXVlRPueLOFG2AAAoIO+2ZVR7qnV8dqTrXZlxFopV0cRNgAAaKfmujLq6n1y\nn+dRWu+KVrsyNu4sT5iw4Yj0AAAAiDU+n/TJJw7Nn5+stWsDQcMwpMGD/br2uq+V1a+szVKuT7Yf\nltnc2tY4xJUNAACCFExXhml2V+pHyW3uOuutrkuYWymEDQAA2mCa0q5dDq1e7VRFxZmQ0VxXhmEY\nGjWoh95fv6/N542l/U06grABAEAr9u41tGpVkg4ePBMyMjICIaOlrox/y8kIKmzE0v4mHUHYAACg\nGYcPG1q1yqk9e858vDElxdS4cfXKz/fL0cqnHmNte3urETYAAPgGj0dasyZJ27efSRMulzR2rE/D\nhvmVFMQ7Z6xsb28XwgYAAGrcleE/VYuRnCwVFdVr5Mh6dQ7xjke87W/SEYQNAEBCa64rw+mUhg2r\n1+jR9erWrf3PHU/7m3QEYQMAkJB8PmnjRofWrUtSbW3gmGFIgwb5NX68T6lhWpEaL/ubdARhAwCQ\nUILpykB4ETYAAAkhlK4Ma17f1M59Xnmr65TaLVm52akJczuFsAEAiHvt6coIp5LSci1bubvRUtis\nVJeKJyXGB0UJGwCAuNWRroxwKSktb3YJbJm3RvPe3qIZ0wvjPnAQNgAAcSccXRnhYJqmlq3c3WzX\nRuDr0rKPdmt4bkZc31IhbAAA4ka4uzI6auc+b6stopJU5qmJ+w3ZCBsAgJhnZVdGR7S18+uZ8+J7\nQzbCBgAgZtnVldFeqd2SgzwvvjdkI2wAAGJOrHRlsCFbAGEDABAzIt2VESo2ZAsgbAAAYkKkuzLa\niw3ZCBsAgCj31VeGVq5MimhXRkcl+oZshA0AQFTyeKQVK6T165Pk9wfuQUSiKyNcEnlDthj7VwUA\niHenuzK2bElq6MWIZFcGOo6wAQCICmd3ZTgcga6MESP8GjnyZMS6MtBxhA0AQES11JWRn+/XtGmS\nVC+fL5IjREcRNgAAEdFWV0bv3obc7sBnNxJ5e/Z4QNgAANgq+K6MwNc27CjTkg92Jez27PGAsAEA\nsE2oXRlrtxzUnOWbE3p79nhA2AAAWO7wYUOrVjlD6sowTVML3/0s4bdnjweEDQCAZTweac2aJG3f\nfiZNBNuVUbrXq0MVx1t9/kTYnj0eWBo2Nm36pxYvflWlpTtUUXFEs2Y9rfHjv9Xi+f/8Z4l+9rPb\nGx0zDEPvvPNXud1pVg4VABBGp7syNm1yyu8PHAu1KyPYbdfjfXv2eGBp2KipqVFOTp4uvfR7euCB\nXwb1GMMw9MYbf9A555zTcIygAQDhZ8UKj7O7MqRAV8awYfUaPbo+pK6MYLddj/ft2eOBpWFjzJiL\nNGbMRZICP9TBSk1NVdeutLcAgFVKSsu1bOXusK3waKkrY9Agv8aP9ym1HXc58vqk6rz0rq3eSkmE\n7dnjQdR9ZsM0Td1ww7Wqqzuh/v0H6KabblVh4dBIDwsA4kZJaXmzW563Z4VHW10ZPXq0f8t3wzB0\n47TBmvXK+oTenj0eRFXYSE/P0D33zNSFFw5SXd1JvfvuH3XHHbfpt799RTk5eSE9l9MZA9sAxonT\nc82c24c5t1+8zLlpmlr20e5WV3i89dFujRqc1eqbuGlKO3caWr3aqSNHAuc5HIGujIkT6091ZRg6\n3ZXRHk6nQ2MLe+nnxUP1xt926vA3tmfv4Xbpqik5Krowq93Pj6acTod8Pn/YnzeqwkafPn3Vp0/f\nhj8XFBTqwIH9Wrp0sR544JGQnislxRXu4aENzLn9mHP7xfqcb/3XEZV94027OYc9NTrkPaH8/unN\nfv2LL6QPPpD27w/82eWSsrKkKVOk3Fw16croqCmjL9DkUX217fMKeapOKK17Fw3ul8YVDYuUlx8L\n+3NGVdhozuDB+dqyZVPIj6uqqlF9ffjTGZpyOh1KSXEx5zZizu0XL3O+79DRoM7be9CrXu4ujY59\n9ZWhVasc+vzzxl0ZF1/sb+jK8HrDN9az57x3mku90wJhz+v9OnwvhAZWXbmL+rCxa9dOpadnhPy4\n+nq/JZeC0DLm3H7Muf1ifc7PdXUK6ryUc5Ibvs+mXRlmk64Mv18NS1zDLdbnHDYsfd2/f5+kwM3B\ngwcPaNeunUpJSVGPHj310ktzdeRIecMtkjfffEO9evVSv34DdOLECb377tvauHGDnn32BSuHCQAJ\nIzc7VVmprkarUM52eoVHOLoyAMnisLFjx2f62c9ul2EYMgxDc+c+J0n67ncv1cyZD6myskJlZYcb\nzvf5Tmru3Od05Ei5unTpogEDcvT88y9q2LDhVg4TABKGYRgqnjSw2dUoga9L3xubozVrksLSlWE1\ndoONDUZZWVX71yVFqczMc+XxHOeym02Skhxyu7sy5zZizu0Xb3NeUlquZR/tbvRh0YyUc1SYNVie\ngxlh68roiLbmPNxdIQjMedyvRgEA2GNEXqaG52Zo5z6vPMfqdORgN+3bnapDn4e3K8Mq4ewKgfUI\nGwCQsAwZtenats6pioozIaN378CW74GujOhjmqaWrWy9K4TdYKMLYQMAEtDevYZWrUrSwYNn3owz\nMgIhY8AAM+xdGeG0c5+31Q+4SuwGG20IGwCQQA4fNrRqlVN79jTuyhg3rr6hKyPaeavrgjyP3WCj\nBWEDABJA064MNenKiBWp3ZKDPI+1udEihn68AAChiseujFC6QhAdCBsAEIdqa6X1650x0ZURqmC6\nQtgNNroQNgAgjvh80saNDq1blxQVXRlWGZGXqRnTC5t0hWS5XSqeSM9GtCFsAEAc8PulbdscWrPG\nqWPHYqMro6O+2RVy9HidUrt1Vs753bmiEYUIGwAQw0xT2rXLodWrY6srI1wMw1BeH3ekh4E2EDYA\nIEbFclcGEgthAwBiTDx0ZSCxEDYAIEbEU1cGEgs/mgDQDnZubR6PXRlILIQNAAiRXVubx3NXBhIL\nYQMAQrBhR5nlW5snSlcGEgdhAwCCZJqmlqzYZdnW5onYlYHEQNgAgCBt+7yiUVtlc9qztXmid2Ug\n/hE2ACBIlVW1QZ0XytbmdGUgERA2ACBIaSldgjovmK3N6cpAIiFsAECQ8vunK8vtavVWSltbm9OV\ngUTEjzWAmGdX54VhGLp6So7mLN8c8tbmdGUgkRE2AMQ0uzovTiu6MCukrc3pygAIGwBiWElpueWd\nF80JZmtzujKAMwgbAGKSaZpatnK3ZZ0XbWlpa3O6MoCmCBsAYtLOfd5Gt06a057Oi/aiKwNoGWED\nQEzyVtcFeV7wnRftRVcG0DrCBoCYlNotOcjzrFvmQVcGEBzCBoCYlJudqqxUV6u3UtrqvGivcHVl\n2LlNPRBJhA0AMckwDBVPGtjsapTA11vuvGiv6mrp/fed2rgxqcNdGXYv2QUiibABIGaNyMsMqfOi\nvWprA8tYt26Vqqoc8vvNDnVlRGrJLhAphA0AMS2Yzov2+mZXRl2dIZcrcMVk8OD2d2VEeskuEAmE\nDQAxr6XOi/ZqrivD4ZBycqQRI3xKT69v93NH25JdwA6EDQA4pbWujMmT6zVkSLI8HlM+X/tfI5qW\n7AJ2IWwAgNruyujUKTzrWKNhyS5gN8IGgIRmd1dGJJfsApFC2ACQkMLVlRGqSCzZBSKNsAEgoVRX\nS2vXOrVpk7PDXRntZdeSXSBaEDYAJITaWmn9eqc2bHDq5MnAsY50ZXSUlUt2gWhD2AAQ177ZlVFb\nGzhmGNKgQe3vygiXcC/ZBaIVYQNAXGquK0OS+vf36+KL69WjB1u+A3YhbACIK611ZUyY4FN2NiED\nsBthA0DcaKsrg49DAJFB2AAQ8+zuyggWW8gDAYQNADErUl0ZwWALeeAMwgaAmBMNXRmtYQt5oDHC\nBoCYEW1dGc1hC3mgKcIGgKgXzV0ZZ2MLeaApwgaAqBWLXRlsIQ80RdgAEHViuSuDLeSBpggbAKJK\nrHdlsIU80JSlYWPTpn9q8eJXVVq6QxUVRzRr1tMaP/5brT5m48YNmjv3OX3xxefKyuqpH/7wJk2d\nepmVwwQQBaK1KyNUbCEPNGXpf741NTXKycnTXXfdG9R/WIcOHdS9996poqKRWrRosYqLr9bjjz+q\n9evXWTlMABHk8UjvvpukV17p1BA0XC5p8mSfbrnlpAoLYydonHZ6C/kst6vR8Sy3i2WvSEiWXtkY\nM+YijRlzkaTAcrC2/PGPb6lXr96aMePnkqQ+fS7Q5s2faunSxRo5coyVQwVgs2jvyugotpAHzoiq\nz2x89tlWFRWNanRs9Oix+s1vZkdoRADCLRa6MsKFLeSBgKgKGxUVR+R2pzU65nan6euvj6uurk7J\nycF9yluSnM4Yu+4aw07PNXNun1icc59PKilxaN06p2pOfXbS6ZQGDw4sYw10ZUTv9xOLcx7rmHP7\nOZ0O+Xz+sD9vVIWN5gVuv4R66TElxdX2SQgr5tx+sTDnfr+0aZO0cqVUVRU45nJJOTnSlClSz56R\nHV+oYmHO4w1zbq/y8mNhf86oChvp6RnyeCobHfN4PDrnnK7q1KlTSM9VVVWj+vrwpzM05XQ6lJLi\nYs5tFAtzbprSzp2GVq926siRxl0ZEyfWN3RleDyRGmFoYmHO4w1zbj+rriJFVdjIzy/UunUfNzr2\nySfrVFBQGPJz1df7LbkUhJYx5/aL1jlv2pVhNunK8PkiOsR2i9Y5j2fMeeyzNGzU1NRo//59On0r\n5ODBA9q1a6dSUlLUo0dPvfTSXB05Uq4HHnhEkjR9+hX6wx/e1Lx5v9Fll12uDRvW66OPVuipp563\ncpgAwiReujIAhJelYWPHjs/0s5/dLsMwZBiG5s59TpL03e9eqpkzH1JlZYXKyg43nH/eeb305JPP\nac6cZ/XWW0uVlZWl++57UCNHjrZymAA6yOOR1qxJ0vbtZ9KEyyWNHevTsGF+JUXVNVQAdjPKyqqi\nd5OBdsrMPFcez3Euu9kkKckht7src26jaJnzeO/K+KZomfNEwpzbLykpYVejAIg2idSVAaDjCBsA\ngubzSRs3OrRuXZJqawPHDEMaNMiv8eN9p7oyAKAxwgaANvn90rZtDq1Z49SxY2eWsfbvHyjk6tEj\n7u7GAggjwgaAFpmmtGuXQ6tXO1VR0bgrY8IEX0NXBgC0hrABoFlNuzLUpCsDAIJB2ADQCF0ZAMKN\nsAFAEl0ZAKzDrw8gwSVSVwaAyCBsAAmKrgwAdiFsAAmGrgwAdiNsAAmCrgwAkULYAOIcXRkAIo2w\nAcQxujIARAPCBhCH6MoAEE0IG0AcoSsDQDTiVw8QB+jKABDNCBtADKutDYQMujIARDPCBhCDfD7p\n44+l99/vpOPHA6tJ6MoAEK0IG0AMOd2V8fHHSfL5pJqawHG6MgBEM8IGEAPO7spwOAy5XIGujPHj\nT9KVASCqETaAKNdcV0Zmpqlp06TMTJ/q6wkaAKIbYQNog2ma2rnPK291nVK7JSs3O1WGDW1YrXVl\nDB0qpad3lsdj+TAAoMMIG0ArSkrLtWzlbpV5axqOZaW6VDxpoEbkZVrymsF0ZThCaOWKVFgCgNMI\nG0ALSkrLNe/tLTLPuktR5q3RvLe3aMb0wrAGDiu6MiIRlgDgbIQNoBmmaWrZyt1NgsaZr0vLPtqt\n4bkZHb5KUFsrrV/f/q6Mlq5c2B2WAKAlhA2gGTv3eRtdDWhOmadGu/YfVW52+0otfD5p40aH1q1L\nUm1t4FioXRkbdpRpyQe7mrlyMUDLVv7LlrAEAG0hbADN8FbXBXneiZCf+3RXxpo1Th07duaNPtSu\njLVbDmrO8s0tXLnY2mLQaDivg2EJAIJF2ACakdotOcjzgv8gxdldGaf17h3Y8j2UrgzTNLXw3c9a\nvXIRjPaEJQAIFWEDcSHcKy5ys1OVlepq9VZKltulnPO7B/V8zXVlZGQEQsaAAaZCHWrpXq8OVRwP\n7UHNCCUsAUB7ETYQ86xYcWEYhoonDWz2A5aBr0vFEwe2GWha68rIz/crhBWsjYTjikQoYQkAOoKw\ngZhm5YqLEXmZmjG9UMs+2q0yzzeCjNul4omtB5lgujI6oqNXJIINSwAQDoQNxCw7lqeOyMvU8NwM\n7dzn1dHjdUrt1lk553dv8fms6MpoTl6fVJ2X3rXVWylZbpeu/NYAvfX3f4UclgAgnAgbiFl2LE+V\nArdU8vq4Wz2no10Z7RnTjdMGa9Yr61u9zTMiL1Mj8jKDDksAYAXCBmKWlctTgxWOroz2GlvYS3dc\nMURLVuxq9cpFMGEJAKxE2EDMsmJ5arDC1ZXRUUUXZmnogHSuXACIaoQNxKxwL08NRji7MsKFKxcA\noh1hAzErXMtTgxXurgwASBSEDcS0jixPDZZVXRkAkCgIG4h5oS5PDZbVXRkAkCj4dYm4EM7PLdjV\nlQEAiYKwAZxid1cGACQKwgYSXiS7MgAgERA2kLCipSsDAOIdYQMJJxq7MgAgnhE2kFDoygAA+xE2\nkBDoygCAyCFsJADTNLVzn1fe6jqldktWbnZqwuydQVcGAEQev2rjXElpuZat3N1o/5CsVJeKJ4Wn\nXTNa0ZUBANGDsBHHSkrLm903pMxbo3lvb9GM6YVxFzjoygCA6EPYiFOmaWrZyt3NblAW+Lq07KPd\nGp6bERe3VOjKAIDoRdiIUzv3eVvdel2Syjw12rX/qHKzY/edmK4MAIh+toSN5cvf1JIlv1dFRYUG\nDszRnXfeo0GD8ps99y9/+bN+/etHZBiGzFN/LU9OTtaKFf+wY6hxw1tdF+R5JyweiTXoygCA2GF5\n2Fix4n298MJzuueemRo8uEBLl76uu+66Q2+88QeltnBtu2vXbnrjjT9IOv2GEfuX+e2W2i05yPNi\n75OSdGUAQGyxPGwsXbpYl1/+fU2depkk6Z57Zmrt2n/ovffe0bXX/rDZxxiGIbc7PDt4Jqrc7FRl\npbpavZWS5XYp5/zuNo6qY+jKAIDYZGnY8Pl8Ki3druuvv7HhmGEYKioapa1bt7T4uJqar3XlldPk\n9/uVl3ehbr31J+rXr7+VQ407hmGoeNLAZlejBL4uFU8cGBMfDqUrAwBim6W/pr1er/x+v9zu9EbH\n3e407d37ZbOPyc7uq/vv/5UGDMjR8ePVWrz4Vf34xzfptdfeVGZmVtCv7XTy19zR+T3kdBpaumKX\nDnvOXOHo4Xbpqik5Krow+Plszem5DvecV1dLH3/s1KefOuT3Sw5HoCtj5Mh6jRrlP9WVkZj/nq2a\nc7SMObcfc24/p9Mhn88f9ueN0N8JzRb/Rl1QUKiCgsKGP+fnF+q664r1pz/9UTfffFvQr5CS4urw\nKOPBJWP76dtjLtC2zyvkqTqhtO5dNLhfmiVXNMI157W10scfS2vXSidPSp07B7oyioqkiy8WXRnf\nwM+5/Zhz+zHn9iovPxb257Q0bKSmpsrhcMjjqWh03OPxyO1OC+o5kpKSlJOTp/3794X02lVVNaqv\nD386i1W901zqnRb4D9br/Tqsz+10OpSS4urwnPt8UkmJQ+vWOVVz6kKMYUiDBweWsaamBsKHxxOm\ngcewcM05gsec2485t59VV5EsDRtJSUnKyxukDRvWa/z4b0kKlE2VlKzXlVdeFdRz+P1+7dnzL40Z\nMy6k166v91tyKQgta++cN+3KCHzI5OyuDJ8vnKOND/yc2485tx9zHvssv41y1VXX6rHHHlZe3oUN\nS19ra2s1deo0SdKjj/5KWVk9dNttP5EkLVr0O+XnF6h372xVVx/T66+/qq++OqRp06ZbPVTYjK4M\nAEgMloeNKVMu0dGjXi1Y8LIqKyuVk5Or2bPnNCxtLS8vk9PpbDj/2LEqPfnkr1VRUaFzzz1XeXmD\n9NJLC9W37wVWDxU2oisDABKHUVZWFXd/fczMPFcez3Euu9kkKckht7trUHNOV0Z4hDLnCA/m3H7M\nuf2SkuJqNQoSDV0ZAJC4+BUPS1VXS2vXOrVpk1P+U2E5OVkqKqrXyJH1p7oyAADxjLABS9TWSuvX\nO7Vhg1MnTwaOOZ3SsGH1Gj26nq4MAEgghA2Elc8nbdzo0Lp1SaqtDRwzDGnQIL/Gj/ephb33AABx\njLCBsPD7pc2bHfr73093ZQSc3ZUBAEg8hA10iGlKpaWGSkqkvXud8vsDoYKuDADAaYQNtNvproyv\nvnLIdWrrAroyAABnI2wgZGd3ZTgcUvfu0uTJ9crL89GVAQBohLCBoLXUlTF+fL0mT5aOHfOzfwkA\noAnCBtrUVldG164OSrkAAC3iLQItoisDABAOhA00QVcGACCcCBto4PdL27Y5tGYNXRkAgPAhbECm\nKe3a5dDq1U5VVJwJGXRlAADCgbCR4E53ZRw8eCZk0JUBAAgnwkaCOrsrQ5JSUkyNG1ev/Hw/XRkA\ngLAhbCTkaGm3AAARxklEQVSYlroyxo71adgwP0tYAQBhx1uLDUzT1M59Xnmr65TaLVm52akybL4/\n0VZXRufOtg4HAJBACBsWKykt17KVu1XmrWk4lpXqUvGkgRqRl2n569OVAQCINMKGhUpKyzXv7S0y\nz1rMUeat0by3t2jG9ELLAgddGQCAaEHYsIhpmlq2cneToHHm69Kyj3ZreG5GWG+p0JUBAIg2hA2L\n7NznbXTrpDllnhrt2n9Uudkdv8xAVwYAIFoRNizira4L8rwTHX4tujIAANGMsGGR1G7JQZ7X/mUg\ndGUAAGIBYcMiudmpykp1tXorJcvtUs753UN+broyAACxhLclixiGoeJJA5tdjRL4ulQ8cWBIHw6l\nKwMAEIsIGxYakZepGdMLteyj3SrzfKNnw+1S8cTgezboygAAxDLChsVG5GVqeG6Gdu7z6ujxOqV2\n66yc87sHdUWDrgwAQDwgbNjAMAzl9XEHfT5dGQCAeELYiCJ0ZQAA4hFhI0rQlQEAiFeEjQijKwMA\nEO8IGxFCVwYAIFHwlmYzujIAAImGsGETujIAAImKsGExujIAAImOsGERujIAAAggbIQZXRkAADRG\n2AgjujIAAGiKsBEGdGUAANAywkYH0JUBAEDbeDtsB7oyAAAIHmEjBHRlAAAQOsJGEOjKAACg/Qgb\nraArAwCAjiNsNIOuDAAAwoewcRa6MgAACC/Cxil0ZQAAYI2EDxt0ZQAAYK2EfSulKwMAAHskXNig\nKwMAAHvZEjaWL39TS5b8XhUVFRo4MEd33nmPBg3Kb/H8Dz/8QAsWvKRDhw4pO7uPbr/9pxo7dlyH\nxkBXBgAAkWH5xx5XrHhfL7zwnG666VYtXPi6Bg7M0V133SGv19vs+Vu3btYjj/ynpk2brkWLXteE\nCRM1c+bd2rPn83a9vt8vbdni0G9/20kffXQmaPTv79f115/UZZcRNAAAsJLlYWPp0sW6/PLva+rU\ny9S37wW6556Z6tKli957751mz1+27A2NGXORrr76OvXpc4Fuvvk25eZeqOXL3wzpdU1T2rnToYUL\nO+kvf0lqKOXq3dvUNdec1JVX+ijlAgDABpbeRvH5fCot3a7rr7+x4ZhhGCoqGqWtW7c0+5itW7fo\n6quva3Rs1KgxWrPm70G/7hdfSO+8k6T9+88coysDAIDIsDRseL1e+f1+ud3pjY673Wnau/fLZh9T\nWVmhtLS0RsfS0tJVWVkR9Os+OftrVR0/oc6dnOqf3VkXX+w/1ZVhSCJphJvT6Wj0/7Aec24/5tx+\nzLn9nE6HfD5/2J83QqtRTBkhXF4wTVOhhIQP1u+Vs1O90ntXyOh9UpPdg5Se3qsd40QoUlJckR5C\nwmHO7cec2485t1d5+bGwP6elYSM1NVUOh0MeT+OrEh6PR253WrOPCVzFqDzr/MomVzta03PgIXVL\nq5Yzya/DXmnWK+t1xxVDVHRhVujfBNrkdDqUkuJSVVWN6uvDn4jRFHNuP+bcfsy5/ay6imRp2EhK\nSlJe3iBt2LBe48d/S1LgKkVJyXpdeeVVzT6moKBQJSWfqLj46oZjGzb8n/LzhwT9ut2zqhr92TSl\nJSt2aeiA9JCuqCA09fV+Sy6/oWXMuf2Yc/sx57HP8hthV111rf70pz/qL3/5s7788gs99dSvVVtb\nq6lTp0mSHn30V3r55Rcazi8uvkbr1n2sJUt+r717v9CCBS+rtHSHrrjiBx0aR5mnRrv2H+3QcwAA\ngNBZ/pmNKVMu0dGjXi1Y8LIqKyuVk5Or2bPnyO12S5LKy8vkdDobzi8oGKKHH35M8+fP0/z5Lyo7\nO1uzZj2jfv36d3gs3uoTHX4OAAAQGqOsrCruyiZuevzDZo/fd+1w5WbT4BVuSUkOud1d5fEc51Kn\nTZhz+zHn9mPO7ZeUZM1qlIRZT5Tldinn/O6RHgYAAAknIcKGYUjFEwfy4VAAACIg7nd9zXK7VDxx\noEbkZUZ6KAAAJKS4DBuzZozTvkNHlXJOsnLO784VDQAAIiguw0bBgAz1TnPxgSIAAKJAQnxmAwAA\nRA5hAwAAWIqwAQAALEXYAAAAliJsAAAASxE2AACApQgbAADAUoQNAABgKcIGAACwFGEDAABYirAB\nAAAsRdgAAACWImwAAABLETYAAIClCBsAAMBShA0AAGApwgYAALAUYQMAAFiKsAEAACxF2AAAAJYi\nbAAAAEsRNgAAgKUIGwAAwFKEDQAAYCnCBgAAsBRhAwAAWIqwAQAALEXYAAAAliJsAAAASxE2AACA\npQgbAADAUoQNAABgKcIGAACwFGEDAABYirABAAAsRdgAAACWImwAAABLETYAAIClCBsAAMBShA0A\nAGApwgYAALAUYQMAAFiKsAEAACxF2AAAAJYibAAAAEsRNgAAgKWSrHriqqoqPfvsk/r449UyDIcm\nTpysn//8brlcrhYf89Of3qpNm/7Z8GfDMHT55d/X3XffZ9UwAQCAxSwLG4888oA8ngo9//yLOnny\npB577BE99dSv9atfPdriYwLh4v/TLbf8WJIpSercuYtVQwQAADaw5DbKl19+oU8+Wav77ntQF144\nWIWFQ3XnnfdoxYr3VVFxpNXHdunSRW63W253mtzuNJ1zzjlWDBEAANjEkrCxdetmnXtuinJzL2w4\nVlQ0SpL02WdbW33s++//VZdd9m1df/1VevnlF3TiRK0VQwQAADax5DZKZWWF3G53o2NOp1MpKd1V\nUVHR4uP+/d+nqmfPnkpPz9S//rVLL744R/v2fan//u8nQx6D08lnX+1yeq6Zc/sw5/Zjzu3HnNvP\n6XTI5/OH/XlDChsvvTRXr7/+SotfNwxDv//9sha/bpqmDMNo8evTpk1v+Of+/QcoPT1Dv/jFDB08\neEC9evUOepzl5ceCPhcd5/P5mXObMef2Y87tx5zbz4qgIYUYNq655jr9v/83rdVzevXqrbS0dHk8\nnkbH6+vrdexYldLS0oJ+vcGDC2Sapg4c2BdS2AAAANEjpLDRvXuqundPbfO8goIhqq4+pp07dzR8\nbqOkZL2kQIAI1s6dO2QYhtLTM0IZJgAAiCKW3Ajr2/cCjRo1Rk888Zi2b9+mzZs/1bPPPqlvf/vf\nG4LDkSPluvbaK7Vjx2eSpAMH9mvRot+ptHSHvvrqkNas+bsee+xhDRs2XP37D7RimAAAwAaW9Ww8\n9NBjevbZJ/WLX8yQYTg0adIU/fzndzd83efzad++vaqtDaw26dSpkzZs+ERvvbVENTU1ysrqoUmT\nvq3rr7/JqiECAAAbGGVlVWakBwEAAOIX64kAAIClCBsAAMBShA0AAGApwgYAALAUYQMAAFiKsAEA\nACwVk2Fj+fI3VVx8uSZPHqdbb71B27dva/X8Dz/8QNdee6UmTx6nH/7wGq1d+w+bRho/Qpnzd999\nWz/5yY80depkTZ06Wb/4xYw2/x2hqVB/zk/74IP/1cUXj9TMmfdYPML4E+qcV1dX65lnntD3vvdd\nTZ48Tv/xH1do3bqPbRptfAh1zt98c7H+4z+u0JQp4/T971+qOXNmq66uzqbRxr5Nm/6pe++9U9On\nT9XFF4/UmjV/b/MxGzdu0E03XafJky/S1Vd/X3/5y59Dft2YCxsrVryvF154TjfddKsWLnxdAwfm\n6K677pDX6232/K1bN+uRR/5T06ZN16JFr2vChImaOfNu7dnzuc0jj12hzvmnn5bokku+ozlzXtbL\nLy9UVlYP3XnnT3XkyBGbRx67Qp3z07766pDmzfuNhg79N5tGGj9CnXOfz6df/GKGDh/+So899pQW\nL16ue+99QJmZWTaPPHaFOufvv/9XvfTSC7r55tv0+uvLdf/9v9KKFX/T/PnzbB557KqpqVFOTp7u\nuuveVjdGPe3QoYO69947VVQ0UosWLVZx8dV6/PFHtX79upBeN+bCxtKli3X55d/X1KmXqW/fC3TP\nPTPVpUsXvffeO82ev2zZGxoz5iJdffV16tPnAt18823Kzb1Qy5e/afPIY1eoc/7gg49q+vQrNXBg\njvr06av77ntQpulXScknNo88doU655Lk9/v1X//1oG6++TY2LmyHUOf8z39+W9XVxzRr1tMqKChU\nz549NXTov2nAALZXCFaoc75t22YNGTJUU6b8u3r27KmRI0fr29/+DldOQzBmzEW65ZbbNWHCRJlm\n252ef/zjW+rVq7dmzPi5+vS5QFdc8QNNnDhFS5cuDul1Yyps+Hw+lZZuV1HRyIZjhmGoqGiUtm7d\n0uxjtm7doqKi0Y2OjRo1Rtu2bbZ0rPGiPXN+ttraGvl8PqWkpFg1zLjS3jlfuPC3crvduvTSy+0Y\nZlxpz5z/4x+rlZ9fqGeeeVyXX/4dXX/9VXrttYXy+63ZojvetGfOCwqGqLR0R0O4OHBgv9at+4fG\njh1ny5gT0WefbVVR0ahGx0aPHhv07//TLNsbxQper1d+v19ud3qj4253mvbu/bLZx1RWVjTZ1j4t\nLV2VlRWWjTOetGfOzzZv3hxlZmY1CX1oXnvmfPPmT/Xee3/SokVv2DHEuNOeOT948IBKSjboO9+Z\nqqeffl779u3T7NmPq76+XjfccIsdw45p7ZnzSy75ro4e9WrGjFtkmqb8fr++970rdN11N9gw4sRU\nUXFEbnfj91C3O01ff31cdXV1Sk5ODup5YipstMwM6t5Tw9mmKSn489Gc4Ob8tdcWaeXKv2nOnPnq\n1KmTDeOKZ83P+ddff61HH31I9977AFePwq7ln3O/36+0tDT98pf/KcMwlJt7oY4cKdMbb/yesNEh\nLc/5xo0b9NprC3X33fdr0KB8HTiwT88997TS09OZc1sFbr+E8r4bU2EjNTVVDodDHk/jqxIej6dJ\n8jotcBWj8qzzK5tc7UDz2jPnpy1e/JoWL35Vzz8/T/37D7BymHEl1Dk/cGC/Dh8+pHvvvbPhHuzp\n/584cYwWL17OZzja0J6f8/T0DHXq1KnRL9y+ffupsrJCPp9PSUkx9evVdu2Z8wULXtZ3vnNpw63C\n/v0HqKamRk899WvChkXS0zPk8Zz9HurROed0DekvkDH1mY2kpCTl5Q3Shg3rG46ZpqmSkvUqLBzS\n7GMKCgqbfDBxw4b/U35+8+ejsfbMuSQtXvyqXn31fzR79hzl5l5ox1DjRqhzfsEF/fTKK0u0cOFi\nLVr0hhYtekPjxk3Q8OFFWrToDWVl9bBz+DGpPT/nhYVDtX///kbH9u79UunpGQSNILRnzmtra+Vw\nNH7bMgxDpmkG9WFHhC4/v7DRvyNJ+uSTdSooKAzpeZz33HP/w2Ecl+XOOaerfve7l5SV1UPJycma\nP3+edu/epXvvfVAul0uPPvor7djxWcMHWjIzszR//jx16dJFKSkpeuutpVq5coXuu+9Bud3uCH83\nsSHUOX/99Ve0YMHLeuCB/2r4m0dNTY0kg1spQQplzp1Op9xud6P/ffLJWpmmdMUVP2jyyxnNC/Xn\nvE+fvlqy5DVVVFSod+/ztW3bFs2b9xtdeeXVGjp0WIS/m9gQ6pxXVBzR8uVLdd55veRyubRjxza9\n8MLzGj58pCZNmhLh7yY21NTU6Isv9qiy8ojeeecPGjw4X8nJneXznVS3bt300ktz9de/vqcJEyZJ\nks4/P1uvvvo/qq6uVo8ePbRixd+0dOnruvPOX6p37/ODft2Yi99Tplyio0e9WrDgZVVWVionJ1ez\nZ89pCA7l5WVyOp0N5xcUDNHDDz+m+fPnaf78F5Wdna1Zs55Rv379I/UtxJxQ5/ztt5fL5/PpwQfv\nbfQ8N974I914449sHXusCnXO0XGhznlWVg/Nnv2C5syZrRtuuEaZmVn6wQ+u0bXX/jBS30LMCXXO\nb7jhFjkcDv32ty/pyJEypaa6NX78BP3oRz+O1LcQc3bs+Ew/+9ntMgxDhmFo7tznJEnf/e6lmjnz\nIVVWVqis7HDD+eed10tPPvmc5sx5Vm+9tVRZWVm6774HNXJkaB/4N8rKqrj2BAAALMP1VQAAYCnC\nBgAAsBRhAwAAWIqwAQAALEXYAAAAliJsAAAASxE2AACApQgbAADAUoQNAABgKcIGAACwFGEDAABY\n6v8HfrrbIuofMCkAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f462b86ecd0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Visualize the solution regression line\n",
    "\n",
    "fig, ax = plt.subplots(1, 1)\n",
    "fig.set_facecolor('#EAEAF2')\n",
    "plt.plot(x, y, 'o', label='t')\n",
    "plt.plot([0, 1], [f(0), f(1)], 'b-', label='f(x)', alpha=0.5)\n",
    "plt.plot([0, 1], [0*w, 1*w], 'r-', label='fitted line', alpha=0.5, linestyle='--')\n",
    "plt.xlabel('input x')\n",
    "plt.ylabel('target t')\n",
    "plt.title('input vs. target')\n",
    "plt.ylim([0,2])\n",
    "plt.gcf().set_size_inches((10,3))\n",
    "plt.grid(True)\n",
    "plt.show"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'target_value' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-35-d882cf7e284f>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      3\u001b[0m \u001b[0mfig\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0max\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mplt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msubplots\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0mfig\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mset_facecolor\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'#EAEAF2'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 5\u001b[0;31m \u001b[0mplt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mplot\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0marange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtarget_value\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msize\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtarget_value\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'o'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0malpha\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m0.2\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      6\u001b[0m \u001b[0mplt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mxlabel\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Iteration'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      7\u001b[0m \u001b[0mplt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mylabel\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Cost'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mNameError\u001b[0m: name 'target_value' is not defined"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD8CAYAAAB0IB+mAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAEVBJREFUeJzt3H9MVff9x/HX1SsuJsOfVC6QCLbLqN91IMhtSKl8sTJ+WMtuTMVqZEXT2miMW62Smsza6rbbuNpKtlnbLXbc+KsKaqyVicUvU9NYL0xtu6GTW61yofhF469GsVe+f3wzGFN7rsC9WD/Px38n53PPefuJeebmXu6xtbRcahcA4L7Xr68HAACEB8EHAEMQfAAwBMEHAEMQfAAwBMEHAENYBn/BgrkaM2a0xo9/9Lbn29vbtWTJIjmdScrMTNexY0d6fUgAQM9ZBn/atBnatKnijuc/+miPfL4GHTp0RG+8sVqLF/+iVwcEAPQOy+Cnpz+mIUOG3vH87t0faurUZ2Sz2TRunFMXL17UV1819+qQAICes/f0As3NfsXExHUcx8TEqqnJr5Ejo29ZW1a2Th7Pe5KkkydPKDExsae3BwCjfPHFF6qvP9Wt1/Y4+HejqKhYRUXFkqS8vCx5vd5w3h4AvvOSk1O6/doe/5VOdHSM/P6zHcd+f6McjpieXhYA0Mt6HPzc3Dy9//5Gtbe3y+v9RJGRkbf9OAcA0LcsP9KZM6dYBw8e0PnzrUpKStTixUt048YNSdKzz87WxIk52rt3j5zOJA0aNEirV/8h5EMDAO6eZfDXrl33redtNptef31Vrw0EAAgNfmkLAIYg+ABgCIIPAIYg+ABgCIIPAIYg+ABgCIIPAIYg+ABgCIIPAIYg+ABgCIIPAIYg+ABgCIIPAIYg+ABgCIIPAIYg+ABgCIIPAIYg+ABgCIIPAIYg+ABgCIIPAIYg+ABgCIIPAIYg+ABgCIIPAIYg+ABgCIIPAIYg+ABgCIIPAIYg+ABgCIIPAIYg+ABgCIIPAIYg+ABgCIIPAIYIKvjV1VVKT0+R05mk0tJVt5w/e/aMXK5JmjAhQ5mZ6dq79y+9PigAoGcsgx8IBFRSslAbN5brwIHDqqjYquPH67usefPNlXrqKZeqqw/onXfWqaRkYcgGBgB0j2Xw6+q8SkgYrfj4BEVERMjlmqLKyl3/scqmy5cvS5IuXbqokSOjQzErAKAH7FYLmpubFBsb13HscMSors7bZc3ixS9r6tSf6k9/Wquvv/5aW7fuuO21ysrWyeN5T5J04UJrD8YGANytXvnStqJiqwoLZ+jo0Xpt2LBF8+Y9r5s3b96yrqioWFVVNaqqqlFUVFRv3BoAECTL4EdHO9TYeLbjuKnJL4cjpsuaDRvKVFDgkiSlpT2qa9euq7WVd/AAcC+xDP7Ysany+Xw6ffqU2tratG1buXJy8rusiY2N0/79NZKkEyeO6/r1axoxYkRoJgYAdIvlZ/h2u11u90oVFroUCAQ0ffpMJSY+LLd7hZKTU5Sbm69XX/21Xnxxvt5++/ey2WwqLV0jm80WjvkBAEGytbRcau+LG+flZcnr9VovBAB0SE5OUVVVTbdeyy9tAcAQBB8ADEHwAcAQBB8ADEHwAcAQBB8ADEHwAcAQBB8ADEHwAcAQBB8ADEHwAcAQBB8ADEHwAcAQBB8ADEHwAcAQBB8ADEHwAcAQBB8ADEHwAcAQBB8ADEHwAcAQBB8ADEHwAcAQBB8ADEHwAcAQBB8ADEHwAcAQBB8ADEHwAcAQBB8ADEHwAcAQBB8ADEHwAcAQBB8ADBFU8Kurq5SeniKnM0mlpatuu2bHjgplZKTp8cedeuGFWb06JACg5+xWCwKBgEpKFmrLlh2KiYnVT37y38rJydcPf5jYscbnO6nVq1fpgw/2aMiQoTp37lxIhwYA3D3Ld/h1dV4lJIxWfHyCIiIi5HJNUWXlri5rPJ4/a9as5zRkyFBJUlRUVGimBQB0m+U7/ObmJsXGxnUcOxwxqqvzdlnT0HBSkjRpUrZu3gxo0aKXNWFC9i3XKitbJ4/nPUnShQutPZkbAHCXLIMfjEDgG/l8Ddq+/UP5/Y0qKMhTTc3HGjx4SJd1RUXFKioqliTl5WX1xq0BAEGy/EgnOtqhxsazHcdNTX45HDFd1jgcscrNzdeAAQM0alS8HnzwIfl8Db0/LQCg2yyDP3Zsqnw+n06fPqW2tjZt21aunJz8Lmvy8yfp4MH9kqTW1lY1NJzUqFHxIRkYANA9lh/p2O12ud0rVVjoUiAQ0PTpM5WY+LDc7hVKTk5Rbm6+srImat++amVkpKl///565ZXlGjZseDjmBwAEydbScqm9L26cl5clr9drvRAA0CE5OUVVVTXdei2/tAUAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQwQV/OrqKqWnp8jpTFJp6ao7rtu5c4ceeCBSR47U9dqAAIDeYRn8QCCgkpKF2rixXAcOHFZFxVYdP15/y7orVy7r3XfXKDV1XEgGBQD0jGXw6+q8SkgYrfj4BEVERMjlmqLKyl23rHO7V2j+/J9r4MDvhWRQAEDPWAa/ublJsbFxHccOR4yamvxd1hw7dkSNjY3Kzs791muVla1TdnamsrMzde7cuW6ODADoDntPL3Dz5k0tXbpEpaVrLNcWFRWrqKhYkpSXl9XTWwMA7oLlO/zoaIcaG892HDc1+eVwxHQcX7lyWfX1f5fLNUmpqT9Sbe1hzZw5jS9uAeAeYxn8sWNT5fP5dPr0KbW1tWnbtnLl5OR3nI+MHKz6+lOqrf1MtbWfKTU1TR7PJiUnp4R0cADA3bEMvt1ul9u9UoWFLj322DgVFLiUmPiw3O4Vqqz8MBwzAgB6ga2l5VJ7X9w4Ly9LXq+3L24NAN9Zyckpqqqq6dZr+aUtABiC4AOAIQg+ABiC4AOAIQg+ABiC4AOAIQg+ABiC4AOAIQg+ABiC4AOAIQg+ABiC4AOAIQg+ABiC4AOAIQg+ABiC4AOAIQg+ABiC4AOAIQg+ABiC4AOAIQg+ABiC4AOAIQg+ABiC4AOAIQg+ABiC4AOAIQg+ABiC4AOAIQg+ABiC4AOAIQg+ABiC4AOAIQg+ABiC4AOAIYIKfnV1ldLTU+R0Jqm0dNUt59es+Z0yMtKUmZmuKVMm68yZL3t9UABAz1gGPxAIqKRkoTZuLNeBA4dVUbFVx4/Xd1nzyCM/1p49Naqp+VhPPlmg115bGrKBAQDdYxn8ujqvEhJGKz4+QREREXK5pqiycleXNRkZ4zVo0CBJ0rhxafL7G0MzLQCg2+xWC5qbmxQbG9dx7HDEqK7Oe8f169eX6Yknsm97rqxsnTye9yRJFy603uWoAICesAz+3diyZZOOHv2btm/ffdvzRUXFKioqliTl5WX15q0BABYsgx8d7VBj49mO46YmvxyOmFvW1dTs01tv/Vbbt+/WwIEDe3dKAECPWX6GP3Zsqnw+n06fPqW2tjZt21aunJz8Lms+/fSoXnppgTyeTYqKigrZsACA7rN8h2+32+V2r1RhoUuBQEDTp89UYuLDcrtXKDk5Rbm5+Vq27Je6evWqZs/+mSQpLi5OHs/mkA8PAAieraXlUntf3DgvL0te752//AUA3Co5OUVVVTXdei2/tAUAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQwQV/OrqKqWnp8jpTFJp6apbzl+/fl3PPfesnM4k5eZm6csvT/f6oACAnrEMfiAQUEnJQm3cWK4DBw6romKrjh+v77Jm/foyDR48RJ98clRz5szT8uWvhGxgAED3WAa/rs6rhITRio9PUEREhFyuKaqs3NVlTWXlLhUWPiNJmjz5p9q//3/U3t4emokBAN1it1rQ3Nyk2Ni4jmOHI0Z1dd47rrHb7fr+9yN1/vx5DR8+vMu6srJ18njekyT94x+fKzk5pafz3xdaW/9Xw4eP6Osx7gnsRSf2ohN70emf/zzR7ddaBr83FRUVq6ioWJKUnZ2pqqqacN7+nsVedGIvOrEXndiLTtnZmd1+reVHOtHRDjU2nu04bmryy+GIueOab775RpcvX9KwYcO6PRQAoPdZBn/s2FT5fD6dPn1KbW1t2ratXDk5+V3W5OTka/PmjZKknTu3KyMjUzabLTQTAwC6pf+iRS8v+7YF/fr10+jRozV37nP64x/X6umnCzV5coHc7hW6cuWKHnroBxoz5r9UXv6+fvWrZfr880+1cuVbGjJkqOXNk5LG9ta/4zuPvejEXnRiLzqxF526uxe2lpZL/DkNABiAX9oCgCEIPgAYIuTB57EMnaz2Ys2a3ykjI02ZmemaMmWyzpz5sg+mDA+rvfiXnTt36IEHInXkSF0YpwuvYPZix44KZWSk6fHHnXrhhVlhnjB8rPbi7NkzcrkmacKEDGVmpmvv3r/0wZSht2DBXI0ZM1rjxz962/Pt7e1asmSRnM4kZWam69ixI0FdN6TB57EMnYLZi0ce+bH27KlRTc3HevLJAr322tI+mja0gtkLSbpy5bLefXeNUlPH9cGU4RHMXvh8J7V69Sp98MEe7d//iZYvf72Ppg2tYPbizTdX6qmnXKquPqB33lmnkpKFfTRtaE2bNkObNlXc8fxHH+2Rz9egQ4eO6I03Vmvx4l8Edd2QBp/HMnQKZi8yMsZr0KBBkqRx49Lk9zf2xaghF8xeSJLbvULz5/9cAwd+rw+mDI9g9sLj+bNmzXqu4y/foqKi+mLUkAvu/4VNly9fliRdunRRI0dGh3/QMEhPf+xb/9Jx9+4PNXXqM7LZbBo3zqmLFy/qq6+aLa8b0uDf7rEMTU3+O67598cy3G+C2Yt/t359mZ54Ijsco4VdMHtx7NgRNTY2Kjs7N9zjhVUwe9HQcFINDSc1aVK28vImqLq6KtxjhkUwe7F48csqL9+spKRETZ/+tH7zm5XhHvOe0NzsV0xM517FxMR+a0/+JayPVkBwtmzZpKNH/6bt23f39Sh94ubNm1q6dIlKS9f09Sj3hEDgG/l8Ddq+/UP5/Y0qKMhTTc3HGjx4SF+PFnYVFVtVWDhDc+fO1+HDhzRv3vP6618PqV8//v4kGCHdJR7L0CmYvZCkmpp9euut36qsbLMGDhwYzhHDxmovrly5rPr6v8vlmqTU1B+ptvawZs6cdl9+cRvM/wuHI1a5ufkaMGCARo2K14MPPiSfryHco4ZcMHuxYUOZCgpckqS0tEd17dp1tba2hnXOe0F0dIz8/s698vsbb9uT/xTS4PNYhk7B7MWnnx7VSy8tkMez6b79nFay3ovIyMGqrz+l2trPVFv7mVJT0+TxbLovn64azP+L/PxJOnhwvySptbVVDQ0nNWpUfB9MG1rB7EVsbJz27///h6idOHFc169f04gR5j1FMzc3T++/v1Ht7e3yej9RZGRkUN9nhPQjHbvdLrd7pQoLXQoEApo+faYSEx+W271Cyckpys3N14wZRZo373k5nUkaOnSo1q5dF8qR+kwwe7Fs2S919epVzZ79M0lSXFycPJ7NfTx57wtmL0wRzF5kZU3Uvn3VyshIU//+/fXKK8s1bNhw64t/xwSzF6+++mu9+OJ8vf3272Wz2VRauua+fIM4Z06xDh48oPPnW5WUlKjFi5foxo0bkqRnn52tiRNztHfvHjmdSRo0aJBWr/5DUNfl0QoAYAi+6QAAQxB8ADAEwQcAQxB8ADAEwQcAQxB8ADAEwQcAQ/wfpruRaQt91DMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff8bcd75da0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Visualize the evolution of the target function value during iterations.\n",
    "\n",
    "fig, ax = plt.subplots(1, 1)\n",
    "fig.set_facecolor('#EAEAF2')\n",
    "plt.plot(np.arange(target_value.size), target_value, 'o', alpha = 0.2)\n",
    "plt.xlabel('Iteration')\n",
    "plt.ylabel('Cost')\n",
    "plt.grid()\n",
    "plt.gcf().set_size_inches((10,3))\n",
    "plt.grid(True)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Mini-batch Gradient Descent"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In code, general batch gradient descent looks something like this:\n",
    "\n",
    "```python\n",
    "nb_epochs = 100\n",
    "for i in range(nb_epochs):\n",
    "    grad = evaluate_gradient(target_f, data, w)\n",
    "    w = w - learning_rate * grad\n",
    "```\n",
    "\n",
    "For a pre-defined number of epochs, we first compute the gradient vector of the target function for the whole dataset w.r.t. our parameter vector. \n",
    "\n",
    "**Stochastic gradient descent** (SGD) in contrast performs a parameter update for each training example and label:\n",
    "\n",
    "```python\n",
    "nb_epochs = 100\n",
    "for i in range(nb_epochs):\n",
    "    np.random.shuffle(data)\n",
    "    for sample in data:\n",
    "        grad = evaluate_gradient(target_f, sample, w)\n",
    "        w = w - learning_rate * grad\n",
    "```\n",
    "\n",
    "Mini-batch gradient descent finally takes the best of both worlds and performs an update for every mini-batch of $n$ training examples:\n",
    "\n",
    "```python\n",
    "nb_epochs = 100\n",
    "for i in range(nb_epochs):\n",
    "  np.random.shuffle(data)\n",
    "  for batch in get_batches(data, batch_size=50):\n",
    "    grad = evaluate_gradient(target_f, batch, w)\n",
    "    w = w - learning_rate * grad\n",
    "```\n",
    "\n",
    "Minibatch SGD has the advantage that it works with a slightly less noisy estimate of the gradient. However, as the minibatch size increases, the number of updates done per computation done decreases (eventually it becomes very inefficient, like batch gradient descent). \n",
    "\n",
    "There is an optimal trade-off (in terms of computational efficiency) that may vary depending on the data distribution and the particulars of the class of function considered, as well as how computations are implemented."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Loss Funtions\n",
    "\n",
    "Loss functions $L(y, f(\\mathbf{x})) = \\frac{1}{n} \\sum_i \\ell(y_i, f(\\mathbf{x_i}))$ represent the price paid for inaccuracy of predictions in classification/regression problems.\n",
    "\n",
    "In classification this function is often the **zero-one loss**, that is, $ \\ell(y_i, f(\\mathbf{x_i}))$ is zero when $y_i = f(\\mathbf{x}_i)$ and one otherwise.\n",
    "\n",
    "This function is discontinuous with flat regions and is thus extremely hard to optimize using gradient-based methods. For this reason it is usual to consider a proxy to the loss called a *surrogate loss function*. For computational reasons this is usually convex function. Here we have some examples:\n",
    "\n",
    "### Square / Euclidean Loss\n",
    "\n",
    "In regression problems, the most common loss function is the square loss function:\n",
    "\n",
    "$$ L(y, f(\\mathbf{x})) = \\frac{1}{n} \\sum_i (y_i - f(\\mathbf{x}_i))^2  $$\n",
    "\n",
    "The square loss function can be re-written and utilized for classification:\n",
    "\n",
    "$$ L(y, f(\\mathbf{x})) = \\frac{1}{n} \\sum_i (1 - y_i f(\\mathbf{x}_i))^2  $$\n",
    "\n",
    "\n",
    "### Hinge / Margin Loss (i.e. Suport Vector Machines)\n",
    "\n",
    "The hinge loss function is defined as:\n",
    "\n",
    "$$ L(y, f(\\mathbf{x})) = \\frac{1}{n} \\sum_i \\mbox{max}(0, 1 - y_i f(\\mathbf{x}_i))  $$\n",
    "\n",
    "The hinge loss provides a relatively tight, convex upper bound on the 0–1 Loss.\n",
    "\n",
    "\n",
    "<img src=\"https://raw.githubusercontent.com/DataScienceUB/DeepLearningfromScratch2018/master/images/loss_functions.png\">\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Logistic Loss (Logistic Regression)\n",
    "\n",
    "This function displays a similar convergence rate to the hinge loss function, and since it is continuous, simple gradient descent methods can be utilized. \n",
    "\n",
    "$$ L(y, f(\\mathbf{x})) = \\frac{1}{n} \\sum_i log(1 + exp(-y_i f(\\mathbf{x}_i))) $$\n",
    "\n",
    "\n",
    "### Sigmoid Cross-Entropy Loss (Softmax classifier)\n",
    "\n",
    "Cross-Entropy is a loss function that is very used for training **multiclass problems**. We'll focus on models that assume that classes are mutually exclusive. \n",
    "\n",
    "In this case, our labels have this form $\\mathbf{y}_i =(1.0,0.0,0.0)$. If our model predicts a different distribution, say  $ f(\\mathbf{x}_i)=(0.4,0.1,0.5)$, then we'd like to nudge the parameters so that $f(\\mathbf{x}_i)$ gets closer to $\\mathbf{y}_i$.\n",
    "\n",
    "C.Shannon showed that if you want to send a series of messages composed of symbols from an alphabet with distribution $y$ ($y_j$  is the probability of the  $j$-th symbol), then to use the smallest number of bits on average, you should assign  $\\log(\\frac{1}{y_j})$  bits to the  $j$-th symbol. \n",
    "\n",
    "The optimal number of bits is known as **entropy**:\n",
    "\n",
    "$$ H(\\mathbf{y}) = \\sum_j y_j \\log\\frac{1}{y_j} = - \\sum_j y_j \\log y_j$$\n",
    "\n",
    "**Cross entropy** is the number of bits we'll need if we encode symbols by using a wrong distribution $\\hat y$:\n",
    "\n",
    "$$ H(y, \\hat y) =   - \\sum_j y_j \\log \\hat y_j $$ \n",
    "\n",
    "In our case, the real distribution is $\\mathbf{y}$ and the \"wrong\" one is $f(\\mathbf{x}_i)$. So, minimizing **cross entropy** with respect our model parameters will result in the model that best approximates our labels if considered as a probabilistic distribution. \n",
    "\n",
    "Cross entropy is used in combination with **Softmax** classifier. In order to classify $\\mathbf{x}_i$ we could take the index corresponding to the max value of $f(\\mathbf{x}_i)$, but Softmax gives a slightly more intuitive output (normalized class probabilities) and also has a probabilistic interpretation:\n",
    "\n",
    "$$ P(\\mathbf{y}_i = j \\mid \\mathbf{x_i}) = - log \\left( \\frac{e^{f_j(\\mathbf{x_i})}}{\\sum_k e^{f_k(\\mathbf{x_i})} } \\right) $$\n",
    "\n",
    "where $f_k$ is a linear classifier. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Advanced gradient descend\n",
    "\n",
    "\n",
    "### Momentum\n",
    "\n",
    "SGD has trouble navigating ravines, i.e. areas where the surface curves much more steeply in one dimension than in another, which are common around local optima. In these scenarios, SGD oscillates across the slopes of the ravine while only making hesitant progress along the bottom towards the local optimum.\n",
    "\n",
    "<img src=\"https://github.com/DataScienceUB/DeepLearningfromScratch2018/blob/master/images/ridge2.png?raw=true\">\n",
    "\n",
    "Momentum is a method that helps accelerate SGD in the relevant direction and dampens oscillations. It does this by adding a fraction of the update vector of the past time step to the current update vector:\n",
    "\n",
    "$$ v_t = m v_{t-1} + \\alpha \\nabla_w f $$\n",
    "$$ w = w - v_t    $$\n",
    "\n",
    "The momentum $m$ is commonly set to $0.9$.\n",
    "\n",
    "### Nesterov\n",
    "\n",
    "However, a ball that rolls down a hill, blindly following the slope, is highly unsatisfactory. We'd like to have a smarter ball, a ball that has a notion of where it is going so that it knows to slow down before the hill slopes up again.\n",
    "\n",
    "Nesterov accelerated gradient (NAG) is a way to give our momentum term this kind of prescience. We know that we will use our momentum term $m v_{t-1}$ to move the parameters $w$. Computing \n",
    "$w - m v_{t-1}$ thus gives us an approximation of the next position of the parameters (the gradient is missing for the full update), a rough idea where our parameters are going to be. We can now effectively look ahead by calculating the gradient not w.r.t. to our current parameters $w$ but w.r.t. the approximate future position of our parameters:\n",
    "\n",
    "$$ w_{new} = w - m v_{t-1} $$\n",
    "$$ v_t = m v_{t-1} + \\alpha \\nabla_{w_{new}} f $$\n",
    "$$ w = w - v_t $$\n",
    "\n",
    "### Adagrad\n",
    "\n",
    "All previous approaches manipulated the learning rate globally and equally for all parameters. Tuning the learning rates is an expensive process, so much work has gone into devising methods that can adaptively tune the learning rates, and even do so per parameter. \n",
    "\n",
    "Adagrad is an algorithm for gradient-based optimization that does just this: It adapts the learning rate to the parameters, performing larger updates for infrequent and smaller updates for frequent parameters.\n",
    "\n",
    "$$ c = c + (\\nabla_w f)^2 $$\n",
    "$$ w = w - \\frac{\\alpha}{\\sqrt{c}} $$ \n",
    "\n",
    "\n",
    "### RMProp\n",
    "\n",
    "RMSProp update adjusts the Adagrad method in a very simple way in an attempt to reduce its aggressive, monotonically decreasing learning rate. In particular, it uses a moving average of squared gradients instead, giving:\n",
    "\n",
    "$$ c = \\beta c + (1 - \\beta)(\\nabla_w f)^2 $$\n",
    "$$ w = w - \\frac{\\alpha}{\\sqrt{c}} $$ \n",
    "\n",
    "where $\\beta$ is a decay rate that controls the size of the moving average.\n",
    "\n",
    "\n",
    "<img src=\"https://github.com/DataScienceUB/DeepLearningfromScratch2018/blob/master/images/g1.gif?raw=true\">\n",
    "\n",
    "(Image credit: Alec Radford) \n",
    "\n",
    "<img src=\"https://github.com/DataScienceUB/DeepLearningfromScratch2018/blob/master/images/g2.gif?raw=true\">\n",
    "\n",
    "(Image credit: Alec Radford) \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Once deleted, variables cannot be recovered. Proceed (y/[n])? y\n"
     ]
    }
   ],
   "source": [
    "%reset\n",
    "\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "\n",
    "import numpy as np\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.datasets.samples_generator import make_regression \n",
    "from scipy import stats \n",
    "import random\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW0AAAFpCAYAAABXkHk0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJztnW+Ibed13tfaM3OvJMeWcKUibLm1SY1BNaYBkTa00D8xVElDQ1IC8YeWkMCl0EACgRBX0HwogYAhFNqUIrDJF5P0g2NsWpfYhoBbqEMcY1LbsoMIMXEIDY7xX0n3zpz99sPcFM3dz3PvXtr7nD3vzO8HBt931ln7PWfOWbN1nme9K1trAQAAfTBsvQEAAJgPRRsAoCMo2gAAHUHRBgDoCIo2AEBHULQBADqCog0A0BEUbQCAjqBoAwB0BEUbAKAjjre46BsffUN76sknLqy1NspY2WZvWu9bqBwmVuZYIVauu6MCpuurHCqQleAVriiux+EI22B/9Yt/IYU31X7eUvfZw3Q9K7G5n9jzVXFfbHJ84cWvfK219oT84avYpGg/9eQT8dH//KsX1s7OXpGx4+50srY7uy1jd+N0fTdOHx8RMYp19XiXQz0+ImJsIjZ0bGu76dpg/nil+CSY94+Kbe6/qVRsqejrfcj9Fh7vKO9NXm9hVVlhE6UtVF5Kt7dSDvV4U5TE27WyBxeb4/QNm3kkY4c4ma7ldC0iYhim60di7Xz95qzHV3McHU/XIiLe8c9+5ivyB/fuYU4QAABcDijaAAAdQdEGAOgIijZMOH3d2dZbALiUOC3rkGwiRO52t+Ob3/yTe9a0EFkRF1WsEwHVehuMI0SsN6076BwrCIaVvNKV4vIaRUwV7tGIWcoxo+VUTSVvBZd3XwwL9UnneKjktXqzyG3zlgRDfbXZecc1BM5p7Sjl3ZnYU/GaCdHzfutSiDSi5Vy40wYA6AiKNgBAR1C0AQA6gqINADATp5Edkm2EyPFOfOvlP7mwZrsGlQh4ZATDk2XCnhcixZoR8FQOJ8rthDKzMy36SlTbGaFtVELktPnS7s3uQafQsfaoARVbyHtFGuSHQhuoEwzV8mDbrNUeNEcihxU4xeXcczsSSQbd5Cj3cOSEWrUvJ0QKMVOLqTrH+dr0w5SjNlLk+J3pmhE+58KdNgBAR1C0AQA6gqINANARFG0AgI6gaMOEr53Sxg6gcGaFQ7KJe6TlGHceuqi2WueGcIpUXB474/KoODfOpHPD5BX1zudVe5Ch0jVRco+YWLWHCF24ncuj4iqRLe+Fz8H2H5l1qPgHvHtkmVOk4ko5Nq6UkntkYazbg3SamOemYo9NFaw4WI4q540bZ8tcuNMGAOgIijYAQEdQtAEAOoKiDQAwE6WFHZpthMgh4vSRi0KXezFOhVCmhMGIiDOhzJ25vEL9csKe2psT8FSOisDpzpBWeV2ruIw1eeX8+oLAeb6PeXnPY+e/6SsCp3uNt0aJWY7qHZRqWXc5lOjoREAtLpq8IocTAdV+XawSWY8L4qLLq8RMKy6K5ROj3h6bHMci3gmfc+FOGwCgIyjaAAAdQdEGAOgIijYAQEdQtAEAZqIMDIdmE/fIWbT4xtnFg8SVS8Stu9hKW7h0jxScG949Mn8PKodzTOjnZl4H2So+3z3iBzHMz2GdLXJVsy+nySGp3BVVBhhERKj5AS6HbmOf3xbuXSnCHWHdGMoRMn+/vo193rVcDtvyLp7HyZDyTVxxq5wUf8/3wp02AEBHULQBADqCog0A0BEUbQCAmVxfIbK1+Muzi23s7sU4LbSFK4HSnnstlu8UBM7TpqUvHTtf2HMCp8pRaRWv7KHagr4TZwmPlfsBcxZxK506PT/v3jBnt89+uDvXopB3ML+9I5FjDdFSiWqVvE6UUwJeZQ9OtDzJ6S5c7A0nWkohcv7eKkcbKLjTBgDoCIo2AEBHULQBADqCog0A0BEUbQCAmbju5kOyiXtk1yK+cfra29jdEITb41Q5d0MQZGxpD8ZpIvLW2u61sjy2adOyc1c0EStH1UdEk9dzeZ3qPc3tYwvKecH94a+3LVlxlJTdJ9N4f73p+9LHFvKKceOpRpCHdscMeSYitdvFO0KEe2TQ7/dKW7mKvenymve1infDHObCnTYAQEdQtAEAOoKiDQDQERRtAICZKC3s0GwkRLb4+j3nad82beEVwXCpuHjH/EJOlQhohb2T6dqoTj6OUKKcFBHd9Vz7t1h3efUe5guONr4iIu5JnPTXW3avkmuc3l0QHW17eyGvFhKNYChjXd5pT7cXLZXA6fagTl43exjUHlze6fqJvFbEDSEiVkTLCCdEbtzGnplvyczfzcwvZuYXMvPnluYEAADNGnfaZxHxC621z2bm6yPiDzLzE621L66QGwAAXsXiO+3W2p+31j579/9/OyJeiIg3L80LAABTVhUiM/OtEfF9EfF7a+YFAIBzVivamfk9EfGhiPj51tq3xM9vZeZnMvMzL3379lqXBQA4GK67+ZCs4h7JzJM4L9gfbK39topprT0fEc9HRDz+tsfa107vXPj57RUcIbdH1U5tHA/j9Kl7R0jBPSL3UHCEOIeGyltqYy8MGrCuFPc3vpBjb23sl9O96lwMOrhWEKSrpNSabvamXB6FAQ3a+aFz5GBHboi8850mLq/KsTP7va1a9Afddu/2dnM4naxdBvdIRsT7I+KF1tqvLc0HAACeNW5P/n5E/MuI+CeZ+bm7//vhFfICAMA9LP56pLX2v6J0dBsAALxWLucXgQAAlxClhR2aTXZw2lp87fTiF/Sn5sUYC23hrRXERSkY6j1IIVI83l3PCpGqhdy1vKu87j9wCm3stfO05wuRpfOt9zaN/cD3JBXRUT18hWnstTOy3X7nn6ctRUcXK4VTIxiq1vRSK73LqwROI5yKs75zGCN2N2dfbydz6OvNhTttAICOoGgDAHQERRsAoCMo2gAAHUHRBgCYiTMgHJJN3COtDfHK6SP3rBnnhnBTOCfEWIit5C05QlSsdbuoN8D81nTn0NADE4zbZY0WdPmcjYuh0Klt5mLo2O2PhJBUOpazWg9E7hzMBVVLdqU13cZW2s3nt7xX3C61IQgLY4ddhDLMmOcxyL3hHgEAuDZQtAEAOoKiDQDQERRtAICZ+CHdh2MzIXJ3dlGIVCJiRO3c61G1sVsRULWm70mIdO3UhVi9Xy0ONaFzWKFOaDAu1oqL4szzijBoBceF4qLd756wIuDsBC5vIYVrYxc5UkwKP88x7/EuNk1dU6+PbyEXb4rChPVVhEgnIioh0rSmD6qNfeFxB9xpAwB0BEUbAKAjKNoAAB1B0QYA6AiKNgDATJwB4ZBs5B7JODt76OKaG4Ig3SNrtKZXBhsUprGr2J1W9EclOBthedyJH9hYtTEdqxwWVTeHiq+4VRytYEGptLzvDzVtfP6jszqlWzlCKg4UN6xAOk1cYrEtV9eG6fWGI/MCCWeLy5tHy1wpNrY0MEGvj7SxAwBcbyjaAAAdQdEGAOgIijYAQEdQtK8Jp9+dr9SdffeSHk7dIaffnv9ann7nUqip146z0+l0dce1dY9EG2J39vDFJXPux77cI6N0eZjD/9VZHkYAVi4PFyvzVs4TMZ9xdz1VuF0OVbjt9dS6ex57GoJQcaUclGF+4c6hVrj1GSH6PawdISaxjDVOE7EJf/aIWDtyDpZCXpHDuVJSrOeRLtwyb+4i4mQaax0ouEcAAK41FG0AgI6gaAMAdARFGwBgJkoLOzTbtLHHELt72tiV4Bjhppu72OkLOp7OF/ZG024uY11e1dJdECLdHpTQNk7PV7/7AyEiuhZ0kaMkOJrrOeTrcxUExyqV9nY3gECuuyEIQtgr5E1TKeTQBSOGDiqH62IXtdELkWLNCKfDyXyBc1Dt8UcRoyibw8lUnIyISHGmRA7ugzsP7rQBADqCog0A0BEUbQCAjqBoAwB0BEUbJpx9hzb2LeB1v/w4U8Eh2aiNPSfuETdUYNxN151rQg0bKLlHzgot5LaNXTg33H7F9eQAg4jl7hHnCDE5VAFxLfalwQa4R84xt0vqda+5R9z1VN5Cy7tzj6g9OEdIyT0i9nts9ivyKudHhK4H1j0i8tpY8zlSbfPD8Q0dPBPutAEAOoKiDQDQERRtAICOoGgDAMzk2gqRrWXszi5e2olqSjiwImApVqy5NnbV6m1ESyUkWiFSiXJODK20m4sctbOwC7FhBMqKOOna/JeKjocWLRfeAlnBsXLchRUtVRu7a3lXj3d5xZqpKmMlVoiO9kxvKUS6WJHX7EG10g8nEaFqhzE8qLZ5MzpgNtxpAwB0BEUbAKAjKNoAAB1B0QYA6AiKNkxgGjuARhkYDs1G7pGI3e2La059PaQjxLa8q9hCa/oa7hF5PeOOkO34xcntsnAX3CMl58ee3COL3SdFSm3llccX3AallveKI8TtQcTKdvVirHJ02OcmYpX75Px6qj1e521irkEea4eYa5tXR2uU3ECCVe60M/MDmfkXmfn5NfIBAIBmra9HfiMinl0pFwAAGFYp2q21T0XE19fIBQAAHoRIAICZOH3qkBxMiMzMWxFxKyLi+PWPxu6Vi1/Qu7OslQBXEgwPLC6q5+HP0573+PMfiCWnZMszqwvT403edcTFhaLlFT9PW+HFRSF+WZGrcE63EuAKexjtJPTpmmptjzBnWRdESydwNpnXiIjis5jHIdvYlWgZYURL0dpe4WB32q2151trz7TWnjl++JFDXRYA4ErB1yMAAB2xluXvNyPif0fEOzLzq5n5M2vkBQCAi6zynXZr7T1r5AEAgPvD1yMwYfcybewAimvlHnk1523sFwtDZbDBeGf+AILxjtnDJXCEKJeGb3kXeyi4POwAA+OukYV7X4MN3JT3Qt7eqA02cFPTF7ZIO0fIbbFmHA9yDyavem+7ifBNtZsbh8Yg3B/q8RErOU3kQBLjmCkcPzEX7rQBADqCog0A0BEUbQCAjqBoAwDMxB2dfEg2ESKjTcU5JxjK87SdYCgESjvlXcQeWlyUOZwgq3LY2GXnadu2ctcKX8ph1mfmrdDdedqFFvSIqE1NV7mtwCnWXLGSZ2SbNm3xHlbnW9vYs0sgWt7IaMoI4d5rN8T12rLCz502AEBHULQBADqCog0A0BEUbQCAjqBoAwDMxHVjH5LN2tjvdYvYaezCVWInlotYO1xB5pWh8npuAIFsjzex+3KEyL3t0fmxhgNF5j2g02RfVNrKy+4T4f6ouUeMa6I0jV04OtzvWO1tZ5wm6npisnlERIr3SY4mr9ibi1V5hxumcLvrqeETNnYe3GkDAHQERRsAoCMo2gAAHUHRBgCYievcPiQbtbG3ifBYOffatrwX2tiV6GgFTnG+cKmNvXCWtRVDldjn9iBjC3mrbezq/O6KMFj4JLR4qZDY5fjuosdnvG7+tcxrmTEdbm0l2uGGyz7NawRDKVBW2thdXtECPrrnLFrIR/OeUi3kY8wXDNOJoUIEHOwrL4TeY/N2tQJ75Xrz4E4bAKAjKNoAAB1B0QYA6AiKNgBAR1C0AQBmcm2nsUebPvlVWtOlI8RsQblShEvEXc9OQhfrbr+qLdy2xyunSaHd3OVVqrd1uzhHiPglVVwe3s2xzOWx1CXi82pKrhK5qh+f4/y8MU5dKRERTThQ8mh+u3kzrdfqrs86ZkQbupvyrvOaiedu6IJglK+824PY73HqoSjugvKoANrYAQCuDRRtAICOoGgDAHQERRsAYCbXdhp7a1PR0IpfqkXaKbgy1oiWst3c/EIKrd4yR+HcayU4uutZMVRdzzw3ffa2bit34mJN8JvGVh6/Rhv7vqjsTbWxe+HVCZHTdfdaSjGzIFoOJ+79M/9M71AT0t05+qr920xY15850/KuPsvmXHH13IYTXVPSHAkgj8tYeKvMnTYAQEdQtAEAOoKiDQDQERRtAICOoGgDAMzEdhYfkM3a2O91M9h280KbtWw3t63X6lour4h1Lg8Va/ewrDW9MqzAt7HPb0GvtJvvzxHyrUKsu94XFj0+42+vsIc1mL7GrpW+cj01VGA81YMYlKukFYYV2CMB1GfDTW6XFFwpbsCDGq5wrD/P7qiKQbhKrEttJtxpAwB0BEUbAKAjKNoAAB1B0QYAmMn1PU877iOi3YtsyV5+HSkGVCaIFyidT12Knd/yXmFf51uf51ai43xxcamIuAZr7KEiZvrzu3UbukaJxcvzqvdamsxKSHTSony/m1tMNyl+NuZzr9rb8zjNczYsrF8K7rQBADqCog0A0BEUbQCAjqBoAwB0BEUbAGAm13YIgmKp42GvVIYg7MmBsi/WGCqwr6nnUMcOQShNip++JzJ0G/tloOQ0UbGFKpiFye/7YpU77cx8NjO/nJkvZuYvrZETAACmLC7amXkUEb8eET8UEU9HxHsy8+mleQEAYMoad9rfHxEvttb+uLV2JyJ+KyJ+dIW8AABwD2sU7TdHxJ++6t9fvbsGAHCluFZCZGbeiohbERFHD79h+nPz52P7lygiRJusOmvXxV5mkVW1LFeFRSVyIU5uQ0Vw9Dkq7fHbo9rY7UT4hS3vV0WI/LOIeMur/v3U3bULtNaeb60901p7ZrjR15sCAOCysEbR/v2IeHtmvi0zb0TET0bER1fICwAA97D465HW2llm/mxE/E6c/8fHB1pr2x/FBgBwBVnlO+3W2sci4mNr5AIAAA9t7AAAM7lW7pF7uVfdtS+F+rNS+FNjVWQ1JfnI7GLhtIqKku2cJkqzbqOZei1ezUp7fX2i93yniHarzGedSejbT2OPmDqo/PXWEO6nv9NVnCby82nel8q5URhsYD9HS3HT2MXzyGrFXFi/5qYEAIBLCkUbAKAjKNoAAB1B0QYA6AiK9mskT+a3s1amRVfElkrscFLIe6nbmOcLeDWx7wcuwR4u8+s+n329h0ufo8rns7CHttCUsAbbuEcyJgpzHmsPQQqHRIrR9hERgzgXYBx1rHJpuF/eIF6l8cy9MYRz40y/4aT7Y9fkPuSz2Ok9N+U1GZss3OOpONx+vFMqIGpv3pmw7EyS8wP6lxVNV6APWbjd66vW/Ws53/2hctjf8TB9T9g/+s4pUnBNLHWgOEeH+nyqz7Lfg1gyr4OqPS6vcqVU4E4bAKAjKNoAAB1B0QYA6AiKNgDATMbTrXewkRCZORUERte+rb7IN7uW4qIRCAYhn42mLVy1t9shCKo93gmnShiUDesRcSpyGFFE7W08Na/DiRBORzN5e6fXc1St6W7KuxLPnDg5Xa9NFT/sIIblbeH68ZW8FXHRuTEqwqAU5gqCo3N51PIqs4KOlQNNCs/NDkFwexO1amk7PnfaAAAdQdEGAOgIijYAQEdQtAEAOoKiDRMuw8Tp68gQT2y9BXgA13cIQk5VVdWuHqFdHnokQESIlnVn8lA59LV07Fg4un84Mq4U4Qhxf0WbcqWYN5AaeGDz7kTeURdud3yAdJsYp4lCuU/OmRYx70oRec36UlfJKsMDjMtDFm7h/LB5V3CESJeHy6vcETZWvKcqeY3TRLtSdF7Vbj7cdHuYPwTBtcfL683/deqcyx4OAACHhKINANARFG0AgI6gaAMAzOQan6edMUxEBS1yqbbuimAYRjzTAmVBDDUijmpZb+a8AvXcXIvrKETHNAKnOh8hzaR5JVq6ifDh2vxLOdQ+tDIjjyVwsaVp84/ND17IvgZgRIR8D7ocpUnoBSFS7cGfvS2WnFNJXM+ehV1oN6+0pivR0YmI7nmo+KXuLO60AQA6gqINANARFG0AgI6gaAMAdARFGyYMN2ljB1CMd7bewZZDECaqqm86nqzYacbzXR46r4lUOVxLtxyCYLagHB3OBaGcIi5W7a3o/JCF2022X8GBMjdvBbuHPbH0cPuK0yQi5C1XzT2yvI19X44QHTvf5VFpY7fT3KV7xLhEXBu7iHfXmwt32gAAHUHRBgDoCIo2AEBHULQBAGYy3rnG52nfKwik0S6UKDKmeeGUsGLUXpXXnSsgW8jPjLAnXlH1+IiIFBPS3R5STU037fHq3GEnyqkzvfcpIsrcRuCs5O2Nkui4hmAo8+rlytR0GevyqhZy99wOKS5WBM5Cu3pExKAEVSNmzoU7bQCAjqBoAwB0BEUbAKAjKNoAAB1B0YYJRw/Txg6guLbT2DMjju5pk3YOC6mcO3VaOC/sBHHh/hidc0OsH9ppopwbo/vtCYdGcy3ool29nerCXWoLt+4R5VYp/JE4cGv63ijcLlk3hvpsrDB0Qbs8CnswsXI4gou9DI4QGav3MB3qcp+9mdi5cKcNANARFG0AgI6gaAMAdARFGwBgJtd3GvsQcfTQPW3sriVbCIlDZQq5eYbqxU/R0n0eux/RUgl7gxU4RaxpnZViZuU8bZPXiYBaXDQ51LnplZb3At2dp23PXV+eoyIY7u08bRlr9qCESLffA4qLNtbUJDn9vXpu+r05lzw4M38iM7+QmWNmPrNsKwAA8CCWfj3y+Yj48Yj41Ap7AQCAB7Do65HW2gsREemO6AMAgFVBiAQA6IgH3mln5icj4knxo+daax+Ze6HMvBURtyIiTt7w6OwNwuE5fl3G2Xe3b9cFuGy0s+WDeZfywMu31t69xoVaa89HxPMREY+86U3t6ObFn6dRX5XS6g7CTzHdfNjtx2kymHbzUVyv4h5xscpV4twRlVg9lOC8cC/NMZfSwIQKh25535N7ZJ3BBvPdGKUp7yrWtZDLWJN3oXtEDR+weQuOENmKH/53JB0oW7pHAADgsCy1/P1YZn41In4gIv57Zv7OOtsCAADFUvfIhyPiwyvtBQAAHgBfjwAAzERpYYdmo/O0WxwdX1TLhiP992M8nq67FnL1pf9oWtOVoODyylZvJ3DuREu3yyvETCVknv9ALLlzEOR52m4PhVg3Nb0ypb2wN32tQuxlZpXztJddz05CrwiGag8mr2xZd+dpC7HOCpxSiJxvbLAiohItC4Kji3et+3PhThsAoCMo2gAAHUHRBgDoCIo2AEBHULQBAGbiuncPyTZd9Nni6PiVC0ujGS2ew/TvynCkY1ubSrVuYrl68UfXTq1inStFOClsm7Zwiij3yfkPxFLFPWLS1twjztky/5wS3CN3Obh7pNAWXnGPqLfEKu6R+c4N6QixE+ELjpDCHqx7JKcf/nSTTmbCnTYAQEdQtAEAOoKiDQDQERRtAICZXFshMmOcCJHDqL/hVwJlM1/kN5FjMAfrjkK0dGdv64nlhTb2gsBpRcBSrBBDjRJZOQu7JFAacdIJopXrSS6rQLmC4GjjCyKgFBfXOE9bbMIKhoUp7/L874IQuUYbuzyfXwiLEVpwjIjIQQiRJnYu3GkDAHQERRsAoCMo2gAAHUHRBgDoCIo2AMBMlIHh0GzXxn500T3SjMsjlXvEvHDKPeJiB7HeRMt8REQ7VnlNrMprWtNl27xpFS+5UpSTohlnjHB5WNeGa4WXbfPzhzk4nONl7h4uAxVHiHJi3BflxjAp5D4KsW5ggspRmSrvXB7qudkJ66rd3Lo8pm8U7/woxOIeAQAABUUbAKAjKNoAAB1B0QYAmInTyA7JRtPYxzg+efnC2mja2AcpRJrJ7U3EmrxSMLSx0+tZMVTFmknzRzJvQeB0reKq5d1peur8bys4zj/re5V2dSdmzs5b2MQKWLFudgKXt5DCbaEyjV22x8+/Xqk1vSAYhlozsfLx5no2tiAiqtiIiCGnR264682FO20AgI6gaAMAdARFGwCgIyjaAAAdQdEGAJjJtXaPHB2/dGFtEM6PiFprunKgrNHyvtg9UnClOAuBdI+41nSV17lS1PVcXmdvkM/ZWVD0sgwttbyr1YVujhWodKZXhyDoFnJnQREOC3emQE5fzHS/uJJzQ+S1Ld0qtuLyWME9Iiep19rYB9rYAQCuNxRtAICOoGgDAHQERRsAYCZOnzokmwmRD51cFCJPRbt6hD503L1wSsy0beGjEheNGCr34NrNC6KlUJLsm0LltcKgak13YqjKURA4Tbw9T1smLgqfs/Me+J5kYXuyF/vmq7dK7DtnvrCnY3VeKaq5WPX8KuKieX1qremVM7JFC7p4/H2vJ3MgRAIAXBso2gAAHUHRBgDoCIo2AEBHULQBAGbiDAiHZBP3yElmPH5ycmHttjk1//Z4Olk7a3dMbGGoQGG4gm4hr7hSKm3shbyFlveSQ6PSHh8R0m2yhiOk4EDxe9uW0oH3BZdIhHNjrOAeKbWxz29NVzmcG0NN1qi1m1cGJhQmtw9TN8j99nZT7OO4craB4HK+0wEAQELRBgDoCIo2AEBHULQBAGaitLBDs8kOjjPj8ZMbF9Zum8OTb4/T9TMrWqpYnfe03Z6s3RGPP48tCJGF87R1+3dBtLSC4fy8tRZ095znC5Hy8YV2/DVoC+9V7DnUS6m0f1dzyPX55167g9B1C3lFtJwvLto9lM7Tnq6fGCHyxjB9n5wYEdGJizeHk8kaQiQAwDViUdHOzPdl5pcy8w8z88OZ+dhaGwMAgClL77Q/ERHvbK29KyL+KCLeu3xLAADgWFS0W2sfb639ldv80xHx1PItAQCAY83vtH86Iv7HivkAAC4Vp8YEcUge6B7JzE9GxJPiR8+11j5yN+a5iDiLiA/eJ8+tiLgVEfHo44/EG48vuhlOjRtDvUhn5nWT7hGjOFdcKXoPOvZMtN2fmrZ7lWNnHBNyGERlWIF1u1SGIMx3lfjYfbWxbz95XeGHEqjgakGYP6xAt4VXWt5XmMYu8g7GuXEkrudcF8rRcSycH+c5lCNEO6vU9W66vOZ9reKPF75VH1i0W2vvvt/PM/OnIuJHIuIHW/N/hlprz0fE8xERb/7eN27/5woAoEMW+bQz89mI+MWI+IettZceFA8AAMtY+p32f4qI10fEJzLzc5n5X1bYEwAAGBbdabfW/tZaGwEAgAdDR+Q14fU3pm37jkdu8E3XFpycfGfrLVxL/vo9R2rcD2VgODSbnD1ylBGPndzjHhmdc2O6trMuj+nfIBe7E8t3nCNErJ+aM010rM47FlwpKsdonDFjmx7Uftp04VZ7GEMXbhUboR2L2sGoAAAIpElEQVQvY+V+YI2BCYW8e6Ps/pgiC3ch72DOE1FujMG4MeRojoJzo5L3JHUJUs6Nyh6800S/L1XhviFyHJm31FFhby52LtxpAwB0BEUbAKAjKNoAAB1B0QYAmInSwg7NZkMQ/trxxUs7sU6tu1jdFq73oITPnRH2VA4nGCrh0+1B5XDatH5u5nUQz8OJiOp6Lm8lhxUt5arG5Zi7B3do/iGp3BU5oc21/qvm68G0ZFfERSWUueehcriWbpXXCYbqei5WiYNO7FM5rLgonsfJMH8P7npukMJcuNMGAOgIijYAQEdQtAEAOoKiDQDQERRtmPBkoa0X4DrhOrcPyTbukch47J4hCM65URmCcCZeUDcEQbpHTF61N+8eUWvO7TJdawUHi28rF7Gu5V2staYLt2+bn5f3PHapI0TjXuOtqbQsV++glHPDuzzU9fTe1JZtXpHDOSnUfl1sirxueIB6jStuDuUScTmce8Q5Zo5F/NIhCNxpAwB0BEUbAKAjKNoAAB1B0QYAmMm1FSJzjDh56eKljwf9Ytw4mq43EysGlsfOnEWsxMWKYFg5p9vnnS+GKhGwEut0OrUHO8/bXm8+SmitfA62/8isQ0WLMtqXFOtsjlLeKa6FXIuW84W9SqxvY1927nVJ4HT7NWe3q8H0uaONHQDg2kDRBgDoCIo2AEBHULQBADqCog0THj/ZRJ8GuPS4zu1Dso17pA1x45WHLqyNcSpjlVOkCUeJiz02f5ZkXudKETmacaWovTl3RcXBohwWa7hH3BAEVbgrLhHXrq5WK+4R10rfG841IWMLLg8/SEHtQVMZgrDYPbKCc0PtLUfj5hAujzSTOVQOm9d8OGQO3CMAANcHijYAQEdQtAEAOoKiDQAwE6d7HZJNhMij4Ua84eG3Xljb7V6Rsbvx9nRtp0XL3ek01gmcat0LkUq0lKE6h9EdlJi5Rl4l96kWf7cHl9cJhrI13W1tYd4Khz4mwolqc3Ft6ZW8VjAUuW1e9ZYotGn7N5B6fKH92+xB7reS18YqQfZExrr1o+GmWNOxc+FOGwCgIyjaAAAdQdEGAOgIijYAQEdQtGHCyXdpYwdQOGPDIdnGPXJ0Mx599K0X1s7OtHtkFE6R3dnUJRJhnCajcY+IdfV4l2M8NXmbiHUt+m3aP9sG7buouDwqrpRQsVks3GIfrs1/7uMdzkBQorK3PW2itIXKS1lwWPgc6vF7cnnYvNM3bKa2QCnnxpDG5SGcG0dHxvlxMnV+qMdHeEeIdI8cT9cqcKcNANARFG0AgI6gaAMAdARFGwBgJkoLOzTbCJEnD8WjTz19YW13+2UZu7szXT8TaxER49md6eNPTawSOJ1oKWLVWoQWIitiaGtaiFR5Xay6XjvThwarHC3mx7r4ZhrZlfjqaO6QYsUaAuU+KAmAtXsoJcyluQ/LELG5QqzYgxPlVA4nGMrYgghohUghOqo1m9eKlg/r6x3fmKwd39Cxc+FOGwCgIyjaAAAdQdEGAOgIijYAQEdQtAEAZuIMCIdkG/fIzYfi0e99x4W18c7U+RERcfby1P2xe8U4QkQO6zQ5ncYq94lbd7G7Quy4m647d4Ua/ODeQNIRMlbcIzXnh8oxFmIdJadJIe8hca4LHWsmVRRyDyaHinXXUw4U6x4Zpjmcw0LlcC3kam/D0dSJEaEdGkdizcWqNRt7omOdI2S4Ifb20Ibukcz895n5h5n5ucz8eGa+adFuAADgviz9euR9rbV3tdb+TkT8t4j4dyvsCQAADIuKdmvtW6/65+ui1EoAAABVFn+nnZm/EhH/KiK+GRH/ePGOAAAuKePpHfu99qHI1u5/c5yZn4yIJ8WPnmutfeRVce+NiIdaa79s8tyKiFt3//nOiPj8a9rx5efxiPja1pvYI1f5+fHc+uSqPLe/2Vp74kFBDyzac8nMvxERH2utvXNG7Gdaa8+scuFLxlV+bhFX+/nx3PrkKj83xVL3yNtf9c8fjYgvLdsOAADcj6Xfaf9qZr4jIsaI+EpE/OvlWwIAAMeiot1a+xev8aHPL7nuJecqP7eIq/38eG59cpWf24TVvtMGAID9w9kjAAAdsVnRvsot8Jn5vsz80t3n9+HMfGzrPa1FZv5EZn4hM8fMvBKKfWY+m5lfzswXM/OXtt7PmmTmBzLzLzLzyllsM/Mtmfm7mfnFu+/Jn9t6T4dgyzvtq9wC/4mIeGdr7V0R8UcR8d6N97Mmn4+IH4+IT229kTXI85OJfj0ifigino6I92Tm0/d/VFf8RkQ8u/Um9sRZRPxCa+3piPh7EfFvrtjvTrJZ0b7KLfCttY+31s7u/vPTEfHUlvtZk9baC621L2+9jxX5/oh4sbX2x621OxHxW3FuX70StNY+FRFf33of+6C19uettc/e/f/fjogXIuLN2+5q/2xyNOtfcU1a4H86Iv7r1psAy5sj4k9f9e+vRsTf3Wgv8BrJzLdGxPdFxO9tu5P9s9ei/aAW+NbacxHx3N0W+J+NCNkCfxmZ096fmc/F+X/CffCQe1vK3KMLAC4Dmfk9EfGhiPj5e/4L/kqy16LdWnv3zNAPRsTHoqOi/aDnlpk/FRE/EhE/2DrzVRZ+b1eBP4uIt7zq30/dXYMOyMyTOC/YH2yt/fbW+zkEW7pHrmwLfGY+GxG/GBH/vLX20tb7gfvy+xHx9sx8W2beiIifjIiPbrwnmEFmZkS8PyJeaK392tb7ORSbNddk5oci4kILfGvtStzhZOaLEXEzIv7y7tKnW2tXosU/M38sIv5jRDwREd+IiM+11v7ptrtaRmb+cET8h4g4iogPtNZ+ZeMtrUZm/mZE/KM4Pwnv/0bEL7fW3r/pplYiM/9BRPzPiPg/Ef9/Rt6/ba19bLtd7R86IgEAOoKOSACAjqBoAwB0BEUbAKAjKNoAAB1B0QYA6AiKNgBAR1C0AQA6gqINANAR/w882vmMOGw1OQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff8bccebc50>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# the function that I'm going to plot\n",
    "def f(x,y):\n",
    "    return x**2 + 5*y**2\n",
    "\n",
    "x = np.arange(-3.0,3.0,0.1)\n",
    "y = np.arange(-3.0,3.0,0.1)\n",
    "X,Y = np.meshgrid(x, y, indexing='ij') # grid of point\n",
    "Z = f(X, Y) # evaluation of the function on the grid\n",
    "\n",
    "plt.pcolor(X, Y, Z, cmap=plt.cm.gist_earth)\n",
    "plt.axis([x.min(), x.max(), y.min(), y.max()])\n",
    "plt.gca().set_aspect('equal', adjustable='box')\n",
    "plt.gcf().set_size_inches((6,6))\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [],
   "source": [
    "def target_f(x):\n",
    "    return x[0]**2.0 + 5*x[1]**2.0\n",
    "\n",
    "def part_f(x, \n",
    "           f, \n",
    "           i, \n",
    "           h=1e-6):\n",
    "    w1 = [x_j + (h if j==i else 0) for j, x_j in enumerate(x)]\n",
    "    w2 = [x_j - (h if j==i else 0) for j, x_j in enumerate(x)]\n",
    "    return (f(w1) - f(w2))/(2*h)\n",
    "\n",
    "def gradient_f(x, \n",
    "               f, \n",
    "               h=1e-6):\n",
    "    return np.array([round(part_f(x,f,i,h), 10) for i,_ in enumerate(x)])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 8.73043864e-04 -4.00026038e-12]\n",
      "Val: 0.000001, steps: 475\n"
     ]
    }
   ],
   "source": [
    "def SGD(target_f, \n",
    "        gradient_f, \n",
    "        x, \n",
    "        alpha_0=0.01,\n",
    "        toler = 0.000001):\n",
    "    alpha = alpha_0\n",
    "    min_val = float('inf')\n",
    "    steps = 0\n",
    "    iteration_no_increase = 0\n",
    "    trace = []\n",
    "    while iteration_no_increase < 100:\n",
    "        val = target_f(x)\n",
    "        if min_val - val > toler:\n",
    "            min_val = val\n",
    "            alpha = alpha_0\n",
    "            iteration_no_increase = 0\n",
    "        else:\n",
    "            alpha *= 0.95\n",
    "            iteration_no_increase += 1\n",
    "        trace.append(x)\n",
    "        gradient_i = gradient_f(x, target_f)\n",
    "        x = x - (alpha *  gradient_i)\n",
    "        steps += 1\n",
    "    return x, val, steps, trace\n",
    "\n",
    "x = np.array([2,-2])\n",
    "x, val, steps, trace = SGD(target_f, gradient_f, x)\n",
    "print(x)\n",
    "print('Val: {:.6f}, steps: {:.0f}'.format(val, steps))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " [1.90552394e-05 1.27080227e-05]\n",
      "Val: 0.000000, steps: 241\n"
     ]
    }
   ],
   "source": [
    "def SGD_M(target_f, \n",
    "        gradient_f, \n",
    "        x, \n",
    "        alpha_0=0.01,\n",
    "        toler = 0.000001,\n",
    "        m = 0.9):\n",
    "    \n",
    "    alpha = alpha_0\n",
    "    min_val = float('inf')\n",
    "    steps = 0\n",
    "    iteration_no_increase = 0\n",
    "    v = 0.0\n",
    "    trace = []\n",
    "    while iteration_no_increase < 100:\n",
    "        val = target_f(x)\n",
    "        if min_val - val > toler:\n",
    "            min_val = val\n",
    "            alpha = alpha_0\n",
    "            iteration_no_increase = 0\n",
    "        else:\n",
    "            alpha *= 0.95\n",
    "            iteration_no_increase += 1\n",
    "        trace.append(x)\n",
    "        gradient_i = gradient_f(x, target_f)\n",
    "        v = m * v + (alpha *  gradient_i)\n",
    "        x = x - v\n",
    "        steps += 1\n",
    "    return x, val, steps, trace\n",
    "\n",
    "x = np.array([2,-2])\n",
    "x, val, steps, trace2 = SGD_M(target_f, gradient_f, x)\n",
    "\n",
    "print('\\n',x)\n",
    "print('Val: {:.6f}, steps: {:.0f}'.format(val, steps))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7ff8bcd97a58>,\n",
       " <matplotlib.lines.Line2D at 0x7ff8bcda5550>]"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlUAAADFCAYAAAB9/wfYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xl8VPW9//HXd7Zksi+EEJKQyCKLCIIgaC1aaRWrrXWrXvf+bG1te9XWarW2vV5vvb1t7WZrbXFpq221qHXDBRdwqSCrgAIiAgkECATIvs32/f1xhpBAINuQmeD72c7jzHzPmXM+yVF58z3f8z3GWouIiIiI9I0r3gWIiIiIHA0UqkRERERiQKFKREREJAYUqkRERERiQKFKREREJAYUqkRERERiQKFKREREJAYUqkRERERiQKFKREREJAY88TjooEGDbGlpaTwOLSIiItIjy5cv322tzetqu7iEqtLSUpYtWxaPQ4uIiIj0iDGmvDvb6fKfiIiISAwoVImIiIjEQJ9DlTGm2BizwBiz1hizxhhzYywKExERERlIYjGmKgTcbK1dYYxJB5YbY1611q6Nwb5FREREBoQ+91RZa3dYa1dE39cD64DCw32noq6qr4cVERERSSgxHVNljCkFJgGLO1l3nTFmmTFmWW2win+seiOWhxYRERGJq5iFKmNMGvAUcJO1tu7A9dba2dbaKdbaKcZ6+L8Vt7OltiJWhxcRERGJq5iEKmOMFydQ/d1a+6+uts9PKSRiQ3xt3rdpCjbFogQRERGRuIrF3X8GeAhYZ639VXe+k5OSSkrNVWxv2sSP3vkR1tq+liEiIiISV7HoqfoUcCVwhjFmZfT1+cN9wQBXnTCLll2zeKX8FR54/4EYlCEiIiISP32eUsFa+2+cnNQjs8YXcM8rM5g8spnfvfc7hmcO57Mln+1rOSIiIiJxEbcZ1UfkpVKam4qv5hIm5E3gtrdvY+WulfEqR0RERKRP4haqjDHMHJvP4o31/PzU3zAkdQjfnv9tNtdujldJIiIiIr0W12f/nTFmMIFwhPXbI9z/2ftxGzfXv3Y9u5t3x7MsERERkR6La6iaPCwbr9uwePNeitOL+cPMP7C3ZS/ffO2bNAYb41maiIiISI/ENVT5fW4mFmWxZPNeAI4bdBz3nHYPH1V/xM1v3EwwHIxneSIiIiLdFtdQBXDSMTm8X1FLUyAEwIyiGfz45B/zzvZ3uO3t2whFQnGuUERERKRrCRGqQhHLivKatrYLRl3ALVNu4ZXyV/jxOz8mYiNxrFBERESka32ep6qvTizJxhhYsaWaU0cNamu/6riraAm38Lv3fkeSJ4kfT/8xzuTtIiIiIokn7qEqPdnLyLw0Vm2tOWjddROuoznUzIPvP0iyO5lbp96qYCUiIiIJKe6hCmBicRZvrN+Ftfag0HTDpBtoCbXwt3V/I8mdxI2Tb1SwEhERkYQT9zFV4ISq3Q0BttU0H7TOGMOtU2/l4mMv5qEPHuIXy36hBzCLiIhIwkmInqoTirIAWLW1lqLslIPWG2P40fQf4XP7eHTto7SGWrlj+h24TEJkQhEREZHECFWjh6Tj87hYVVHDORMKOt3GGMP3p36fZHcyD33wEC3hFu465S7cLnc/VysiIiJysIQIVT6PizFD0lm7ve6w2xljuHHyjSR7krlv5X0EwgH+99P/i9fl7adKRURERDqXEKEKYFxBBq+s3dnpYPX2jDF8Y+I38Hv83LPsHuqD9fzqtF+R4j34sqGIiIhIf0mYQUljCzLY2xhgZ11rt7a/+rirufPkO3l3+7t8Zd5X9BBmERERiauECVXjhmYAsG7H4S8BtnfhsRdy7xn3srl2M1e8eAVltWVHqDoRERGRw0uYUDVmSDoAa3sQqsB5VuBDZz5Ec6iZK1+6kpW7Vh6J8kREREQOK2FCVXqyl5LclC4Hq3fm+LzjefTsR0n3pfPVV77Ky5tfPgIVioiIiBxawoQqgLFDMnrcU7XPsIxhPHr2o4zLHcctb93CvSvu1YOYRUREpN8kVKgaNzSDsj2NNLSGevX9XH8uD575IBeMuoAH3n+AGxfcSEOgIcZVioiIiBwssUJVQQbWwvrKOgg0wb+ug58MgVWPd3sfPrePO0++k9tPup23K97mihevYGvd1iNYtYiIiEiChaqx0TsA1+6oh/Uvwup/gtsLS2b3aD/GGC4bexl/+tyf2N2ym0teuIT5W+YfiZJFREREgAQLVUMzk8n0e53B6pvfgqRM+PR3Ydty2L2hx/ubVjCNx895nOL0Ym5ccCO/WPoLgpHgEahcREREPukSKlQZYxhXEB2sXvY2lJwCEy4F44LVc3q1z6L0Ih49+1EuHX0pj6x9hGtevoYdDTtiXLmIiIh80iVUqAJnZvWays2wdxMc82nIKIChk2DLol7v0+f2ccf0O/jFab9gY81GLp57MQu2LIhh1SIiIvJJl4ChKp2J4bXOh9JTnWXBRKhcDdb2ad+zSmfxz3P/SUFqATcsuIE7F95JU7CpjxWLiIiIJGSoymCEawcWFwwe5zQOmQAttVBT3uf9l2SU8PfP/51rx1/Lvzb8iwufu1CzsIuIiEifxSRUGWMeNsbsMsZ80Nd9jRycxjCzi3rfYOfOP4CCCc5yx6q+7h5wLgfedOJN/HnWn7FYrn75au5dcS/BsAaxi4iISO/EqqfqL8CsWOwo2etmlG8PO1z5+xsHHwfGDTtWx+IQbU7MP5Env/Ak5404jwfef4Avz/2yeq1ERESkV2ISqqy1bwF7Y7EvgCJTxcfB3P0N3mTIG+OMq4qxNF8ad33qLu6beR8NwQaueukqfrr4pzQGG2N+LBERETl69duYKmPMdcaYZcaYZVVVVYfeMNhCVmg361tyqGtpdzkufxxUfXjE6ptRNINnznuGS8dcymMfPsb5z57PWxVvHbHjiYiIyNGl30KVtXa2tXaKtXZKXl7eoTesdR4ps9Xmsb6yfn97zgio2QrBliNWY6o3lR9M+wGPnP0Ifo+fb73+LW6YfwMV9RVH7JgiIiJydEi4u/+odu7w22rz+HBH3f723BGAheqyI17CCYNP4IkvPMFNk2/i3R3vct4z53HfyvtoDjUf8WOLiIjIwJR4oaqmDIDapKGsO7CnCmDvxn4pw+f2ce3x1/L8l55nZslM/rjqj3zpmS8xr2weto/zZYmIiMjRJ1ZTKjwGLAJGG2MqjDHX9npn1eXg9pE7ZBjrOvRUDXeWe/onVO2Tn5rPz2f8nIfPephUXyrfe/N7XPHSFSzfubxf6xAREZHEFqu7//7DWltgrfVaa4ustQ/1emf1lZBewJihWayvrCcSifYK+bMhJRf2fByLknts6pCpPHHuE9x1yl1UNlRyzcvX8J/z/5NNNZviUo+IiIgklsS7/NdYBal5jC1IpykQZmt1u8fI5IxwngkYJ26Xm/NHnc/cC+Zy4+QbWVq5lPOfO587/n0HW+q2xK0uERERib/EDFVpgxkzJAOAdTvajavKHdHvl/864/f4+erxX+XFC17k8rGXM69sHl985ovc8e87KK/r+6N0REREZOBJzFCVOohj89NxGTqOq8oZDvXbIZgYd+HlJOdw69RbefnCl7l87OW8UvaKwpWIiMgnVGKFqkgEGndDah5+n5vSQal8WNkuVGUNc5Y1W+NT3yEM8g/ilqm38NKFL3Hl2CvbwtXNb9zM6qrYzwIvIiIiiSexQlVzNdgwpA4GYOyQDD5sP61CVomzrEnM8UuD/IP43tTv8dKFL3HNcdewaPsiLn/xcq566Spe3/I64Ug43iWKiIjIEZJYoapxl7NMHQTAmCHplO9porE15LRn7wtVZf1fWw8M8g/iOyd+h1cvfpXvT/0+u5p2cdOCm/jiM1/k8Q8f13MFRUREjkIJFqqizwRMc3qqxhQ4g9XbeqvShoDb1zbreqJL9aZyxbgrmHv+XO457R4ykzK5e/HdnDHnDP5n0f+wfu/6eJcoIiIiMeKJdwEdNOzrqXKeDTi2IB1wBqufWJINLhdkFifs5b9D8bg8nFV6FmeWnMnq3auZs34Oz258ljkfzWFi3kS+PPrLnFlyJsme5HiXKiIiIr2UYD1Vu51ldExVYZafTL+XNdvbDVbPLoGagdFTdSBjDBPzJnL3qXfz+sWvc+vUW6ltreWOf9/BGU84vVcrd63UY3BEREQGoMTqqWrcBcbtzJ6OE0LGF2bwwbba/dtklcD2lXEqMHYykzK5ctyVXDH2CpZULuHpj5/muY3PMeejOQxLH8YXRnyBL4z4AoVphfEuVURERLohwUKVM0cVrv0daOMLM/nzv8sIhCL4PC6np6p5L7TWQ1J6HIuNDWMM0wqmMa1gGo3TG3ml7BWe3/Q89628j/tW3sfkwZM5q/QsPlfyOfJS8uJdroiIiBxC4l3+S+0YHMYPzSQQjvDRzuhg9QSdqyoWUr2pnD/qfB4+62HmXTiPb5/wbeoCdfx0yU+Z+cRMrn7pav6x7h9UNVXFu1QRERE5QOKFqpTcDk3HF2YC7L8EmFHkLOu29Wdl/W5o2lC+PvHrPH3e0zxz3jNcP/H6DgHrqpeu4sH3H2RD9QaNwRIREUkAiXX5r7kaMjuOISrJTSE92cP722q5FCAzGqpqj76eqkMZkTWC60+4nutPuJ5NNZuYVz6PBVsW8NsVv+W3K35LYVohM4pmcFrRaUwZMoUkd1K8SxYREfnESbxQFR2kvo8xhvFDM/f3VKUPcQaz1x7dPVWHMjxrONdnXc/1E69nZ+NO3t72Nm9WvMnTG57msQ8fw+/xM23INKYPnc70gukMzxyOMSbeZYuIiBz1EidUWdtpqAIYX5jBXxeVEwxH8LrdkF4AtRWxPX44CB/8y7kDsegkGDYttvs/AvJT87no2Iu46NiLaAm1sKRyCW9ufZOF2xfyRsUbAOT585hWMI3pBdOZVjCNIalD4lu0iIjIUSpxQlVrvfPcv05DVSaBUIQNOxsYNzTDuQQYyzFVtdvg7xfBrrX728Z+Ac6fDb6U2B3nCEr2JDOjaAYzimYAUFFfweIdi1m8YzELty9k7qa5AJRklDBp8CQmD57MpMGTKMkoUU+WiIhIDCROqGqpcZbJWQetaj9Y3QlVhbBtRWyOG2yBf17h3E146T+geDos/zMsuBsePR+uega8/tgcqx8VpRdRlF7EhcdeSMRG2FC9gXd3vMuynctYsHUBz3z8DAA5yTmckHcCk/MnMzFvImNyxmhmdxERkV5InFDVXO0sO+mpKs1NJS3JGaz+5anFkFEI656HSKTDnFa9svBe2L4CLvk7jDnHaZvxPcgZDk/+P3juBrhgNgzg3hyXcTE6ZzSjc0Zz9XFXE7ERymrLWLFrBe/teo8VO1cwf+t8ANzGzciskYwfNJ5xueMYP2g8o7JG4XV74/xTiIiIJLYBEapcLsO4oRms3jdYPbMYwgFo2t328OVeH3Ph72HMuTD23I7rxl8AezbCgp/AiM/ACZf1/jgJxmVcDM8azvCs4Vx07EUAVDVVsXr3atbsXsOaPWt4tfxVntrwFAA+l4/ROaMZlzuOY7OP5djsYxmVPYpUb2o8fwwREZGEMiBCFcCkYVk8/O/NtATDJO+bdqG2om+havGfoLUWTr+98/Wfvhk2vg4v3wbDPwMZBb0/VoLLS8lj5rCZzBw2EwBrLRUNFazZs6YtaM3dNJfGYGPbdwrTCtsC1r5lcXoxXpd6tURE5JNnwISqKSU5/OnNTayuqOWkjHahqnBy744XicB7f4MRM2HI+M63cbngvPvg/lPghe86Y64G8GXAnjDGUJxeTHF6MbNKZwFO0NreuJ0N1Rv4qPqjtuWbFW8SsREAPMZDUXoRpZmlHJNxDKWZpZRmlFKaWUp2UrYGxYuIyFErAUPVwQPVAU4sccLWsvK9nDS12Gnsyx2AZW85E4h+7r8Pv13uCDjjh/DKD+GDp+D4i3p/zAHOGENhWiGFaYWcXnx6W3truJVNNZvYULOBzbWbKasto6yujHe2vUMwEmzbLsOXwTGZxzAsfRhF6UUUphW2LQenDMZlEmuCfxERkZ5IrFDl8R/yTrucVB/D81JZXlYNp40AT3Lf5qpa+RgkZ8Loc7redvo3Yc3TzmXAkTMP2Zv2SZXkTmJs7ljG5o7t0B6OhNneuL1D0CqrK2PpzqXM3TQXy/7H63hd3rbAVphWSGF6IUNShpCfmk9+Sj6DUwbjc/v6+0cTERHptgQKVTWH7KXaZ0pJNq+s3UnEgiujsPehKhyE9S85c1F5uzF9gMsN5/4aZp8Or9/lvJcuuV3utkuI++bP2icQDrCjcQfb6rdR0VBBRUNF2/sP9nxAbWvtQfvLSc4hPyW/LWgNSR1Cfko+eSl55CbnkuvPJSspSz1eIiISFwkUqjqfTb29KSU5zFlWwabdDYzMLOz95b8ti5wB6qPP7v53CibCtG/Au/fDxMugeGrvji0A+Nw+SjJKKMko6XR9Q6CBXU27qGyqZGfjTnY2RV+NO9nesJ0VO1dQF6g76Htu4yY7OZtB/kFtQWvfMic5h0H+QWQnZ5PpyyQzKRO/x69xXiIiEhMJFKpqugxVk6PjqpaXVzMysxg2zu/dsda/BO4kZ6qEnvjMD2DNMzD3O3DdG+BOnF/f0SbNl0aaL43hWcMPuU1TsIldTbuoaq5iT8se9jRHX+3eb6zdyJ7mPR3GdrXnc/nISsoiIymDrKQsspKyyExyAlf7z+m+dNK8Tk37lrrLUURE2kucVNBcDTnHHHaTEXmpZKd4WVZWzSW5hVBf6VzK6+nElB+9DMfMAF8P51lKSoez/w/mXAVL/gQnf6tn35eYSvGmOHcXZpYedjtrLfXB+ragVdNaQ21rbafLzbWbqQ0470OR0GH3m+xO3h+yokEr3ZdOqjeVNG9aWxBL8aaQ4knB7/Hj9/qdZfS1rz3Zk6zLliIiA1xMQpUxZhbwW8ANPGit/b8e76S5GvyTujoOJ5Zks7y8GoYXAhbqd0DWsO4fp3Yb7N0EU7/W4xIBGPtFGHUmzL8bRn++yyC4TzhiWVVRw5LNe/loZz0bdzUQrt/F1JaFjI18TI6po9Uks9NTyIbUyVRmTmRQRirDclIYlptCcU4Kw3JSyE316XJVDxljyPBltN192B3WWppCTdS01lDTWkNjoJH6YD0NgQYagg1ty/pAPY3B/euqmqqc9cGGDnN6dUf7sNU+cPk9fpI8SSS5k/C5fW3LZHdy2+f2r8Nts++91+3FYzy4Xe7e/EpFRKQTfQ5Vxhg3cB/wOaACWGqMec5au/bw3zxAS02nz/070IklOby2bhe1vnwywRms3pNQtWWRsyw5uUfltTEGzvmVM3fV09+Ar7zoDGTvhLWWpWXV/HPpVl7/cCc1Tc4lqGPTg9zqe4LTA/PwmCCNSdk0+HLxhirJCizCVTeHhvpU3mIKj7WezK8j44ng9GJkJHsYMTiNkXlpjBicxoi8NEYOTqM424/HfZiejnAIAg1Ozx7WufPRk9S738EngDGGVG8qqd5UCtMKe7WPcCRMY6iRpmATzaFmmkPNHd43h5ppCrX7HDzgc3T7mtYaApEAgXCAllALgXCA1nArgUigzz+ny7jwurx4XV48Ls/B76Phy+s+xPpOtvW4PLiMC4/Lg9u4cbvcuI27Q9u+9y7jwm3c+79jom0ut/Pe1cX66Pt9+zQYZ2mcpYt276Pr3S53h3aD0V9URCQmYtFTdRLwsbV2E4Ax5nHgPKD7oSochGBTt0LVtOE5ALxXm8rp4PQ89UT5O+BLh/zje/a99rKK4fP3wNPXOc8OPPU7HVYHgiGeW7KZR95Yz/adteS4w1xUmsFJY9OYbNfhX/QbaG2kZfgs7IgzIa0AfziCDYeob2nEVq7BblvFyTtWc3JwKWFfBjVZ46nwj2VLUyZV77eyp6GVtS1B1gIG8BjITfUyKNXHUF8jeZEqsoNVpLbswhuoxRWoBxudwmDfTAZurzONhS8FfGnO5dC2ZfS9x0fEuIngJowhHI4QCYeIRMLOKxzGhkPYcBAbCUMkBOEQkUgQImFMJISNhDGRICa63mKJ4CKCIWINEQzWuLAYLC6scZZt9bazr8XQ/mextP8j0bSbqmH/+47bHM6h/3zt/h+8JnrEA7/hBtIMpPV4/y4gOfpyOL9HS/igZaTt/b72MJEOn+1B74l+K0SEJiJYrKHtO7bd/pxtD2yHMJZgu/0NNMYY2v4X/YfgwLZ9Z7St/YA2FwbatTvLtiPsf3/A9zrU0Un7gW0H/pNlMDj/Nwe1Hbh9l/vqpL1920FHNs6yY2tnuv735+B/9w7+TndaurXFIX7vh99PN/4b0MkmR2rf3fqO9Fn+9NO6vW0sQlUhsLXd5wpg2oEbGWOuA64DGDbsgJ6lluhdXMkZXR5sQmEm6UkeFuzwRUPV1i6+cYDyRTBsWrcHmdtQiFBVFcHKSkI7dxKsrCS8ezfh2jrCa44jvOD3hNPnEW4KEGlsJNTUgisY4DjgZ53srwoAn/OavxxYfpijJ0VfAGvJZy353f5BIQzsvz8u/TBbNkVfsdO+z+zAf+0NTrjQhScREUl0m4Pd/2tivw1Ut9bOBmYDTJkypWMXREuNs0zO7HI/HreL6SNymb+5jv/2Z0PNlu4X0bgHqtZ1Oiu6tZZgeTnNq1fT+tFHtG7cROumjQS3VjiPtGnHeL24s7JwpWfiNtV4A+WEx81kyR7Y3BAmMzON6eMKGVmci9vvxwRqMO89hKt2I2bMLJh+HSbJj3G7we3BeNzgdmM8nv1tbtf+y4oGTEstfPwqbFsG21c4lz1tyAksxg2puZAzHAaPwQ4eR0POcexILqGqMcKu+lYaAmGaA2EaA2GaghGaAxEixkZ/9g6/CDJNI1mRWrJsNX7bQrIJ4jMhfCaMy+3B5fHhcntxeby4PD6MNxmXJxnjTcbtTcblTcLl8+P2JeP2JePx+vF4XHjdLrwug8ftwuMyeN0u3C7T4djs6+2KhJ2/SXZIZAf+Nc1EfzntlxzcdiQv7Rxm39Ye/Pvd32YP6KE4uFcuEQ20vxe73IruItI3Y4yBn/+jW9vGIlRtA4rbfS6KtnVfS3Six26EKoBTRw7i1bU7aR1WTFJPQlXbeKpPARAoL6d+wQIaFy6kZdVqwrVOHcbrxVdaSvKYsWScfTbegqF4C4bgyR+Cd0g+royMtj8QA9tWwUNnsS38Kv/I/QGXnnUaV0wvccJCOATLHoJX/wtyk+CaP8Fx53e/3g4Gw7BR+z9aC6EW55WU6TynsJ2s6KvjHOfSnwZaABERkb6JRahaCowyxhyDE6YuBS7r0R5a913+62aoGjUIgEqTT0nN5u4fZ8siWhv91M75N/Xz/4fAZue7vuHDSfvcZ/FPnIh/4kSShg/HeLr+1Sz8eDc/fLaarObv86j/l7zo+j6m9hpYdbzTg/b+HOdOwxEznQczZxR0v9auGOM80ucQj/URERGR/tXnUGWtDRljvg3Mwxkm87C1dk2PdrKvpyqp6zFVAMMHpVKSm8Ka5mxKGt90Ls+5Dn3nm41EaJg/nz0/f57mbdng/iup06aRfdllpJ1+Gr7i4kN+tzN7Glr5yQvrePq9bQzLSeGGay4ndfAF8NqdsPQB5/IVwLBT4MyfOFMv6O4iERGRo1pMxlRZa18EXuz1Dnp4+c8Yw8wx+Sxeksbn3QFoqISMoZ3VRf1rr7H73ntp3fAx3tQQg780hcyb78WTl9erUhes38UtT6ymtjnAf54xkm99ZiTJ3ui4jYv/DMEW5/E5qXndGngvIiIiR4fEmFG9h6EKYObYwcxelOf0jVWXHxSqWjduZOfdd9O4cBG+ESMYevPVZGz9Kebqr0EvAlUwHOHuF9bxl4VljM5P59FrT2JsQSehyZsMuSN6vH8REREZ2BLjuRgtdWBczrxI3XTSMTlU+6JjlGrK29qttex95FE2f+l8mj9YQ/4Pf8jwZ58hc3jQubOu6KQel7e7oZXLH1zMXxaWcc0ppTz77U91HqhERETkEytxeqqS0g87LupAXreL8eOOg7UQ3LMZLxCurWX7bbfTsGABaaefTsHdP8GTm+t8oXwhFEyEpO4HN4APttXy9UeXs7uhld9ccgJfmtS72bVFRETk6JY4oaoHl/72+fyk4exck0WwfD35lZVs/drXaC0rJ/8HPyD7yiv2zwMUbIFty+Gknj3vb96aSm58/D1yUnw8+Y1TOL6o5zWKiIjIJ8OADlUnj8hlpRlKxofrKHvkMiJ1dQx7YDap06d33HD7Cgi3Qskp3d73Y0u2cMfT7zOhKIsHr57CoDQ9J09EREQOLTFCVWudM4FlD7ldBndyKZFnVmBTvJT+7VGSx3Yy3WX5O85yWNcPUbbW8vv5H/PLVz/i9NF5/OHyyaT4EuPXJCIiIokrQQaq966nKlRdTcqLmyEMS669qfNABc7z/gaPg5Scw+4vErHc+dwafvnqR1wwqZAHrpqiQCUiIiLdMmBDVSQQoOIb1xOpaaJ4xl7e2baVpkDo4A3DIdi6uMteqtZQmBsef4+/LirnuhnDuefiiXjdifHrERERkcSXGKmhF6Gq6le/pnnVKobeeTspeQHyWsv527vlB2+4830INBx2PFVdS5CrH17C3NU7uP3sMfzg82NxuTQDuoiIiHRf/ENVJAKt9T2afbzhrbfY+5e/kH355WScfyn40pmRvZc/vrmJ2qZgx43LFzrLQ4SqytoWvvzHRSwrq+bXl0zk66dp4k4RERHpufiHqtY6wHa7pypUVcX2224nafRoBt96i/NMvUGjOCVzN7XNQX760rqOX9j8NmQf0+ljbD7aWc+F9y9k694m/vyVqZw/qSgGP5CIiIh8EsU/VPXwETU7f/ZzIg0NFP7ql7iSotMcDB5H2t51fPXUUh5fupUX39/htIcCUPY2jDjjoP08tbyC837/DoFwhH9+/WQ+Pap3zwIUERERgUSYUqG1zll2I1Q1LV1K3dy5DPrm9SSNaHeZbtg0WPk3vjvZxdKyLL7zz5WkJXmY4VvvjKdqF6r2Nga4+4V1PLWigmnH5HDvf0wiPyM51j+ViIiIfMLEP1Tt66lKOvyCMgZ9AAALI0lEQVSYKhsKUfmTu/EUFJD7tQNmRi92JvtM2r6EB666hMsfXMw1f17CX4e9zKnGTU3+yWytqOHlDyr527vlNAbC/OcZI7lx5ig8usNPREREYiBxQlUXPVXVc+bQun49hb/5NS6/v+PKQaPAnwNbFpM7+Sqeuv4UfvLCWvJWLmApI/nyz98FnOFXZ40bwnfPPJZj89OPxE8jIiIin1ADIlRFWlvZ/Yf7SZk6lfSzzjp4A2Ng2HTYsgiA1CQPPz0ZWL2V1RN+yI8Gj2NwehLTh+eSl67HzYiIiEjsJUCo6npMVc1TTxHevZtB99yz/yHJBzpmBqx/ESrfhyHHw6rHweVlwqxrmdDFTOoiIiIifRX/AUVdjKmywSB7H3wI/8SJpEw76dD7mXgpeFPg3fuhcTesegxGz+ry0TQiIiIisZAAPVW14EsDd+el1L7wAsHt28n/0Q8P3UsF4M+GEy6HFX+FnR9AoBFO+/4RKlpERESko8QIVYe49GcjEfbMfoCk0aNJO/30rvf16e9C1YdQ/g6c80vnMqCIiIhIP4h/qGo9dKhqWryYwKZNDP35zw7fS7VPxlC4Zq4z6afHF+NCRURERA4tMcZUHWI8Vc0TT+LKzOz8jr/DUaASERGRfpYYoaqTnqpwTQ31r71G5rnn7n8cjYiIiEiCSthQVfv8XGwgQNZFF8ahKBEREZGeSYBQVXdQqLLWUvPkkySPG0fy2LFxKkxERESk++IbqqyN9lR1HFPV8sEaWtevJ+vii+JUmIiIiEjPxDdUBRrBhg/qqaqb+zzG6yXjnHPiVJiIiIhIz8Q3VHXy3D8biVA37xVSTz0Vd0bndwWKiIiIJJo+hSpjzMXGmDXGmIgxZkqPd9Aafe5fuykVWlavJlRZScasHk6jICIiIhJHfe2p+gC4AHirV9/upKeqbt4r4PWSdsYZfSxNREREpP/0aUZ1a+06oHuznXemLVRltTU1LFhA6rRpuNPT+1KaiIiISL/qtzFVxpjrjDHLjDHLqqqqnMbmamfpd0JV6+bNBMrKuvecPxEREZEE0mVPlTHmNWBIJ6vusNY+290DWWtnA7MBpkyZYgFornFW+rMBaHjzTQCFKhERERlwugxV1trPHrGjt0RDVXRMVcObb5I0aiS+osIjdkgRERGRIyG+Uyo0Vzt3/rncRJqbaV62nNRTPx3XkkRERER6o69TKpxvjKkATgZeMMbM69EOmmvaxlM1LVuODQZJPeWUvpQkIiIiEhd9vfvvaeDpXu+gubrtzr/GRYswXi8pU07sS0kiIiIicRHnGdX391Q1LlyIf/JkXH5/XEsSERER6Y04j6mqAX824ZoaWj/8kNTp0+JajoiIiEhvxX+genIWTStWAJAydWpcyxERERHprfiFKmvbLv81LV2G8flIPv74uJUjIiIi0hfxC1XBZggHwJ9N07Jl+CdMwJWUFLdyRERERPoifqEq+oiaCCm0rF2Lf+qUuJUiIiIi0lfxC1XR2dSbK+ohHCZl0qS4lSIiIiLSV3HsqYqGqrI9APgnTIhbKSIiIiJ9FffLf80bKvCVluLOyopbKSIiIiJ9FdfLf9ZC87qN+CdOjFsZIiIiIrEQ156qYKOb8N5qkifq0p+IiIgMbPELVY1VtNQ4j6TxT1BPlYiIiAxscQxVu2lpyACvl6RjR8WtDBEREZFYiF+oathFS7WXpFEjcfl8cStDREREJBbiFqpswy5aqiIkjxsXrxJEREREYsYTrwOHdlURbkKhSkRERI4KceupaqmoA8CvUCUiIiJHgfiEqkiYlj0GjCFp9Oi4lCAiIiISS3EKVSFaaz34huTg8vvjUoKIiIhILMUpVAVprfWSdExxXA4vIiIiEmvxCVWhIIF6t+anEhERkaNGXEJVpKUFMCSNHR+Pw4uIiIjEXFxClW1tBSD5+BPjcXgRERGRmItPT1UgiHFbvCWl8Ti8iIiISMzFp6cqECIp14Nxu+NxeBEREZGYi0+oCkXwFeTE49AiIiIiR0R8QlUYkko1nYKIiIgcPeL2mBrfsWPjdWgRERGRmOtTqDLG/MIY86ExZrUx5mljTFZ3v5t0/El9ObSIiIhIQulrT9WrwHhr7QTgI+D2bn3LWHzHn9LHQ4uIiIgkjj6FKmvtK9baUPTju0BRd75n3AbjT+vLoUVEREQSSizHVP0/4KVDrTTGXGeMWWaMWWbjNpJLRERE5MjwdLWBMeY1YEgnq+6w1j4b3eYOIAT8/VD7sdbOBmYDnDh+rO1VtSIiIiIJqstQZa397OHWG2OuAc4FZlpruxWWTHJqt4oTERERGSi6DFWHY4yZBdwKnGatbYpNSSIiIiIDT19HN/0eSAdeNcasNMb8MQY1iYiIiAw4feqpstaOjFUhIiIiIgOZ7sMTERERiQGFKhEREZEYMN28YS+2BzWmHljf7weWI2kQsDveRUjM6bwefXROj046r0dWibU2r6uN+jSmqg/WW2unxOnYcgQYY5bpnB59dF6PPjqnRyed18Sgy38iIiIiMaBQJSIiIhID8QpVs+N0XDlydE6PTjqvRx+d06OTzmsCiMtAdREREZGjjS7/iYiIiMSAQpWIiIhIDPRrqDLGzDLGrDfGfGyMua0/jy19Y4x52BizyxjzQbu2HGPMq8aYDdFldrTdGGPujZ7n1caYyfGrXA7FGFNsjFlgjFlrjFljjLkx2q7zOoAZY5KNMUuMMaui5/W/o+3HGGMWR8/fP40xvmh7UvTzx9H1pfGsXw7NGOM2xrxnjJkb/axzmmD6LVQZY9zAfcDZwDjgP4wx4/rr+NJnfwFmHdB2G/C6tXYU8Hr0MzjneFT0dR1wfz/VKD0TAm621o4DpgPfiv47qfM6sLUCZ1hrJwInALOMMdOBnwG/jj6ztRq4Nrr9tUB1tP3X0e0kMd0IrGv3Wec0wfRnT9VJwMfW2k3W2gDwOHBePx5f+sBa+xaw94Dm84C/Rt//FfhSu/ZHrONdIMsYU9A/lUp3WWt3WGtXRN/X4/zHuhCd1wEten4aoh+90ZcFzgCejLYfeF73ne8ngZnGGNNP5Uo3GWOKgHOAB6OfDTqnCac/Q1UhsLXd54pomwxc+dbaHdH3lUB+9L3O9QATvTwwCViMzuuAF71MtBLYBbwKbARqrLWh6Cbtz13beY2urwVy+7di6YbfALcCkejnXHROE44GqktMWGduDs3PMQAZY9KAp4CbrLV17dfpvA5M1tqwtfYEoAjnKsGYOJckfWCMORfYZa1dHu9a5PD6M1RtA4rbfS6KtsnAtXPf5Z/ocle0Xed6gDDGeHEC1d+ttf+KNuu8HiWstTXAAuBknMu1+5732v7ctZ3X6PpMYE8/lyqH9yngi8aYMpyhM2cAv0XnNOH0Z6haCoyK3q3gAy4FnuvH40vsPQdcHX1/NfBsu/aroneLTQdq211OkgQRHWPxELDOWvurdqt0XgcwY0yeMSYr+t4PfA5nvNwC4KLoZgee133n+yJgvtWs0AnFWnu7tbbIWluK82fnfGvt5eicJpx+nVHdGPN5nOvCbuBha+3d/XZw6RNjzGPA6cAgYCfwX8AzwBxgGFAOfNlauzf6h/Xvce4WbAK+Yq1dFo+65dCMMacCbwPvs3+cxg9wxlXpvA5QxpgJOIOU3Th/cZ5jrb3LGDMcp5cjB3gPuMJa22qMSQYexRlTtxe41Fq7KT7VS1eMMacD37PWnqtzmnj0mBoRERGRGNBAdREREZEYUKgSERERiQGFKhEREZEYUKgSERERiQGFKhEREZEYUKgSERERiQGFKhEREZEY+P+0yy8hUXlBVwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff8bcda6400>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x2 = np.array(range(len(trace)))\n",
    "x3 = np.array(range(len(trace2)))\n",
    "plt.xlim([0,len(trace)])\n",
    "plt.gcf().set_size_inches((10,3))\n",
    "plt.plot(x3, trace2)\n",
    "plt.plot(x2, trace, '-')"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}