02_Dimensionsreduktion.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dimensionsreduktion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Überblick"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wir betrachten Methoden, um lineare Modelle bei Regressionsproblemen zu vereinfachen\n",
    "bzw. bei hochdimensionalen Datensätzen Dimensionsreduktionen durchzuführen.\n",
    "Das wesentliche Werkzeug dazu ist die **Singulärwertzerlegung**\n",
    "\n",
    "Die Singulärwertzerlegung kommt ursprünglich aus der linearen Algebra.\n",
    "Dort zeigt man, dass jede Matrix $A\\in\\mathbb{R}^{m\\times n}$ sich in\n",
    "\\begin{equation*} \n",
    "A = U\\Sigma V^T\n",
    "\\end{equation*}\n",
    "mit\n",
    "\\begin{equation*} \n",
    "\\quad\n",
    "U\\in\\mathbb{R}^{m\\times m}, \n",
    "\\quad\n",
    "V\\in\\mathbb{R}^{n\\times n},  \n",
    "\\quad\n",
    "\\Sigma = \\text{diag}\\big(\\sigma_1, \\ldots, \\sigma_r, 0, \\ldots,0 \\big)\\in\\mathbb{R}^{m\\times n}\n",
    "\\end{equation*}\n",
    "und\n",
    "$\\sigma_1 \\ge \\ldots \\ge \\sigma_r > 0$,\n",
    "$r \\le \\min(m,n)$,\n",
    "zerlegen lässt, wobei $U,V$ orthonormal sind.\n",
    "\n",
    "Geometrisch bedeutet das, dass man immer orthonormale Basen\n",
    "$V$ im Urbild-Raum und $U$ im Bildraum finden kann, so dass sich die zu $A$ gehörige\n",
    "lineare Abbildung bezüglich dieser\n",
    "Basen als Diagonalmatrix $\\Sigma$ darstellen lässt.\n",
    "\n",
    "Die Singulärwertzerlegung stellt somit eine Verallgemeinerung der Diagonalisierbarkeit\n",
    "quadratischer Matrizen (und damit der Eigenwert- und Eigenvektor-Berechnung) dar.\n",
    "\n",
    "Andererseits kann man die Berechnung von Singulärwerten und Singulärvektoren aber auch\n",
    "als restringiertes konvexes Optimierungsproblem beschreiben. Für den ersten Spaltenvektor $v_1$  von $V$ gilt\n",
    "\\begin{equation*} \n",
    "v_1 = \\text{argmax}_{\\|v\\|_2^2=1}f(v) = \\text{argmax}_{\\|v\\|_2^2 \\leq 1}f(v),\n",
    "\\quad\n",
    "f(v) = \\|A v\\|_2^2,\n",
    "\\quad\n",
    "f(v_1) = \\sigma_1^2.\n",
    "\\end{equation*}\n",
    "Allgemein gilt mit $V_k = \\text{span}(v_1,\\ldots,v_k)$\n",
    "\\begin{equation*} \n",
    "v_k = \\text{argmax}_{\\|v\\|_2^2=1, v\\bot V_{k-1}}f(v) = \\text{argmax}_{\\|v\\|_2^2\\leq 1, v\\bot V_{k-1}}f(v),\n",
    "\\quad\n",
    "f(v) = \\|Av\\|_2^2,\n",
    "\\quad\n",
    "f(v_k) = \\sigma_{k}^2.\n",
    "\\end{equation*}\n",
    "\n",
    "Für die numerische Berechnung von Singulärwerten und -vektoren haben\n",
    "wir damit die Möglichkeit, Algorithmen aus beiden Bereichen anzuwenden.\n",
    "Welche Verfahren dabei effizienter sind, hängt stark von der konkreten Aufgabenstellung\n",
    "ab."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Modellvereinfachung"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Regressionsproblem"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wir betrachten wieder das lineares Regressionsproblem aus dem letzten Abschnitt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-24T09:28:12.780405Z",
     "iopub.status.busy": "2021-09-24T09:28:12.780093Z",
     "iopub.status.idle": "2021-09-24T09:28:13.236974Z",
     "shell.execute_reply": "2021-09-24T09:28:13.236630Z"
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAam0lEQVR4nO3df5Bd5X3f8fdnV0K4NakyKzUwEmulHbVNYtcJvQVU2mSDPamlkDAd8wdp47WhYxkjdcwkHqfyHyVjjUdN/sgIGxuxJNjsJDGTMdiAR0rrSb0BT1Y4KwzINm1HBUeoUiyxKRCCkVntt3+ce9HV0bn3nnvvuT/OvZ/XzM6995xnzz5nxHzvw/d8n+dRRGBmZuU3MegOmJlZMRzQzcxGhAO6mdmIcEA3MxsRDuhmZiNizaD+8IYNG2LLli2D+vNmZqV05MiRlyJiY9a5gQX0LVu2sLS0NKg/b2ZWSpL+qtG5XCkXSd+XdFTS05IuisJKfEbSMUnPSrqqmw6bmVn72hmh/2JEvNTg3HZga/XnGuCe6quZmfVJUQ9FbwTmI3EYWC/pioKubWZmOeQN6AH8d0lHJO3MOL8JeLHu84nqsQtI2ilpSdLSmTNn2u+tmZk1lDegXxcRV5GkVnZJ+vnUeWX8zkWLxETEXERUIqKycWPmQ1ozM+tQroAeESerr6eBrwBXp5qcAK6s+7wZOFlEB83MLJ+WAV3S35d0We098EvAd1LNHgVmq9Uu1wKvRMSpwnsLLC7Cvn3Jq5mZnZenyuUngK9IqrX/44j4U0m3AUTEAeAgsAM4BrwO3NKLzi4uwnveAz/6EVxyCfzZn8G2bb34S2Zm5dMyoEfE88C7M44fqHsfwK5iu3axhYUkmJ87l7wuLDigm5nVlGotl5mZZGQ+OZm8zswMukdmZsNjYFP/O7FtW5JmWVhIgnnW6Hxxsfl5M7NRVaqADkmQbhSonWM3s3FWuoCepTYqP37cOXYzG1+lD+j1o/I1a5L8OjjHbmbjp/QBvb7yBeDDH4bpaefQzWz8lD6g1ypfannz2dmLA7kflJrZOCh9QG9V+ZJ+ULp/PywvO7ib2egpfUCH5pUv9SmZs2dh925YXXUVjJmNnlJNLOpE/WSkiQlYWTkf3BcWBt07M7PijMQIvZn6lMzLL8Pv/m5yfHUVpqYG2TMzs2KN/AgdkqC+Zw+sX5+M0iF5XV4eaLfMzAo1FgG9ZmYG1q1L0i/r1rlO3cxGy8inXOrlWQvGzKysRj6gp2vQm1XEmJmV2UgH9EaLdXmikZmNotwBXdIksAT834i4IXVuBngEeKF66OGI+FRBfexY1oYY4BUZzWw0tTNC/xjwHPBjDc4/kQ70g5ZeFmBmxrsemdnoyhXQJW0Gfhn4NPAbPe1RgRo9BJ2cTOrQJydd6WJmoyPvCH0/8AngsiZttkl6BjgJfDwivptuIGknsBNgenq6vZ52KOshaLLf9flX59TNbBS0DOiSbgBOR8SRaq48y1PAOyLiNUk7gK8CW9ONImIOmAOoVCrRYZ+7srCQTP+PSF7n5+GBB5xTN7PyyzOx6DrgVyV9H3gQuF7SH9Y3iIhXI+K16vuDwFpJG4rubBHSG01D9oNTM7OyaTlCj4g9wB54q5rl4xHx6/VtJF0O/CAiQtLVJF8UQzmxPp1XB7j/fufUzaz8Oq5Dl3QbQEQcAG4CPippBfghcHNEDCSlkkd9Xn1x8eKcuplZGbUV0CNiAViovj9Qd/xu4O4iO9Yv6Zy6yxjNrKzGanGuLOmculMuZlZWIz31P49t25Jt6R56CN7/fo/Ozay8xj6gLy7CHXckFS5PPAHvepeDupmV09inXBqt92JmVjZjE9AXF2HfvuS1Xi2HPjGRVLlkbUvX6HfNzIbJWKRcGi2jC+dz6Lt3J6P0O+64MO3S7HfNzIbJWIzQW6VVlpeTiUWrqxefd0rGzMpiLAJ6q9LEZudd1mhmZTEWKZdWe4k2O+99SM2sLDSoGfqVSiWWlpYG8re75eV2zWxQJB2JiErWubEYoRfJD0nNbFiNRQ69SH5IambDygG9TVkPSdN16q5bN7NBcMqlTemHpEePwq5dScnjunVJTXttKYHJSbj1VpiddVrGzHrPD0W7sLgIv/AL8OabyeeJCXjve5OAf+5cckyCSy91rt3MitHsoWjulIukSUnflvS1jHOS9BlJxyQ9K+mqbjpcFgsL5wM3JAH9/e9PUjG1zTIinGs3s/5oJ4f+MeC5Bue2k2wKvRXYCdzTZb9KYWYmSbNMTMDatfC5z8HOnclo/CMfSc55QpKZ9UuuHLqkzcAvA58GfiOjyY3AfHXbucOS1ku6IiJOFdfV4dNo0lFti7vZWderm1n/5H0ouh/4BHBZg/ObgBfrPp+oHrsgoEvaSTKCZ3p6up1+Dq36/UnbOWdmVrSWKRdJNwCnI+JIs2YZxy562hoRcxFRiYjKxo0b2+immZm1kieHfh3wq5K+DzwIXC/pD1NtTgBX1n3eDJwspIcjwHXpZtYPLVMuEbEH2AMgaQb4eET8eqrZo8BuSQ8C1wCvjHr+PC8vFWBm/dLxTFFJt0m6rfrxIPA8cAy4D7i9gL6NhHaWCvBI3sy60dZM0YhYABaq7w/UHQ9gV5EdGxW1pQJqI/RG5Yv1I3nPMDWzTngtlx6rlTbu3ds83ZIeyd97bxLgPVo3s7y8lksf5ClfrI3k33gjmV1aP8PUo3Qzy8Mj9CFRG8l7hqmZdcoj9CHiGaZm1g0H9CHkGaZm1gmnXMzMRoQDupnZiHBAH1KNJhl58pGZNeIc+hBqtFyAlxEws2Y8Qh9CjZYLaGcZATMbPw7oQ6g2yShdi97ouJkZOOUylJrthJR13MwMQMm6Wv1XqVRiaWlpIH971CwuOsibjQtJRyKiknXOI/SS84NSM6txDr3k/KDUzGoc0EvOD0rNrKZlykXSpcDjwLpq+y9HxJ2pNjPAI8AL1UMPR8SnCu2pZfKDUjOryZNDPwtcHxGvSVoLfFPSoYg4nGr3RETcUHwXrRUv5mVmkG+T6ABeq35cW/0ZTGmMmZk1lCuHLmlS0tPAaeDrEfFkRrNtkp6RdEjSzzS4zk5JS5KWzpw503mvrSGv9WI2vnKVLUbEOeBnJa0HviLpnRHxnbomTwHvqKZldgBfBbZmXGcOmIOkDr3LvluKSxjNxltbVS4R8TKwALwvdfzViHit+v4gsFbShoL6aDm5hNFsvLUM6JI2VkfmSHob8F7gf6baXC5J1fdXV6+7XHhvrSmXMJqNtzwplyuAByRNkgTqP4mIr0m6DSAiDgA3AR+VtAL8ELg5BrWmwBjrpISxftkAcPmjWZl5LZcxVp9zn5wECVZWnH83G2bN1nLxTNExVp9zf/NN59/Nys6Lc42hWpplaioZjWeN0J1/NysfB/Qxky5t3L8flpedQzcbBQ7oYyZd2ri8DHv2nD/vQG5WXs6hjxmXNpqNLo/Qx0ye0kbvgGRWTg7oY6jZ6oxePsCsvJxysQt4+QCz8nJAtwtk5di9gqNZOTjlYhdI59jBKRizsvAI3S6QfiDqFIxZeXiEbm/JeiBaS8HUjrnM0Wx4eYRub8kajddSMHv3Ot1iNuw8Qre3NBqNexNqs3JwQLe3dLKeupkNj5YBXdKlwOPAumr7L0fEnak2Au4CdgCvAx+KiKeK7671mkfjZuWVZ4R+Fri+ugH0WuCbkg5FxOG6NttJNoXeClwD3FN9NTOzPmn5UDQSr1U/rq3+pLc5uhGYr7Y9DKyXdEWxXTUzs2ZyVblImpT0NHAa+HpEPJlqsgl4se7zieoxMzPrk1wBPSLORcTPApuBqyW9M9VEWb+WPiBpp6QlSUtnzpxpu7NmZtZYW3XoEfEysAC8L3XqBHBl3efNwMmM35+LiEpEVDZu3NheT61UvP6LWf/lqXLZCLwZES9LehvwXuB3Us0eBXZLepDkYegrEXGq8N5aKXgJXrPByDNCvwL4hqRngb8kyaF/TdJtkm6rtjkIPA8cA+4Dbu9Jb60UvP6L2WC0HKFHxLPAz2UcP1D3PoBdxXbNyqp+xumaNXD8eDJq9yjdrLe8losVrjbj9MMfhgi4774kBeN8ullvOaBbT2zbBtPTSdrFqRez/nBAt57J2v3IzHrHi3NZz3ixL7P+ckC3nvJiX2b945SLmdmIcEA3MxsRDuhmZiPCAd3MbEQ4oNvQ8IJeZt1xlYsNhbk52LULVldh3Tov6GXWCY/QbeAWF2H3blhZSQL62bOeVWrWCQd0G7iFhWR5gJqJCc8qNeuEA7oN3MxMkmaZmIC1a+Fzn8tOtzjHbtacc+g2cHmWCPCmGWatOaDbUGi1REDWphnp9ouLXjfGxlueLeiuBOaBy4FVYC4i7kq1mQEeAV6oHno4Ij5VaE9trNVvmlG/cmMtiE9NwR13eARv4y3PCH0F+M2IeErSZcARSV+PiO+l2j0RETcU30Wz7LRMfRpmYiIZva+uNh7Bm426PFvQnQJOVd//raTngE1AOqCb9VQ6LVOfholIgrrktddtfLVV5SJpC8n+ok9mnN4m6RlJhyT9TIPf3ylpSdLSmTNn2u+tWZ36DTTWrUuqY/buTUby4IoYGz9K9nfO0VB6O/DnwKcj4uHUuR8DViPiNUk7gLsiYmuz61UqlVhaWuqw22aJrAehroixUSbpSERUss7lqnKRtBZ4CPijdDAHiIhX694flPR5SRsi4qVOO22WR1Z1TJ6KGHBVjI2ePFUuAv4AeC4ifq9Bm8uBH0RESLqaJJWzXGhPzXJqVBFTz6N4G0V5RujXAR8Ajkp6unrsk8A0QEQcAG4CPippBfghcHPkzeWYFSzPRKW8o3izMslT5fJNQC3a3A3cXVSnzLrVaqJSnlG8Wdl4pqiNpTyj+E44L2+D5IBuY6sWcGtL9XYbgJ2Xt0FzQLexVXQAdl7eBs3L59rYSgfg+fn8k5GylvKtn+jkvLwNgkfoNrbqH4xOTsIXvpDsmtRqtN5oZN+rvLxZXg7oNrbqA/C3vgWPPJKsCdMqXdIstdKqusaslxzQbazVgu9v/3YSzAHWrGmeLnHJow0rB3Qbe/V7mkpwyy3NR9lOrdiwckC3sZcecc/Otv4dp1ZsGDmg29jziNtGhQO6GR5x22hwHbqZ2YhwQDfLIWsikdmwccrFrAWv0WJl4RG6WQvz8/DGGxdOJDIbRg7oZk0sLiZLAtQmHU1Otj+RyOka65c8W9BdCcwDlwOrwFxE3JVqI+AuYAfwOvChiHiq+O6a9dfCQrK+CySTjm69tb10i9M11k95RugrwG9GxE8B1wK7JP10qs12YGv1ZydwT6G9NBuQ+hUUL70036Sjelnrvpj1Sp4t6E4Bp6rv/1bSc8Am4Ht1zW4E5qv7iB6WtF7SFdXfNSutvJOOGu1UlF7R8fhxmJuD5WVPYrLitVXlImkL8HPAk6lTm4AX6z6fqB67IKBL2kkygmd6errNrpoNRqtJR43SKrUgv38/HDoEjz0G9957Ph+/bh184xsO6lac3AFd0tuBh4A7IuLV9OmMX4mLDkTMAXMAlUrlovNmZdQorVIL8pOTybnaAmA1Z88mFTQO6FaUXAFd0lqSYP5HEfFwRpMTwJV1nzcDJ7vvntnwy1pOtz7Ir64OuIM2NvJUuQj4A+C5iPi9Bs0eBXZLehC4BnjF+XMbF43y7PW5cwnefDN5hSTIr13b/kNWs2byjNCvAz4AHJX0dPXYJ4FpgIg4ABwkKVk8RlK2eEvhPTUbYuk8ezrIQ/Z7p1usSIoYTCq7UqnE0tLSQP622aA0qoYxy0vSkYioZJ3zWi5mfeJJRtZrnvpv1ieeZGS95oBu1if1s069ubT1glMuZn3SatZpUfn1xcWkvh2SKhqndcaHA7pZHzWaddpJfj3rC2BuDm6//fwkpi98wbNRx4kDutkQyMqvt7vcAMDu3RfOSM1zLRsdDuhmQyBrtmkzjR6wppcXaHUtl1GOFgd0syGQd1XHmtoXwNmzyezTqSl417uSBb9qx37lV+ATn2i+QmSrNI8Dfrk4oJsNiVarOqbb7t9/PsVyxx1JQG7nS6FVmsd18+XjskWzklpeTtaEWV29MCDv2ZMv8LYqo3TdfPl4hG5WUu3m3dPpk1Zpnnavb4PntVzMSixvjrvT9Ilz6MPHa7mYjai8efd2yyLbvb4NB+fQzcbAsC07sLgI+/Ylr1Ycj9DNxkC7ZZFQTLol6xqunumdPDsW3Q/cAJyOiHdmnJ8BHgFeqB56OCI+VWAfzazPmgXdbvP2naZ/rLU8I/QvAncD803aPBERNxTSIzMrXLuj4kYli/PzyfowKyutr9MocHdaPeMHtK21DOgR8bikLX3oi5n1SLuj4nTQnZpKvhDeeANqhXGtrpN1jX37kuOdpH+cpmmtqBz6NknPACeBj0fEd7MaSdoJ7ASYnp4u6E+bWSvtjorTOffaF0ItmEutr1N/jampZDZrfUDesyd//52myaeIKpengHdExLuBzwJfbdQwIuYiohIRlY0bNxbwp80sj1pw3bs3/+i2lh6pBeT6KpmPfCTfdWozV5eXu5t1OmxVOsOq6xF6RLxa9/6gpM9L2hARL3V7bTMrTrs15ek0x/79SWBOV6zkSZ10O+u0kyqdcdR1QJd0OfCDiAhJV5OM+pe77pmZDVQ6zbG8fGGapJ28dhEB2ZOcWstTtvglYAbYIOkEcCewFiAiDgA3AR+VtAL8ELg5BrWegJkVptWout28tgNy7+Wpcvm1FufvJilrNLMR0q/Fu1yOWBzPFDWzhpqNqjtNo9QHcHA5YpEc0M2sY90+aP3gB9vfS9Wj+cYc0M2sqSKDaH3e/Y034K//On/app2HsOMa+B3QzayhomdozszAmjVJQI+Agwfhs5+9uBwyS56HsIuLyfIE99+ftOt1GmfYvjgc0M2soaJnaG7bBrfcAvfemwT0c+cuLodspNVD2NqXTzvLE3RjGJcj8HroZtZQL2Zozs7CpZe2f8362a779yeBun499azlCSYn4fjxC9sVtRb7MO656hG6mTXUixma3Vyz1jZrZFw/gl+zBrZvh0OH4L774IEHknaNfrcTw7jnqgO6mTXViwlB7VwznadulAbKWlDssccuHkGfPQurq8lrN+mYYVyOwAHdzIZWVp662cg4/UWRbnf0aBLMIXmdmuquf+m/N+iHpA7oZjZQzYJg1mh8z558I+OsEfTCAkxMJMF8YiJ5IFvkfQz6IakDupkNTKsg2GyTjDyVMekR9MwMrFvXm7z3wkJx6ZxOOaCb2cC0KotstUlGJwHzgx9MXmdniw24U1PFpnM64YBuZgOTp1KkNsret6/9mvhaOmdqCr797Qv3Q52dLfZelpd7l87JywHdzAamnUqRdssEa+mcWhqkXi8mHPUynZOXA7qZDVTeEsZ2ywRr6Zx0MM+zH2onhqGM0QHdzEqjnfr12oi+NkKfmEgmHN16a+v8eaflh3n618vSxjw7Ft0P3ACcjoh3ZpwXcBewA3gd+FBEPFVsN83M2pN+oJpnATDobflhr0sb84zQv0iyI9F8g/Pbga3Vn2uAe6qvZmYD1cks16IXJOvXtSHH4lwR8TjwN02a3AjMR+IwsF7SFUV10Mysn3qxIFk/rg3F5NA3AS/WfT5RPXYq3VDSTmAnwPT0dAF/2sysWL18uNnrB6dFBHRlHIushhExB8wBVCqVzDZmZoPWiwXJ+nHtItZDPwFcWfd5M3CygOuamVkbigjojwKzSlwLvBIRF6VbzMzGXVGbazSSp2zxS8AMsEHSCeBOYC1ARBwADpKULB4jKVu8pTddNTMbTnlqy/uxGmPLgB4Rv9bifAC7CuuRmVmJ5A3UvS5ZBO8pambWlbx7i/a6ZBE89d/MrCt5Fw3rx1ovDuhmZl1oJ1D3smQRHNDNzLrW60Cdl3PoZmYjwgHdzGxEOKCbmY0IB3QzsxHhgG5mNiIc0M3MRoSSmfsD+MPSGeCvcjbfALzUw+4Mq3G9bxjfex/X+wbfe957f0dEbMw6MbCA3g5JSxFRGXQ/+m1c7xvG997H9b7B917EvTvlYmY2IhzQzcxGRFkC+tygOzAg43rfML73Pq73Db73rpUih25mZq2VZYRuZmYtOKCbmY2IoQnokt4n6X9JOibpP2ecl6TPVM8/K+mqQfSzF3Lc+3+o3vOzkv5C0rsH0c+itbrvunb/UtI5STf1s3+9lOfeJc1IelrSdyX9eb/72Cs5/nv/B5Iek/RM9d5HYp9iSfdLOi3pOw3Odx/jImLgP8Ak8H+AfwRcAjwD/HSqzQ7gECDgWuDJQfe7j/f+r4Afr77fPgr3nue+69r9D5LNyG8adL/7+G++HvgeMF39/A8H3e8+3vsngd+pvt8I/A1wyaD7XsC9/zxwFfCdBue7jnHDMkK/GjgWEc9HxI+AB4EbU21uBOYjcRhYL+mKfne0B1ree0T8RUT8v+rHw8DmPvexF/L8mwP8J+Ah4HQ/O9djee793wMPR8RxgIgYlfvPc+8BXCZJwNtJAvpKf7tZvIh4nOReGuk6xg1LQN8EvFj3+UT1WLttyqjd+/qPJN/iZdfyviVtAv4dcKCP/eqHPP/m/wT4cUkLko5Imu1b73orz73fDfwUcBI4CnwsIlb7072B6jrGDcsWdMo4lq6nzNOmjHLfl6RfJAno/7qnPeqPPPe9H/itiDiXDNZGRp57XwP8C+A9wNuARUmHI+J/97pzPZbn3v8t8DRwPfCPga9LeiIiXu1x3wat6xg3LAH9BHBl3efNJN/O7bYpo1z3JemfA78PbI+I5T71rZfy3HcFeLAazDcAOyStRMRX+9LD3sn73/tLEfF3wN9Jehx4N1D2gJ7n3m8B/mskieVjkl4A/hnwrf50cWC6jnHDknL5S2CrpJ+UdAlwM/Boqs2jwGz1SfC1wCsRcarfHe2BlvcuaRp4GPjACIzQalred0T8ZERsiYgtwJeB20cgmEO+/94fAf6NpDWS/h5wDfBcn/vZC3nu/TjJ/5kg6SeAfwo839deDkbXMW4oRugRsSJpN/DfSJ6C3x8R35V0W/X8AZIqhx3AMeB1km/x0st57/8FmAI+Xx2trkTJV6XLed8jKc+9R8Rzkv4UeBZYBX4/IjLL3cok57/7XuCLko6SpCF+KyJKv6yupC8BM8AGSSeAO4G1UFyM89R/M7MRMSwpFzMz65IDupnZiHBANzMbEQ7oZmYjwgHdzGxEOKCbmY0IB3QzsxHx/wHM6xhaVnck1wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import autograd.numpy as np\n",
    "from autograd import jacobian\n",
    "\n",
    "import scipy as sp\n",
    "import matplotlib.pyplot as plt\n",
    "import sympy as sy\n",
    "\n",
    "seed = 17\n",
    "\n",
    "from IPython.display import Math\n",
    "%precision 5\n",
    "np.set_printoptions(precision=4)\n",
    "\n",
    "\n",
    "m = 100\n",
    "\n",
    "delta = 1e-1\n",
    "\n",
    "h = lambda x : 5 - 7*x + 3 * x**2\n",
    "\n",
    "np.random.seed(seed)\n",
    "x = np.random.rand(m)\n",
    "y = h(x) + delta * np.random.randn(*x.shape)\n",
    "\n",
    "%matplotlib inline\n",
    "plt.plot(x, y, 'b.');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "benutzen aber als Modellfunktion\n",
    "\\begin{equation*} \n",
    "g(x,w) = w_1 + w_2\\,(x-1) + w_3(x + 1)\n",
    "\\end{equation*}\n",
    "bei unverändertem Loss\n",
    "\\begin{equation*} \n",
    "l(w) = \\frac{1}{m}\\sum_{i=1}^m l_i(w),\n",
    "\\quad\n",
    "l_i(w) = \\frac{1}{2} \\big( g(x_i,w) - y_i \\big)^2. \n",
    "\\end{equation*}\n",
    "Die Modellfunktion ist \"überparametriert\":\n",
    "\n",
    "- mit $g$ können beliebige Geraden erzeugt werden\n",
    "\n",
    "- dafür reichen aber bereits zwei Parameter"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Mit\n",
    "\\begin{equation*} \n",
    "X = \n",
    "\\begin{pmatrix}\n",
    "1 & x_1 - 1 & x_1 + 1\\\\\n",
    "\\vdots & \\vdots & \\vdots\\\\\n",
    "1 & x_m - 1 & x_m + 1\n",
    "\\end{pmatrix}\n",
    "\\in \\mathbb{R}^{m\\times 3},\n",
    "\\quad\n",
    "\\end{equation*}\n",
    "und\n",
    "\\begin{equation*} \n",
    "w = \n",
    "\\begin{pmatrix}\n",
    "w_1 \\\\ w_2 \\\\w_3\n",
    "\\end{pmatrix},\n",
    "\\quad\n",
    "y = \n",
    "\\begin{pmatrix}\n",
    "y_1\\\\\n",
    "\\vdots\\\\\n",
    "y_m\n",
    "\\end{pmatrix}\n",
    "\\in \\mathbb{R}^m\n",
    "\\end{equation*}\n",
    "können wir $l$ umschreiben zu\n",
    "\\begin{equation*} \n",
    "l(x)=\\frac{1}{2m} \\|Xw - y\\|_2^2 = \\frac{1}{2m}(Xw - y)^T (Xw - y)\n",
    "\\end{equation*}\n",
    "und erhalten\n",
    "\\begin{equation*} \n",
    "l'(w) = \\frac{1}{m}X^T(Xw - y),\n",
    "\\quad\n",
    "l''(w) = \\frac{1}{m}X^T X.\n",
    "\\end{equation*}\n",
    "Unsere Zielfunktion ist also nach wie vor quadratisch und konvex.\n",
    "\n",
    "Wir wenden jetzt nacheinander die Algorithmen aus dem letzten Abschnitt an."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### GD"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Direkte Anwendung von GD liefert folgende Ergebnisse."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "code_folding": [
     6,
     22,
     31
    ],
    "execution": {
     "iopub.execute_input": "2021-09-24T09:28:13.263263Z",
     "iopub.status.busy": "2021-09-24T09:28:13.262936Z",
     "iopub.status.idle": "2021-09-24T09:28:13.743802Z",
     "shell.execute_reply": "2021-09-24T09:28:13.743468Z"
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "w = [-0.9954 -2.5079 -1.5126], l(w) = 0.055993\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAEGCAYAAAB4lx7eAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAmRElEQVR4nO3de5yOdf7H8df3noNFB1Pp9GPY5JBDYSYJHUQl6SQ2Um02qZR0stmSYkkpZbNqQ7u1uw4dUNKZSoVRRpRDJCGyiSRFxsz9/f3xnVsz456Ze+Y+zXXf7+fj4bEzc1/3dX+vVZ+5+lyf7+djrLWIiIh3+eK9ABERCY8CuYiIxymQi4h4nAK5iIjHKZCLiHhcajw+9KijjrL169ePx0eLiHhWbm7udmtt7ZI/j0sgr1+/PkuWLInHR4uIeJYxZmOwn0ckkBtjNgC7gQIg31qbHYnziohI+SJ5R97RWrs9gucTEZEQ6GGniIjHRSqQW+BtY0yuMaZ/sAOMMf2NMUuMMUu+//77CH2siIhEKpC3t9a2Bi4AbjbGnFnyAGvtRGtttrU2u3btgx66iohIJUUkkFtrvy38323ALKBNJM4rIiLlCzuQG2NqGmMODXwNnAesCPe8weRu3MmE99aRu3FnNE4vIuJJkahaOQaYZYwJnG+qtfbNCJy3mNyNO+kzOYe8fD/pqT6m9GtLVr2MSH+MiIjnhB3IrbXrgVMisJYy5azfQV6+H7+F/fl+ctbvUCAXEcFD5YdtTziS9FQfKQbSUn20PeHIYq8r7SIiySouW/QrI6teBlP6tSVn/Q7annBksbtxpV1EJJl5JpCDC+YlA3jO+h1s+XGv0i4ikrQ8FciLKnoXnpriI9VnKPDboGkXEZFE5tlAXvThZ0GBn15tMjm+VvWD0i4iIonOs4E88PBzf76ftFQf3VvXOSiAB1IvCu4iksg8G8jLevgJegAqIsnDs4EcDn74WVTJuvMZSzfr7lxEEpKnA3lZiqZeUnyGl3I3k1/gx2cMIy5pzpWnZcZ7iSIiEeGZDUEVFUi93HFeY3pm1yW/wN2d5/stw15ZoY1DIpIwEjaQgwvmN3c8ke6t6+BzvWAA8PstOet3xHFlIiKRk9CBPCCrXgYjLmlOqs/gA9LTVGsuIokjYXPkJV15WiaNjz1UDzxFJOEk7B25mmiJSLJIyDvyYDXkgOrKRSQhJWQgD9a7HFBjLRFJSAkZyEtu3w882Az2MxERr/NWIH/3XTjhBKhfv8zDStu+P6xbM95YsZULmh934GfqxyIiXuedQF5QAP36wdatcPfd8Oc/Q40apR4erHf5iDkrycv388mGH2h87KGA8uYi4n3eqVpJSYH58+HSS2H4cDjpJHjpJbA2pLcHy5uXlksXEfES7wRygLp1Ydo0F9Br1YKePaFTJ1ixoty3Bpv5mVEjHZ9xm4SUNxcRr/JWIA8480zIzYUnn4Tly6FlS7j1VthZes140d4rgXLEEXNW4rcWn88wrFszpVVExJO8GcgBUlPhpptg7Vro3x8mTIBGjWDSJJdPDyLQeyWrXkaxtIq1lp178mJ8ASIikeHdQB5w5JHuznzpUmja1AX1Nm1g4cIy3xYs1SIi4kXeD+QBp5wC77/vcujffQft28PVV8O33wY9PJBq6dUmk+6t68R2rSIiEZQ4gRzAGOjVC9asgXvvhRdegMaN4eGHYd++oG+ZsXQz0z/eRJ/JOerLIiKelFiBPKBmTRg5ElatclUtQ4ZAixbw+uvFDlP5oYgkgsQM5AENGsDLL/Plcy+y89cCuPBC6NYNvvwScHnyVJ/BACk+c1CeXB0URcQLvLOzs5JyN+6kz5c1sb3G0vfT1xg8fzopzZrB7bfju+Zml47BFv5vifdp16eIeEBi35HzW/pkny+NSdmX8u9n34Y+fWDMGBqddSoXLp+HtZaCguKpFaVdRMQrEj6QlywzPDm7CfzrX5CTA3Xq8PicscycMpiW339VLLWi8kQR8QpjQ+xVEknZ2dl2yZIlMfu8Ujsc+v1sGPskR49+gOo//oDp1w9GjYLatYO+T50SRSSejDG51trsg36eDIG8XLt2wYgR8MQTruJl+HAYMADS0g4copy5iMRbaYE84VMrITn8cBg7Fj77zO0Kve02aNXK9T8vpJy5iFRVCuRFnXQSvPUWzJoFv/ziatB79ICNG0vNmRctUVS5oojEg1Irpdm7192lP/ig63k+ZAhLe13Pom/3HMiRT128iWGvrKDAb0lL9YG15PstqT5Dz+y6dG9dR+kXEYkY5cgr65tvYPBgeP55qFfPBffu3cnd9CNXPL2IfP9v//8VVqQf+LpamnLpIhI5Uc+RG2NSjDGfGmPmROqcVULdujB9umvIddhhLtXSuTNr5+XgL/JLMMVnSEtxu0TBBXTl0kUkFiKZIx8ErI7g+aqWs85yrXInTIBPP6VX/4t4YN5Eau37mVSf4a+XNGda/9PpfVqm6s9FJKYikloxxtQBngNGAXdYa7uVdbynUivB7NgB992Hffppfj30cLYNGUa9wQPdXFHKqFsXEQlDVHPkxpiXgNHAocBdwQK5MaY/0B8gMzMza+PGjWF/btwtW+ZGzH34IWRluTr0du3ivSoRSVBRy5EbY7oB26y1uWUdZ62daK3NttZm1y7cOel5LVu6QdDTpsH//ueGWVxzDWzdGu+ViUgSiUSOvD1wsTFmAzAdOMcY898InNcbAsMsvvgC7rnHVbc0agSPPAJ5eaotF5Goi2j5oTHmbEpJrRTl+Rx5Wb76Cm6/HV59lV9POJGBba5mXv2scrf1K68uIuUpLbWS8P3IY65BA5g9G958k73XD2DS9PuZ2+BUHux8PTnrdwQN0kX7uGgzkYhUVES36Ftr3y/vbjxpdOnC1+8uYkyn6zjtmxW8OXkA3V+cAD//fNChRfu45BVYpi7WDFERCZ16rURR64bH0OmZR3hpylx+urQnxz01zg2DnjLFbfsvFOjjos1EIlIZCuRRllUvg7492nHUS1Nh0SI4/ni46io44wy3wajwmCn92mozkYhUinqtxJrfD88+C0OGwPbtcP31MHJkqcMsREQC1I+8qvD54E9/grVrXd/zZ55x5Yrjx0N+Pln1Mri544kK4iISMgXyeKlVCx57zA2zyM52O0RbtYL33ov3ykTEYxTI461pU3j7bZg501W0nHMO/OEPsGlTvFcmIh6hQF4VGAOXXQarVvHt7UPYP/tV/I2buDmie/ceOEy7REUkGAXyKiR326+cU/NMOv7pSd48IRvuv9+Nn5s5k9wNP9Bncg5j316jGnMRKUaBvAoJbAzafNjRDLz4bmY99l83zOLyyzn68ouot/VrDX8WkYMokFchJQc8Z3bv6mrNx4/nuHUree2fAxk2bxJH5O9RjbmIHKA68iqm1Dry7dv5ftBgjpr2HPlHHEnaQ6Ohb98DwyxEJPFp+HKi+PRTGDgQFiyArCy+uGcU8zIaaAORSBLQhqBE0aqVm0g0ZQp5W76lyeVdOPbWG7jt8df1AFQkSamNrRcZA1deybO1mpE/chTXfTyL89cu4rO9A2H8XyE9Pd4rFJEY0h25h2U1y+SJTn25oN+TfFzvZNpNHAMtWsCbb8Z7aSISQwrkHhbomnj5FR05/J034PXX3QsXXAAXXwzr1sV3gSISEwrkHlesydYFF8Dnn8OYMa5nS7Nmbo5okGEWIpI4FMgTSO7GnUxYsIncP/Rz3RV79YLRo90wi6lTiw2zEJHEoUCeIAJzPw9s4c/7HTz3HCxcCMcdB336wJlnuvJFEUkoCuQJoujcz2Jb+E8/HT7+GCZPhjVrICsLbrzRDbUQkYSgQJ4gSm7vL7aF3+eD665z6ZZbb3VBvVEjmDAB8vMBdVYU8TLt7EwgIY+JW7XKBfR589h+QmOW3TmcW7YeRl6+n/RUH1P6tdUuUZEqSDs7k0DIY+KaNiV38gsM7HEve3f8SOebe/HoS6M5dtc2dVYU8SDt7Ewygbv2LT/u5bUTT+ftzNbc+PFMbsp5kU7rPmZi+56cfu3D8V6miFSAUitJJFDZkpfvJzXFB9ZS4Lekpfp4KOtwmo8byYnz34D69d080Usvde0ARKRKKC21ojvyJFK0sqWgwE+vNpkcX6v6bzn1S193G4luvRW6d4fOneFvf3NzRUWkylKOPImUrGzp3rrOwTn1jh1drfn48bBkCZx8Mtx+O+zaFb+Fi0iZlFpJMmVVthz02vbtMHQoTJwItWu7XaLXXuvKGUUk5jRYQspUNH9+UAni0qUu3bJgAWRnu7v1tm3ju2CRJKTyQylTqTtDAVq3dsMs/vtf2LLF7Ra99lr43//itl4R+Y0CuQDBd4YW2+1pjOvXsmYNDBkC06a53aGPPgp5efFevkhSUyCXA7q3rkOvNplM6efSJsWacAW27h96qMuVr1gBZ50Fgwe7B6IaZiESNwrkciA/Pv3jTcxYuhkoJ9UC0LAhvPoqvPYa+P2uF/oll8BXX8XhCkSSmwK5BA3aZTbhKqprVzfM4uGH4d13Xc25hlmIxJQCuQQN2oExcnec17j8JlrVqsGf/+zy51dc4VIvTZq4PLqGWYhEncoPBahA58RQLFwIAwe6ssUOHVy5YsuWEVmnSDJT+aGUKeTOiaFo184Ns5g0Cb74wg2zuOkm2KGuiiLREHYgN8b8zhjzsTFmuTFmpTFmeCQWJh6XkgL9CmeHDhzognrDhsWGWYhIZETijnwfcI619hSgJdDFGKNtf+JkZMC4cbB8ObRqBbfc4jYYvf9+vFcmkjDCDuTWCZQopBX+0RMuKa5ZM5g7F2bMgJ9+cs25rrgCvvkm3isT8byI5MiNMSnGmGXANuAda+3iIMf0N8YsMcYs+f777yPxseI1xrj2uKtWwQMPwOzZ0LgxjBwJv/4a79WJeFZEArm1tsBa2xKoA7QxxjQPcsxEa222tTa7du3akfhYqaLKHeRcowbcf797EHrhhXDffa7+/OWXVa4oUgkRrVqx1v4IvA90ieR5xTsCu0QP2tofTL168OKLMG+eC+6XXQZdusDq1bFbsEgCKDeQG2NaGGMWFPm+tTHm3SLf1zbG1Cr8ujrQGfgiCmsVDyh3a38w55wDy5bBE0+4ssWTT4Y779QwC5EQhXJHvhJoYIxJKfx+LHBXkdePA94zxnwGfILLkc+J7DLFKwK7RH0GjDFk1EgP7Y2pqa5Mce1a6NsXHn/cdVf8179cLxcRKVW5gdxa68cF82bGmMuBTdbapUVe/8xa28pae7K1trm1dkQU1ytVXFa9DIZ1a4bPGAr8lhFzVpadXimpdm03keiTT+DEE+FPf3L9zxcf9PxcRAqFmiPPAdoDDwD3RG01khB27snDby2WCqRXSsrKgo8+gv/8x5Uotm3r7tQ1zELkIBUJ5COBWdbaLVFcjySAkDsnlscYuOoq14zr7rthyhSXbhk7VsMsRIoIqWmWMaYhMB9oaK39JdwPVdOsxBfRJlwBa9fC7bfD66+7+vO//Q3OPz8y5xbxgHCbZg0C/hKJIC7JIaJNuAIaNXKDLObMgYICV6p46aWwfn3kPkPEg8oM5MaYBsaYL4Dq1trnYrQmkbJdeKEbNffQQ27bf9OmMHQo/KL7DElOZQZya+1X1tom1trrYrUgkZBUq+by5mvXQs+eMGqUG2Yxfbp2h0rSUT9y8bbjj3eVLR995EoXe/d2Q6GXL4/3ykRiRoFcqoRy+7OUp317V3v+9NOuKVfr1nDzzRpmIUlBgVziKnfjTu6d9Tm9Jy4KrT9LWVJSoH9/+PJLF8Sffto9IH3qKfdwVCRBKZBL3AQabE1dvIm8Alux/ixlychwfVs+/RROOQUGDHAbjD74IDILF6liFMglbgINtgKPJg1hbiAqqUUL11nxxRdh506XO+/dW8MsJOEokEvcFN0Bmp7qo/dpmUzp1/ag2vOw8ufGQI8erjXu/fe7nudNmrgqFw2zkAQR0s7OSNPOTgkobwdoIP2Sl+8nPdUXNNBXyIYNcNddbuTc73/vuixefLEL+CJVXLg7O0WiorwdoKH0N6/QHXv9+vDSS24jUfXqbmdoly5uWpGIRymQS5VWVgOusCpeOnVywyzGjXMtclu00DAL8SwFcqnSsuplMKVfW+44r3GxtEpEKl7S0mDQILc79NprNcxCPEuBXKq8YOmXiFa8HH00TJrkxsydcMJvwyw+/jgi6xeJNgVy8aTSKl6Ayle4ZGfDggXw73/Dpk1w2mkuqH/3XYRXLxJZqloRzypZ8RJqhUtIvdJ374a//tXl0KtXd6WLAwe6dIxInKhqRRJOyZRLqBUufSbnlP9w9NBDYcwY+PxzaNfOPQg9+WR4551oXpJIpSiQS8IIZcRcKMG+mMaN3USiV1+F/fvhvPPgsss0zEKqFAVySRilVbgUVal5osZAt26wciWMHu3uyps2hfvu0zALqRKUI5ekE8iRZ9RIZ+eevIrPFd2y5bdh0HXqwKOPwh/+QO6mHyM/p1SkiNJy5ArkkpQisvX/o4/cA9Bly9jdph1XnXIlnx9ZPzKtBESC0MNOkSIqnCsPpkMHWLIE/vEPUr9YzczJt/LAW09RY/eu8FvxilSAArkkpZK58owa6SHVnx/U1yUlBW64gTUfLGF6Vlf6LHuDdyf2p9vC2RpmITGj1IokrdyNO5m5dDPbdu9j/trvyS8ov/68rHRM7sadfPnOAi585mEOzfnIDbUYPx7OOCOWlyUJTKkVkSBmLN3M3FXfhZRmKS8dk1Uvg179unHowg/ghRfghx/gzDPhyith8+ZYXI4kKQVySVoV7dcScumiMdCzp2uNO2wYzJzphlmMHq1hFhIVSq1I0gqkSvbn+0nxGXpm16V76zplVpuEtL2/pK+/dsMsZs6EBg1cl8Vu3TTMQipM5YciQVQqMFfW3Llw661u7FyXLq6PS+PG0f1MSSjKkYsEUd6Eoojq3BmWL3cBfNEiaN4cBg+Gn36K/mdLQlMgF4mlosMs/vhHGDvWDbN47jkNs5BKUyAXKUeFZoKG6uijYfJkN2aufn03oahdO/jkk8h9hiQNBXKRMoTc9rayTj0VFi50d+QbNrhhFv36wbZtkf0cSWgK5CKlyN24k3Fz17Jvf+W38od0N+/zwTXXuHTLnXe6oN6wocul798f3kVIUgg7kBtj6hpj3jPGrDbGrDTGDIrEwkTiKXAnvmDddizuX5SKzgSt8N38YYfBI4+4YRannw633+52h86dG9a1SOKLxB15PnCntfYkoC1wszGmaQTOKxI3RXdx+gy0b3hUhTsaVroxV5Mm8MYbMHs27NsH554L3bu7enSRIMIO5NbardbapYVf7wZWA/8X7nlF4qnkcOfbOjeqcIlipYZYBBgDF13khlk8+CC89RacdJLbKbpnTwWvRhJdRDcEGWPqAx8Aza21P5V4rT/QHyAzMzNr48aNEftckWgIdbNQWceVHGJR6WEWmze7YRZTp0Ldum6YRc+e2h2aZKK+s9MYcwgwHxhlrZ1Z1rHa2SmJorSOiEUD+Mpvd/Hikm/YX2CxuJ4u1dIqOXziww/d7tBly+Dss+GJJ6BFi8hfmFRJpQXy1AidPA2YAUwpL4iLJJLS8uB9Juewb/9vDbmKskBe4bEVDuRnnOGGWUyaBPfeCy1bwk03wYgRcMQR4V6OeFQkqlYM8Ayw2lr7WPhLEvGOYHnwkl0Vg/EZU7GceVEpKXDjjfDlly6IP/WU2x36j39omEWSikTVSnvgauAcY8yywj9dI3BekSovq14GU/q15Y7zGh9IlQSCe+BfLp+BtBRDinFplVSfYcQlzcPv73LEEfD3v8Onn7q+LTfdBNnZbpaoJBV1PxSJgmAPOVd8uwsD5bbKrRRr4cUX3YaizZvdMIsxY+D/VECWSNTGViROyhsRF1G//AIPP+yCeGqqy6PfcQdUqxadz5OYUhtbkTip9MagyqhZ0z34XL3abSS65x5o1gzmzHF37ZKQFMhFoiysjUEVdKC3i68WzJoFb78N6eluc9GFF8KaNVH7bIkfpVZEYiCUTUMV2SQU7D1TF29i2Csr8FtbPIWzf797KPrAA7B3L9x2Gwwd6nq7iKcoRy5SBVUmfx7sPWv+t5uhL3+Ov/BfZ5+BO89rzM0dT/ztjd9951It//wnHHusy6VfdZXrviieoBy5SBVUmfx5yffMWLq58E78t2OC1qkfcww884wbZlGvnptQ1L6922AknqZALhJHlcmfl3yPAfxF/svaZyi7Tr1NG1i4kK8fncCeNeuwbdpomIXHKbUiEmeVzZHPWLoZAzQ7/nBGzFlJXr4fn3Gbja48LbPc9/eZnEO1n3czKOd5+i6ZjalZE4YPhwEDIC2tUuuS6IpqrxURqbysehmVCpQzl24+kCcf1q1ZhboqBtIzv1aryaiz/0S1G/vTZ+pj7kHoxImsvXcUfVZXi03tu4RNqRURDyqZJ9+5J4+bO54YNNgGGzdXMj3T5Ow28Oab8Mor8OuvNOpzGeNeGMnxP34X/dp3CZvuyEU8KBCI9+f7y8ytl1YVE+gRc1Dq5OKL4bzz2DJsFGeOe4S5k5fwzOk9OP2PY2J4dVJRypGLeFQoOewJ761j7Ntr8FtIMXBHyZLEMnyWs4Lf3fsXGr07BzIzYexYuPxyDbOII5UfiiSYrHoZpaZTAsLZVXpy2+Y0mvcqzJ8PtWq5iUSdOsGKFWGtO1iqR8KjO3KRBFfR6pOgx+fnu2EWQ4fCrl2usmX4cMgIfr7SPjOmDcQSkKpWRJJUIFAGHliWN3806Oi6LbuZcWwHajz+Mv3nPsfREybAtGkwahRcd50bdlHOOQJrKLkBSoE8fEqtiCS4QGAd+/Ya+kzOKTOlESzQ5m7cSe+Ji5i6eBOTV/1Eh8zLWTXnfWjaFG64AU49FRYsKPMcAZVN9SgdUzYFcpEEV5E2AKWNrttf8FsKdn++n/d+dxy8/767K9+2DTp0cH1btmw56BwZNdIPBOFgE5XKU5FfRMlKqRWRBBdqqSJQalliWoohrzCYHziHMdCrl2uRO3o0PPIIvPwyWUOHMvXqa1m05WcyaqQf2HVasvwxVErHlE+BXCTBlVozXsbxUDynPq3/6QdaAhw0qq5mTRg5Evr2daPm/vIXWj/zDK3HjWMCjcMOwhX5RZSsVLUiIsWEWllSajXMW2/BoEGwZg27Op7LFU168uXhx5MWRpWK+r44qloRkZCEksooM9iffz589hmMH8/hw4fz+kfvs+zyazFDh9KqkkG4sv1okoUedopIMaFUlpT7ADU93aVZ1q7Fd9VVtJ4+iVadT4P//Af8/gOHqRolMhTIRaSYUCpLQi4jPPZYN5EoJ8dt87/mGra2yGb1nPdUjRJBypGLSKVUeMfo1zt4adAo7pz3L47Ys4vczpcx4KTufF+jVkh9YJQnV45cRCqovMBZ4TLCDTt5vlkn5jRoy6AF0+j73mzmffgWf+twJc+3uajMapSwH8AmOAVyETlINHqiZNRIx29hd7WajDynH8fecQtnPDmK++ZN4q5vPqR6p79DvU5B3xvKA9ipizcx7JUVFPgt1dKi28elqv3CUI5cRA5SmaHQ5dm5J49AA1wfsPGYehw+fx68/DLV9++Dzp1dm9wNGw56b3k5+dyNOxn2ygry/RYL5O2P3jCMqpjbVyAXkYOE0/62rHNWS3PnTE8rsjv0kktg1Sq3qeiNN+Ckk+CBB2DPngPvDTyA7dUmk+6t6xx07pz1O4oPoPaZYq0BAiJRJRONX3Lh0sNOEQkqGumDcs/5zTcweDA8/7yrcnnsMejeHYwpM91T9DWfMfTr8HueXbSh2LEAvSflHNghOu36ym9O6jP5t/PEshWvHnaKSIVEYxNOueesWxemT4ebboJbb4UePdjc6nR2PfQIOWnHlJonL9mGINhd87c/7iUv39Ww5+X7mbl0c6Wur6ItD2JBgVxEqp6zziJ35lzm3PIAg97/N8d2OZPUnn/kubpd2JFWM2i6p+QviZL9WWYs3Vzs+HByEcF+IcXzAagCuYjERXmBL2fTLp5r2ZVZjTow+MP/0Pul51hQazYfXXs7h11/Q5nBsrS75peWfMP+AktaiuHyILn2cK4lnpOPFMhFJOZCCXyBB667axzG8K4D+eGqvlz7/ON0fGwovD8Txo+Hdu1K/Yxgd809s+tigctLdnAMU876Hezb7y9WMRPLQK6qFRGJuVAqP4pWqmAtj39fg1PPHcr6JybBd99B+/Zw9dXw7bdBPyNQoTJ18SbumfU5vSflMO3jTcwskWKJhIwa6QdSNf7C72NJgVxEYi7U8sasehkcX6s6+X7rgn6B5Y3mZ8OaNXDvvfDCC9C4MYwZA/v2HXhf0Vrve2Z9ztTFm6JaMrhzTx6+wiJ5n3Hfx5ICuYjEXEVGvgUN+oFhFqtWQadOcPfd0KIFvP46UPyOvyhD5OriS1tjehyGX6iOXESqvHIrQt58E267zd2pX3ghKwYPp8fcbQeCuQ9ITTH0zK578ISjin5WJd8XiaqW0urIww7kxph/At2Abdba5qG8R4FcRCIuL889AB0+HPbtY+t1A3i127UccmQGO/fkhRRAo1V9EqnzlhbII5FaeRboEoHziIhUXpFhFvTuzXFPjaP/9V25ct2H3Hx2g5ACZ7S230d7W3/Ygdxa+wHwQwTWIiISvmOPhWefhUWL4Pjj4aqroEMHWLq03LdGo8dMNM8bEJEcuTGmPjCnrNSKMaY/0B8gMzMza+PGjWF/rohImfx+F9SHDIHt2+H6691D0tq1S31LtHZoVukceeHJ61NOIC9KOXIRiakff4QRI1wO/ZBDXB59wABI9daeyGjmyEVEqrZatVwnxeXLITsbBg2Cli3h3Xdj8vHRHjKtQC4iyaNpU3j7bZg1C375xdWg9+wJYaR6ywvSsRhEEXYgN8ZMAxYBjY0xm40x14W/LBGRKDEGLr3UbSYaMQJeew2aNHHplr17K3SqUIJ0LAZRRKJqpbe19jhrbZq1to619plILExEJKqqV4f77oMvvoCLL3ZTiU46CWbMgBCfHYYSpKNdsQJKrYhIssvMdBOJ3nsPDjsMevSAc8+FlSvLfWsoQboi7QgqS1v0RUQC8vPh6afdnfpPP8Ett7g79Vq1Sn1LLAdKRLX8sKIUyEWkStu+3QXzp5+GI4+E0aOhb19ISYnrslR+KCISqqOOgqeegtxc9yD0+uvhtNPcbtEqSIFcRKQ0rVrBBx/A1KmwdaubSHTNNe7rKkSBXESkLMZA796uRe5f/uIejDZqBI884jouVgEK5CIioTjkEHjwQVfN0rEj/PnPbpjFG2/Ee2UK5CIiFXLiiTB79oFpRHTtChddBOvWxW1JCuQiIpVxwQXw+ecuxTJ/PjRr5lIvP/8c86UokIuIVFZ6Otx1l8uf9+4NDz3khkFPnRry7tBIUCAXEQnXcccVH2bRpw+ccQZ8+mlMPl6BXEQkUtq2hcWLYfJkN3IuKwtuuMFtMIoiBXIRkUjy+eC661wgv+02eOYZaNgQ/v531wIgGh8ZlbOKiCS7wDCLzz5zd+YDB7oNRps3R/yjFMhFRKKpaVN45x2YORMaNHD59Ajz1sA6EREvMgYuu8z9iQLdkYuIeJwCuYiIxymQi4h4nAK5iIjHKZCLiHicArmIiMcpkIuIeJwCuYiIxxkbw1aLBz7UmO+BjSEcehQQ3W4zVVeyXnuyXjck77Un63VDxa+9nrW2dskfxiWQh8oYs8Ramx3vdcRDsl57sl43JO+1J+t1Q+SuXakVERGPUyAXEfG4qh7IJ8Z7AXGUrNeerNcNyXvtyXrdEKFrr9I5chERKV9VvyMXEZFyKJCLiHhclQjkxpguxpg1xph1xpghQV43xpgnCl//zBjTOh7rjLQQrrtP4fV+ZoxZaIw5JR7rjIbyrr3IcacaYwqMMT1iub5oCeW6jTFnG2OWGWNWGmPmx3qN0RLCP++HG2NeNcYsL7z2vvFYZ6QZY/5pjNlmjFlRyuvhxzdrbVz/ACnAV8AJQDqwHGha4piuwBuAAdoCi+O97hhddzsgo/DrCxLhukO99iLHvQu8DvSI97pj9HdeC1gFZBZ+f3S81x3Da78HeLjw69rAD0B6vNcegWs/E2gNrCjl9bDjW1W4I28DrLPWrrfW5gHTgUtKHHMJ8G/r5AC1jDGRH3wXW+Vet7V2obV2Z+G3OUCdGK8xWkL5OwcYCMwAtsVycVEUynVfCcy01m4CsNYm07Vb4FBjjAEOwQXy6IydjyFr7Qe4aylN2PGtKgTy/wO+KfL95sKfVfQYr6noNV2H+62dCMq9dmPM/wGXAf+I4bqiLZS/80ZAhjHmfWNMrjHmmpitLrpCufa/AycB3wKfA4Ostf7YLC+uwo5vVWH4sgnys5I1kaEc4zUhX5MxpiMukHeI6opiJ5RrHwfcba0tcDdoCSGU604FsoBOQHVgkTEmx1q7NtqLi7JQrv18YBlwDtAAeMcY86G19qcory3ewo5vVSGQbwbqFvm+Du43ckWP8ZqQrskYczIwGbjAWrsjRmuLtlCuPRuYXhjEjwK6GmPyrbUvx2SF0RHqP+vbrbW/AL8YYz4ATgG8HshDufa+wEPWJY7XGWO+BpoAH8dmiXETdnyrCqmVT4CGxpjfG2PSgV7A7BLHzAauKXy62xbYZa3dGuuFRli5122MyQRmAlcnwB1ZUeVeu7X299ba+tba+sBLwACPB3EI7Z/1V4AzjDGpxpgawGnA6hivMxpCufZNuP8SwRhzDNAYWB/TVcZH2PEt7nfk1tp8Y8wtwFu4J9v/tNauNMbcWPj6P3BVC12BdcAe3G9uTwvxuocBRwJPFt6Z5tsE6BIX4rUnnFCu21q72hjzJvAZ4AcmW2uDlq15SYh/538FnjXGfI5LN9xtrfV8e1tjzDTgbOAoY8xm4H4gDSIX37RFX0TE46pCakVERMKgQC4i4nEK5CIiHqdALiLicQrkIiIep0AuIuJxCuQiIh6nQC4CGGPeM8acW/j1SGPME/Fek0io4r6zU6SKuB8YYYw5GmgFXBzn9YiETDs7RQoVTuM5BDjbWrs73usRCZVSKyKAMaYFcBywT0FcvEaBXJJe4TSWKbhJLb8YY86P85JEKkSBXJJaYavYmcCd1trVuA58D8R1USIVpBy5iIjH6Y5cRMTjFMhFRDxOgVxExOMUyEVEPE6BXETE4xTIRUQ8ToFcRMTj/h/IzaWdzdSrEwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY8AAAEICAYAAACnL3iHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAdKklEQVR4nO3deZhU1Z3/8fe3blXvTUOzbwKKoqigiIC7jhpX5OeSaEajoxE0OupkTEYTn5lkZp78nJh1kriMRjQaRR13nZhoxl0UbAQEFxBRZJNNutl6rzN/1K2meqOq6Oq+VdWf1/PU07dO3ar+Hq/w4dS591xzziEiIpKOUNAFiIhI7lF4iIhI2hQeIiKSNoWHiIikTeEhIiJpCwddQE8ZMGCAGz16dNBliIjklAULFmx2zg1s295rwmP06NFUVVUFXYaISE4xs1UdtetrKxERSZvCQ0RE0pb34WFm083s7pqamqBLERHJG3kfHs6555xzsyoqKoIuRUQkb+R9eIiISOYpPEREJG0KDxERSVveh0dXJ8yfWLCGh+d9keGqRERyW96HR1cnzJ9ZvI7HqlZnuCoRkdyW9+HRVZ5Bc1Q3zBIRSaTwSMILhRQeIiJtKDyS8EIaeYiItJX34dHVCfNwKESz7vMuItJK3odHVyfMQyHTyENEpI28D4+uCis8RETaUXgkETKFh4hIWwqPJDTyEBFpT+GRRChkNCk8RERayfvw6PrZVkZUZ1uJiLSS9+HR1bOtvJDR1BzNcFUiIrkt78Ojq7yQoW+tRERaU3gk4YWMpqhGHiIiiRQeSXghQ9khItKawiMJzzTyEBFpS+GRRHzOw+mMKxGRFnkfHl09VdcLGaCVdUVEEuV9eGTiVF1AK+uKiCTI+/DoKo08RETaU3gkEVZ4iIi0o/BIImQKDxGRthQeSYQ9hYeISFsKjyQ08hARaU/hkYTOthIRaU/hkUQ8PJqaFR4iInF5Hx5dvkjQ/9pK9/QQEdkt78OjqxcJxifMdTdBEZHd8j48uio+YR5VeIiItFB4JBG/SFAjDxGR3RQeSYR0hbmISDsKjyS0PImISHsKjyRCus5DRKQdhUcSGnmIiLSn8EjC0/IkIiLtKDyS0P08RETaU3gkofAQEWlP4ZGEFkYUEWkv78Ojy2tbxcNDCyOKiLTI+/Do6tpWGnmIiLSX9+HRVZrzEBFpT+GRhK7zEBFpT+GRhG5DKyLSnsIjiXAo9p9I4SEispvCIwk/OxQeIiIJFB5JtIw8dLaViEgLhUcS8ZGHbgYlIrKbwiOJ+MhDt6EVEdlN4ZFEfFVdjTxERHZTeCThebHw0MhDRGQ3hUcSGnmIiLSn8EgivjxJVGdbiYi0UHgkEQ+PJq2qKyLSQuGRhJ8dus5DRCRB3odHV+/nYWZ4IaM5Gs1wZSIiuSvvw6Or9/MA/PDIYFEiIjku78MjEzzTyENEJJHCIwVhjTxERFpReKQgpDkPEZFWFB4pCIdMZ1uJiCRQeKQgNvJQeIiIxCk8UhBWeIiItKLwSEHITGtbiYgkUHikIOyZVtUVEUmg8EiBp5GHiEgrCo8UeCHTqroiIgkUHinwQqZVdUVEEig8UqCRh4hIawqPFHghzXmIiCRSeKTA03UeIiKtKDxSEFtVV+EhIhKn8EiBRh4iIq0pPFKg8BARaU3hkQJPq+qKiLSi8EiBRh4iIq0pPFJQGA5R36ibQYmIxCk8UlBSEKa2sTnoMkREsobCIwVFEY9dDQoPEZE4hUcKSgo86jTyEBFpEQ66gL1hZqXAHUAD8Kpz7qHu/H3FEY9dDU045zCz7vxVIiI5IWtGHmY228w2mtnSNu2nm9kyM1thZjf7zecBjzvnZgLndHdtxQUeUQcNzZo0FxGBLAoP4H7g9MQGM/OA24EzgPHAN81sPDACWO3v1u3fJxVHPABqNe8hIgJkUXg4514HvmrTPAVY4Zxb6ZxrAB4BZgBriAUI7KEPZjbLzKrMrGrTpk17XVtJgR8emvcQEQGyKDw6MZzdIwyIhcZw4EngfDO7E3iuszc75+52zk12zk0eOHDgXhdR7IeHzrgSEYnJ9gnzjmannXNuJ3B5TxWhr61ERFrL9pHHGmBkwvMRwLqeLqJYX1uJiLSS7eHxLrC/mY0xswLgIuDZdD7AzKab2d01NTV7XUTLnIdGHiIiQA+Eh5n9p5mV+NuzE9qPMLOrEp7PAd4GxpnZGjP7tnOuCfh74C/AR8BjzrkP0vn9zrnnnHOzKioq9roPRRGNPEREEqUUHmZ2ipk9aGbFZvaafwptKu+rJDZHscsPkBozO8nMfgp8ABzn71dAbCJ8pHMu4pwb4Zy7l9ib/+ScO8A5t59z7id708muKimITQ1p5CEiEpPqyGMisBi4AnjSOZfq36KHAR/625P85+Occzc55+qAOjMb7J+G+7/AhakW3pOKNfIQEWklnfBYBFwMPAPgz0O85m9PMjNnZv3NzDOzpf5IoxKo9j/jSGAesDPhc7cCffztp/3Pzzo6VVdEpLV0wmMJsK9z7nO/rRoo97evA94B+gFnAS8553YBy4HR/j7jgR8CB5rZcX7bcOALf3spsYDJqExMmMdHHlocUUQkJml4mFmE2OjAY/coAqAGKDGz/sBQ4C1i4TELuNPfZwmwL4BzbqZzLuqcu8U594aZlQHbnHP1/uvNQIOZlZNBmZgwj3iGFzJ2NTRlsDIRkdyVyshjPLEznWqBonijcy6+SuBM4F5gOzAB8Jxzy/19HPBQ/GyrNoYBP2vTVgjUpdOBnmBmlEQ8ahu0MKKICKR2hflEYJFzbqs/n1HkT3YDRImtans8sQv4vgfcnPhm59wbHX1oPGDi/BHMJudcY5p96BFFBR61jRp5iIhAaiOP+JlWAC8Cxya81gC84F+PsQ0oBZ7fy1pOAv60l+/tdiUFnk7VFRHxJR15OOduTHj6O+Afgb/6rx2WsN+9xL6+2lt/C/ygC+/vkJlNB6aPHTu2S59TrFvRioi0SOsKc+fcQuCVVC8STJV/keDTzrllmfxcyMyEOcRO19V1HiIiMWmvquucm518r7Q/swF4INOfm0nFEd3HXEQkLtsXRswapYVhtu7Kyrl8EZEep/BI0YThFazYuIOtOxuCLkVEJHAKjxQdPbY/AO+s3BJwJSIiwVN4pGjCiL6UFHjM/VThISKS9+GRibWtACJeiGn79udPS9azcXvWXQQvItKj8j48MnWqLsBNpx/IzoYmrn3oPZ15JSK9Wt6HRyaNG1LOzy6YSNWqrVzz0Hs0NmutKxHpnRQeaZo+cRj/PuMQXv54I999dBHNURd0SSIiPS7tiwQFLpk2ih31TfzHCx9TVhjm1vMOxcyCLktEpMcoPPbS1Sfsx/a6Rm5/5VPKi8L88MyDFCAi0msoPLrge18bx466Ju554zPKiyJcf/L+QZckItIj8j48MrWqbiefzY+mH8z2+iZ++dJyygrDXHHsmIz/HhGRbJP3E+aZPFW3I6GQcdv5Ezjt4MH82/Mf8vC8L5K/SUQkx+V9ePSEsBfiN988nJPGDeSHTy3h8QVrgi5JRKRbKTwypDDsceclR3Ds2AH80+OLeWbR2qBLEhHpNgqPDCqKeNxz6WQmj67kHx9bzAtL1gddkohIt1B4ZFhxgcfsvzuSiSMquG7OQv764YagSxIRyTiFRzcoKwxz/xVTOHhYH6556D1eW74p6JJERDJK4dFN+hRFeOCKqYwdVMasB6qYu2Jz0CWJiGSMwqMbVZRE+OOVUxnVv4Rv/6GKdz//KuiSREQyIu/DI1P389hblaUFPHTlNIb2LeLy+95l4RdbA6lDRCST8j48uvsiwVQMLC/k4Sun0b+sgEtnz2fR6urAahERyYS8D49sMaSiiIdnTqNfSQHf+v08jUBEJKcpPHrQ8L7FPDJrGv1KC7j03vm8pwARkRyl8Ohhw/oW8+hV06gsiwXIglUKEBHJPQqPAAytKObRWUcxsLyQS++dR5XOwhKRHKPwCMiQiiLmzJzG4D5FXDZ7vk7jFZGcovAI0JCKIubMmsbgiliAzP9MASIiuUHhEbDBfYp4ZOY0hlYU8Xf3zeedlVuCLklEJCmFRxYY1Cc2AhnWt5jL73uXt7SUiYhkOYVHlhhUHpsDGdW/hMvvf1er8YpIVsv78Ah6eZJ0DCwvZM7MaRw4pJyr/7iA599fF3RJIiIdyvvwyIblSdLRr7SAh66cyuH79OX6OQt5rGp10CWJiLST9+GRi8qLIvzhiikcM3YA//T4+zzw9udBlyQi0orCI0uVFIT5/WWTOXX8YP7lmQ+489VPgy5JRKSFwiOLFYY97rh4EudMHMZP//wxv3hxGc65oMsSESEcdAGyZxEvxK8uPIziiMdvX15BTW0jP55+MKGQBV2aiPRiCo8c4IWMW887lD7FYe554zO+2tnAL79xGAVhDRxFJBgKjxwRChm3nDWeAWWF3PrCx9TUNnLnJUdQVqhDKCI9T/90zTFXnbAft10wgbmfbuHie95hy476oEsSkV5I4ZGDvjF5JP91yRF8/OV2vn7X26zZuivokkSkl1F45KhTxg/mj1dOZdOOes6/cy7LN2wPuiQR6UUUHjnsyNGV/PfVR+EcXHDnXN0TRER6jMIjxx04pA9PfOdoBpQVcvE983h2sdbDEpHup/DIAyMrS3jymqM5bGRsPazbX1mhiwlFpFspPPJE35ICHrxyCjMOG8bP/rKMHzy5hMbmaNBliUie0kUCeaQw7PHrCw9jn8oSfvvyCtZW13LHxZMoL4oEXZqI5BmNPPKMmXHj18Zx2/kTePvTLXz9rrdZV10bdFkikmfyPjxy6WZQmfSNI0dy/+VTWLu1lnPveIula3tX/0Wke+V9eOTazaAy6dj9B/D4d44mHApxwV1zeU5nYolIhuR9ePR244aU8/S1x3DIsAqum7OQn/9lGdGozsQSka5RePQCA8sLeXjmNC46ciS/e2UFsx5cwPa6xqDLEpEcpvDoJQrCIW4971D+9ZyDeWXZRs6/cy6rtuwMuiwRyVEKj17EzLjs6NE8cMUUNm6vZ8btb/HmJ5uDLktEcpDCoxc6ZuwAnrn2GAaVF/Kt2fP43cufaB5ERNKi8OilRvUv5alrjuGcicP4+YvLufKBKmp2aR5ERFKj8OjFSgvD/PrCw/j3GQfzxiebOOu3b7Bkja4HEZHkFB69nJnxraNG89hVRxGNOs6/ay6PzP9CCyuKyB4pPASAw/fpx/PXH8fUMZXc/OQSbnxsMTvqm4IuS0SylMJDWlSWFnD/5VO44eT9eXrRWs7+zRssXl0ddFkikoUUHtKKFzK+e+oBPDLrKBqaopx/51zueu1TnY0lIq0oPKRDU8ZU8sINx3Pq+MH8xwsfc+ns+WzcVhd0WSKSJRQe0qmKkgh3XDyJW887lKpVX3H6f77BXz/cEHRZIpIFFB6yR2bGN6fsw/PXHcvgPkVc+UAVNz62mJpaXRMi0pspPCQlYweV88y1x3Dd34zl6UVrOe1Xr/PKso1BlyUiAVF4SMoKwiFu/No4nrrmaMqLwlx+37vc9Pj7bNMKvSK9jsJD0jZhRF+ev/5YvnPifvz3gtWc9qvXeW35pqDLEpEepPCQvVIY9rjp9AN54jtHU1Lgcdns+dzwyEI2ba8PujQR6QEKD+mSw/fpx/9cfxw3nLw/Lyz5kpN/8SoPz/tC14WI5DmFh3RZUcTju6cewAv/cBzjh/Xhh08t4ev/9TYff7kt6NJEpJsoPCRj9htYxpyZ0/jF1yeyctMOzvrNm/zomaVU72oIujQRyTCFh2SUmXH+ESN4+cYTuXjqPjz4zipO/PmrPPj25zQ1R4MuT0QyJCfDw8z2NbN7zezxoGuRjvUrLeDfZhzCn244joOG9OGfn/mAs37zJm+t0G1vRfJBj4eHmc02s41mtrRN++lmtszMVpjZzXv6DOfcSufct7u3UsmEA4f04eGZU7nrkiPY1djExb+fx6Wz57N0rW46JZLLghh53A+cnthgZh5wO3AGMB74ppmNN7NDzez5No9BPV+ydIWZcfohQ3jpuydwy5kH8f6aas7+7ZtcN2chq7bsDLo8EdkL4Z7+hc65181sdJvmKcAK59xKADN7BJjhnLsVOLuHS5RuUhTxmHn8vnzjyJHc/fqn3PvmZ7ywZD0XTRnJtSeNZWhFcdAlikiKsmXOYziwOuH5Gr+tQ2bW38zuAg43sx/sYb9ZZlZlZlWbNukK6GxRURzh+6cdyOvfP4kLjxzJI/NXc8Jtr3LLU0tYs3VX0OWJSAp6fOTRCeugrdOrzJxzW4Crk32oc+5u4G6AyZMn66q1LDOoTxE/OfdQrj5hP+567VMeq1rNo++u5vxJI7jmpP0Y1b806BJFpBPZMvJYA4xMeD4CWBdQLdLDRlaW8JNzD+W175/ExVP34alFaznp569y7cPvsUi3wRXJSuZcz/+D3J/zeN45d4j/PAwsB04G1gLvAn/rnPsgU79z8uTJrqqqKlMfJ91ow7Y6Zr/5GQ/P/4LtdU0cObofVx63L6ccNBgv1NEgVUS6i5ktcM5Nbtfe0+FhZnOAE4EBwAbgR865e83sTODXgAfMds79JEO/bzowfezYsTM/+eSTTHyk9JAd9U08+u5qZr/5GWuraxndv4RLpo3i/Ekj6FdaEHR5Ir1C1oRHUDTyyF1NzVH+/MGX3PfW5yxYtZWCcIizJwzl4qmjmLRPX8w0GhHpLgoPhUde+Gj9Nh6at4qn3lvLzoZmDhrahwuOGME5E4cxsLww6PJE8o7CQ+GRV3bUN/HsonXMmf8FS9bW4IWMEw4YyLmHD+fU8YMpinhBlyiSFxQeCo+89cmG7Ty5cC1PL1zL+po6ygvDnHnoUM6aMJSj9utPxMuWkwpFck+vDQ9NmPcezVHHvJVbeOK9tfx56Xp2NjTTpyjMKeMHc8YhQzlu/wEakYikqdeGR5xGHr1LXWMzb3yymT8v/ZK/frSBmtpGSgo8Tho3iL85cBDHHzBQcyQiKegsPLLlCnORjCqKeJw6fjCnjh9MY3OUd1Zu4YWlX/LiBxv4nyXrATh4WB9OHDeQEw4YxOH79NXXWyJp0MhDepVo1PHh+m28tnwTry3bxIIvttIcdZQXhpkyppKp+1YydUx/Dh7Wh7DCRERfWyk8pCPb6hqZu2Izry3fzLyVW1i5ObZEfGmBxxGjK5k6ppIjRvXj0OEVlBZqoC69T68ND02YSzo2bqtj/udfMW/lV8z7bAvLN+wAIGSw/6ByJoyoYOLIvkwc0ZdxQ8opCGt0Ivmt14ZHnEYesje+2tnA4tXVLF5T7f+s4audDQAUhEMcOKSccYPLGTeknAOH9OGAIWUMLCvUVe+SNxQeCg/JAOcca7bWtoTJR+u38/GX29m8o75ln8rSAsYNLueAwWWMHlDK6AGljOlfyoh+xZpHkZyjs61EMsDMGFlZwsjKEs6eMKylfcuOepZ9GQuSZV9u5+MN23nyvbVsr29q2Sccir13dP+SWKj0L2VY32KG9S1ieN9iKoojGrFIzlB4iGRA/7JCjh5byNFjB7S0OefYvKOBz7fs5LPNO/l8805/exfvrPyK2sbmVp9RUuAxtKKIYX2LGd63mKEVxQwsL2z1GFBWQGFYFzpK8BQeIt3EzFr+0j9ydGWr15xzbNpRz/rqOtZV17K2upb1NbHtddW1fLS+9VdhifoUhRPCpJB+JQX0LYlQURxp2Y49L6Cf366vyyTT8j48Es62CroUkRZmxqDyIgaVFzFxZN8O92loirJlZz2btu9+bN7hb/s/P1i3jepdDdTUNhLdw/RleWGYsqIwpYVhyhIesedey2vlfltpYZiSAo+iiEdR2KMoEqIo4lEYCVEcibXrosreTRPmInkgGnVsr2uiuraB6l2NVNc2Ur3L397VSHVtAzvqmtjZ0MT2uiZ21jexs76ZHfVNLY/mPaVPB7yQURSOhUrsEdsuCIeIeCEKvBARz4h4If/hb4d3vxb24vu23y/shfBCEDIjHNq97YWMUMgIhwzPYtte/BF/3RLaWvaj9XMzzMAwLARGLNRD8Taj5fWQJbzWy+alNGEuksdCIaOiJEJFSYRR/dN/v3OO+qZoLEjqYmFS19hMXWOU2sZmf7uZuqYo9Y3N1DY0U9cUez2+X3yfhuYoDU2x922ri203NkdpbHZtfu5uz0WtAgUDo2U7/loskPxti4VfPKRiwbQ7oOKZFI+mxJBqeS3+uxLbEvZteYe1/pz7Lz+SEf1KMtp/hYeIYGYtI4gBZT27YKRzjqaoHyhNjgY/VJqaHc3O0Rx1RJ2jqTn2szm6u7056oi2eR7fvzmK3x6lOUrLfk3+e5xzRB04vwbnwBH7GU3Yjr8Wb4vG3oADom1ew7Vva/mM2Ntiryf+zvh7/deBln1j27sbXcJ/s/b77X6tJY79jYJu+IpR4SEigTKzlq+q0K3pc4ZmvEREJG0KDxERSVveh4eZTTezu2tqaoIuRUQkb+R9eDjnnnPOzaqoqAi6FBGRvJH34SEiIpmn8BARkbQpPEREJG0KDxERSVuvWdvKzDYBq/by7QOAzRksJ0j50pd86QeoL9kqX/rS1X6Mcs4NbNvYa8KjK8ysqqOFwXJRvvQlX/oB6ku2ype+dFc/9LWViIikTeEhIiJpU3ik5u6gC8igfOlLvvQD1JdslS996ZZ+aM5DRETSppGHiIikTeEhIiJpU3jsgZmdbmbLzGyFmd0cdD3pMrPPzWyJmS0ysyq/rdLMXjKzT/yf/YKusyNmNtvMNprZ0oS2Tms3sx/4x2mZmZ0WTNUd66QvPzaztf6xWWRmZya8lpV9MbORZvaKmX1kZh+Y2Q1+e84dlz30JRePS5GZzTezxX5f/tVv797jErsVoh5tH4AHfArsS+z+ZouB8UHXlWYfPgcGtGm7DbjZ374Z+GnQdXZS+/HAJGBpstqB8f7xKQTG+MfNC7oPSfryY+B7HeybtX0BhgKT/O1yYLlfb84dlz30JRePiwFl/nYEmAdM6+7jopFH56YAK5xzK51zDcAjwIyAa8qEGcAf/O0/AP8vuFI655x7HfiqTXNntc8AHnHO1TvnPgNWEDt+WaGTvnQma/vinFvvnHvP394OfAQMJwePyx760pls7otzzu3wn0b8h6Obj4vCo3PDgdUJz9ew5/+5spEDXjSzBWY2y28b7JxbD7E/QMCgwKpLX2e15+qx+nsze9//Wiv+lUJO9MXMRgOHE/tXbk4flzZ9gRw8LmbmmdkiYCPwknOu24+LwqNz1kFbrp3XfIxzbhJwBnCtmR0fdEHdJBeP1Z3AfsBhwHrgF3571vfFzMqAJ4B/cM5t29OuHbRle19y8rg455qdc4cBI4ApZnbIHnbPSF8UHp1bA4xMeD4CWBdQLXvFObfO/7kReIrY0HSDmQ0F8H9uDK7CtHVWe84dK+fcBv8PfBS4h91fG2R1X8wsQuwv24ecc0/6zTl5XDrqS64elzjnXDXwKnA63XxcFB6dexfY38zGmFkBcBHwbMA1pczMSs2sPL4NfA1YSqwPl/m7XQY8E0yFe6Wz2p8FLjKzQjMbA+wPzA+gvpTF/1D7ziV2bCCL+2JmBtwLfOSc+2XCSzl3XDrrS44el4Fm1tffLgZOAT6mu49L0GcKZPMDOJPYWRifArcEXU+ate9L7IyKxcAH8fqB/sD/Ap/4PyuDrrWT+ucQ+9qgkdi/lL69p9qBW/zjtAw4I+j6U+jLg8AS4H3/D/PQbO8LcCyxrzfeBxb5jzNz8bjsoS+5eFwmAAv9mpcC/+K3d+tx0fIkIiKSNn1tJSIiaVN4iIhI2hQeIiKSNoWHiIikTeEhIiJpU3iIiEjaFB4iIpI2hYdIQMzsFDN7MOg6RPaGwkMkOBOJXRksknMUHiLBmQgs9NcYut/M/r+/5pJI1gsHXYBILzaR2EqnfwF+75z7Y8D1iKRMa1uJBMBfDnwzsAq4yjn3dsAliaRFX1uJBGM8sWX/m4DmgGsRSZvCQyQYE4G5xO4Tc5+ZDQ64HpG0KDxEgjERWOqcWw7cBDzmf5UlkhM05yEiImnTyENERNKm8BARkbQpPEREJG0KDxERSZvCQ0RE0qbwEBGRtCk8REQkbf8HF49lY3x1k1oAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "npar = 3\n",
    "\n",
    "def g(x, w):\n",
    "    return w[0] + w[1] * (x - 1) + w[2] * (x - 2) \n",
    "    #return w[0] + w[1] * x + w[2] * x**2 + w[3] * x**3\n",
    "\n",
    "def l(w):\n",
    "    return ((g(x, w) - y)**2).mean()\n",
    "\n",
    "l1 = jacobian(l)\n",
    "\n",
    "def GD(w0, l1, gamma = 1.0, nit = 20):\n",
    "    ww = [w0]\n",
    "    w  = w0.copy()\n",
    "    \n",
    "    for k in range(nit):\n",
    "        w = w - gamma * l1(w)\n",
    "        ww.append(w)\n",
    "    \n",
    "    return ww\n",
    "\n",
    "\n",
    "def evalw(w, x, y):\n",
    "    # Fit\n",
    "    plt.figure(1)\n",
    "    plt.plot(x, y, '.');\n",
    "    xp = np.linspace(x.min(), x.max())\n",
    "    plt.plot(xp, g(xp, w), 'r');\n",
    "    plt.xlabel('$x$')\n",
    "    plt.ylabel('$y$', rotation=0)\n",
    "\n",
    "def evalwl(w, l, x, y):\n",
    "    # Konvergenzgeschwindigkeit\n",
    "    plt.figure(2)\n",
    "    plt.semilogy(list(map(l, w)))\n",
    "    plt.xlabel('$k$')\n",
    "    plt.ylabel('$l(w^{(k)})$', rotation=0)\n",
    "    \n",
    "\n",
    "w0 = np.zeros(npar)\n",
    "w = GD(w0, l1, gamma = 0.1, nit = 300)\n",
    "evalw(w[-1], x, y)\n",
    "evalwl(w, l, x, y)\n",
    "print(\"w = {}, l(w) = {:f}\".format(w[-1], l(w[-1])))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ändern wir den Startwert so erhalten wir zwar einen ähnlichen Wert für $l$, aber\n",
    "einen anderen Parametersatz $w$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-24T09:28:13.864850Z",
     "iopub.status.busy": "2021-09-24T09:28:13.863966Z",
     "iopub.status.idle": "2021-09-24T09:28:14.113663Z",
     "shell.execute_reply": "2021-09-24T09:28:14.113360Z"
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "w = [-0.6512 -2.828  -1.1768], l(w) = 0.056096\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAEGCAYAAAB4lx7eAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAm1ElEQVR4nO3deXjU1fXH8feZLCiCkgJuRUBlE3AjEalQK4oiClKpFgVpURFFFCmKG4iKilvBrajFlbYgtSKoiOK+VAEhVJGwi0ARF6S4ooRk7u+Pm+GXhEkyyWyZyef1PDxm+c537n3Ak5vzPfdcc84hIiKpK5DsAYiISHQUyEVEUpwCuYhIilMgFxFJcQrkIiIpLjMZb9qkSRPXsmXLZLy1iEjKys/P/9o517T815MSyFu2bMnixYuT8dYiIinLzDaE+3pMArmZrQe+B4qBIudcXizuKyIiVYvliry7c+7rGN5PREQioIedIiIpLlaB3AGvmFm+mQ0Nd4GZDTWzxWa2eMuWLTF6WxERiVUg7+qc6wT0Aoab2fHlL3DOTXHO5Tnn8po23e2hq4iI1FBMArlzbnPJf78CZgGdY3FfERGpWtSB3Mz2MrOGoY+BU4Bl0d43nPwN25j85lryN2yLx+1FRFJSLKpW9gNmmVnoftOdcy/H4L5l5G/YxsBHF1BYFCQ7M8C0IV3IbZET67cREUk5UQdy59w64MgYjKVSC9ZtpbAoSNDBzqIgC9ZtVSAXESGFyg+7HNKY7MwAGQZZmQG6HNK4zPeVdhGRuiopW/RrIrdFDtOGdGHBuq10OaRxmdW40i4iUpelTCAHH8zLB/AF67by2Tc/Ke0iInVWSgXy0kqvwjMzAmQGjOKgC5t2ERFJZykbyEs//CwuDnJO5+Yc2GjP3dIuIiLpLmUDeejh586iIFmZAfp1arZbAA+lXhTcRSSdpWwgr+zhJ+gBqIjUHSkbyGH3h5+lla87n7lkk1bnIpKWUjqQV6Z06iUjYDyTv4mi4iABM8b37ciAY5sne4giIjGRMhuCqiuUehl1SlvOzjuIomK/Oi8KOsY9t0wbh0QkbaRtIAcfzId3b0W/Ts0I+F4wAASDjgXrtiZxZCIisZPWgTwkt0UO4/t2JDNgBIDsLNWai0j6SNsceXkDjm1O2/0b6oGniKSdtF2Rq4mWiNQVabkiD1dDDqiuXETSUloG8nC9ywE11hKRtJSWgbz89v3Qg81wXxMRSXWpFci//Rb23htKlRKGU9H2/XG9O/DSss/p1fGAXV9TPxYRSXWpE8idg969oV49uO8+6NCh0svD9S4fP6eAwqIgi9b/j7b7NwSUNxeR1Jc6VSvBIPTvD0uWwJFHwsiR8M03Eb88XN68oly6iEgqSZ1AnpEBl10Gq1fDRRfB/fdD69bwyCNQXFzly8Od+ZlTP5uA+U1CypuLSKpKnUAe0qQJPPQQ5OdDu3YwdCh07gzvv1/py0r3XgmVI46fU0DQOQIBY1zvDkqriEhKSr1AHnL00fDOO/DUU/Dll9C1KwwaBJs3V/iSUO+V3BY5ZdIqzjm2bS9M4OBFRGIndQM5+OqVc86BVatgzBh4+mlo0wbuvBN27Kj0peFSLSIiqSi1A3nIXnvBrbfC8uXQowdcey107Ahz5vhqlzBCqZZzOjenX6dmCR6wiEjspEcgDzn0UJg9G+bNg8xM6NMHTjvNr9grMHPJJmZ8sJGBjy5QXxYRSUnpFchDTjkFli6FiRP9Q9COHWH0aPjuuzKXqfxQRNJBegZygKwsGDUKVq/m63794c9/ZuehrWHqVF+Tjs+TZwYMAzICtlueXB0URSQVpG8gL5H/czbd2gzkt3+cREG9X8DgwXDccbBokb8gtN2/3Lb/UAfFia+sUtpFRGq1tA/kofTJh/u34azz7uK1a++CDRugc2f2HDaURt9txQHFxWVTK0q7iEiqSPtAXrrMMDMrk5xLhvjdoaNH0+6153h9ysVctGgW9a24TGpF5YkikirMVVCeF095eXlu8eLFCXu/Cjscrl7Nt5dcxj5vvspPh7Zmz8kPQM+eFb5OnRJFJJnMLN85l7fb1+tCIK/Siy/6Jlxr18IZZ8CkSb6UsZRwpw4pmItIIlUUyNM+tRKR00+HZcv8jtA33oD27f1O0R9+2HWJcuYiUlspkIfUqwdXX+03D/XvDxMmQNu2MH06OFdhzrx0iaLKFUUkGZRaqcj778OIEb7LYrducP/95P+iZZkc+fSFGxn33DKKg46szAA4R1HQkRkwzs47iH6dmin9IiIxo9RKdR13HCxc6Pudr1oFubnk3n4dww9vtOvB57jnllEUdDj8wc47ix1BB4XFjukLte1fRBIjZoHczDLM7D9mNidW90y6jAwYUlKuOGIEPPqoP8zigQdYuOZLgqV+m8kIGFkZfpcogEO5dBFJjFiuyK8AVsTwfrVHo0Zw772+f0teHowYwfmX9ePX/11KwCAzYNzStyNPDf0V5x7bXPXnIpJQMcmRm1kzYCpwGzDKOde7sutTIkdeEefguefgT3+C9etZe3xPfrrtTg7vduSuS1RvLiLxENc6cjN7BrgdaAhcFS6Qm9lQYChA8+bNczds2BD1+ybVTz/57ooTJvjgfs01vuqlfv1kj0xE0lTcHnaaWW/gK+dcfmXXOeemOOfynHN5TZs2jfZtk2/PPWHsWFi5Evr2hZtvhsMOg2eeqfAwCxGReIhFjrwrcIaZrQdmACea2T9icN/U0Lw5zJgBb73lc+lnnw0nneQ3GKFWuCISfzGtIzezE6ggtVJaSufIK1NU5MsVx46Fb7/lq0EX0rvxSXydtVeV2/qVVxeRqqiOPBEyM2HYMF+uePHFNPnbo7z84EX0/8/LFBfurLAUsXTv83OnzGfMrI+1gheRiMU0kDvn3qpqNV4nNG4Mkyez8sW3Wdu0BbfP+wuz/zaKk/63Juzlpfu4aDORiFSXVuRx1P7UbmS8/Rbzxt1Ha/uJdmedBgMHwmeflbku1MdFm4lEpCYUyOMst+Uv6HnzCLLXrIYbboCZM30zrttvh59/9te0yGHakC7aTCQiNaKmWYn26af+UOjZs33P80mToE+fXWeG6qGniFREDztri4MPhlmz4JVXIDvb16D36uXr0fGr8+HdWymIi0jEFMiT5eST4aOPfA+XBQvg8MPhyivh22+TPTIRSTEK5MmUlQVXXOHLFQcPhnvugTZt4IknIBhM9uhEJEUokNcG++7rNxJ98AE/NGsBF1zAj52O8f3QS9EuUREJR4G8FslveijH9LqRUb1H8eMn66FLFzj/fPjiizKbhlRjLiKlKZDXIgvWbWVHMTzb4UR6XPQw+ecMhWnToE0bfr7jLtyOHTr8WUR2o0Bei5Q+4LlwrwZwxx1QUADHH0/Xh+/gpccvp/un+aoxF5EyVEdey1RYRz53Lj9fNoI9Pv2Eb3qcSqOHHoBWrZI3UBFJuLgeLFFdCuQ1VFgI990H48f7j0eN4j+DLuX9L3doA5FIHaANQekgOxtGj/bliuecA3fcwS+PPYq1kx5m4CPz9QBUpI5SIE9FBxwAU6fyzANP80WDxtwzZyLTnryKtS+/k+yRiUgSKJCnsIP79KD/hfdwTa8RtNy2md8POxOGDoUtW5I9NBFJIAXyFJbbIod/XHQczUdfzn/nf4iNHOl3hbZuDfffDzt3JnuIIpIAetiZblasgJEjfVOuDh38w9GTTkr2qEQkBvSwsw7I37CNyV9kkf/Xp3yb3O3boUcP+N3vYP36ZA9PROJEgTxNlNnC/9hC8o86HpYvh1tvhZdfhsMOgxtv9MFdRNKKAnmaKH3u564t/HvsAWPGwKpVcOaZvv68XTt4+mlIQkpNROJDgTxNlN7ev9sW/mbNYPp0ePtt+MUvoH9/6N4dli7ddYk6K4qkLj3sTCMRHRNXXOxb5o4Zg/vmGz7uM4DVl45m7LufUVgUJDszwLQhXbRLVKQW0sPOOiCiY+IyMuCSS/jwjUVM73QaHZ6fzkl9u3HWojlQXKzOiiIpSIG8jgmlUJ5e9yM39LiE0wffx8p9W3LrvAeZM3Ukv/qsQJ0VRVKMUit1SKiypbAoSGZGAJyjOOjIyjAeqb+Bo+67lYZfbfZ9XO66Cw46KNlDFpFSlFqRMpUtxcVBzs47iFGntGXaRb/i19cPo+Gna3yJ4uzZvrrlttvg55+TPWwRqYICeR1SvrKlX6dmZXPq9evDTTf53aG9esHYsdC+vQ/sKlcUqbUUyOuQ3BY5TBvSxa/Cw1Sm7CpBtH3gmWfgtdd8cD/zTOjZ0wd4Eal1lCMXoGz+vEwJ4s6d8NBDMG4c/PgjXH65T7/ss0+yhyxS5yhHLpUKuzMUICsLRoyANWvgggvg3nuhTRt4/HEIBpM6ZhHxFMgFCL8ztMxuz6ZN4a9/hcWL/VmhF14Ixx4L8+cne+gidV5msgcgtUe/Ts2wkv8C4VMtnTrBv//tt/xffTUcdxz84Q9wxx3+5CIRSTityGVXfnzGBxuZuWQTUEmqBcAMBg70zbiuuw5mzPDplrvugh07kjQLkbpLgVzCBu1Km3CFNGgAEyZAQYFvwnXNNXD44TB3buInIVKHKZBL2KBdValiGa1awfPPw0svQSAAp58OvXv7B6QiEncqPxQgws6JkSgs9OeFjh/vd4X+6U9+Y1HDhrEbrEgdVVH5oQK5xMcXX/j8+ZNPwv77+/z5wIF+xS4iNRK3OnIz28PMPjCzj8yswMxujvaekgb23x+eeAIWLIDmzX1lS9euvnxRRGIqFsujHcCJzrkjgaOAU82sSwzuK+kgVGv+xBPw6afQuTMMGQJffZXskYmkjagDufN+KPk0q+SPOizJ/wsEYPBgWL0arrwSpk715Yr33utbAIhIVGKSsDSzDDP7EPgKeNU5tzDMNUPNbLGZLd6yZUss3lZSzd57w913w8cfQ5cu/kHokUf65lwiUmMxCeTOuWLn3FFAM6CzmXUMc80U51yecy6vadOmsXhbqaWqPMi5XTtfqvj8834D0cknQ79+PvUiItUW0xIC59w3wFvAqbG8r6SO0C7Ria+sYuCjCyoO5mbQp4/fTDRhAsybB4cdBjfc4LssikjEqgzkZna4mb1X6vNOZvZGqc+bmlmjko/3BHoAK+MwVkkBlW7tD2ePPXyZ4qpV8Lvfwa23+hX7P/+pwyxEIhTJirwAONTMMko+nwhcVer7BwBvmtlSYBE+Rz4ntsOUVBHaJRowMDNy6mdH9sJmzWDaNHj3Xd9p8Zxz4IQT4KOP4jpekXRQZSB3zgXxwbyDmf0O2OicW1Lq+0udc0c7545wznV0zo2P43illsttkcO43h0ImFEcdIyfU1BxeiWcbt1g0SLfMregwHdbvPRS2FrFyl6kDos0R74A6ArcBFwft9FIWti2vZCgczgiTK+Ul5EBQ4f6Xi3Dh8OUKdC6NTz4IBQVxWXMIqmsOoH8VmCWc+6zOI5H0kBEnRMjkZPj+7Z8+CEcdZQP6rm58PbbsRyuSMqLNJCvxO/gvDOOY5E0Ua3OiZHo2BFef90fCP3NNz533r8/bNwYi+GKpLxIA/kVwHXOOdWFSURyW+QwvHur6IN4iJmvalmxAm66ydegt2sHt9wCP/0Um/cQSVGVBnIzO9TMVgJ7OuemJmhMIhWrXx9uvBFWrvQ9z8eNg/btYdYslStKnVVpIHfOfeKca+ecuzBRAxKJSIsW8PTT8MYb/qSifv38DtGCgmSPTCTh1BxaUlv37vCf/8ADD0B+vu/dMnKkz6WL1BEK5FIrVNmfpTKZmXDZZb5cccgQX+nSujU8+igUF8d+sCK1jAK5JFX+hm2MmfUx506ZX3V/lqo0aQIPP+xX5u3awUUX+X7o778f20GL1DIK5JI0oQZb0xdupLDYRd6fpSpHHw3vvAPTp/sj57p2hUGDYPPm2AxcpJZRIJekCTXYCtWaGFFuICrNDM4911e3jBnjH4y2aQN33ulb54qkEQVySZrSO0CzMwOce2zzsBuIosqfN2jgOyouXw49esC11/oNRnPmqFxR0oa5JPxjzsvLc4t1CK/gg/SCdVvpckjjsJuHQumXwqIg2ZmB6HeKvvIKXHGFX6n36gX33ANt20YxA5HEMbN851xe+a9rRS5JVdUO0Ej6m1drxX7KKbB0KUyaBO+951fno0fDd99FOxWRpFEgl1qtsgZcNa54ycry54WuXu0fgv75zz5/PnUqBINxnI1IfCiQS61WUQOumFS87LcfPP44fPABtGwJgwfDccf5fugiKUSBXGq9cOmXmFa8HHOMrzWfOhU2bIDOneGCC+DLL2MyfpF4UyCXlFRRxQtQswqXQAD+8Ad/dujo0fCPf/h0y6RJUFgYhxmIxI6qViRlla94iWmFy+rVvmfLSy/5XaL33ecflIokkapWJO2UT7lEUuECEVa5tGkDc+f6evOiIujZE/r2hU8+icdURKKiQC5pI5Ij5kKr9oirXE4/HZYtgzvu8KcUtW/vd4r+8EOcZiFSfQrkkjYiOWIu0lV7GfXqwTXX+HTL738PEyb4dMv06dodKrWCArmklao2GEV1MPSBB8Lf/+43Eu23HwwcCMcf7/uhE2UrAZEo6GGn1Dmhh6Q59bPZtr2wwvYAlSouhieegOuug61b2XLuH+mzf0++ym4Ym1YCImHoYadIidwWOXQ5pDHj5xTUvAd6RoY/xGLNGrjiChrP+DvzJg/hvMUvECzcGX0rXpFqUCCXOql8rnzmkk0RpUV2S580agT33MOKee9ScEBrxr/2V+Y8MYIeXyyP/yRESiiQS50UypUHzH/+r8X/rXJ1XlnFS4cev6Lem68z9+bJtKjnaDugL5x9tt8pKhJnCuRSJ+W2yGFc7w4EzCh2sDOCfi1VVbzktvwFp427lHqrV8L48fDii7665aabYPv2BMxK6ioFcqmztm0vJFjqYX9V/VoirnjZc0+44Qbf87xvX7j5ZjjsMHjmGZUrSlwokEudVaZfS4YxoIITikIiqVMvo3lzmDED3noL9tnHp1pOOslvMBKJIZUfSp1W1QlFMVNUBFOmwNix/hCLSy/1K/UclShK5FR+KBJGVRuIYiYz0wfvNWvg4oth8mRo3doH9+Li+L63pD0FcpFEatzYB/ElS6BDBx/UjznG7xYVqSEFcpEqxGXr/ZFH+tz5jBmwZQt06+a3/H/2WezeQ+oMBXKRSlS7W2J1mEH//r66ZexYmDkT2raF22+HHTti9z6S9hTIRSqQv2Eb9762mh07q9ktsbr22gtuuQVWrPCHV1x/vU+7vPCCyhUlIgrkImGEVuLvrf0ah/8fpSZnglYrLXPwwfDss/DKK5CdDWecAb16+RW7SCWiDuRmdpCZvWlmK8yswMyuiMXARJKp9C7OgEHX1k2q3dGwxmmZk0+Gjz6Ce+6B+fPh8MPhqqvg229rOBtJd7FYkRcBVzrnDgO6AMPNrH0M7iuSNOUPdx7Zo021SxRrdIhFSFaWPzN0zRoYPNgfAt2mjW+dGwxWaxyS/qIO5M65z51zS0o+/h5YAfwy2vuKJFN1dnFWlD4p/cMgI2Bs/uYnpi/cWL0KmH33hUcegUWL4NBD4YILoEsXWLgwmulJmonpzk4zawm8A3R0zn1X7ntDgaEAzZs3z92grnCSBkLpk8KiYJkDJUI7Rr//aSfz122l4PPvKC52OHxPl3pZNTh8wjmYNg2uvho+/9yv1G+/HfbfP06zk9om7js7zawBMBMYWT6IAzjnpjjn8pxzeU2bNo3V24okVbj0SSi4/3neKh5+Zx0fbfqWopIgDuCAwppUwJjBeefBqlVw7bU+qLdpAxMnQmFhrKcmKSQmgdzMsvBBfJpz7tlY3FMkFYTriBgK7pX9rhswq3YFzC4NG/qVeEEB/OY3/kHoEUfAyy/X7H6S8mJRtWLAY8AK59yk6IckkjrC5dJ3HVpRck3AICvDyDCfVskMGOP7doy+v0vr1r7W/MUX/QPQXr18yeLatdFOS1JM1DlyM+sGvAt8DIQep1/vnJtb0WvU/VDSXfkDnnPqZ7Ns87cY0K9Ts9g36SoshPvu8wdaFBbCqFEwZgw0aBDb95GkqihHrja2InFW0QPRuPj8c58//9vf4MAD4a67YMAAn1+XlKc2tiJJElU9eXUdcABMnQrvv+8/Pu8835BryZL4vacknQK5SJxFfERcLP3qV/DBB/Doo35TUV6eb5m7ZUv831sSTqkVkQSo7CSiWJ1SlL9hGzOXbNo9D//NNz53/sADPmc+fjwMG+YPu5CUohy5SC1U0/x5+eA/feFGbpj9McUl/ztnZwZ46qJy91q+3G/7f/VV313x/vvhxBPjMzGJC+XIRWqhmuTPyzfjmr5wI+OeW7YriEMF92rfHubNg9mzYft2fxD0WWfB+vUxnZMkngK5SBLVJH9ePvi/tOxzguV+s67wXmbQty8sX85nV41h54tzCbY7DG680Qd3SUkK5CJJVJ3mXCGlNxyZGR0O2Nt/bpBhcEr7/XZPq5ST/+VPnLRHV35zwUPMbXWsz5u3awf/+teuwyzicsSdxIVy5CIpKJROCTpHdmaAcb07sG17YcQPTCe/uZaJr6wi6Hzwv3u/b+n35F2+D/oJJ7D8mlvo9973ial9l4gpRy6SRrZtLyTo3K70yrbthQzv3iriYFs+pdPizF6Qnw8PPghLl9Lu9N9w/dzJNNz+ffxr3yVqCuQiKag6ufVwKZKwKZ2MDF+WuGYNX593AQP/8xJvPnIx5330El1aNErArKSmlFoRSVGR1J9H0x5g+cv/Zu/rrqLZhwvhqKN8ueKvfx3DGUh1KbUikmZyW+RUmU6Jpj1A+1O70WzJfHj6adi6FY4/3vdt2bQpqnHrIWrsKZCLpLGalDeWCbRmcPbZsHIljBsHs2ZB27Zw223w88+R3aPc12t0ILVUSnt0RdJYKBceaQuAClMx9euTf8FICtp1p8/fJ5Ezdiw89pg/FLpv3zLdFStL54T7DUHVMNHTilwkzYUOuwgdQ1eZilIx0xdupP9f53PT0h/5Ve4wVk+bBXvuCWeeCT17wooVVd4Dat5ATOmYymlFLpLmqvPAMxRodxYFdwXa/A3bGPfcMoqCvjCisCjIqwd0pM2HH8JDD/mUyxFHwOWXw4037naPnPrZTH5z7a7fCKrzG0J1x19XKZCLpLnqpDPCBdrJb66lOPj/1W27zhvNyoIRI+Dcc/1pRPfeC9OmkXv77Uy7oC8L1m8jp3424+cU7BaEqxOIlY6pmlIrImmuuumM8qmYLoc0pl6WbwkQ9rzRpk1hyhRYtAhatYILLyT396cyfI8tbNteGPWhGknp555iVEcuUgdUp+d5uFQGsNvrw97TOZg+Ha6+GjZvZmu/c/jtQb3ZvGcjsqJIi8SqZ3uqUz9yEYlI+T4so05py/DurcpcU2Xe+ocfYMIEmDiR4qxsFg4cxh5XjqJTm/0TPJv0og1BIhKRSFIZVW40atDAB/KCAjJO7M5xU+6mU+/jYe7cMpepGiU29LBTRMqIpLIkXHVLWK1awfPPw0sv+dOJTj+d9cf+hh9uv5sdh7RSNUqMKLUiIjVS3bz1kjVf8uqlY7n0nWnUK9rJ/DMGcdnBvfg+u36FKRwpS6kVEamWqtIekfR6KW3+pu/5a95vOfGiKTzf4QR+M/sJXn/kYn5X8AbZGVRZjRJJGqaupmqUWhGR3cRjE05O/WyCDrY0yOGq00bS8IrL6PbAeCbOmcTNm9+lwW8fhBa7LTYjHs+us0uDjnpZ8U3V1LYqGq3IRWQ30XRNrMi27YWEOrIEgLWHdGCvJYvgiSdo8NlG6NwZhgyBr76q9nhK7z51QOHO+B2GURsbfymQi8hu4rEJJ7SxKMMgO6vknoEADB4Mq1fDlVfC1KnQpg3cdx/s3LnbeALmzynNqZ9d5t4L1m0tcwB1IGC7WgOUDrSxSL3E44dctPSwU0TCikf6oMp7rlzpq1vmzYP27X1A79EDqDx1Ujr1EjBjSLeDeXL++t02NcUiXRR6r1DFTiKrbSp62KkcuYiEVd2eKDG5Z7t2vlRxzhwf0E8+mU+6ncz22+9i284GBJ1PnZTvuVK+ZLKiVXPoa4VR9GypSeOveFMgF5HaxQz69GFJmzzeGnYdl/x7BhkndIY/DuORpifwvWWHTfeU/yFRvs591RffE+r9FXTslp6pjnA/kJL5AFSBXESSoqrAN3/zj/zl2LN5ul13rn/7Cc54/H4+OHAmb144miYXDKo0WIZbNS9YtxUDHP7h4LbthTGdSzI3NymQi0jCRRL4Qg84t+zThKvOvIbPB57PoOmT6HnLSHj7WX8Y9JFHVvge5VfNOfWzyQgYwaD7/4etMbJg3VZ27AyWqZhJZCBX1YqIJFwklR+hVfU5nZuDc9z5fRPy+t7GhgmToKAAOnWCSy/1B0OHEapQmb5wI9fP+pibXigg6ByBgDGud4eYBtqc+tmEykaCRJe2qQmtyEUk4SLt1RJKiRQFHUEHO4LGnC59GL5mMNx0E0yeDP/8J9xyCwwdCpk+pJVe8QfLFeYZLqZpFfBpmoD53HvAYpu2iYRW5CKScKHV9qhT2laZTw5b056T40sTP/zQp1eGD4fcXHj7baDsir80Iz6HU5QeY3YSDr9QHbmI1HqVPhh1Dp59FkaNgo0boX9/ll4xht/P3bQrmAeAzAzj7LyD6NepWaU/OGpafVLV62JR1RK3gyXM7HGgN/CVc65jJK9RIBeRmNu+He6+G+64A8z47NI/8cLJA9i7UUO2bS+s8elIscilx+q+8ex++CRwagzuIyJSc/Xrw403+t2hp5/OLydO4JJhfRiwOZ/hJxwaUeCM1/b7eG/rjzqQO+feAf4Xg7GIiESvRQv417/g9dd9cO/XD045BZYvr/Kl8TroOd4HSMckR25mLYE5laVWzGwoMBSgefPmuRs2bIj6fUVEKlVUBA8/DDfcAN9/D5df7lftjRpV+JJ47dCs1Tnykpu3pIpAXppy5CKSUF9/DWPHwpQp0KSJP0/0/PMhIyPZI6sWnRAkInVXkyZ+ZZ6f79vkXnQRHHssvP9+skcWEwrkIlJ3HH00vPsuTJsGX3wBXbvCoEGweXNc3zbeR9BFHcjN7ClgPtDWzDaZ2YXRD0tEJE7MYMAAX91y/fXw9NN+lX7nnbBjR7VvV1WQTsSJQrGoWjnXOXeAcy7LOdfMOfdYLAYmIhJXDRrAbbf5apYePeDaa6FjR3jxxYhvEUmQTsSJQkqtiEjdduihMHs2vPyyf/jZuzecdhqsWlXlSyMJ0vEuPQQFchERr2dPWLoUJk6E996Dww+H0aPhu+8qfEkkQbo6fWVqSr1WRETK+/JLnz9//HHYbz+fPx80yB8WXU4iTwZS+aGISKT22w8eewwWLoSWLWHwYDjuOFi0aLdLc1vkMLx7q6Se3alALiJSkc6dfa35k0/C+vX+8wsv9Cv2WkSBXESkMoEA/PGPsHq1z5n//e++XPGee2DnzmSPDlAgFxGJzN57w113wccf+zTLqFFwxBHwyivJHpkCuYhItbRtC3Pnwgsv+KZcPXvCb38L69YlbUgK5CIi1WXm682XLfMHWbz2GrRvD2PGwA8/JHw4CuQiIjVVrx5cc43Pn599tu+q2K4dPPWUP4IuQRTIRUSideCB/iHov//tSxcHDIDjj/eHQyeAArmISKx07QoffOD7nq9cCbm5MGyY74ceRwrkIiKxlJHh+52vXu1PJHrkEWjdGv7yF/9wNA4UyEVE4iEnB+69Fz76yK/ML7/c90PftCnmb6VALiISTx06wKuvwsyZvtPiAQfE/C0yY35HEREpywz69fN/4kArchGRFKdALiKS4hTIRURSnAK5iEiKUyAXEUlxCuQiIilOgVxEJMUpkIuIpDhzCWy1uOtNzbYAGyK4tAkQ324ztVddnXtdnTfU3bnX1XlD9efewjnXtPwXkxLII2Vmi51zeckeRzLU1bnX1XlD3Z17XZ03xG7uSq2IiKQ4BXIRkRRX2wP5lGQPIInq6tzr6ryh7s69rs4bYjT3Wp0jFxGRqtX2FbmIiFRBgVxEJMXVikBuZqea2SozW2tm14b5vpnZ/SXfX2pmnZIxzliLYN4DS+a71MzeN7MjkzHOeKhq7qWuO8bMis3srESOL14imbeZnWBmH5pZgZm9negxxksE/973MbMXzOyjkrmfn4xxxpqZPW5mX5nZsgq+H318c84l9Q+QAXwCHAJkAx8B7ctdcxrwEmBAF2BhssedoHkfB+SUfNwrHeYd6dxLXfcGMBc4K9njTtDfeSNgOdC85PN9kz3uBM79euDOko+bAv8DspM99hjM/XigE7Csgu9HHd9qw4q8M7DWObfOOVcIzAD6lrumL/A35y0AGplZ7A++S6wq5+2ce985t63k0wVAswSPMV4i+TsHuByYCXyVyMHFUSTzHgA865zbCOCcq0tzd0BDMzOgAT6Qx+fY+QRyzr2Dn0tFoo5vtSGQ/xL4b6nPN5V8rbrXpJrqzulC/E/tdFDl3M3sl8CZwMMJHFe8RfJ33gbIMbO3zCzfzP6QsNHFVyRz/wtwGLAZ+Bi4wjkXTMzwkirq+FYbDl+2MF8rXxMZyTWpJuI5mVl3fCDvFtcRJU4kc78XuMY5V+wXaGkhknlnArnAScCewHwzW+CcWx3vwcVZJHPvCXwInAgcCrxqZu86576L89iSLer4VhsC+SbgoFKfN8P/RK7uNakmojmZ2RHAo0Av59zWBI0t3iKZex4woySINwFOM7Mi59zshIwwPiL9t/61c+5H4Eczewc4Ekj1QB7J3M8H7nA+cbzWzD4F2gEfJGaISRN1fKsNqZVFQGszO9jMsoFzgOfLXfM88IeSp7tdgG+dc58neqAxVuW8zaw58CwwKA1WZKVVOXfn3MHOuZbOuZbAM8ClKR7EIbJ/688BvzazTDOrDxwLrEjwOOMhkrlvxP8mgpntB7QF1iV0lMkRdXxL+orcOVdkZpcB8/BPth93zhWY2SUl338YX7VwGrAW2I7/yZ3SIpz3OKAx8GDJyrTIpUGXuAjnnnYimbdzboWZvQwsBYLAo865sGVrqSTCv/NbgCfN7GN8uuEa51zKt7c1s6eAE4AmZrYJuBHIgtjFN23RFxFJcbUhtSIiIlFQIBcRSXEK5CIiKU6BXEQkxSmQi4ikOAVyEZEUp0AuIpLiFMhFADN708xOLvn4VjO7P9ljEolU0nd2itQSNwLjzWxf4GjgjCSPRyRi2tkpUqLkNJ4GwAnOue+TPR6RSCm1IgKY2eHAAcAOBXFJNQrkUueVnMYyDX9Sy49m1jPJQxKpFgVyqdNKWsU+C1zpnFuB78B3U1IHJVJNypGLiKQ4rchFRFKcArmISIpTIBcRSXEK5CIiKU6BXEQkxSmQi4ikOAVyEZEU93/fWHt86iSXeQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY8AAAEICAYAAACnL3iHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAfDklEQVR4nO3deZRU1b328e+vqrp6ppuhGWxmccKBQUZRM9wYpxCJGocYo2LAaMxNsm5yY+Jaec2b916Tm0mTcB3BWdEYjZo4DxEjiDSIgMqMQCNDiwzN0FPVfv+o003TA10F3X2qTj+ftWpV1a5TVb/tER722efsMuccIiIiqQj5XYCIiGQehYeIiKRM4SEiIilTeIiISMoUHiIikrKI3wV0ll69ernBgwf7XYaISEZZuHDhp865kqbtgQ8PM5sMTB42bBhlZWV+lyMiklHMbH1L7YE/bOWce845N72oqMjvUkREAiPw4SEiIu1P4SEiIilTeIiISMoUHiIikjKFh4iIpEzhISIiKVN4tOGvC8t5dP4Gv8sQEUkrCo82PPv+JzxettHvMkRE0orCow2RkBGLx/0uQ0QkrSg82hAOGXUx/dqiiEhjCo82RMJGLK7wEBFpTOHRhnAopPAQEWlC4dGGSMioU3iIiBxE4dGGcEiHrUREmlJ4tCESMmpjOttKRKQxhUcbNPIQEWlO4dGGrHBIcx4iIk0oPNqgkYeISHMKjzYkzrbSnIeISGMZGR5mNtTMZprZkx39XRp5iIg0lzbhYWazzGybmS1r0n6Oma0ws9VmdhOAc26tc+7azqhL13mIiDSXNuEB3A+c07jBzMLADOBcYDhwuZkN78yiwqEQzkFcASIi0iBtwsM5Nwf4rEnzOGC1N9KoAWYDFyT7mWY23czKzKysoqLisOqKhA1Aow8RkUbSJjxaUQo0/jGNcqDUzHqa2Z3AKDP7aWtvds7d7Zwb45wbU1JSclgFhEP14aFJcxGRehG/C2iDtdDmnHPbge90RgGRkEYeIiJNpfvIoxwY0Oh5f+CTziygPjxi+k0PEZEG6R4eC4BjzGyImUWBy4BnU/kAM5tsZnfv2rXrsAoIhxP/iTTyEBE5IG3Cw8weA+YBx5lZuZld65yrA24EXgI+Ap5wzn2Qyuc6555zzk0vKio6rLoaRh4KDxGRBmkz5+Gcu7yV9ueB5zu5nAaaMBcRaS5tRh7pSiMPEZHmAh8eRzznobOtRESaCXx4HPmchzdhrrOtREQaBD48jpTmPEREmlN4tEFzHiIizSk82qC1rUREmgt8eBzphHn9nIdGHiIiBwQ+PI50wrxhzkMT5iIiDQIfHkeq/rCVRh4iIgcoPNqgs61ERJpTeLRBZ1uJiDQX+PBoryvMazXnISLSIPDh0V5XmGvkISJyQODD40gduM5Dcx4iIvUUHm3QnIeISHMKjzZoVV0RkeYUHm3QnIeISHMKjzZo5CEi0lzgw+PI17by5jximjAXEakX+PA44rWttKquiEgzgQ+PIxXRYSsRkWYUHm0I61RdEZFmFB5tyNJvmIuINKPwaEMoZJhBTFeYi4g0UHgkIRIyzXmIiDQS+PA40lN1ITHvoTkPEZEDAh8eR3qqLiSuMtfIQ0TkgMCHR3vQyENE5GAKjyREQkatrjAXEWmg8EiCRh4iIgdTeCRBZ1uJiBxM4ZGESDikkYeISCMKjyRo5CEicjCFRxIScx6aMBcRqafwSEI4ZFrbSkSkEYVHEiJhnW0lItJY4MOjfZYn0RXmIiKNBT482md5EqNOcx4iIg0CHx7tQXMeIiIHU3gkIaIrzEVEDqLwSEIkrDkPEZHGFB5JyAoZNXWa8xARqafwSEJedoR9NXV+lyEikjYUHkkoyA6zpzrmdxkiImlD4ZGE/GiEvdUaeYiI1FN4JKEgJ8L+2pjOuBIR8Sg8klCQHQFgr+Y9REQAhUdS8uvDQ4euREQAhUdSFB4iIgdTeCShIDsMQGWVwkNEBLpAeLTHqrr50fqRh07XFRGBLhAe7bGqbkFOIjz26LCViAjQBcKjPRRozkNE5CAKjyTk61RdEZGDKDySUD/y0GErEZEEhUcSsiMhwiFjj862EhEBFB5JMTPyo2HNeYiIeBQeSSrMydLKuiIiHoVHkvKzNfIQEamn8EhSfnZEZ1uJiHgUHkkqyI7obCsREY/CI0mlxbms3rpHv2UuIoLCI2lnDe9DZXUd89Zu97sUERHfKTySNGlYL/KjYV5cttnvUkREfKfwSFJOVpizT+zLXxduYsHHn/ldjoiIrzo8PMzsdjPL8x7PatR+qpld19Hf355+Pnk4pd1zufb+BSwtP/wl3kVEMl1S4WFmXzKzh8ws18zeNLNwku/rATjn3D4vQHaZ2RfM7NfAB8AZ3nZRM5tjZpHD7UhnKM6L8uDUcXTLzeKKe9/h/Y07/S5JRMQXyY48RgDvA1OBp5xzyV5qPRL40Hs82nt+nHPuJ865KqDKzPo452qA14BLky3cLwN65DF7+gSK8rL45sz5vLdhh98liYh0ulTCYzFwBfAMgJkNMbM3vcejzcyZWU8zC5vZMm+k0QPY6X3GWGA+sLfR5+4AunmP/+Z9ftrr3z2P2dMn0j0vyrdmvssiBYiIdDGphMdSYKhz7mOvbSdQ6D3+HvAO0B04H3jFObcPWAkM9rYZDvwMON7MzvDaSoEN3uNlJAImI5QW5zJ7+gR6FCQCZOF6BYiIdB1thoeZZZEYHYQ5MIoA2AXkmVlPoB/wNonwmA7c4W2zFBgK4Jyb5pyLO+duds69ZWYFwG7nXLX3egyoMbNCMsRRxbk8Pn0iJYXZfGvmfMp0FpaIdBHJjDyGAx8B+4Gc+kbnXP2l1tOAmUAlcAoQds6t9LZxwCP1Z1s1cRTwmyZt2UBVWwWZWb6ZPWBm95iZr4e6+hblMHv6BPp0y+Fbs97l3XUKEBEJvmTCYwSw2Dm3AwibWU6j1+LAV4Gngd3Aj4A7G7/ZOfeWdwiLJu0rnXNr6p+b2SNAMfBe4+3M7BwzW2Fmq83sJq/5QuBJ59w07/t91adbIkD6FuVw9X3vMl9XoYtIwCUbHu97j18GTm/0Wg3wgnOujkR45AN/P8xalgOPNW7wTgmeAZxLYgR0uZkNB/oDG73N0uJHNnp7AdKvKIer71vA3NWf+l2SiEiHaTM8nHP/4Zx73Hv6Z+CqRq+NdM790ns80zk3MIXTeJsaBdzXpG0csNo5t9Y7nXc2cAFQTiJADtkHM5tuZmVmVlZRUXGYZSWvd2EOs6dPZGCPPK65fwH/XLGtw79TRMQPKV1h7px7D3gj2YsEk2VmURKn6q5t8lIpB0YYkAiNUuAp4CIzuwN47hD13u2cG+OcG1NSUtKeJbeqpDCbx6ZP4OiSAqY/uJBXPtzaKd8rItKZUl6exDk36whGF619Zo1z7sEWXrKWN3d7nXPXOOeud8490p61tIce+VEemzaBE/oVcv3DC3l+qRZTFJFgSfeFEcuBAY2e9wc+8amWlBTlZfHwt8czckAxNz66iGcWb/K7JBGRdpPu4bEAOMa7mj0KXAY8m8oHmNlkM7t7167OX8iwMCeLB6aOY9yQHvzg8cU8Ubax7TeJiGSAtAkPM3sMmAccZ2blZnatdxbXjcBLJK41ecI590Eqn+uce845N72oqKj9i05CfnaE+64ex+nDevGfTy7hkfnrfalDRKQ9pc0qts65y1tpfx54vpPLaVe50TD3fGsMNzyyiJufXkZ1bZyppw/xuywRkcOWNiOPoMvJCnPnN0/l7BP78H///iF3vrmm7TeJiKSpwIeHn3MeTUUjIf78jdFMHnEUv3phOX98bZXfJYmIHJbAh4ffcx5NZYVD3HbpSC4cXcrvX1nJb19aQWIJMBGRzJE2cx5dSThk/PbiEUTDIf78xmqq62L87LwTMGvpshYRkfSj8PBJKGT899dOJhoJcc9b66iui3PL5BMJhRQgIpL+FB4+CoWMX3z1RLK9ANlfE+NXF51CWAEiImku8OFhZpOBycOGDfO7lBaZGT877wTyohFuf20Ve2vquO3SUUQjgZ+OEpEMFvi/odJtwrwlZsYPzzqWm887geeXbmHag2Xsr0mLleZFRFoU+PDIJNPOHMqtF57MnFUVXDXrXSqrav0uSUSkRQqPNHP5uIHcftkoFm3YwRX3zmfH3hq/SxIRaUbhkYa+OuIo7rryVJZvqeTSu+exbXebP+suItKpAh8e6XSFeSr+7YQ+3H/NWDbt2M/Fd85j42fNfgZeRMQ3gQ+PTJgwb81pR/fi4W+PZ9f+Wr5+5zxWb9vjd0kiIkAXCI9MN2pgd2ZPn0Bd3HHJXfNYtimzRlAiEkwKjwxwQr9uPHHdBHIiIS6/5x3KPv7M75JEpItTeGSIoSUF/OX60+hVkM2VM9/lzZUVfpckIl2YwiODlBbn8sR1ExncK59r71+g30UXEd8oPDJMSWE2j183gdGDuvP92Yu57+11fpckIl1Q4MMjU0/VPZRuOVk8OHUcXx7eh1889yG/e1m/CSIinSvw4ZHJp+oeSk5WmP+9YjSXjR3An15fzc+eXkYsrgARkc4R+FV1gywSDnHrhSfTsyDKjDfWsGNvDbddNpKcrLDfpYlIwAV+5BF0ZsaPzz6en39lOC9+sIWr79OCiiLS8RQeATH19CHcdulIyj7ewWV3v0NFZbXfJYlIgCk8AmTKqFLuvWoMayv2cvGdc9mwXethiUjHUHgEzOeP682j0xLrYV1051w++CQ4Z5mJSPpQeATQqIHdefI7E4mEjEvveoc5uhpdRNpZ4MMjiNd5JGNY70KevmES/bvnMvX+BfylbKPfJYlIgAQ+PIJ6nUcy+hbl8JfvTGTi0T358ZNLuO3VlbqYUETaReDDo6srzMli1tVjufjU/tz26ir+88kl1MbifpclIhlOFwl2AVnhEL+5+BRKi3O5/bVVbNldxf9eMZrCnCy/SxORDKWRRxdhZvzwrGP5n4tOYe6a7Vxy1zts1W+ji8hhUnh0MZeMHcCsq8eyYftevjbjbVZurfS7JBHJQAqPLuhzx5bw+HUTqYs7LrpjLnNXf+p3SSKSYRQeXdRJpUU8/d1J9CvK4Vuz3uWxdzf4XZKIZBCFRxdWWpzLk9efxqRhvfjpU0v5f3//UMu6i0hSFB5dXLecLGZeNYarTxvMvf9ax/QHy9hTXed3WSKS5gIfHl31CvNURMIhbvnqifxyykn8c2UFF98xl/IdWlRRRFoX+PDoyleYp+rKCYO4/5qxbNq5nykz3mbh+h1+lyQiaSrw4SGpOeOYEp6+YRL52REuv+cd/vbeJr9LEpE0pPCQZob1LuBvN0xi1IBifvD4Yn7z0nLimkgXkUYUHtKi7vlRHrp2PJeNHcCMN9bw7QfL2LVfP28rIgkKD2lVNBLi1gtP5pdTTmLOygqmzHib1dt0RbqIKDykDWbGlRMG8ei0CVRW1TJlxlxe/mCL32WJiM8UHpKUcUN68OyNpzO0JJ/pDy3kD6+s1DyISBem8JCkHVWcyxPXTeSi0f25/bVVTH9oIZVVmgcR6YoUHpKSnKwwv/36KdwyeThvrNjGBX9+m+Vbdvtdloh0MoWHpMzMuHrSEB759ngqq+uYMuNtnlxY7ndZItKJFB5y2CYM7ck//v10Rg4o5kd/eZ+fPLmEqtqY32WJSCdQeMgR6V2Yw8PXjufGLwzj8bKNTJnxNus+3et3WSLSwRQecsQi4RA/Ovs47rtmLFt2VzH5T//i+aWb/S5LRDpQ4MNDq+p2ni8c15t//PsZDOtdwA2PLOKWZz+guk6HsUSCKPDhoVV1O1epdzrv1ElDuH/ux0yZMVdXpYsEUODDQzpfNBLi55OHM+vqMWzdXcVX/vQvHp2/Aed0UaFIUCg8pMN88fg+vPj9Mxg7uAc/e3op1z+8iJ37avwuS0TagcJDOlTvbjk8cM04bj7vBF5bvpVzbnuLeWu2+12WiBwhhYd0uFDImHbmUJ66fhK50TDfuPcd/usfH+qaEJEMpvCQTnNy/yL+/r3T+eb4Qdzz1jrO/+NbvL9xp99lichhUHhIp8rPjvDLKSfx0LXj2FcT48I75vK7l1dQUxf3uzQRSYHCQ3xxxjElvPiDM5kyspQ/vb6aKTO0wKJIJlF4iG+KcrP43SUjuPvKU9lWmbgy/fcvr9CFhSIZQOEhvvvyiX15+Yef4/yT+/HH11dz7u1v8e66z/wuS0QOQeEhaaFHfpTbLhvFA1PHUVMX55K75vHTp5aya79+bEokHSk8JK187tgSXv7hmUw7YwiPL9jAWb9/kxeWbtbV6SJpRuEhaScvGuHm84fzzHdPp6Qwm+sfWcRV9y1gTcUev0sTEY/CQ9LWyf2LeOa7k/j5V4bz3vodnHPbHG594SP2VNf5XZpIl6fwkLQWCYeYevoQXv/R55kyspS73lzLF3/7T/723iYdyhLxkcJDMkJJYTa/+foInr7hNPoW5fCDxxdzyV3zdIW6iE8UHpJRRg3szt9umMSvLjyZtRV7uWDG29z46CLWb9dP34p0JusqQ/8xY8a4srIyv8uQdlRZVcvdc9Zy71vrqIvH+eaEQXzvi8fQIz/qd2kigWFmC51zY5q1Kzwk023bXcUfXl3F4ws2kB+N8J3PH801kwaTF434XZpIxlN4KDwCb/W2Sn794gpe+XArPfOjXPe5oXxzwiCFiMgRCFR4mNlQ4GagyDl3cTLvUXh0HQvXf8Ztr67irVWf0jM/yvQzh3LlRIWIyOFoLTw6fcLczGaZ2TYzW9ak/RwzW2Fmq83spkN9hnNurXPu2o6tVDLVqYN68NC14/nr9adxYmkRt76wnDN+/QZ3vrlG14iItJNOH3mY2ZnAHuBB59xJXlsYWAmcBZQDC4DLgTBwa5OPmOqc2+a970mNPKQtC9fv4PbXVjFnZQWFORGuGD+Iq08bTN+iHL9LE0l7rY08On0c75ybY2aDmzSPA1Y759YCmNls4ALn3K3AVzq5RAmYUwd158Gp41i8cSf3zFnL3XPWMPNfa5k84iimnTGUE/p187tEkYyTLtd5lAIbGz0v99paZGY9zexOYJSZ/fQQ2003szIzK6uoqGi/aiUjjRxQzIwrRvPmj7/AFeMH8cLSLZx7+1tcOXM+ry/fSiyeefN/In7xZcLcG3n8vdFhq68DZzvnvu09vxIY55z7Xnt9pw5bSVM799XwyPwN3D/3YyoqqyktzuUb4wdyyZgBlBRm+12eSFpIm8NWrSgHBjR63h/4xKdapIsozovy3S8MY9oZQ3nlw608Mn89v3lpBX94ZSVnn9SXK8YPZOLQnpiZ36WKpJ10CY8FwDFmNgTYBFwGfKM9PtjMJgOThw0b1h4fJwEUjYQ4/5R+nH9KP9ZU7OGx+Rv4y8Jy/rFkM4N65jFlZClfG1XK4F75fpcqkjb8ONvqMeDzQC9gK/B/nHMzzew84DYSZ1jNcs79V3t+rw5bSSqqamM8v3Qzf11Uztw123EORg8s5muj+zP5lH4U52kJFOkaAnWR4OFQeMjh2rxrP88s/oSnF21ixdZKssLGmceUcM5JfTlreB8FiQSawkPhIUfIOceHm3fz9KJNvLBsC5t27icSMiYe3bMhSHoX6toRCZYuGx6N5jymrVq1yu9yJCCccyzdtIsXlm3hxWVbWPfpXsxg9MDufP7YEj53XAknHVVEKKTJdslsXTY86mnkIR3FOcfKrXt4YdlmXl++jSXluwDoVRDlzGMSQXLGMSVaKl4yksJD4SGd5NM91by1qoJ/rqhgzsoKduyrBeD4voWMH9KDCUN7Mm5ID3oW6FoSSX8KD4WH+CAWdywp38ncNdt5Z+12yj7ewf7aGADH9ilg3JAejBzQnZEDihnaK1+HuSTtKDwUHpIGamNxlpTvYv667byz9jMWrd/RsNJvYU6EEf2LGTkgcTtlQJEm4MV3XTY8NGEu6SwWd6yp2MPijTsTtw07WbG1smGdrV4FUY7v240T+hVyfN9uHN+vkGG9C8iOhH2uXLqKLhse9TTykEyxvybGsk92saR8F8s372b5lkpWbq2kui4OQDhkDO2Vz9ElBQwpyWdIr3yOLslnSK8CuudlaTkVaVfpvraViHhyo2HGDu7B2ME9GtrqYnE+3r6P5Vt289Hm3azYsoeV2yp59aOt1DVaDbgoN4uhJfkM6ZlP/x559C/OpbR7LqXFufQrztGIRdqNwkMkA0TCIYb1LmBY7wK+cspRDe11sTjlO/az9tM9rK3Yy7pPE7d5a7ezZfEmGh9YMIOSguyGMOnbLYeSwmx6d8umd2EOvQsT991yIxq9SJsUHiIZLBIOMbhXPoN75fPF4w9+raYuzpZdVZTv3MemHfvZtHN/w/3STbt49aOtVNXGm31mNBKipCARKj3zoxTnRemel+XdN3qcn0X3vCjFeVka0XRBCg+RgIpGQgzsmcfAnnktvu6cY091Hdsqq6morGZbZTXbdldRsaeait2J55/srOLDT3azY19twynGLcnJClGQnUVhToSCbO+WE6Gw/j4nQkF2VkNbXjRMbjRMblaYHO+WGw2TEwl592GdtpzmAh8eWpJdpGVmRmFOFoU5WRxdUtDm9lW1MXbsq2HH3lp27qthx75aduyrYee+GnZX1VFZVcee6jr2VNWyp7qOjZ/tSzyvTryW6i81ZkdCiVDxgqX+eTQSIhoOkRU2ssKhRs9DZEVaaPO2zY4ceB4JGyEzIiEjFDr4PmxGOHTgFgmFCIUgEgoRDkE4FEpsE26+bcggZInPNkscKjQOtCfaghGKOttKRDqcc46q2jiV1bXsqapjb3WMqroYVbUx9tfE2F+beFxVG2e/11ZVF6PKe21/bdx7PUZtLE5tzFFTF6c2FqcmlrivrXOJx3WJtppYnHT9680aQiYRLvVB0xA83jZmzYPHaPReO/i9ifcl7jEant9/zVj6d295BNp2rTrbSkR8YmaJw1TRML0LO+97Y/FEyDQETCxOTV2cWNwlbs5RF3PEnaMu7ojHD76v364ufohtnCMWi1MXdzgHDkfcQdx5z737hjYatyW2dRzYttl7abSda/u9DZ8PiSckDmG2N4WHiARWOOSFFprQb2/tH0ciIhJ4Cg8REUlZ4MPDzCab2d27du3yuxQRkcAIfHg4555zzk0vKiryuxQRkcAIfHiIiEj7U3iIiEjKFB4iIpIyhYeIiKSsyyxPYmYVwPrDfHsv4NN2LMdPQelLUPoB6ku6CkpfjrQfg5xzJU0bu0x4HAkzK2tpbZdMFJS+BKUfoL6kq6D0paP6ocNWIiKSMoWHiIikTOGRnLv9LqAdBaUvQekHqC/pKih96ZB+aM5DRERSppGHiIikTOEhIiIpU3gcgpmdY2YrzGy1md3kdz2pMrOPzWypmS02szKvrYeZvWJmq7z77n7X2RIzm2Vm28xsWaO2Vms3s596+2mFmZ3tT9Uta6Uvt5jZJm/fLDaz8xq9lpZ9MbMBZvaGmX1kZh+Y2fe99ozbL4foSybulxwze9fM3vf68guvvWP3S+InDXVregPCwBpgKBAF3geG+11Xin34GOjVpO1/gJu8xzcBv/a7zlZqPxMYDSxrq3ZguLd/soEh3n4L+92HNvpyC/CjFrZN274A/YDR3uNCYKVXb8btl0P0JRP3iwEF3uMsYD4woaP3i0YerRsHrHbOrXXO1QCzgQt8rqk9XAA84D1+AJjiXymtc87NAT5r0txa7RcAs51z1c65dcBqEvsvLbTSl9akbV+cc5udc4u8x5XAR0ApGbhfDtGX1qRzX5xzbo/3NMu7OTp4vyg8WlcKbGz0vJxD/8+VjhzwspktNLPpXlsf59xmSPwBAnr7Vl3qWqs9U/fVjWa2xDusVX9IISP6YmaDgVEk/pWb0fulSV8gA/eLmYXNbDGwDXjFOdfh+0Xh0TproS3Tzmue5JwbDZwLfNfMzvS7oA6SifvqDuBoYCSwGfid1572fTGzAuCvwA+cc7sPtWkLbenel4zcL865mHNuJNAfGGdmJx1i83bpi8KjdeXAgEbP+wOf+FTLYXHOfeLdbwOeJjE03Wpm/QC8+23+VZiy1mrPuH3lnNvq/YGPA/dw4LBBWvfFzLJI/GX7iHPuKa85I/dLS33J1P1Szzm3E/gncA4dvF8UHq1bABxjZkPMLApcBjzrc01JM7N8Myusfwx8GVhGog9XeZtdBTzjT4WHpbXanwUuM7NsMxsCHAO860N9Sav/Q+35Gol9A2ncFzMzYCbwkXPu941eyrj90lpfMnS/lJhZsfc4F/gSsJyO3i9+nymQzjfgPBJnYawBbva7nhRrH0rijIr3gQ/q6wd6Aq8Bq7z7Hn7X2kr9j5E4bFBL4l9K1x6qduBmbz+tAM71u/4k+vIQsBRY4v1h7pfufQFOJ3F4Ywmw2Ludl4n75RB9ycT9cgrwnlfzMuDnXnuH7hctTyIiIinTYSsREUmZwkNERFKm8BARkZQpPEREJGUKDxERSZnCQ0REUqbwEBGRlCk8RHxiZl8ys4f8rkPkcCg8RPwzgsSVwSIZR+Eh4p8RwHveGkP3m9l/e2suiaS9iN8FiHRhI0isdPoScK9z7mGf6xFJmta2EvGBtxz4p8B64Drn3DyfSxJJiQ5bifhjOIll/+uAmM+1iKRM4SHijxHAXBK/E3OfmfXxuR6RlCg8RPwxAljmnFsJ/AR4wjuUJZIRNOchIiIp08hDRERSpvAQEZGUKTxERCRlCg8REUmZwkNERFKm8BARkZQpPEREJGX/Hx1YU/qRTi0QAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "w0 = np.ones(npar)\n",
    "w = GD(w0, l1, gamma = 0.1, nit = 300)\n",
    "evalw(w[-1], x, y)\n",
    "evalwl(w, l, x, y)\n",
    "print(\"w = {}, l(w) = {:f}\".format(w[-1], l(w[-1])))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "GD zeigt dabei unverändertes Konvergenzverhalten."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Newton"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Die Iterationsvorschrift für das einfache Newton-Verfahren lautet\n",
    "\\begin{equation*} \n",
    "w^{(k+1)} = w^{(k)} -  l''\\big( w^{(k)} \\big)^{-1} l'\\big( w^{(k)} \\big).\n",
    "\\end{equation*}\n",
    "In unserem Beispiel ist \n",
    "\\begin{equation*} \n",
    "l''(w) = \\frac{1}{m}X^T X, \n",
    "\\quad\n",
    "X = \n",
    "\\begin{pmatrix}\n",
    "1 & x_1 - 1 & x_1 + 1\\\\\n",
    "\\vdots & \\vdots & \\vdots\\\\\n",
    "1 & x_m - 1 & x_m + 1\n",
    "\\end{pmatrix}.\n",
    "\\end{equation*}\n",
    "Da die Spalten von $X$ linear abhängig sind, hat $X$ keinen vollen Rang so dass die Hessematrix $l''$ singulär (nur positiv semidefinit)\n",
    "ist. Somit ist das einfache Newton-Verfahren nicht anwendbar."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Singuläre Hesse-Matrizen treten bei Optimierungsproblemen häufig auf, weshalb es eine ganze Reihe von modifizierten Newton-Varianten gibt (gedämpfte Newton-Verfahren, Quasi-Newton, Gauß-Newton, Trust-Region, siehe auch [scipy.optimize.minimize](https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.minimize.html)), die \n",
    "dieses Problem auf unterschiedliche Art und Weise umschiffen.\n",
    "\n",
    "Wir wenden BFGS an, ein Quasi-Newton-Verfahren dass adaptiv\n",
    "spd-Näherungen $B^{(k)}$ der Hesse-Matrix benutzt. Für\n",
    "$w^{(0)}, B^{(0)}$ gegeben berechnet man\n",
    "\\begin{align*}\n",
    "B^{(k)} p^{(k)} \n",
    "&= -l'(w^{(k)}) \\\\\n",
    "w^{(k+1)} \n",
    "&= w^{(k)} + \\gamma^{(k)} p^{(k)} \\\\\n",
    "B^{(k+1)} \n",
    "&= B^{(k)} + \\frac{ y^{(k)} {y^{(k)}}^T }{ {h^{(k)}}^T y^{(k)} } \n",
    "- \\frac{ u^{(k)}{u^{(k)}}^T }{ {h^{(k)}}^T u^{(k)} }\n",
    "\\end{align*}\n",
    "mit\n",
    "\\begin{equation*} \n",
    "h^{(k)} = w^{(k+1)} - w^{(k)},\n",
    "\\quad\n",
    "y^{(k)}= l'(w^{(k+1)}) - l'(w^{(k)}),\n",
    "\\quad\n",
    "u^{(k)} = B^{(k)} h^{(k)}.\n",
    "\\end{equation*}\n",
    "$\\gamma^{(k)}$ wird dabei durch Liniensuche bestimmt."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wir benutzen die BFGS Implementierung von [SciPy](https://www.scipy.org/) und erhalten für zwei unterschiedliche Startwerte folgende Ergebnisse."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-24T09:28:14.116744Z",
     "iopub.status.busy": "2021-09-24T09:28:14.116361Z",
     "iopub.status.idle": "2021-09-24T09:28:14.261938Z",
     "shell.execute_reply": "2021-09-24T09:28:14.261637Z"
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Optimization terminated successfully.\n",
      "         Current function value: 0.055920\n",
      "         Iterations: 8\n",
      "         Function evaluations: 9\n",
      "         Gradient evaluations: 9\n",
      "w = [-1.0151 -2.532  -1.5169], l(w) = 0.055920\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAEGCAYAAAB4lx7eAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAmd0lEQVR4nO3deXhURdbH8W91FgVEzbAoigmCgGwKJAZUdBQ3UBwQdxl3xAURl1FGFFBcQAVRHBVZFB0BRwFFGRdcUNyCEAUEWUQUROcdEOMKY0i63j8qHULSSTrp7nTf7t/neXjMcvt2lcBJce45VcZai4iIeJcv1gMQEZHwKJCLiHicArmIiMcpkIuIeJwCuYiIx6XG4k0bN25sW7RoEYu3FhHxrPz8/B+stU3Kfz0mgbxFixYsXbo0Fm8tIuJZxpiNwb4ekUBujPkG+BUoBoqstTmRuK+IiFQvkivy4621P0TwfiIiEgI97BQR8bhIBXILLDDG5BtjBgW7wBgzyBiz1BizdOvWrRF6WxERiVQgP9pa2xXoDQw2xhxb/gJr7WRrbY61NqdJkwoPXUVEpJYiEsittd+X/HcL8CKQG4n7iohI9cIO5MaYBsaYhoGPgZOBleHeN5j8jQU8unA9+RsLonF7ERFPikTVyn7Ai8aYwP1mWmtfj8B9d5O/sYABU/MoLPKTnupjxsDuZGdlRPptREQ8J+xAbq3dABwegbFUKW/DNgqL/Pgt7Czyk7dhmwK5iAgeKj/s3rIR6ak+Ugykpfro3rLRbt9X2kVEklVMWvRrIzsrgxkDu5O3YRvdWzbabTWutIuIJDPPBHJwwbx8AM/bsI3vftqhtIuIJC1PBfKyyq7CU1N8pPoMxX4bNO0iIpLIPBvIyz78LC72c15uJgfsW69C2kVEJNF5NpAHHn7uLPKTluqjf9fmFQJ4IPWi4C4iicyzgbyqh5+gB6Aikjw8G8ih4sPPssrXnc/5dLNW5yKSkDwdyKtSNvWS4jPMzt9MUbEfnzGM7tuRC7plxnqIIiIR4ZmGoJoKpF5uPLktZ+ccRFGxW50X+S0j561U45CIJIyEDeTggvng4w+hf9fm+NxeMAD4/Za8DdtiODIRkchJ6EAekJ2Vwei+HUn1GXxAeppqzUUkcSRsjry8C7pl0nb/hnrgKSIJJ2FX5NpES0SSRUKuyIPVkAOqKxeRhJSQgTzY3uWANtYSkYTkrUD+yy/QsCGUqUAJpnz7fuDBZrCviYh4nXcCubXQpw/suSc8/DC0a1fppZW174/s04HXVv6H3h2blX5N+7GIiNd5J5D7/XD22TByJBx2GFx3HYwaBXvvHfTyYHuXj56/isIiP0u++ZG2+zcElDcXEe/zTtVKSgoMGQLr1sEll8CECdCmDUyf7oJ8NYLlzSvLpYuIeIl3AnlAkyYwZQp88gkcfDBceikcdRQsWVLly4Kd+ZlRPx2fcU1CypuLiFd5L5AH5OTAhx+6Ffk330C3bjBwIGzZEvTysnuvBMoRR89fhd9afD7DyD4dlFYREU/ybiAH8Png4otduuXGG+Hpp126ZeJEKCqqcHlg75XsrIzd0irWWgq2F8ZgAiIi4fN2IA/Ye28YNw5WrIDcXBg6FDp3hoULK31JsFSLiIgXJUYgD2jXDt54A158EX7/HXr2hHPOgU2bKlwaSLWcl5tJ/67NYzBYEZHISKxADq5ZqF8/+OILGD0a5s+HQw+Fu+6C//2vwuVzPt3Mc59sYsDUPO3LIiKelHiBPKBePRgxAlavdo1EI0dC+/bw0kuuuYjKW/lFRLwkcQN5QFYW+Q88wUvjnmFH2h5wxhnQqxesWUP3lo1I9RkMkOIzFfLk2kFRRLzAO52dtbRrJ8Q/Ue+sB3gtbSWZD42FTp048NIrabDP8fyUsmeF/VuC7aCo8kQRiUcJvyIvmz75n9/wyp/PKu0O3W/qYyx4fCD9P38bf1HRbqkVpV1ExCsSPpAHLTNs2hSmTGHNS2/y3b77Mf7VCcx+9hZ6/rap6teJiMQhY0se/NWlnJwcu3Tp0jp7v6p2OMz/ehsFT0zjz9PGk7ZtK1x+Odx7LzRpUuF12ilRRGLJGJNvrc2p8PVkCOQh+flnV6L48MPQoIErXbzmGkh1jxGUMxeRWKsskCd8aiVk++xTsTu0S5fS7lDlzEUkXimQlxfoDp07F377rbQ79Jg9dgTNmZctUVS5oojEglIrVdmxAx54AMaMAWP47pobeOWkCzji0APIzspg5uJNjJy3kmK/JS3VB9ZS5Lek+gxn5xxE/67NlX4RkYhRjjwcGzfCTTfBnDluD/QJE8g//BjOnZxHkX/X/z8D2DIf75GmXLqIRE7Uc+TGmBRjzGfGmPmRumfcyMqC2bPhrbdc63+/fjQ+px9ZP3xbekmKz5CW4rpEwQV05dJFpC5EMkc+FFgdwfvFnxNOgGXLYMIEmq9ZxuvTBjN84ZPsu3MHd/XtyKxBR3J+t0zVn4tInYpIasUY0xx4GrgHuNFa26eq6z2XWglmyxZ+uO4mGj0/g6LGTUkb/wAMGAA+n+rNRSQqop1aeQi4Baj0FGRjzCBjzFJjzNKtW7dG6G1jqGlTGj/3T0xeHmkHZ8FFF0GPHpCfv9tJRCIi0RZ2IDfG9AG2WGvzq7rOWjvZWptjrc1p0qRJuG8bP3Jz4eOP4amn4Kuv4IgjYNAgSIQfViLiCZFYkR8N/MUY8w3wHNDTGPNsBO7rHT4fXHKJ24zr+utdUG/TBh55hPyvtqq2XESiKuxAbq291Vrb3FrbAjgPeMda+9ewR+ZF++wDDz4Iy5dDTg5cdx17HZXLB1NeqPYEIjUTiUhtJfx+5DHRvj0sWMCrdz1OpwdHM2vWcP596DGs7HgP2VnHVLi87D4uaiYSkZqKaIu+tfbd6ipWkoYx7Hfx+fS5ahITegzghPWL+eulveCeeyqcHVp2H5fCYsvMxTpDVERCp71Woig7K4Mnr/4z6aPv4Mt3l5Byam+4/Xbo0AFefrn07NDA3udqJhKR2lAgj7JAKWKnow/b1R26557Qty+ceiqsXUt2VgYzBnZXM5GI1Ir2WomFnTvh0Udh1Ci3Mdf118OIEdCwoZqJRKRS2jQrHv33v3Drra5ccf/94f77S7tDRUTK08ES8Wi//eDJJ2HxYjjooF3doZ9+GuuRiYiHKJDHg9xcyMuDadNg/XpXg37llfDDD7EemYh4gAJ5vPD54LLLWPbWJyzrfzF22jTXHfroo1BUBKhpSESCU0NQHMnfWMCAF1ZT2Pos2g3szqyVs9j72mth8mTWDr+HAStTdPiziFSgFXkcKdsYtCajOf8cPcWVLP70E23PO50H5oyl6c8/qMZcRHajQB5HAo1BpXXkrRrDmWfC6tV8P/QWTvoyj3emXsm1eS9w5AENYj1cEYkTKj+MM1XVkX/+wXLqD7+FVu8vgJYt4aGHyO90NHlf/6i6c5EkoDryRPLmmzB0KKxezaJW2dx5wiC+a3qQ8uYiCU515InkpJNg+XI+uPpWumxezWtTB3PDm9PIX7kx1iMTkRhQIPeqtDTqDbuZXldP4eUOx3Hl4jlccllvePbZ0s24RCQ5KJB7WHZWBhOv781/H36c1S8uID3rILjwQnWHiiQZBXKPC+yu2K7fSa47dOpU+PJL1x161VWwTWWKIolOgTyB5H/7M4+2/DPL3voErrvOBfXWreGxx0q7Q0Uk8SiQJ4jAcXHjF6zlvBdWk3/DKFi2DDp3hsGD3Qp90aJYD1NEokCBPEGU7Qot7fzs2BHefhteeAEKCuDPf4YLLoDNm2M9XBGJIAXyBFGhKzRwupAxcNZZsHo1jBwJc+fCoYfCmDHwxx+xHbSIRIQaghJISKcLff013HQTvPgitGoFDz0EffroZCIRD1Bnp+zuzTfZcfVg6n31Jd8ddTyXdb6AL/dupp0VReKYOjtlN/ltcsg9Zxz3HH8Z+yzN45VJV3Pzwumkbf9dOyuKeIz2I08ygRTKdz/t4HebwpTc/sxrfzzDFk3n6sWz6f/FQn7PGgPHtXL5dRGJe1qRJ5GyJYqz8zeT6jOkGPgloxF/THmS2f94nr0OzqTl0EFwzDHw2WexHrKIhEAr8iRStkSxuNjPebmZHLBvvV0POLtlwtVnwlNPwa23utrzQYPgrrugceNYD19EKqEVeRIpX6LYv2tzBh9/yO4PNn0+uPxyWLcOrr0WpkxxZ4eqO1QkbqlqJclUVWYY9HsrV7p2/4UL4fDDYeJEOPbYGIxcRFR+KFUK5M+DHu5srTs79Kab4Ntv4fzz4f77oXnz2A5aJMmo/FCqFLTFP8AYOPtsWLMGRoxQd6hInFEgFyB4i3/+xgIeXbie/I0F7qL69WH0aNfuf9JJMHy428/l3/+O7eBFkpyqVqRU/67NMSX/BSpPtRx8sGvxX7DA5c/79IHTToMJE9y2uSJSp7Qil9L8+HOfbGLOp25nxCpTLQEnnwwrVsC4cW6L3I4dXdnib7/V8QxEkpsCuQQN2pXuplheerp7CLp2rXsIOnYstG0LM2fq7FCROqJALkGDdnZWBjMGdufGk9uGtolWs2YwfTp89JH7eMAAV6a4bFldTEEkqan8UIAQt8ANVXHxru7QH3+EK6903aGNKlnVi0hIolZ+aIzZ0xjziTFmuTFmlTHmznDvKXUvcIhzRLavTUmBgQNdd+jgwTB5susOffxxF+RFJKIikVr5A+hprT0c6Az0MsZ0j8B9xesyMlwn6GefwWGHwTXXQHY2vP9+rEcmklDCDuTWCZQppJX80lMu2aVTJ3jnHXj+eZdqOfZYl0P/7rtYj0wkIUTkYacxJsUYswzYArxprV0c5JpBxpilxpilW7dujcTbipcEukNXr3bdoXPmuOqWsWPVHSoSpogEcmttsbW2M9AcyDXGdAxyzWRrbY61NqdJkyaReFvxogYNXHfoF1/AiSe6B6KdOsGrr8Z6ZCKeFdHyQ2vtT8C7QK9I3le8pUJrfzAtW8JLL8Frr7mtc087zXWIrl9fZ+MUSRSRqFppYozZt+TjesCJwJpw7yveVPYUogFT86oO5gC9ernu0AcegPfegw4d3B4u6g4VCVm1gdwY08kY82GZz7saY94pc0kzYKExZgWwBJcjnx/5oYoXhNTaX156Ovztb65c8bzz3K6Khx4Ks2apO1QkBKGsyFcBrYwxKSWfjwf+FvimtXaFtbaLtfYwa21Ha+3oaAxUvCHQJeozYIwho3566C9u1gyefho+/BD22w8uuACOOw6WL4/aeEUSQbWB3FrrxwXzDsaYM4FN1tpPoz4y8aTsrAxG9umAzxiK/ZbR81dVn14p76ij4JNPXCPRF19A166usejHH6MzaBGPCzVHngccDdwBDI/aaCQhFGwvxG8tlhqkV8pLSYErrtjVHTppkusOfeIJdYeKlFOTQH438KK1Vl0cUqWQd04MRaA7dNkyt03uVVfBEUe49IuIAKEH8jW4Vvz7ojgWSRA13jkxFJ06uQOgn3sOtm6FHj3gwgvh++/Dv7eIx4UayIcCt1prf4/mYCRxRHQTrgBj4Nxz3dmht93mWv7btnUHQas7VJJYlYHcGNPKGLMGqGetfbqOxiRStQYN4O673YPQnj1h2DC3Yn/ttViPTCQmqgzk1tqvrLWHWmsvr6sBiYSsVSuYN88FcGPg1FPh9NPVHSpJRycESVwIqa2/Mr16weefu+7Qd99Vd6gkHQVyibkat/UHU7Y79Nxz1R0qSUWBXGIqf2MBD721jj921rCtvzLNmsEzz8AHH0DTpuoOlaSgQC4xE1iJf7j+ByzuD2PYdecBRx8NS5a4BqJVq9QdKglNgVxipuwGWz4DR7duHLm6c3DdoYMGuXTL1VerO1QSlgK5xEzZDtD0VB/Xn9gmaBAP60EowJ/+BP/4hzs7tEMH1x2amwsffRTmDETig7ExeBCUk5Njly5dWufvK/Enf2MBeRu20b1lo0qD+ICpeRQW+UlP9QVdsVd3j91YC//6l3sw+t138Ne/wn33wQEHRHJaIlFhjMm31uaU/3pqLAYjEpCdlVFl8A22v3ng+vyNBcz9dDMvLP2WIr+tNNDvxhi35/npp8O998K4ce6kopEjYehQV/0i4jFKrUhcq2wDrsBKfebiTRQW25pXvDRoAPfc47pDjz8ebrnFdYe+/noUZyMSHQrkEtcq24ArsFIPJAYNtax4adUKXn7ZHf5sLfTuDX37wldfRXQeItGkQC5xL9gGXOUflJ7fLZMZA7sD1O7BaO/erjt07Fh4+233UPT22+F37RMn8U8PO8Wzyj/kDOXBaEi+/96lWmbMgObNXR79nHNcfl0khip72KkVuXhW+ZV6qAc/V1vOeMAB8Oyz8P770Lixezjas6dbsYvEIQVySRihnExUo31devSApUtdI9GKFdC5MwwZou5QiTsK5JIwQjmZKNRVe6mUFLjySvjyS9dI9Nhjrjt08mR1h0rcUCCXhFLdyUS1Pk/0T3+CRx+FTz+F9u1dcFd3qMQJPeyUpBN4SJpRP52C7YWhdYSWVb479MIL4b77yC/cM/QOU5FaqOxhpwK5JKWIVLj89pvb93zcOIrT0xnf7Vymde2D2WOPyG7+JVJCVSsiZdQ4V15it4qXvfZy3aGrVrGp4xHc8vY0Xp02hO7rloS3p7pIDSmQS1IqnyvPqJ9ebSNRpRUvhxzCj8/NYdB5d+KzfqY/P4oBY4bChg11NBtJdgrkkpQCFS7n5WZyTOsm3PHKqmpLEqtaxWdnZXDl2CG8PmsBm4eNYt+P3nMPRUeMgO3b62pakqQUyCWpzfl0M2998d+Q0izVVbxkZ2Vw9SkdaD72DneYxVlnwd13u7NDn39eZ4dK1CiQS9Kq6cZbodSplyrbHdqokTsQ+oQTYOXKiM9DRIFcktZuG2+lGC4o2XirqgBdXZ16BYHu0McfdwdAd+4M110HBbU87UgkCJUfSlKr0elC4dq2zR1gMWmSazAaMwYuvdR1j4qEQOWHIkHUeIUdjkaNXHdofj60awdXXAHdusHHH0f/vSWhKZCLVCPsw5/L69wZ3nsPZs6E//s/OOoouPhi97FILSiQi1ShRrsl1oQxcP75sGYN3HorPPec24xr3DgoLIzMe0jSUCAXqUJtO0BDttde7hDolSvh2GPh5pvhsMPgjTci+z6S0BTIRSqRv7GA73/aQarP1Hy3xDL3CCkt07o1zJ/vfhUXQ69e0K+fukMlJKnh3sAYcxDwDLA/4AcmW2sfDve+IrFUdlOt1BQf5+YexJldm9fooWitNuY67TQ48USYMME1E7Vv71bpt94K9euHOStJVJFYkRcBN1lr2wHdgcHGmPYRuK9IzJRNqRQX+zlw33o1rmypdVpmjz3g73+HtWvhzDN3dYe+8IK6QyWosAO5tfY/1tpPSz7+FVgNHBjufUViqSYHUFSWPil7jxSf4fufdjBz8abQK2AOPNAdAL1okas7P+ccdYdKUBFtCDLGtAAWAR2ttb+U+94gYBBAZmZm9saNGyP2viLREEqzUGXpk8Brf92xk483bGPVf36huNhicVsB7JFWwz3Qi4vd8XK33Qa//AKDB8Odd8K++0ZquuIBlTUEhZ0jL/MGewFzgOvLB3EAa+1kYDK4zs5Iva9ItGRnZVQbaCtLnwyYmscfO3ft41KWBQpLrg05kKekwNVXu1X5iBHwj3+4OvQxY+Cyy8CnuoVkFpHffWNMGi6Iz7DWzo3EPUW8IFgKpvxmXMH4jKlxBQzgukMfe8zt39K27a7u0Ly8Ws9BvC/sQG6MMcA0YLW19sHwhyTiHcF2RAwE98BfLp+BtBRXwmiAVJ9hdN+O4W0L0KWL21nx2WfduaFHHgmXXKLu0CQVdo7cGNMDeB/4HFd+CDDcWvtqZa/RplmS6Mof8JxRP52V3/+MAfrXsIyxWr/+6o6ce/BB2HNPuOMOGDIE0tIi9x4SF3T4skiMROSg51CsWwfXXw+vvebKFSdOhJNOivz7SMxo90ORGIl6m39Amzbw6qvwyiuwcyecfDKccQZ8/XV03k/ihgK5SJTVpCY9Ivr0gVWr3B4uCxa4LXNHjtTZoQlMqRWROlAXB1jkbyxgzqebd8/Db94Mt9wCs2ZBZiaMH++6RY2JyhgkupQjF4lTtQny5V8zc/EmRrz0OcUlf53TU33MuqJMLv79990D0OXLoWdPlz/v0CFKM5JoiXpDkIjUXG0ehJZ/zcg+HRg5b2VpEIddufjSex1zjKs9nzwZbr8dDj8crr3WVbioO9TzlCMXiaHaPAgt/5rXVv4Hf7l/WQfNxaemwjXXwJdfwhVXYCdOZPvBrfjm/ong9yPepUAuEkO1eRBatuHIGEOHZnu7zw2kGDi5/X67p1XKa9SI/L/fS//LJ/LFXvvTYthQfu96BCxevNtlET/iTqJGOXKRGKtNjnzm4k2MnLcSv7Wl6ZWC7YUh3+PRhesZv2Atfr/ljNXvcvdH/6TBti2uO3TsWPL/l143te9SI6ojF4lT2VkZDD7+kBoFyoLthfitLU2vFGwvrNE9Sv8l4DO8dvgJrHtvCQwb5rbNbdOGHfeNw/9HYfRr3yUiFMhFPCjc/dLL7xHTpUMmjB3r9jo/+mh6PH4vrz41hGM2Lqub2ncJi1IrIh4Vzn7pVbIW5s/nj2uvY49N31DQqw8Zjz8CLVpEfhJSI0qtiCSYUFIytdoewBg4/XT2WLsa7rmHjEXvuO7QO+5Qd2icUiAXSWC1qYopTcX8dwcMHw5r1kDfvu5EonbtYM6cas8OrariRdUwkafUikiCq0lVTFWpmLX/mk+j4X+j8Ya17uzQhx8O2h1a1T3qbCfIBKXUikiSChx2kbdhW7Wr4MpSMTMXb+K05T6OPOt+Rve6mqKl+a479IYb4KefQrpHdd+rilbxVVOLvkiCq8kqOJCK2VnkL03F5G8sYOS8lRT5LZgUpnc+jQOvvITLX3/SrcpnzHBnh156Kfh8Qe9R1f0jOf5kpRW5SIKrySo42NF1eRu2UezflYL1GUPnLq1h0iS3f0vr1jBwIHTvDosXV7gHULqaDnb/SI4/WWlFLpLgaroKDgTXQMDs3rIRe6T5KNzpx1f+vNGuXeGDD9yq/JZbXDC/9FKyx4wh+/hDKl1N12RFXZtVfLLRw06RJBDuA0+gwusr3PPXX+Guu+Chh6BePbjjDh7v2IsH3tmA37p9YG48uS2Djz8kquNPZNrGViSJ1WQVHCyVUb5evdK89f33w+WXu7NDb7yRSw6ZxCe5F7Mo8/CwVtM1XcUnG+XIRWQ3odSeV5m3btvWnR06bx71/EU8NfM2Fnw4kRdOOaBCMFY1SmRoRS4iuwk8kKwqlVFt3toY+Mtf3AHQ48fT6t574aTufH/VUF4+5a8c0e5AAFWjRIhy5CJSKzXKW3/7LT9eM5Q/zX+RzXs35b6Tr6Dh+efw3JJvw86fJxPlyEWkRqoL1DXKWx90ELNuvJ8PGh7BqDef4JHZ97B+3dssP/IS1mQ0Dyl/HuomYcn4UFSBXEQqiEYTTveWjXi4ZWf6XDqRi5a/zvC8mcz/4lqW97sQO3IUXaq4fyjjCRy2Uey37JGWXKkaPewUkQqi1oRjLcW+FJ7N6cOqhUswl11G5znT6XJiLjz1VKVnh1Y3nrLdpxYo3BndxqF4e0irQC4iFdRm18Tq5G3YVhpoi/2WD3428MQTsGQJtGoFl10GRx3lPq9kPD7jzinNqJ9e4d5lD6D2+dw15YNtJAJw4F8H4xesZcDUvLgI5kqtiEgFoVSu1FSllS7Z2bt3h+bmuqA+Zgw0bVo6npF9OpSmTkbPX0Xb/RuWjitw78IiPz5jGNjjYEbPX1WhqSkS6aJg/zqIdQpHgVxEgop0E06VPxx8PrjwQujXb1d36Jw5fDt0GPN79CO3zf6l55RaKgbQ8veuLBUT+FphGAE4HrcMUCAXkTpT7Q+Hhg1Lu0N/HnQNB40eTs/GkxhzylWcPGRAlQG0/L3LX7v2/34lsPeX31IhPVOTOQT7gRTLihkFchGJiSoDX9u2PDtqEsse+ye3vz2F6TOGs/7b93jh3vtZVFi/2mAZLNjmbdiGASzu4WDB9sJaj738D41Yb7WrQC4idS6UwNe9VWMeaXckvQ7uwqAlLzL44xdIPaEbnW79O3S/pdr3KB9sM+qnk+Iz+P2W9LTIpkTyNmzjj53+3SpmFMhFJKGF8sAwsKqe++lmHk8/j9kdenLbu09y6h13wPTp8OCDLqduTND3CKz4M+qns/L7n5mdvxm/tfh8hpF9OkQ00GbUTydQM+On9mmb2lIgF5E6F+oDw0BKpMhv+a5hE4b8ZRh/DLySM54cC/37w4knwsSJ7lDoMsqu+P3ldiEx2LDSKsEUbC/EZ1zu3WfCS9vUhurIRaTO1eSkoPI17Zn9T4XPPnMBfOlSOOwwuOkm+Pnn0teUXfGXZYhcXXxlY0yPQSWLNs0SkbhX6YPRrVvhtttg6lRXcz52LFx0Efnf/rzbitwHpKYYzs45iP5dm1f5g6O21SfVvS4SVS2VbZoVdiA3xjwJ9AG2WGs7hvIaBXIRiailS2HIEMjLg27d4JFHyG96SGmOvGB7Ya1PR4pELj1S960skEcitTId6BWB+4iI1E5ODnz4oXsI+s030K0b2XfdzOAOe3NBt8wKJxxVJlp7zET7AOmwA7m1dhHwYwTGIiJSez4fXHwxrFsHN94ITz8NbdrAww/Dzp0h3SIae8xE874BEcmRG2NaAPOrSq0YYwYBgwAyMzOzN27cGPb7iohUas0aGDoUFiyADh3cw9GePat9WbQ6NOM6R15y8xZUE8jLUo5cROqEtfDyy3DDDfD113DmmTB+PGRlxXpktRLNHLmISHwyBvr2hVWrYPRodyh0u3bu4x07Yj26iFEgF5HEV68ejBjh0i19+sCoUdC+Pbz4olu1R1m0D6IIO5AbY2YBHwNtjTGbjTGXhz8sEZEoyMyE55+Ht9+GBg1cd+gpp8Dq1bW+ZXVBui4OoohE1cr51tpm1to0a21za+20SAxMRCRqevaEZctcRcsnn+zqDv3llxrdJpQgHe3SQ1BqRUSSVWoqXHcdfPklXHIJTJjgyhWffrrSs0PLCyVIR7v0EBTIRSTZNWkCU6bA4sXQooUL6kcf7bpFqxFKkK7JvjK1pb1WREQC/H745z9h2DDYsgUuvxzuvdcF+0rU5clAKj8UEalOoDt07VrXHTp9OrRu7ZqJioqCviQ7KyPkLQCiRYFcRKS8ffaBceNgxQrIzXUdol26wMKFsR5ZUArkIiKVadcO3ngD5s6F335z1S7nnAObNsV6ZLtRIBcRqYoxcMYZ8MUXcOed8MorcOihcPfd8L//xXp0gAK5iEho6tWDkSNdd+ipp7pO0fbtYd68OukOrYoCuYhITWRlwezZ8NZbLrj36we9e7sAHyMK5CIitXHCCa479KGH3MlEnTrBzTfXuDs0EhTIRURqKy3NVbSsW+fKFsePh7Zt4ZlnQu4OjQQFchGRcDVt6g6AXrzYpV4uvhh69ID8/Dp5ewVyEZFIOeII+OgjeOop+Oor9/mgQbB1a1TfVoFcRCSSfD63X8u6de5koqeecptxPfJIpd2hYb9lVO4qIpLs9tnH5cxXrICcHLfTYpcusHlzxN9KgVxEJJratXMHQM+dC61aQbNmEX+L1IjfUUREdhfoDj3jjKjcXityERGPUyAXEfE4BXIREY9TIBcR8TgFchERj1MgFxHxOAVyERGPUyAXEfE4Y2NwsoUxZiuwMYRLGwM/RHk48SpZ556s84bknXuyzhtqPvcsa22T8l+MSSAPlTFmqbU2J9bjiIVknXuyzhuSd+7JOm+I3NyVWhER8TgFchERj4v3QD451gOIoWSde7LOG5J37sk6b4jQ3OM6Ry4iItWL9xW5iIhUQ4FcRMTj4iKQG2N6GWPWGmPWG2P+HuT7xhgzseT7K4wxXWMxzkgLYd4DSua7whjzkTHm8FiMMxqqm3uZ644wxhQbY86qy/FFSyjzNsYcZ4xZZoxZZYx5r67HGC0h/HnfxxjzijFmecncL43FOCPNGPOkMWaLMWZlJd8PP75Za2P6C0gBvgJaAunAcqB9uWtOBV4DDNAdWBzrcdfRvI8CMko+7p0I8w517mWuewd4FTgr1uOuo9/zfYEvgMySz5vGetx1OPfhwH0lHzcBfgTSYz32CMz9WKArsLKS74cd3+JhRZ4LrLfWbrDWFgLPAX3LXdMXeMY6ecC+xpjIH3xXt6qdt7X2I2ttQcmneUDzOh5jtITyew4wBJgDbKnLwUVRKPO+AJhrrd0EYK1NprlboKExxgB74QJ5dI6dr0PW2kW4uVQm7PgWD4H8QODbMp9vLvlaTa/xmprO6XLcT+1EUO3cjTEHAmcAk+pwXNEWyu95GyDDGPOuMSbfGHNRnY0uukKZ+z+AdsD3wOfAUGutv26GF1Nhx7d4OHzZBPla+ZrIUK7xmpDnZIw5HhfIe0R1RHUnlLk/BAyz1ha7BVpCCGXeqUA2cAJQD/jYGJNnrV0X7cFFWShzPwVYBvQEWgFvGmPet9b+EuWxxVrY8S0eAvlm4KAynzfH/USu6TVeE9KcjDGHAVOB3tbabXU0tmgLZe45wHMlQbwxcKoxpsha+1KdjDA6Qv2z/oO19nfgd2PMIuBwwOuBPJS5XwqMtS5xvN4Y8zVwKPBJ3QwxZsKOb/GQWlkCtDbGHGyMSQfOA14ud83LwEUlT3e7Az9ba/9T1wONsGrnbYzJBOYCFybAiqysaudurT3YWtvCWtsCmA1c4/EgDqH9WZ8HHGOMSTXG1Ae6AavreJzREMrcN+H+JYIxZj+gLbChTkcZG2HHt5ivyK21RcaYa4E3cE+2n7TWrjLGXFXy/Um4qoVTgfXAdtxPbk8Lcd4jgUbAYyUr0yKbALvEhTj3hBPKvK21q40xrwMrAD8w1VobtGzNS0L8Pb8LmG6M+RyXbhhmrfX89rbGmFnAcUBjY8xmYBSQBpGLb2rRFxHxuHhIrYiISBgUyEVEPE6BXETE4xTIRUQ8ToFcRMTjFMhFRDxOgVxExOMUyEUAY8xCY8xJJR/fbYyZGOsxiYQq5p2dInFiFDDaGNMU6AL8JcbjEQmZOjtFSpScxrMXcJy19tdYj0ckVEqtiADGmE5AM+APBXHxGgVySXolp7HMwJ3U8rsx5pQYD0mkRhTIJamVbBU7F7jJWrsatwPfHTEdlEgNKUcuIuJxWpGLiHicArmIiMcpkIuIeJwCuYiIxymQi4h4nAK5iIjHKZCLiHjc/wO2uWPsAKQKkQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from scipy.optimize import minimize\n",
    "\n",
    "w0 = np.zeros(npar)\n",
    "w = minimize(l, w0, method = 'BFGS', jac = l1, options={'disp': True} ).x\n",
    "evalw(w, x, y)\n",
    "print(\"w = {}, l(w) = {:f}\".format(w, l(w)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-24T09:28:14.265478Z",
     "iopub.status.busy": "2021-09-24T09:28:14.264949Z",
     "iopub.status.idle": "2021-09-24T09:28:14.335599Z",
     "shell.execute_reply": "2021-09-24T09:28:14.335298Z"
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Optimization terminated successfully.\n",
      "         Current function value: 0.055920\n",
      "         Iterations: 7\n",
      "         Function evaluations: 8\n",
      "         Gradient evaluations: 8\n",
      "w = [-0.6817 -2.8653 -1.1836], l(w) = 0.055920\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAEGCAYAAAB4lx7eAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAmf0lEQVR4nO3deXhURdbH8W91FgRFzbAoCgmCgmwKJCwqOoIbIA6Iu7ihiCIijs7IiAKKC6ggizuioCPgqODGiOKCu0EIArKDKIjOOyJGXGAMSdf7R6VDSDpJJ92dzu3+fZ6Hxyy3b1cJnBTnnlNlrLWIiIh3+WI9ABERCY8CuYiIxymQi4h4nAK5iIjHKZCLiHhccizetH79+rZp06axeGsREc/Kycn50VrboOTXYxLImzZtytKlS2Px1iIinmWM2RLs6xEJ5MaYb4BfgQIg31qbFYn7iohIxSK5Iu9urf0xgvcTEZEQ6GGniIjHRSqQW2ChMSbHGDM42AXGmMHGmKXGmKXbt2+P0NuKiEikAvkJ1tqOQC9gqDHmpJIXWGunWWuzrLVZDRqUeugqIiJVFJFAbq39vvC/PwAvA50jcV8REalY2IHcGLO/MaZu4GPgdGBVuPcNJmdLLo8s2kTOltxo3F5ExJMiUbVyCPCyMSZwv9nW2jcjcN995GzJZcD0bPLy/aQm+5g1qCuZGWmRfhsREc8JO5BbazcDx0ZgLOXK3ryDvHw/fgt78v1kb96hQC4igofKD7s2q0dqso8kAynJPro2q7fP95V2EZFEFZMW/arIzEhj1qCuZG/eQddm9fZZjSvtIiKJzDOBHFwwLxnAszfv4LufdyvtIiIJy1OBvLjiq/DkJB/JPkOB3wZNu4iIxDPPBvLiDz8LCvxc2Dmdww6uXSrtIiIS7zwbyAMPP/fk+0lJ9tG/Y+NSATyQelFwF5F45tlAXt7DT9ADUBFJHJ4N5FD64WdxJevO5y7bptW5iMQlTwfy8hRPvST5DC/lbCO/wI/PGMb2bcvFXdJjPUQRkYjwTENQZQVSLzed3pLzspqQX+BW5/l+y+hXV6lxSETiRtwGcnDBfGj3I+nfsTE+txcMAH6/JXvzjhiOTEQkcuI6kAdkZqQxtm9bkn0GH5CaolpzEYkfcZsjL+niLum0PLSuHniKSNyJ2xW5NtESkUQRlyvyYDXkgOrKRSQuxWUgD7Z3OaCNtUQkLnkrkP/yC9StC8UqUIIp2b4feLAZ7GsiIl7nnUBuLfTpA/vtB1OmQKtWZV5aVvv+6D5tWLDqP/Rq26joa9qPRUS8zjuB3O+H886D0aPhmGPghhtgzBg48MCglwfbu3zs/NXk5ftZ8s1PtDy0LqC8uYh4n3eqVpKSYNgw2LABrrgCJk2CFi1g5kwX5CsQLG9eVi5dRMRLvBPIAxo0gCefhM8/hyOOgIED4fjjYcmScl8W7MzPtDqp+IxrElLeXES8ynuBPCArCz75xK3Iv/kGunSBQYPghx+CXl5875VAOeLY+avxW4vPZxjdp43SKiLiSd4N5AA+H1x+uUu33HQTPPOMS7dMmQL5+aUuD+y9kpmRtk9axVpL7q68GExARCR83g7kAQceCBMmwMqV0Lkz3HgjtG8PixaV+ZJgqRYRES+Kj0Ae0KoVvPUWvPwy/P479OgB558PW7eWujSQarmwczr9OzaOwWBFRCIjvgI5uGahfv1gzRoYOxbmz4ejj4a77oL//a/U5XOXbeP5z7cyYHq29mUREU+Kv0AeULs2jBoFa9e6RqLRo6F1a3jlFddcRNmt/CIiXhK/gTwgI4OcB57glQnPsjulFpx9NvTsCevW0bVZPZJ9BgMk+UypPLl2UBQRL/BOZ2cV7d0J8U/UPvcBFqSsIn3yeGjXjsMHXsP+B3Xn56T9Su3fEmwHRZUnikhNFPcr8uLpk//5Da//+VzYuBGuuIJDpj/KW49dzdmr3sWfn79PakVpFxHxirgP5EHLDAu7Q9e98jbfH9yQB/89iRdnjaDH79+W/zoRkRrI2MIHf9UpKyvLLl26tNrer7wdDnO+3kHuE0/x56cmkrJju+sOveceaNCg1Ou0U6KIxJIxJsdam1Xq64kQyEOyc6crUZwyBQ44wJUuDhkCye4xgnLmIhJrZQXyuE+thOygg/Z2h3bq5LbJ7dAB3n8fUM5cRGouBfKSAt2h8+bBb79B9+5wwQWcuN/uoDnz4iWKKlcUkVhQaqU8u3fDAw/AuHFgDN9d91deP+1iOh19GJkZacxevJXRr66iwG9JSfaBteT7Lck+w3lZTejfsbHSLyISMcqRh2PLFrj5Zpg71+2BPmkSOceeyAXTssn37/3/ZwBb7ONaKcqli0jkRD1HboxJMsZ8YYyZH6l71hgZGfDSS/DOO671v18/6p/fj6Y/7t2MK8lnSElyXaLgArpy6SJSHSKZIx8OrI3g/WqeU06B5cth0iQar1vOgqeuZ+Sipzl4z27u6tuWOYOP46Iu6ao/F5FqFZHUijGmMfAMcA9wk7W2T3nXey61EswPP/DjsJuo/8Is9tRvSMqE++HSS8HnU725iERFtFMrk4FbgDJPQTbGDDbGLDXGLN2+fXuE3jaGGjak/r+eg8WLSWnW1B0I3a0b5OTscxKRiEi0hR3IjTF9gB+stTnlXWetnWatzbLWZjVo0CDct605OneGzz6DGTPgq69cDfrgwRAPP6xExBMisSI/AfiLMeYb4HmghzHmuQjc1zt8Prci37DBHTM3Y4Y7O/Shh8j5artqy0UkqsIO5NbaW621ja21TYELgfestZeEPTIvOuggePBBWLECsrLghhs44PjOfPzkixWeQKRmIhGpqrjfjzwmWreGhQt5467HaPfgWObMGcm/jz6RVW3vITPjxFKXF9/HRc1EIlJZEW3Rt9a+X1HFSsIwhkMuv4g+1z7O5G4Xc8qmxVwysKfbWbHE2aHF93HJK7DMXqwzREUkdNprJYoyM9J4esifSRl7JxsXfU5S715w++3Qpg289lrR2aGBvc/VTCQiVaFAHmWBUsR23Y513aFvvw21akHfvtC7N6xfT2ZGGrMGdVUzkYhUifZaiYU9e+CRR2DMGLcx1403wqhRULeumolEpEzaNKsm+u9/4dZbXbnioYfC/ffDJZeUOghaRAR0sETNdMgh8PTTsHgxNGkCl13mukOXLYv1yETEQxTIa4LOnSE72wX1TZtcDfo118CPP8Z6ZCLiAQrkNYXPBwMHsvztxSzvfzn2qafgqKPg4YchPx9Q05CIBKeGoBokZ0suA15cS95R59JqUFfmrJrDgcOGwbRprL/tXgasStLhzyJSilbkNUjxxqB1aY3559gnXcnizp20vPAsHpg7jkN2bleNuYjsQ4G8Bgk0BhXVkTevD+ecA2vX8v3wWzht42LenX4t12e/wHGH7R/r4YpIDaHywxqmvDryLz9eQZ2Rt9D8o4XQrBlMnkxOuxPI/von1Z2LJADVkceTt9+G4cNh7Vo+bJ7JnacM5ruGTZQ3F4lzqiOPJ6edBitW8PGQkXTYtpYF04fy17efImfVlliPTERiQIHcq1JSqD3ib/QaMo3X2pzMNYvncsWVveC554o24xKRxKBA7mGZGWlMubE3/53yGGtfXkhqRhN3ALS6Q0USigK5xwV2V2zV7zTXHTp9Omzc6LpDr70WdqhMUSTeKZDHkZxvd/JIsz+z/J3P3cPQ6dNdd+ijjxZ1h4pI/FEgjxOB4+ImLlzPhS+uJefG0e7s0A4dYOhQt0L/8MNYD1NEokCBPE4U7wot6vxs0wbeecd1h+bmwp//DBdfDNu2xXq4IhJBCuRxolRXaOB0IWOKukMZPRrmzYOjj4Zx4+CPP2I7aBGJCDUExZGQThf6+mu4+WZ4+WVo3hwmT4Y+fXQykYgHqLNT9vX22+weMpTaX23ku+O7c2X7i9l4YCPtrChSg6mzU/aR0yKLTudP5J4eV3HQ0mxef3wIf180k5Rdv2tnRRGP0X7kCSaQQvnu593ssj6e7HQ2r7Y6mREfzmTI4pfov2YRv2eMg5Ob6+xQEY/QijyBFC9RfClnG8k+Q5KBX9Lq8ceTT/PSwy9wwBHpNBs+GE48Eb74ItZDFpEQaEWeQIqXKBYU+LmwczqHHVx77wPOLukw5ByYMQNuvdXVng8eDHfdBfXrx3r4IlIGrcgTSMkSxf4dGzO0+5H7Ptj0+eCqq2DDBrj+enjySWjRQt2hIjWYqlYSTHllhkG/t2oV3HADLFoExx4LU6fCSSfFYOQiovJDKVcgfx70cGdrXXfozTfDt9/CRRfB/fdD48axHbRIglH5oZQraIt/gDFw3nmwbh2MGqXuUJEaRoFcgOAt/jlbcnlk0SZytuS6i+rUgbFjXbv/aafByJHQti38+9+xHbxIglPVihTp37ExpvC/QNmpliOOcC3+Cxe6/HmfPnDmmTBpkts2V0SqlVbkUpQff/7zrcxd5nZGLDfVEnD66bByJUyY4LbIbdvWlS3+9ls1z0AksSmQS9CgXeZuiiWlprqHoOvXu4eg48dDy5Ywe7bODhWpJgrkEjRoZ2akMWtQV246vWVom2g1agQzZ8Knn7qPBwxwZYrLl1fHFEQSmsoPBQhxC9xQFRTs7Q796Se45hrXHVqvjFW9iIQkauWHxpj9jDGfG2NWGGNWG2PuDPeeUv0ChzhHZPvapCQYNMh1hw4dCtOmue7Qxx5zQV5EIioSqZU/gB7W2mOB9kBPY0zXCNxXvC4tzXWCfvEFHHMMXHcdZGbCRx/FemQicSXsQG6dQJlCSuEvPeWSvdq1g/fegxdecKmWk05yOfTvvov1yETiQkQedhpjkowxy4EfgLettYuDXDPYGLPUGLN0+/btkXhb8ZJAd+jata47dO5cV90yfry6Q0XCFJFAbq0tsNa2BxoDnY0xbYNcM81am2WtzWrQoEEk3la8aP/9XXfomjVw6qnugWi7dvDGG7EemYhnRbT80Fr7M/A+0DOS9xVvKdXaH0yzZvDKK7Bggds698wzXYfopk3VNk6ReBGJqpUGxpiDCz+uDZwKrAv3vuJNxU8hGjA9u/xgDtCzp+sOfeAB+OADaNPG7eGi7lCRkFUYyI0x7YwxnxT7vKMx5r1ilzQCFhljVgJLcDny+ZEfqnhBSK39JaWmwt/+5soVL7zQ7ap49NEwZ466Q0VCEMqKfDXQ3BiTVPj5ROBvgW9aa1daaztYa4+x1ra11o6NxkDFGwJdoj4DxhjS6qSG/uJGjeCZZ+CTT+CQQ+Dii+Hkk2HFiqiNVyQeVBjIrbV+XDBvY4w5B9hqrV0W9ZGJJ2VmpDG6Txt8xlDgt4ydv7ri9EpJxx8Pn3/uGonWrIGOHV1j0U8/RWfQIh4Xao48GzgBuAMYGbXRSFzI3ZWH31oslUivlJSUBFdfvbc79PHHXXfoE0+oO1SkhMoE8ruBl6216uKQcoW8c2IoAt2hy5e7bXKvvRY6dXLpFxEBQg/k63Ct+PdFcSwSJyq9c2Io2rVzB0A//zxs3w7dusGll8L334d/bxGPCzWQDwdutdb+Hs3BSPyI6CZcAcbABRe4s0Nvu821/Lds6Q6CVneoJLByA7kxprkxZh1Q21r7TDWNSaR8++8Pd9/tHoT26AEjRrgV+4IFsR6ZSEyUG8ittV9Za4+21l5VXQMSCVnz5vDqqy6AGwO9e8NZZ6k7VBKOTgiSGiGktv6y9OwJX37pukPff1/doZJwFMgl5ird1h9M8e7QCy5Qd6gkFAVyiamcLblMfmcDf+ypZFt/WRo1gmefhY8/hoYN1R0qCUGBXGImsBL/ZNOPWNwfxrDrzgNOOAGWLHENRKtXqztU4poCucRM8Q22fAZOOKp+5OrOwXWHDh7s0i1Dhqg7VOKWArnETPEO0NRkHzee2iJoEA/rQSjAn/4EDz/szg5t08Z1h3buDJ9+GuYMRGoGY2PwICgrK8suXbq02t9Xap6cLblkb95B12b1ygziA6Znk5fvJzXZF3TFXtE99mEt/Otf7sHod9/BJZfAfffBYYdFcloiUWGMybHWZpX8enIsBiMSkJmRVm7wDba/eeD6nC25zFu2jReXfku+35YZ6PdhjNvz/Kyz4N57YcIEd1LR6NEwfLirfhHxGKVWpEYrawOuwEp99uKt5BXYyle87L8/3HOP6w7t3h1uucV1h775ZhRnIxIdCuRSo5W1AVdgpR5IDBqqWPHSvDm89po7/Nla6NUL+vaFr76K6DxEokmBXGq8YBtwlXxQelGXdGYN6gpQtQejvXq57tDx4+Hdd91D0dtvh9+1T5zUfHrYKZ5V8iFnKA9GQ/L99y7VMmsWNG7s8ujnn+/y6yIxVNbDTq3IxbNKrtRDPfi5wnLGww6D556Djz6C+vXdw9EePdyKXaQGUiCXuBHKyUSV2telWzdYutQ1Eq1cCe3bw7Bh6g6VGkeBXOJGKCcThbpqL5KUBNdcAxs3ukaiRx913aHTpqk7VGoMBXKJKxWdTFTl80T/9Cd45BFYtgxat3bBXd2hUkPoYacknMBD0rQ6qeTuygutI7S4kt2hl14K991HTt5+oXeYilRBWQ87FcglIUWkwuW339y+5xMmUJCaysQuF/BUxz6YWrUiu/mXSCFVrYgUU+lceaF9Kl4OOMB1h65ezda2nbjl3ad446lhdN2wJLw91UUqSYFcElLJXHlandQKG4nKrHg58kh+en4ugy+8E5/1M/OFMQwYNxw2b66m2UiiUyCXhBSocLmwczonHtWAO15fXWFJYnmr+MyMNK4ZP4w35yxk24gxHPzpB+6h6KhRsGtXdU1LEpQCuSS0ucu28c6a/4aUZqmo4iUzI40hZ7Sh8fg73GEW554Ld9/tzg594QWdHSpRo0AuCauyG2+FUqdepHh3aL167kDoU06BVasiPg8RBXJJWPtsvJVkuLhw463yAnRFdeqlBLpDH3vMHQDdvj3ccAPkVvG0I5EgVH4oCa1SpwuFa8cOd4DF44+7BqNx42DgQNc9KhIClR+KBFHpFXY46tVz3aE5OdCqFVx9NXTpAp99Fv33lrimQC5SgbAPfy6pfXv44AOYPRv+7//g+OPh8svdxyJVoEAuUo5K7ZZYGcbARRfBunVw663w/PNuM64JEyAvLzLvIQlDgVykHFXtAA3ZAQe4Q6BXrYKTToK//x2OOQbeeiuy7yNxTYFcpAw5W3L5/ufdJPtM5XdLLHaPkNIyRx0F8+e7XwUF0LMn9Oun7lAJSXK4NzDGNAGeBQ4F/MA0a+2UcO8rEkvFN9VKTvJxQecmnNOxcaUeilZpY64zz4RTT4VJk1wzUevWbpV+661Qp06Ys5J4FYkVeT5ws7W2FdAVGGqMaR2B+4rETPGUSkGBn8MPrl3pypYqp2Vq1YJ//APWr4dzztnbHfrii+oOlaDCDuTW2v9Ya5cVfvwrsBY4PNz7isRSZQ6gKCt9UvweST7D9z/vZvbiraFXwBx+uDsA+sMPXd35+eerO1SCimhDkDGmKfAh0NZa+0uJ7w0GBgOkp6dnbtmyJWLvKxINoTQLlZU+Cbz21917+GzzDlb/5xcKCiwWtxVArZRK7oFeUOCOl7vtNvjlFxg6FO68Ew4+OFLTFQ8oqyEo7Bx5sTc4AJgL3FgyiANYa6cB08B1dkbqfUWiJTMjrcJAW1b6ZMD0bP7Ys3cfl+IskFd4bciBPCkJhgxxq/JRo+Dhh10d+rhxcOWV4FPdQiKLyO++MSYFF8RnWWvnReKeIl4QLAVTcjOuYHzGVLoCBnDdoY8+6vZvadlyb3dodnaV5yDeF3YgN8YY4ClgrbX2wfCHJOIdwXZEDAT3wF8un4GUJFfCaIBkn2Fs37bhbQvQoYPbWfG559y5occdB1dcoe7QBBV2jtwY0w34CPgSV34IMNJa+0ZZr9GmWRLvSh7wnFYnlVXf78QA/StZxlihX391R849+CDstx/ccQcMGwYpKZF7D6kRdPiySIxE5KDnUGzYADfeCAsWuHLFqVPhtNMi/z4SM9r9UCRGot7mH9CiBbzxBrz+OuzZA6efDmefDV9/HZ33kxpDgVwkyipTkx4RffrA6tVuD5eFC92WuaNH6+zQOKbUikg1qI4DLHK25DJ32bZ98/DbtsEtt8CcOZCeDhMnum5RY6IyBoku5chFaqiqBPmSr5m9eCujXvmSgsK/zqnJPuZcXSwX/9FH7gHoihXQo4fLn7dpE6UZSbREvSFIRCqvKg9CS75mdJ82jH51VVEQh725+KJ7nXiiqz2fNg1uvx2OPRauv95VuKg71POUIxeJoao8CC35mgWr/oO/xL+sg+bik5Phuutg40a4+mrs1KnsOqI539w/Ffx+xLsUyEViqCoPQos3HBljaNPoQPe5gSQDp7c+ZN+0Skn16pHzj3vpf9VU1hxwKE1HDOf3jp1g8eJ9Lov4EXcSNcqRi8RYVXLksxdvZfSrq/BbW5Reyd2VF/I9Hlm0iYkL1+P3W85e+z53f/pP9t/xg+sOHT+enP+lVk/tu1SK6shFaqjMjDSGdj+yUoEyd1cefmuL0iu5u/IqdY+ifwn4DAuOPYUNHyyBESPctrktWrD7vgn4/8iLfu27RIQCuYgHhbtfesk9Yjq0SYfx491e5yecQLfH7uWNGcM4ccvy6ql9l7AotSLiUeHsl14ua2H+fP64/gZqbf2G3J59SHvsIWjaNPKTkEpRakUkzoSSkqnS9gDGwFlnUWv9WrjnHtI+fM91h95xh7pDaygFcpE4VpWqmKJUzH93w8iRsG4d9O3rTiRq1Qrmzq3w7NDyKl5UDRN5Sq2IxLnKVMWUl4pZ/6/51Bv5N+pvXu/ODp0yJWh3aHn3qLadIOOUUisiCSpw2EX25h0VroLLSsXMXryVM1f4OO7c+xnbcwj5S3Ncd+hf/wo//xzSPSr6Xnm0ii+fWvRF4lxlVsGBVMyefH9RKiZnSy6jX11Fvt+CSWJm+zM5/JoruOrNp92qfNYsd3bowIHg8wW9R3n3j+T4E5VW5CJxrjKr4GBH12Vv3kGBf28K1mcM7TscBY8/7vZvOeooGDQIunaFxYtL3QMoWk0Hu38kx5+otCIXiXOVXQUHgmsgYHZtVo9aKT7y9vjxlTxvtGNH+Phjtyq/5RYXzAcOJHPcODK7H1nmaroyK+qqrOITjR52iiSAcB94AqVeX+qev/4Kd90FkydD7dpwxx081rYnD7y3Gb91+8DcdHpLhnY/Mqrjj2faxlYkgVVmFRwslVGyXr3MvPX998NVV7mzQ2+6iSuOfJzPO1/Oh+nHhrWaruwqPtEoRy4i+wil9rzcvHXLlu7s0FdfpbY/nxmzb2PhJ1N58YzDSgVjVaNEhlbkIrKPwAPJ8lIZFeatjYG//MUdAD1xIs3vvRdO68r31w7ntTMuoVOrwwFUjRIhypGLSJVUKm/97bf8dN1w/jT/ZbYd2JD7Tr+auhedz/NLvg07f55IlCMXkUqpKFBXKm/dpAlzbrqfj+t2YszbT/DQS/ewacO7rDjuCtalNQ4pfx7qJmGJ+FBUgVxESolGE07XZvWY0qw9fQZO5bIVbzIyezbz11zPin6XYkePoUM59w9lPIHDNgr8llopiZWq0cNOESklak041lLgS+K5rD6sXrQEc+WVtJ87kw6ndoYZM8o8O7Si8RTvPrVA3p7oNg7VtIe0CuQiUkpVdk2sSPbmHUWBtsBv+XingSeegCVLoHlzuPJKOP5493kZ4/EZd05pWp3UUvcufgC1z+euKRlsIxGAA/86mLhwPQOmZ9eIYK7UioiUEkrlSmWVWemSmblvd2jnzi6ojxsHDRsWjWd0nzZFqZOx81fT8tC6ReMK3Dsv34/PGAZ1O4Kx81eXamqKRLoo2L8OYp3CUSAXkaAi3YRT7g8Hnw8uvRT69dvbHTp3Lt8OH8H8bv3o3OLQonNKLaUDaMl7l5WKCXwtL4wAXBO3DFAgF5FqU+EPh7p1i7pDdw6+jiZjR9Kj/uOMO+NaTh82oNwAWvLeJa9d/3+/Etj7y28plZ6pzByC/UCKZcWMArmIxES5ga9lS54b8zjLH/0nt7/7JDNnjWTTtx/w4r3382FenQqDZbBgm715BwawuIeDubvyqjz2kj80Yr3VrgK5iFS7UAJf1+b1eajVcfQ8ogODl7zM0M9eJPmULrS79R/Q9ZYK36NksE2rk0qSz+D3W1JTIpsSyd68gz/2+PepmFEgF5G4FsoDw8Cqet6ybTyWeiEvtenBbe8/Te877oCZM+HBB11O3Zig7xFY8afVSWXV9zt5KWcbfmvx+Qyj+7SJaKBNq5NKoGbGT9XTNlWlQC4i1S7UB4aBlEi+3/Jd3QYM+8sI/hh0DWc/PR7694dTT4WpU92h0MUUX/H7S+xCYrBhpVWCyd2Vh8+43LvPhJe2qQrVkYtItavMSUEla9rT+/eGL75wAXzpUjjmGLj5Zti5s+g1xVf8xRkiVxdf1hhTY1DJok2zRKTGK/PB6PbtcNttMH26qzkfPx4uu4ycb3fusyL3AclJhvOymtC/Y+Nyf3BUtfqkotdFoqqlrE2zwg7kxpingT7AD9batqG8RoFcRCJq6VIYNgyys6FLF3joIXIaHlmUI8/dlVfl05EikUuP1H3LCuSRSK3MBHpG4D4iIlWTlQWffOIegn7zDXTpQuZdf2domwO5uEt6qROOyhKtPWaifYB02IHcWvsh8FMExiIiUnU+H1x+OWzYADfdBM88Ay1awJQpsGdPSLeIxh4z0bxvQERy5MaYpsD88lIrxpjBwGCA9PT0zC1btoT9viIiZVq3DoYPh4ULoU0b93C0R48KXxatDs0anSMvvHlTKgjkxSlHLiLVwlp47TX461/h66/hnHNg4kTIyIj1yKokmjlyEZGayRjo2xdWr4axY92h0K1auY9374716CJGgVxE4l/t2jBqlEu39OkDY8ZA69bw8stu1R5l0T6IIuxAboyZA3wGtDTGbDPGXBX+sEREoiA9HV54Ad59F/bf33WHnnEGrF1b5VtWFKSr4yCKSFStXGStbWStTbHWNrbWPhWJgYmIRE2PHrB8uato+fzzvd2hv/xSqduEEqSjXXoISq2ISKJKToYbboCNG+GKK2DSJFeu+MwzZZ4dWlIoQTrapYegQC4iia5BA3jySVi8GJo2dUH9hBNct2gFQgnSldlXpqq014qISIDfD//8J4wYAT/8AFddBffe64J9GarzZCCVH4qIVCTQHbp+vesOnTkTjjrKNRPl5wd9SWZGWshbAESLArmISEkHHQQTJsDKldC5s+sQ7dABFi2K9ciCUiAXESlLq1bw1lswbx789purdjn/fNi6NdYj24cCuYhIeYyBs8+GNWvgzjvh9dfh6KPh7rvhf/+L9egABXIRkdDUrg2jR7vu0N69Xado69bw6qvV0h1aHgVyEZHKyMiAl16Cd95xwb1fP+jVywX4GFEgFxGpilNOcd2hkye7k4natYO//73S3aGRoEAuIlJVKSmuomXDBle2OHEitGwJzz4bcndoJCiQi4iEq2FDdwD04sUu9XL55dCtG+TkVMvbK5CLiERKp07w6acwYwZ89ZX7fPBg2L49qm+rQC4iEkk+n9uvZcMGdzLRjBluM66HHiqzOzTst4zKXUVEEt1BB7mc+cqVkJXldlrs0AG2bYv4WymQi4hEU6tW7gDoefOgeXNo1Cjib5Ec8TuKiMi+At2hZ58dldtrRS4i4nEK5CIiHqdALiLicQrkIiIep0AuIuJxCuQiIh6nQC4i4nEK5CIiHmdsDE62MMZsB7aEcGl94McoD6emStS5J+q8IXHnnqjzhsrPPcNa26DkF2MSyENljFlqrc2K9ThiIVHnnqjzhsSde6LOGyI3d6VWREQ8ToFcRMTjanognxbrAcRQos49UecNiTv3RJ03RGjuNTpHLiIiFavpK3IREamAArmIiMfViEBujOlpjFlvjNlkjPlHkO8bY8zUwu+vNMZ0jMU4Iy2EeQ8onO9KY8ynxphjYzHOaKho7sWu62SMKTDGnFud44uWUOZtjDnZGLPcGLPaGPNBdY8xWkL4836QMeZ1Y8yKwrkPjMU4I80Y87Qx5gdjzKoyvh9+fLPWxvQXkAR8BTQDUoEVQOsS1/QGFgAG6AosjvW4q2nexwNphR/3iod5hzr3Yte9B7wBnBvrcVfT7/nBwBogvfDzhrEedzXOfSRwX+HHDYCfgNRYjz0Ccz8J6AisKuP7Yce3mrAi7wxsstZuttbmAc8DfUtc0xd41jrZwMHGmMgffFe9Kpy3tfZTa21u4afZQONqHmO0hPJ7DjAMmAv8UJ2Di6JQ5n0xMM9auxXAWptIc7dAXWOMAQ7ABfLoHDtfjay1H+LmUpaw41tNCOSHA98W+3xb4dcqe43XVHZOV+F+aseDCudujDkcOBt4vBrHFW2h/J63ANKMMe8bY3KMMZdV2+iiK5S5Pwy0Ar4HvgSGW2v91TO8mAo7vtWEw5dNkK+VrIkM5RqvCXlOxpjuuEDeLaojqj6hzH0yMMJaW+AWaHEhlHknA5nAKUBt4DNjTLa1dkO0Bxdlocz9DGA50ANoDrxtjPnIWvtLlMcWa2HHt5oQyLcBTYp93hj3E7my13hNSHMyxhwDTAd6WWt3VNPYoi2UuWcBzxcG8fpAb2NMvrX2lWoZYXSE+mf9R2vt78DvxpgPgWMBrwfyUOY+EBhvXeJ4kzHma+Bo4PPqGWLMhB3fakJqZQlwlDHmCGNMKnAh8FqJa14DLit8utsV2Gmt/U91DzTCKpy3MSYdmAdcGgcrsuIqnLu19ghrbVNrbVPgJeA6jwdxCO3P+qvAicaYZGNMHaALsLaaxxkNocx9K+5fIhhjDgFaApurdZSxEXZ8i/mK3Fqbb4y5HngL92T7aWvtamPMtYXffxxXtdAb2ATswv3k9rQQ5z0aqAc8WrgyzbdxsEtciHOPO6HM21q71hjzJrAS8APTrbVBy9a8JMTf87uAmcaYL3HphhHWWs9vb2uMmQOcDNQ3xmwDxgApELn4phZ9ERGPqwmpFRERCYMCuYiIxymQi4h4nAK5iIjHKZCLiHicArmIiMcpkIuIeJwCuQhgjFlkjDmt8OO7jTFTYz0mkVDFvLNTpIYYA4w1xjQEOgB/ifF4REKmzk6RQoWn8RwAnGyt/TXW4xEJlVIrIoAxph3QCPhDQVy8RoFcEl7haSyzcCe1/G6MOSPGQxKpFAVySWiFW8XOA2621q7F7cB3R0wHJVJJypGLiHicVuQiIh6nQC4i4nEK5CIiHqdALiLicQrkIiIep0AuIuJxCuQiIh73/xCYY/DPcAh3AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "w0 = np.ones(npar)\n",
    "w = minimize(l, w0, method = 'BFGS', jac = l1, options={'disp': True} ).x\n",
    "evalw(w, x, y)\n",
    "print(\"w = {}, l(w) = {:f}\".format(w, l(w)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Die Ergebnisse sind vergleichbar mit denen von GD, allerdings ist\n",
    "die Anzahl der Iterationen (und damit der Aufwand) deutlich geringer."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Normalgleichungen"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wir betrachten jetzt das Normalgleichungssystem\n",
    "\\begin{equation*} \n",
    "X^TXw = X^Ty.\n",
    "\\end{equation*}\n",
    "Da $X$ linear abhängige Spalten hat, ist $X^TX$\n",
    "positiv semidefinit, also nicht regulär. \n",
    "Wie wir oben gesehen haben, existieren aber immer Lösungen dieses\n",
    "Systems."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wie lösen wir jetzt numerisch dieses singuläre Gleichungssystem?\n",
    "\n",
    "Direkte Verfahren (LU, Cholesky, QR) sind prinzipiell anwendbar, leiden aber oft an Stabilitätsproblemen."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Bei den iterativen Verfahren betrachten wir wieder CGLS, wenden es auf unser Beispielproblem an\n",
    "und erhalten für zwei unterschiedliche Startwerte die folgenden Ergebnisse."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "code_folding": [],
    "execution": {
     "iopub.execute_input": "2021-09-24T09:28:14.347447Z",
     "iopub.status.busy": "2021-09-24T09:28:14.344750Z",
     "iopub.status.idle": "2021-09-24T09:28:14.406104Z",
     "shell.execute_reply": "2021-09-24T09:28:14.405810Z"
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "w = [-1.0151 -2.532  -1.5169], l(w) = 0.055920\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAEGCAYAAAB4lx7eAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAmhUlEQVR4nO3de5zV0/7H8dfaMw2V25wuRM2kpDtdxhTiKKTIkdx1KEpJksshOkcRKpSUg05CLpWjC9HhnELkNl0mle4SJfkdSXKpY5rZ6/fHmj1NM3tm9szee/Z8934/Hw8Pc/nu715L+cyaz/fzWctYaxEREe/yxXoAIiISHgVyERGPUyAXEfE4BXIREY9TIBcR8bjkWLxp7dq1bcOGDWPx1iIinpWdnf2DtbZO0a/HJJA3bNiQ5cuXx+KtRUQ8yxizNdjXIxLIjTFfA78AeUCutTYjEvcVEZGyRXJF3tla+0ME7yciIiHQw04REY+LVCC3wAJjTLYxZkCwC4wxA4wxy40xy3fu3BmhtxURkUgF8tOtte2A7sBgY8yZRS+w1k6x1mZYazPq1Cn20FVERCooIoHcWrsj/9/fA68BmZG4r4iIlC3sQG6MqWmMOTzwMdAVWBPufYPJ3rqbJxdtJnvr7mjcXkTEkyJRtXI08JoxJnC/Gdbaf0fgvgfJ3rqb3lOzyMn1k5LsY3r/jrRPT43024iIeE7YgdxauwU4OQJjKVXWll3k5PrxW9if6ydryy4FchERPFR+2LFRLVKSfSQZqJbso2OjWgd9X2kXEUlUMWnRr4j26alM79+RrC276Nio1kGrcaVdRCSReSaQgwvmRQN41pZdfPvTPqVdRCRheSqQF1Z4FZ6c5CPZZ8jz26BpFxGReObZQF744Wdenp8rM9M49qjqxdIuIiLxzrOBPPDwc3+un2rJPnq1q18sgAdSLwruIhLPPBvIS3v4CXoAKiKJw7OBHIo//CysaN35nBXbtToXkbjk6UBemsKplySfYXb2dnLz/PiMYdRFrbi6Q1qshygiEhGeaQgqr0Dq5fauTbksowG5eW51nuu3jJi3Ro1DIhI34jaQgwvmgzufQK929fG5vWAA8PstWVt2xXBkIiKRE9eBPKB9eiqjLmpFss/gA1KqqdZcROJH3ObIi7q6QxpNjzlcDzxFJO7E7Ypcm2iJSKKIyxV5sBpyQHXlIhKX4jKQB9u7HNDGWiISl7wVyH/+GQ4/HApVoARTtH0/8GAz2NdERLzOO4HcWujRAw49FCZOhObNS7y0pPb9ET1a8vaa7+jeql7B17Qfi4h4nXcCud8Pl10GI0bASSfBLbfAyJFwxBFBLw+2d/mo+WvJyfWz7OsfaXrM4YDy5iLifd6pWklKgiFDYNMm6NsXJkyAE0+EadNckC9DsLx5Sbl0EREv8U4gD6hTB555BpYuheOPh+uug9NOg2XLSn1ZsDM/U2uk4DOuSUh5cxHxKu8F8oCMDPj4Y7ci//pr6NAB+veH778PennhvVcC5Yij5q/Fby0+n2FEj5ZKq4iIJ3k3kAP4fNCnj0u33H47vPCCS7dMnAi5ucUuD+y90j499aC0irWW3XtzYjABEZHweTuQBxxxBIwbB6tXQ2Ym3HortGkDixaV+JJgqRYRES+Kj0Ae0Lw5/Oc/8Npr8Ntv0KULXH45bNtW7NJAquXKzDR6tasfg8GKiERGfAVycM1CPXvCunUwahTMnw/NmsEDD8D//lfs8jkrtvPK0m30npqlfVlExJPiL5AHVK8O994L69e7RqIRI6BFC3j9dddcRMmt/CIiXhK/gTwgPZ3sR//B6+NeZF+1Q+Dii6FbN9iwgY6NapHsMxggyWeK5cm1g6KIeIF3Ojsr6MBOiH+gxiWP8Ha1NTSY+DC0bs1x1w2k5pGd+Snp0GL7twTbQVHliSJSFcX9irxw+mSf9fHGWZe5csU+fTh66lP85+kbuHjNu/hzcw9KrSjtIiJeEfeBPGiZYd26MHUqG15fyI6j6vLYvyYwa/owuvy6rfTXiYhUQcbmP/irTBkZGXb58uWV9n6l7XCY/dUufpo8lTOfG0+1XT9Av34wejTUqVPsddopUURiyRiTba3NKPb1RAjkIdmzx5UrTpoENWu6j2+6CZLdYwTlzEUk1koK5HGfWgnZkUfC+PEHukOHDoW2bQu6Q5UzF5GqSoG8qEB36Ny58OuvBd2hZxyyL2jOvHCJosoVRSQWlFopzb598OijMGYMGMO3N93Gm+dezSnNjqV9eiozlmxjxLw15Pkt1ZJ9YC25fkuyz3BZRgN6tauv9IuIRIxy5OHYuhXuuAPmzHF7oE+YQPbJZ3DFlCxy/Qf++xnAFvr4kGrKpYtI5EQ9R26MSTLGfGaMmR+pe1YZ6ekweza8845r/e/Zk9qX9yT9h28KLknyGaoluS5RcAFduXQRqQyRzJEPBdZH8H5Vz9lnw8qVMGEC9Tes5N/PDmb4ouc4av8+HrioFTMHnMpVHdJUfy4ilSoiqRVjTH3gBeAh4HZrbY/SrvdcaiWY77/nh1vuoNar08mtXZdq4x+F3r3B51O9uYhERbRTK48DdwElnoJsjBlgjFlujFm+c+fOCL1tDNWtS+1XXsJkZVHt+HS49lro1Amysw86iUhEJNrCDuTGmB7A99ba7NKus9ZOsdZmWGsz6tSpE+7bVh2ZmfDpp/D88/Dll3DKKTBgAMTDDysR8YRIrMhPB/5kjPkaeAXoYox5OQL39Q6fD/r2dZtx3XqrC+onnghPPEH2lztVWy4iURV2ILfW3mOtrW+tbQhcCbxnrf1z2CPzoiOPhMceg1WrICMDbrmFw07L5KNnZpV5ApGaiUSkouJ+P/KYaNECFizgrQeepvVjo5g5czj/anYGa1o9RPv0M4pdXngfFzUTiUh5RbRF31r7flkVKwnDGI7ucxU9bpzMhE69OXvzEq7pex48+GCxs0ML7+OSk2eZsURniIpI6LTXShS1T0/luUF/JGXUfXzx/jJ8F5zvzhFt0QLmzSs4OzSw97maiUSkIhTIoyxQitj69JNcd+jChQXdoXTvDhs30j49len9O6qZSEQqRHutxML+/fDkkzBypNuY69Zb3Ur98MPVTCQiJdKmWVXRf/8L99zjyhWPOQYeeQT+/OdiB0GLiIAOlqiajj4annsOliyBBg0OdIeuWBHrkYmIhyiQVwWZmZCVBc8+C5s3uxr0gQPhhx9iPTIR8QAF8qrC54Prr2flwiWs7NUH++yzrjv0ySchNxdQ05CIBKeGoCoke+tues9aT06TS2nevyMz18zkiJtvhilT2Dj8IXqvSdLhzyJSjFbkVUjhxqANqfV5adQzrmTxp59oeuWFPDpnLHX3/KAacxE5iAJ5FRJoDCqoI29cGy65BNavZ8fQuzj3iyzemzqQm7NmceqxNWM9XBGpIlR+WMWUVkf++UerqDH8Lhp/uAAaNYLHHye79elkffWj6s5FEoDqyOPJwoUwdCisX8/ixu25/+wBfFu3gfLmInFOdeTx5NxzYdUqPhp0D223r+ftqYO5beGzZK/ZGuuRiUgMKJB7VbVqVB92J90GPcMbLc9i4JI59L2+O7z8csFmXCKSGBTIPax9eiqTbu3Ofyc+zfrXFpCS3gCuuUbdoSIJRoHc4wK7Kzbvea7rDp06Fb74wnWH3ngj7FKZoki8UyCPI9nf7OHJRn9k5TtL4ZZbXFBv0gSeeqqgO1RE4o8CeZwIHBc3fsFGrpy1nuzbRsLKldCmDQwe7FboixfHepgiEgUK5HGicFdoQednq1bw7rswaxbs3g1//CNcfTVs3x7r4YpIBCmQx4liXaGB04WMgUsvhfXrYcQImDsXmjWDMWPg999jO2gRiQg1BMWRkE4X+uoruOMOeO01aNwYHn8cevTQyUQiHqDOTjnYwoXsGzSY6l9+wbendeb6NlfzxRH1tLOiSBWmzk45SPaJGWRePo6HOl/PkcuzeHPyIO5cNI1qe3/TzooiHqP9yBNMIIXy7U/7+M0m8UxmL+a16MywxdMYtGQ2vdYt4rf0MXBWY50dKuIRWpEnkMIlirOzt5PsMyQZ+Dm1Fr8/8xyz//4qhx2fRqOhA+CMM+Czz2I9ZBEJgVbkCaRwiWJenp8rM9M49qjqBx5wdkiDQZfA88/DPfe42vMBA+CBB6B27VgPX0RKoBV5AilaotirXX0Gdz7h4AebPh/06webNsHNN8Mzz7izQ9UdKlJlqWolwZRWZhj0e2vWuHb/RYvg5JNh0iQ488wYjFxEVH4opQrkz4Me7mytOzv0jjvgm2/gqqvgkUegfv3YDlokwaj8UEoVtMU/wBi47DLYsAHuvVfdoSJVjAK5AMFb/LO37ubJRZvJ3rrbXVSjBowa5dr9zz0Xhg93+7n861+xHbxIglPVihTo1a4+Jv/fQMmpluOPdy3+Cxa4/HmPHnDBBTBhgts2V0QqlVbkUpAff2XpNuascDsjlppqCejaFVavhnHj3Ba5rVq5ssVff63kGYgkNgVyCRq0S9xNsaiUFPcQdONG9xB07Fho2hRmzNDZoSKVRIFcggbt9umpTO/fkdu7Ng1tE6169WDaNPjkE/dx795u//NVqyplDiKJTOWHAoS4BW6o8vIOdIf++CMMHOi6Q2uVsKoXkZBErfzQGHOoMWapMWaVMWatMeb+cO8plS9wiHNEtq9NSoL+/V136ODBMGWK6w6dPNkFeRGJqEikVn4HulhrTwbaAN2MMR0jcF/xutRU1wn62Wdw0kkwaJDbv+Wjj2I9MpG4EnYgt06gTKFa/j96yiUHtG4N770Hr74Ku3a5nRX//Gf49ttYj0wkLkTkYacxJskYsxL4HlhorV0S5JoBxpjlxpjlO3fujMTbipcEukPXr3fdobNnu+qWhx9Wd6hImCISyK21edbaNkB9INMY0yrINVOstRnW2ow6depE4m3Fi2rWdN2h69bBOefA3Xe7Fftbb8V6ZCKeFdHyQ2vtT8D7QLdI3le8pVhrfzCNGsHrr8Pbb7utcy+4wHWIbt5caeMUiReRqFqpY4w5Kv/j6sA5wIZw7yveVPgUot5Ts0oP5gDdurnu0EcfhQ8+gJYt3R4u6g4VCVmZgdwY09oY83Ghz9sZY94rdEk9YJExZjWwDJcjnx/5oYoXhNTaX1RKCvzlL65c8cor3a6KzZrBzJnqDhUJQSgr8rVAY2NMUv7n44G/BL5prV1trW1rrT3JWtvKWjsqGgMVbwh0ifoMGGNIrZES+ovr1YMXXoCPP4ajj4arr4azzlJ3qEgZygzk1lo/Lpi3NMZcAmyz1q6I+sjEk9qnpzKiR0t8xpDnt4yav7bs9EpRp50GS5e6RqJ166BdO9dY9OOP0Rm0iMeFmiPPAk4H7gOGR200Ehd2783Bby2WcqRXikpKghtuONAdOnmy6w79xz/UHSpSRHkC+YPAa9ZadXFIqULeOTEUge7QlSvdNrk33ginnOLSLyIChB7IN+Ba8R+O4lgkTpR758RQtG7tDoB+5RXYuRM6dYJrroEdO8K/t4jHhRrIhwL3WGt/i+ZgJH5EdBOuAGPgiivc2aF//atr+W/a1B0Ere5QSWClBnJjTGNjzAagurX2hUoak0jpataEBx90D0K7dIFhw9yK/e23Yz0ykZgoNZBba7+01jaz1varrAGJhKxxY5g3zwVwY+D88+HCC9UdKglHJwRJlRBSW39JunWDzz933aHvv6/uUEk4CuQSc+Vu6w+mcHfoFVeoO1QSigK5xFT21t08/s4mft9fzrb+ktSrBy++6A6vqFtX3aGSEBTIJWYCK/GPN/+Axf1lDLvuPOD002HZMtdAtHatukMlrimQS8wU3mDLZ+D0JrUjV3cOrjt0wACXbhk0SN2hErcUyCVmCneApiT7uPWcE4MG8bAehAL84Q/w97+7s0NbtnTdoZmZ8MknYc5ApGowNgYPgjIyMuzy5csr/X2l6sneupusLbvo2KhWiUG899QscnL9pCT7gq7Yy7rHQayFf/7TPRj99lt3dujDD8Oxx0ZyWiJRYYzJttZmFP16ciwGIxLQPj211OAbbH/zwPXZW3czd8V2Zi3/hly/LTHQH8QYt+f5hRfC6NEwbpw7qWjECBg61FW/iHiMUitSpZW0AVdgpT5jyTZy8mz5K15q1oSHHnLdoZ07w113ue7Qf/87irMRiQ4FcqnSStqAK7BSDyQGDRWseGncGN54wx3+bC107w4XXQRffhnReYhEkwK5VHnBNuAq+qD0qg5pTO/fEaBiD0a7d3fdoWPHwrvvuoeif/sb/KZ94qTq08NO8ayiDzlDeTAakh07XKpl+nSoX9/l0S+/3OXXRWKopIedWpGLZxVdqYd68HOZ5YzHHgsvvwwffgi1a7uHo126uBW7SBWkQC5xI5STicq1r0unTrB8uWskWr0a2rSBIUPUHSpVjgK5xI1QTiYKddVeICkJBg6EL75wjURPPeW6Q6dMUXeoVBkK5BJXyjqZqMLnif7hD/Dkk7BiBbRo4YK7ukOlitDDTkk4gYekqTVS2L03J7SO0MKKdodecw08/DDZOYeG3mEqUgElPexUIJeEFJEKl19/dfuejxtHXkoK4ztcwbPtemAOOSSym3+J5FPVikgh5c6V5zuo4uWww1x36Nq1bGt1Cne9+yxvPTuEjpuWhbenukg5KZBLQiqaK0+tkVJmI1GJFS8nnMCPr8xhwJX3k2TzmPbqSHqPGQpbtlTSbCTRKZBLQgpUuFyZmcYZTepw35tryyxJLG0V3z49lYFjh/D2zIVsHzaSoz75wD0Uvfde2Lu3sqYlCUqBXBLanBXbeWfdf0NKs5RV8dI+PZVB57Wk/tj7YONGuPRSePBBd3boq6/q7FCJGgVySVjl3XgrlDr1Ascdd6A7tFYtdyD02WfDmjURn4eIArkkrIM23koyXJ2/8VZpAbqsOvViAt2hTz/tDoBu0wZuuQV2V/C0I5EgVH4oCa1cpwuFa9cud4DF5MmuwWj0aLj+etc9KhIClR+KBFHuFXY4atVy3aHZ2dC8uTsYukMH+PTT6L+3xDUFcpEyhH34c1Ft2sAHH8CMGfDdd3DaadCnD/zf/0Xm/pJwFMhFSlGu3RLLwxi46ipX3XL33fDKK24zrnHjICcnMu8hCUOBXKQUFe0ADdlhh7k2/zVr4Mwz4c474aSTYMGCyL6PxDUFcpESZG/dzY6f9pHsM+XfLbHQPUJKyzRpAvPnw5tvuu1xzzsPevZUd6iEJDncGxhjGgAvAscAfmCKtXZiuPcViaXCm2olJ/m4IrMBl7SrX66HohXamKtHDzj3XJgwwTUTtWjhjp27+26oUSPMWUm8isSKPBe4w1rbHOgIDDbGtIjAfUVipnBKJS/Pz3FHVS93ZUuF0zKHHOIC98aNcMkl8MADrjt01ix1h0pQYQdya+131toV+R//AqwHjgv3viKxVJ4DKEpKnxS+R5LPsOOnfcxYsi30CpjjjnMHQC9e7OrOL79c3aESVEQbgowxDYHFQCtr7c9FvjcAGACQlpbWfuvWrRF7X5FoCKVZqKT0SeC1v+zbz6dbdrH2u5/Jy7NY3FYAh1Qr5x7oeXnueLm//Q327IHBg+H+++GooyI1XfGAkhqCws6RF3qDw4A5wK1FgziAtXYKMAVcZ2ek3lckWtqnp5YZaEtKn/SemsXv+w/s41KYBXLyrw05kCclwaBBblV+773w97+7OvQxY1x3qE91C4ksIn/6xphquCA+3Vo7NxL3FPGCYCmYoptxBeMzptwVMIDrDn3qKdcd2qwZ3HCD6w7NyqrwHMT7wg7kxhgDPAust9Y+Fv6QRLwj2I6IgeAe+J/LZ6BakithNECyzzDqolbhbQvQpo3LnU+fDjt2wKmnQt++6g5NUGHnyI0xnYAPgc9x5YcAw621b5X0Gm2aJfGu6AHPqTVSWLNjDwboVc4yxjL9+qs7cm78eDj0UBg5EoYMgZSUyL2HVAk6fFkkRiJy0HMovvgCbr0V3nrLpV0mToSuXSP/PhIz2v1QJEai3uYf0KQJ/OtfrkN0/37XHXrxxfDVV9F5P6kyFMhFoqw8NekRccEFsHatq2hZuNB1h44cqbND45hSKyKVoDIOsMjeups5K7YfnIffvt21+M+cCWlpLo9+ySVu90XxHOXIRaqoigT5oq+ZsWQb977+OXn5/zunJPuYeUOhXPyHH7oHoKtWQZcuMGkStGwZpRlJtES9IUhEyq8iD0KLvmZEj5aMmLemIIjDgVx8wb3OOMOdHRroDj35ZLj5ZrjvPnWHxgHlyEViqCIPQou+5u013+Ev8pt10Fx8cjLcdJOrbrnhBuykSew9vjFfPzIJ/H7EuxTIRWKoIg9CCzccGWNoWe8I97mBJANdWxx9cFqlqFq1yL57NL36TWLdYcfQcNhQfmt3CixZctBlET/iTqJGOXKRGKtIjnzGkm2MmLcGv7UF6ZXde3NCvseTizYzfsFG/H7Lxevf58FPXqLmru9dd+jYsWT/L6Vyat+lXFRHLlJFtU9PZXDnE8oVKHfvzcFvbUF6ZffenHLdo+A3AZ/h7ZPPZtMHy2DYMNfyf+KJ7Ht4HP7fc6Jf+y4RoUAu4kHh7pdedI+Yti3TYOxYt9f56afT6enRvPX8EM7YurJyat8lLEqtiHhUOPull8pamD+f32++hUO2fc3ubj1IffoJaNgw8pOQclFqRSTOhJKSqdD2AMbAhRdyyMb18NBDpC5+D5o3d6WK6g6tkhTIReJYRapiClIx/90Hw4fDhg1w0UXuRKLmzWHOnDLPDi2t4kXVMJGn1IpInCtPVUxpqZiN/5xPreF/ofaWje7s0IkTg3aHlnaPStsJMk4ptSKSoAKHXWRt2VXmKrikVMyMJdu4YJWPUy99hFHdBpG7PNt1h952G/z0U0j3KOt7pdEqvnRq0ReJc+VZBQdSMftz/QWpmOytuxkxbw25fgsmiWltLuC4gX3p9+/n3Kp8+nS30+J114HPF/Qepd0/kuNPVFqRi8S58qyCgx1dl7VlF3n+AylYnzG0adsEJk92+7c0aQL9+0PHjrBkSbF7AAWr6WD3j+T4E5VW5CJxrryr4EBwDQTMjo1qcUg1Hzn7/fiKnjfarh189JFbld91lwvm111H+zFjaN/5hBJX0+VZUVdkFZ9o9LBTJAGE+8ATKPb6Yvf85Rd44AF4/HGoXh3uu4+nW3Xj0fe24LduH5jbuzZlcOcTojr+eKZtbEUSWHlWwcFSGUXr1UvMWz/yCPTr584Ovf12+p4wmaWZfVicdnJYq+nyruITjXLkInKQUGrPS81bN23qDoCeN4/q/lyen/FXFnw8iVnnHVssGKsaJTK0IheRgwQeSJaWyigzb20M/OlP0LUrjB9P49Gj4dyO7LhxKG+c92dOaX4cgKpRIkQ5chGpkHLlrb/5hh9vGsof5r/G9iPq8nDXGzj8qst5Zdk3YefPE4ly5CJSLmUF6nLlrRs0YObtj/DR4acwcuE/eGL2Q2ze9C6rTu3LhtT6IeXPQ90kLBEfiiqQi0gx0WjC6dioFhMbtaHHdZPos+pt7smayfx1N7Oq5zXYESNpW8r9QxlP4LCNPL/lkGqJlarRw04RKSZqTTjWkudL4qWMC1m7aBnmuutoM2cabc/JhOefL/Hs0LLGU7j71AI5+6PbOFTVHtIqkItIMRXZNbEsWVt2FQTaPL/loz0GpkyBpUuhUSO4/no47TRYtqzE8fiMO6c0tUZKsXsXPoDa53PXFA22kQjAgd8Oxi/YSO+pWVUimCu1IiLFhFK5Ul4lVrpkZMDHH8PLL7vu0MxMF9THjIG6dQvGM6JHy4LUyaj5a2l6zOEF4wrcOyfXj88Y+nc6nlHz1xZraopEuijYbwexTuEokItIUJFuwin1h4PPB9deCz17HugOnTOHb4YOY36nnmSeeEzBOaWW4gG06L1LSsUEvpYTRgCuilsGKJCLSKUp84fDEUfAo49Cv37sGTiYBqOG06X2ZMacdyNdh/QuNYAWvXfRazf+3y8E9v7yW4qlZ8ozh2A/kGJZMaNALiIxUWrga9aMl0dOZuVTL/G3d59h2vThbP7mA2aNfoTFOTXKDJbBgm3Wll0YwOIeDu7em1PhsRf9oRHrrXYVyEWk0oUS+Do2rs0TzU+l2/FtGbDsNQZ/OovkszvQ+p67oeNdZb5H0WCbWiOFJJ/B77ekVItsSiRryy5+3+8/qGJGgVxE4looDwwDq+q5K7bzdMqVzG7Zhb++/xzn33cfTJsGjz3mcurGBH2PwIo/tUYKa3bsYXb2dvzW4vMZRvRoGdFAm1ojhUDNjJ+Kp20qSoFcRCpdqA8MAymRXL/l28PrMORPw/i9/0Aufm4s9OoF55wDkya5Q6ELKbzi9xfZhcRgw0qrBLN7bw4+43LvPhNe2qYiVEcuIpWuPCcFFa1pT+t1Pnz2mQvgy5fDSSfBHXfAnj0Frym84i/MELm6+JLGmBKDShZtmiUiVV6JD0Z37oS//hWmTnU152PHwrXXkv3NnoNW5D4gOclwWUYDerWrX+oPjopWn5T1ukhUtZS0aVbYgdwY8xzQA/jeWtsqlNcokItIRC1fDkOGQFYWdOgATzxBdt0TCnLku/fmVPh0pEjk0iN135ICeSRSK9OAbhG4j4hIxQS6Q6dNg6+/hg4daP/AnQxueQRXd0grdsJRSaK1x0y0D5AOO5BbaxcDP0ZgLCIiFefzQZ8+sGkT3H47vPACnHgiTJwI+/eHdIto7DETzfsGRCRHboxpCMwvLbVijBkADABIS0trv3Xr1rDfV0SkRBs2wNChsGABtGzpHo526VLmy6LVoVmlc+T5N29IGYG8MOXIRaRSWAtvvAG33QZffQWXXALjx0N6eqxHViHRzJGLiFRNxsBFF8HatTBqlDsUunlz9/G+fbEeXcQokItI/KteHe6916VbevSAkSOhRQt47TW3ao+yaB9EEXYgN8bMBD4Fmhpjthtj+oU/LBGRKEhLg1dfhXffhZo1XXfoeefB+vUVvmVZQboyDqKIRNXKVdbaetbaatba+tbaZyMxMBGRqOnSBVaudBUtS5ce6A79+edy3SaUIB3t0kNQakVEElVyMtxyC3zxBfTtCxMmuHLFF14o8ezQokIJ0tEuPQQFchFJdHXqwDPPwJIl0LChC+qnn+66RcsQSpAuz74yFaW9VkREAvx+eOklGDYMvv8e+vWD0aNdsC9BZZ4MpPJDEZGyBLpDN2503aHTpkGTJq6ZKDc36Evap6eGvAVAtCiQi4gUdeSRMG4crF4NmZmuQ7RtW1i0KNYjC0qBXESkJM2bw3/+A3Pnwq+/umqXyy+HbdtiPbKDKJCLiJTGGLj4Yli3Du6/H958E5o1gwcfhP/9L9ajAxTIRURCU706jBjhukPPP991irZoAfPmVUp3aGkUyEVEyiM9HWbPhnfeccG9Z0/o3t0F+BhRIBcRqYizz3bdoY8/7k4mat0a7ryz3N2hkaBALiJSUdWquYqWTZtc2eL48dC0Kbz4YsjdoZGgQC4iEq66dd0B0EuWuNRLnz7QqRNkZ1fK2yuQi4hEyimnwCefwPPPw5dfus8HDICdO6P6tgrkIiKR5PO5/Vo2bXInEz3/vNuM64knSuwODfsto3JXEZFEd+SRLme+ejVkZLidFtu2he3bI/5WCuQiItHUvLk7AHruXGjcGOrVi/hbJEf8jiIicrBAd+jFF0fl9lqRi4h4nAK5iIjHKZCLiHicArmIiMcpkIuIeJwCuYiIxymQi4h4nAK5iIjHGRuDky2MMTuBrSFcWhv4IcrDqaoSde6JOm9I3Lkn6ryh/HNPt9bWKfrFmATyUBljlltrM2I9jlhI1Lkn6rwhceeeqPOGyM1dqRUREY9TIBcR8biqHsinxHoAMZSoc0/UeUPizj1R5w0RmnuVzpGLiEjZqvqKXEREyqBALiLicVUikBtjuhljNhpjNhtj7g7yfWOMmZT//dXGmHaxGGekhTDv3vnzXW2M+cQYc3IsxhkNZc290HWnGGPyjDGXVub4oiWUeRtjzjLGrDTGrDXGfFDZY4yWEP6+H2mMedMYsyp/7tfFYpyRZox5zhjzvTFmTQnfDz++WWtj+g+QBHwJNAJSgFVAiyLXnA+8DRigI7Ak1uOupHmfBqTmf9w9HuYd6twLXfce8BZwaazHXUl/5kcB64C0/M/rxnrclTj34cDD+R/XAX4EUmI99gjM/UygHbCmhO+HHd+qwoo8E9hsrd1irc0BXgEuKnLNRcCL1skCjjLGRP7gu8pV5ryttZ9Ya3fnf5oF1K/kMUZLKH/mAEOAOcD3lTm4KApl3lcDc6212wCstYk0dwscbowxwGG4QB6dY+crkbV2MW4uJQk7vlWFQH4c8E2hz7fnf62813hNeefUD/dTOx6UOXdjzHHAxcDkShxXtIXyZ34ikGqMed8Yk22MubbSRhddocz970BzYAfwOTDUWuuvnOHFVNjxrSocvmyCfK1oTWQo13hNyHMyxnTGBfJOUR1R5Qll7o8Dw6y1eW6BFhdCmXcy0B44G6gOfGqMybLWbor24KIslLmfB6wEugCNgYXGmA+ttT9HeWyxFnZ8qwqBfDvQoNDn9XE/kct7jdeENCdjzEnAVKC7tXZXJY0t2kKZewbwSn4Qrw2cb4zJtda+XikjjI5Q/67/YK39DfjNGLMYOBnweiAPZe7XAWOtSxxvNsZ8BTQDllbOEGMm7PhWFVIry4AmxpjjjTEpwJXAG0WueQO4Nv/pbkdgj7X2u8oeaISVOW9jTBowF7gmDlZkhZU5d2vt8dbahtbahsBs4CaPB3EI7e/6POAMY0yyMaYG0AFYX8njjIZQ5r4N95sIxpijgabAlkodZWyEHd9iviK31uYaY24G/oN7sv2ctXatMebG/O9PxlUtnA9sBvbifnJ7WojzHgHUAp7KX5nm2jjYJS7EucedUOZtrV1vjPk3sBrwA1OttUHL1rwkxD/zB4BpxpjPcemGYdZaz29va4yZCZwF1DbGbAdGAtUgcvFNLfoiIh5XFVIrIiISBgVyERGPUyAXEfE4BXIREY9TIBcR8TgFchERj1MgFxHxOAVyEcAYs8gYc27+xw8aYybFekwioYp5Z6dIFTESGGWMqQu0Bf4U4/GIhEydnSL58k/jOQw4y1r7S6zHIxIqpVZEAGNMa6Ae8LuCuHiNArkkvPzTWKbjTmr5zRhzXoyHJFIuCuSS0PK3ip0L3GGtXY/bge++mA5KpJyUIxcR8TityEVEPE6BXETE4xTIRUQ8ToFcRMTjFMhFRDxOgVxExOMUyEVEPO7/AfgtZAN26GvqAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def CGLS(w0, X, y, nit = 20):\n",
    "    ww = [w0]\n",
    "    w = w0.copy()\n",
    "    s = y - X.dot(w)\n",
    "    r = X.T.dot(s)\n",
    "    p = r.copy()\n",
    "    \n",
    "    rr = r.dot(r)\n",
    "    \n",
    "    for k in range(nit):\n",
    "        Xp  = X.dot(p)\n",
    "        al  = rr / Xp.dot(Xp)\n",
    "        w   = w + al * p\n",
    "        s   = s - al * Xp\n",
    "        r   = X.T.dot(s)\n",
    "        rrn = r.dot(r)\n",
    "        be  = rrn / rr\n",
    "        p   = r + be * p\n",
    "        \n",
    "        rr = rrn\n",
    "        ww.append(w)\n",
    "        \n",
    "    return(ww)\n",
    "\n",
    "\n",
    "wX = np.eye(npar)\n",
    "X = np.array([g(x, wX[k]) for k in range(npar)]).T\n",
    "\n",
    "w0 = np.zeros(npar)\n",
    "w = CGLS(w0, X, y)\n",
    "evalw(w[-1], x, y)\n",
    "print(\"w = {}, l(w) = {:f}\".format(w[-1], l(w[-1])))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-24T09:28:14.408706Z",
     "iopub.status.busy": "2021-09-24T09:28:14.408344Z",
     "iopub.status.idle": "2021-09-24T09:28:14.473617Z",
     "shell.execute_reply": "2021-09-24T09:28:14.473325Z"
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "w = [-0.6817 -2.8653 -1.1836], l(w) = 0.055920\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAEGCAYAAAB4lx7eAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAmi0lEQVR4nO3de5zV0/7H8dfac6EIc7oQ1aSkO12mC+Iot0pOyV1ISiSJHFIocikUyq2TEEfl6EJ0dOQSjst0GSrdJUqc3ykptzqmmb1+f6zZNc3smdl79t6z57v3+/l49DCX7/7utVSfWX2+6/NZxlqLiIh4ly/eAxARkcgokIuIeJwCuYiIxymQi4h4nAK5iIjHpcbjTWvUqGHr168fj7cWEfGsnJycH621NYt+PS6BvH79+ixbtiweby0i4lnGmM3Bvh6VQG6M+Rb4FcgH8qy1WdG4r4iIlC2aK/LO1tofo3g/EREJgR52ioh4XLQCuQUWGmNyjDEDg11gjBlojFlmjFm2ffv2KL2tiIhEK5CfYq1tA3QDBhtjTit6gbV2irU2y1qbVbNmsYeuIiJSTlEJ5NbaHwr+uw14DWgfjfuKiEjZIg7kxphDjDHVAh8DZwOrIr1vMDmbd/LUoo3kbN4Zi9uLiHhSNHatHAm8ZowJ3G+GtfZfUbjvAXI276TP1Gxy8/ykp/qYPqAjbTMzov02IiKeE3Egt9ZuAk6MwlhKlb1pB7l5fvwW9ub5yd60Q4FcRAQPbT/s2KA66ak+Ugykpfro2KD6Ad9X2kVEklVcSvTLo21mBtMHdCR70w46Nqh+wGpcaRcRSWaeCeTggnnRAJ69aQff79qjtIuIJC1PBfLCCq/CU1N8pPoM+X4bNO0iIpLIPBvICz/8zM/3c2n7ehx9RJViaRcRkUTn2UAeePi5N89PWqqP3m3qFAvggdSLgruIJDLPBvLSHn6CHoCKSPLwbCCH4g8/Cyu673zO51u1OheRhOTpQF6awqmXFJ9hds5W8vL9+IxhTM8WXN6hXryHKCISFZ4pCApXIPUy7OzGXJRVl7x8tzrP81tGzVulwiERSRgJG8jBBfPBnY+jd5s6+FwvGAD8fkv2ph1xHJmISPQkdCAPaJuZwZieLUj1GXxAepr2motI4kjYHHlRl3eoR+OjqumBp4gknIRdkauJlogki4RckQfbQw5oX7mIJKSEDOTBepcDaqwlIgnJW4H8l1+gWjUotAMlmKLl+4EHm8G+JiLidd4J5NZCjx5w8MEwcSI0bVripSWV74/q0ZwFq/5Dtxa1931N/VhExOu8E8j9frjoIhg1Ck44AW66CUaPhsMOC3p5sN7lY+avJjfPz9Jvf6LxUdUA5c1FxPu8s2slJQWGDIENG+Dqq+Gxx+D442HaNBfkyxAsb15SLl1ExEu8E8gDataEZ5+FJUvg2GOhXz84+WRYurTUlwU78zOjajo+44qElDcXEa/yXiAPyMqCTz5xK/Jvv4UOHWDAANi2LejlhXuvBLYjjpm/Gr+1+HyGUT2aK60iIp7k3UAO4PNB374u3TJsGLz4oku3TJwIeXnFLg/0XmmbmXFAWsVay87duXGYgIhI5LwdyAMOOwzGj4eVK6F9e7j5ZmjVChYtKvElwVItIiJelBiBPKBpU3j7bXjtNfj9d+jSBS6+GLZsKXZpINVyaft69G5TJw6DFRGJjsQK5OCKhXr1gjVrYMwYmD8fmjSB++6D//2v2OVzPt/KK0u20GdqtvqyiIgnJV4gD6hSBe6+G9audYVEo0ZBs2bw+uuuuIiSS/lFRLwkcQN5QGYmOY/8jdfHv8SetIPg/POha1dYt46ODaqT6jMYIMVniuXJ1UFRRLzAO5Wd5bS/E+KfqHrBwyxIW0XdiQ9By5Yc0+86Djm8M7tSDi7WvyVYB0VtTxSRyijhV+SF0yd7rI83Tr/IbVfs25cjpz7N289cy/mr3sOfl3dAakVpFxHxioQP5EG3GdaqBVOnsu71d/jhiFo8+s/HmDV9OF1+/67014mIVELGFjz4q0hZWVl22bJlFfZ+pXU4zPlmB7smT+W05yeQtuNHVx36wANQs2ax16lToojEkzEmx1qbVezryRDIQ/Lzz2674qRJcOih7uNBgyDVPUZQzlxE4q2kQJ7wqZWQHX44TJjgqkOzslyb3Nat4YMPAOXMRaTyUiAvqmlTWLgQ5s6F336Dzp3hkks49eA9QXPmhbcoaruiiMSDUiul2bMHHnkExo4FY/j+hlt486zLadfkaNpmZjBj8RZGzVtFvt+SluoDa8nzW1J9houy6tK7TR2lX0QkapQjj8TmzXDrrTBnjuuB/thj5Jx4KpdMySbPv///nwFsoY8PSlMuXUSiJ+Y5cmNMijHmC2PM/Gjds9LIzITZs+Hdd13pf69e1Li4F/V/3N+MK8VnSEtxVaLgArpy6SJSEaKZIx8KrI3i/SqfM86A5cvhsceos245C567kZGLnueIvXu4r2cLZg48ics61NP+cxGpUFFJrRhj6gAvAg8Aw6y1PUq73nOplWC2bePHIcOo8ep09tY8krTxD8MVV4DPp/3mIhITsU6tPA7cDpR4CrIxZqAxZpkxZtn27duj9LZxVKsWNf7xMixeTNqxme6kok6dICfngJOIRERiLeJAbozpAWyz1uaUdp21doq1Nstam1WzZs1I37byaN8ePvsMnn8evv4a2rWDgQMhEX5YiYgnRGNFfgrwF2PMt8ArQBdjzMtRuK93+HzQr59rxnXzzfDCC+7s0CeeIOfr7dpbLiIxFXEgt9aOsNbWsdbWBy4F3rfWXhHxyLzo8MPh0UdhxYp91aHVTmrHx8/OKvMEIhUTiUh5JXw/8rho1gwWLmTBmKdp+egYZs4cyT+bnMqqFg/QNvPUYpcX7uOiYiIRCVdUS/SttR+UtWMlaRhDrasv59xBf+PxTpdzxsbFXNGvq+usWOTs0MJ9XHLzLTMW6wxREQmdeq3EUNvMDJ4f9GfSxtzLV4uWkNK9G9x1FzRvDm+8se/s0EDvcxUTiUh5KJDHWGArYstOJ7rq0HfegYMOgp49oXt3WL+etpkZTB/QUcVEIlIu6rUSD3v3wlNPwejRrjHXzTfD3XdDtWoqJhKREqlpVmX03//CiBFuu+JRR8HDBdWhRQ6CFhEBHSxROR15pCskWrwY6taFq65y1aGffx7vkYmIhyiQVwbt20N2Njz3HHz1lduDft118OOP8R6ZiHiAAnll4fPBNdew/N0lLO/dF/vcc6469KmnIC8PUNGQiASngqBKJGfzTvrMWktuowtpOqAjM1fN5LAbb4QpU1g/8gH6rErR4c8iUoxW5JVI4cKgdRl1+PuYZ92WxV27aHzpeTwyZxy1fv5Re8xF5AAK5JVIoDBo3z7yhjXgggtg7Vp+uOk2zt7wGe9PvY4bs2dx0tGHxHu4IlJJaPthJVPaPvIvP15O1RG30/Djd6BhQ3d2aMtTyP7mJ+07F0kC2keeSN55B4YOhbVr+bBhFmPOuJbva9VV3lwkwWkfeSI56yxYsYKPB42gzXerWTB1MLe88xw5qzbHe2QiEgcK5F6VlkaV4bfR7YZneaP56Vy3eA5XX9MNXn55XzMuEUkOCuQe1jYzg4k3d+e/E59h7WsLSc+sC1deqepQkSSjQO5xge6KTXud5apDp07dXx16/fWqDhVJAgrkCSTnu595qsGfWf7uErjpJhfUi1SHikjiUSBPEIHj4iYsXM+ls9aSc8toWL4cWrWCG2+Etm3ho4/iPUwRiQEF8gRRuCp0X+Vnixbw3nswaxbs2gV//jNcdhls3Rrv4YpIFCmQJ4hiVaGB04WMgQsvhLVrYdQoeO01aNwYHnwQ/vgjvoMWkahQQVACCel0oW++gWHD4PXXXXXo449Djx46mUjEA1TZKQdauJA9N9xIla+/4vuTO3NNq8v56rDa6qwoUompslMOkNO4He0vHs8Dna/h8GXZvDl5ELctmkba7t/VWVHEY9SPPMkEUijf79rD7zaFZ9v3Zl6zzgz/aBqDFs+m95pF/J45Fk5vqLNDRTxCK/IkUniL4uycraT6DCkGfsmozh/PPs/sJ1/l0GPr0WDoQDj1VPjii3gPWURCoBV5Eim8RTE/38+l7etx9BFV9j/g7FAPBl0AL7wAI0a46tCBA+G++6BGjXgPX0RKoBV5Eim6RbF3mzoM7nzcgQ82fT7o3x82bHCFRM8+66pDn35a1aEilZR2rSSZ0rYZBv3eqlWu3H/RIjjxRJg0CU47LQ4jFxFtP5RSBfLnQQ93ttadHXrrrfDdd6469OGHoU6d+A5aJMlo+6GUKmiJf4AxcNFFsG4d3H03zJ0LTZrA2LGqDhWpBBTIBQhe4p+zeSdPLdpIzuad7qKqVWHMGFfuf9ZZMHKk6+fyz3/Gd/AiSU67VmSf3m3qYAr+C5Scajn2WNezZeFClz/v0QPOPRceewwaNYrfBESSlFbksi8//sqSLcz53HVGLDXVEnD22bByJYwf71rktmjhti3+9lsFz0AkuSmQS9CgXWI3xaLS091D0PXr3UPQceNcd8UZM3R2qEgFUSCXoEG7bWYG0wd0ZNjZjUNrolW7NkybBp9+6j7u08f1P1+xokLmIJLMtP1QgBBb4IYqP39/dehPP8F117nq0OolrOpFJCQx235ojDnYGLPEGLPCGLPaGHNvpPeUihc4xDkq7WtTUmDAAFcdOngwTJniqkMnT3ZBXkSiKhqplT+ALtbaE4FWQFdjTMco3Fe8LiPDVYJ+8QWccAIMGuT6t3z8cbxHJpJQIg7k1glsU0gr+KWnXLJfy5bw/vvw6quwY4frrHjFFfD99/EemUhCiMrDTmNMijFmObANeMdauzjINQONMcuMMcu2b98ejbcVLwlUh65d66pDZ892u1seekjVoSIRikogt9bmW2tbAXWA9saYFkGumWKtzbLWZtWsWTMabytedMghrjp0zRo480y44w63Yn/rrXiPTMSzorr90Fq7C/gA6BrN+4q3FCvtD6ZBA3cA9IIFrnXuuee6CtGNGytsnCKJIhq7VmoaY44o+LgKcCawLtL7ijcVPoWoz9Ts0oM5QNeurjr0kUfgww+heXPXw0XVoSIhKzOQG2NaGmM+KfR5G2PM+4UuqQ0sMsasBJbicuTzoz9U8YKQSvuLSk+Hv/7VbVe89FLXVbFJE5g5U9WhIiEIZUW+GmhojEkp+HwC8NfAN621K621ra21J1hrW1hrx8RioOINgSpRnwFjDBlV00N/ce3a8OKL8MkncOSRcPnlcPrpqg4VKUOZgdxa68cF8+bGmAuALdbaz2M+MvGktpkZjOrRHJ8x5PstY+avLju9UtTJJ8OSJa6QaM0aaNPGFRb99FNsBi3icaHmyLOBU4B7gJExG40khJ27c/FbiyWM9EpRKSlw7bX7q0MnT3bVoX/7m6pDRYoIJ5DfD7xmrVUVh5Qq5M6JoQhUhy5f7trkXn89tGvn0i8iAoQeyNfhSvEfiuFYJEGE3TkxFC1bugOgX3kFtm+HTp3gyivhhx8iv7eIx4UayIcCI6y1v8dyMJI4otqEK8AYuOQSd3bonXe6kv/Gjd1B0KoOlSRWaiA3xjQ0xqwDqlhrX6ygMYmU7pBD4P773YPQLl1g+HC3Yl+wIN4jE4mLUgO5tfZra20Ta23/ihqQSMgaNoR581wANwa6d4fzzlN1qCQdnRAklUJIZf0l6doVvvzSVYd+8IGqQyXpKJBL3IVd1h9M4erQSy5RdagkFQVyiauczTt5/N0N/LE3zLL+ktSuDS+95A6vqFVL1aGSFBTIJW4CK/FPNv6Ixf1hjHjfecApp8DSpa6AaPVqVYdKQlMgl7gp3GDLZ+CURjWit+8cXHXowIEu3TJokKpDJWEpkEvcFK4ATU/1cfOZxwcN4hE9CAX405/gySfd2aHNm7vq0Pbt4dNPI5yBSOVgbBweBGVlZdlly5ZV+PtK5ZOzeSfZm3bQsUH1EoN4n6nZ5Ob5SU/1BV2xl3WPA1gL//iHezD6/ffu7NCHHoKjj47mtERiwhiTY63NKvr11HgMRiSgbWZGqcE3WH/zwPU5m3cy9/OtzFr2HXl+W2KgP4Axruf5eefBgw/C+PHupKJRo2DoULf7RcRjlFqRSq2kBlyBlfqMxVvIzbfh73g55BB44AFXHdq5M9x+u6sO/de/YjgbkdhQIJdKraQGXIGVeiAxaCjnjpeGDeGNN9zhz9ZCt27Qsyd8/XVU5yESSwrkUukFa8BV9EHpZR3qMX1AR4DyPRjt1s1Vh44bB++95x6K3nUX/K4+cVL56WGneFbRh5yhPBgNyQ8/uFTL9OlQp47Lo198scuvi8RRSQ87tSIXzyq6Ug/14OcytzMefTS8/DL8+99Qo4Z7ONqli1uxi1RCCuSSMEI5mSisvi6dOsGyZa6QaOVKaNUKhgxRdahUOgrkkjBCOZko1FX7PikpcN118NVXrpDo6adddeiUKaoOlUpDgVwSSlknE5X7PNE//Qmeego+/xyaNXPBXdWhUknoYackncBD0oyq6ezcnRtaRWhhRatDr7wSHnqInNyDQ68wFSmHkh52KpBLUorKDpfffnN9z8ePJz89nQkdLuG5Nj0wBx0U3eZfIgW0a0WkkLBz5QUO2PFy6KGuOnT1ara0aMft7z3HW88NoeOGpZH1VBcJkwK5JKWiufKMqullFhKVuOPluOP46ZU5DLz0XnzWz7RXR9Nn7FDYtKmCZiPJToFcklJgh8ul7etxaqOa3PPm6jK3JJa2im+bmcF144bwr5kL2Tp8NEd8+qF7KHr33bB7d0VNS5KUArkktTmfb+XdNf8NKc1S1o6XtpkZDDqnOXXG3eMOs7jwQrj/fnd26Kuv6uxQiRkFckla4TbeCmWf+j6Fq0OrV3cHQp9xBqxaFfV5iCiQS9I6oPFWiuHygsZbpQXosvapFxOoDn3mGXcAdKtWcNNNsLOcpx2JBKHth5LUwjpdKFI7drgDLCZPdgVGY8dCv36uelQkBNp+KBJE2CvsSFSv7qpDc3KgaVO49lro0AE++yz27y0JTYFcpAwRH/5cVKtW8OGHMGMG/N//wcknQ9++7mORclAgFylFWN0Sw2EMXHYZrFsHI0bAK6+4Zlzjx0NubnTeQ5KGArlIKcpbARqyQw91h0CvWgWnnQa33QYnnABvvx3d95GEpkAuUoKczTv5YdceUn0m/G6Jhe4RUlqmUSOYP9/9ys+Hrl2hVy9Vh0pIUiO9gTGmLvAScBTgB6ZYaydGel+ReCrcVCs1xccl7etyQZs6YT0ULVdjrnPPhTPPhMcec8VEzZq5VfqIEVC1aoSzkkQVjRV5HnCrtbYp0BEYbIxpFoX7isRN4ZRKfr6fY46oEvbOlnKnZQ46CO64A9avhwsu2F8dOmuWqkMlqIgDubX2P9bazws+/hVYCxwT6X1F4imcAyhKSp8UvkeKz/DDrj3MWLwl9B0wxxzjDoD+6CO37/zii1UdKkFFtSDIGFMf+AhoYa39pcj3BgIDAerVq9d28+bNUXtfkVgIpViopPRJ4LW/7tnLZ5t2sPo/v5Cfb7G4VgAHpYXZAz0/3x0vd+ed8MsvMHgw3HsvHHFEtKYrHlBSQVDEOfJCb3AoMAe4uWgQB7DWTgGmgKvsjNb7isRK28yMMgNtSemTPlOz+WPv/j4uhVkgt+DakAN5SgoMGuRW5XffDU8+6fahjx0L11wDPu1bSGZR+d03xqThgvh0a+3caNxTxAuCpWCKNuMKxmdM2DtgAFcd+vTTrn9L48b7q0Ozs8s9B/G+iAO5McYAzwFrrbWPRj4kEe8I1hExENwDf7l8BtJS3BZGA6T6DGN6toisLUDr1q6z4ssvu3NDTzoJrr5a1aFJKuIcuTGmE/Bv4Evc9kOAkdbat0p6jZpmSaIresBzRtV0Vv3wMwboHeY2xjL9+qs7cu7RR+Hgg+Gee2DIEEhLi957SKWgw5dF4iQqBz2HYsMGuPlmWLDAbVecNAnOOiv67yNxo+6HInES8zL/gOOPh7fegjffhL174eyz4fzz4ZtvYvN+UmkokIvEWDh70qOiRw9Yvdr1cFm40LXMHTVKZ4cmMKVWRCpARRxgkbN5J3M+33pgHn7rVrj9dpg5E+rVgwkTXLWoMTEZg8SWcuQilVR5gnzR18xYvIW7X/+S/IK/zumpPmZeWygX/+9/uwegK1ZAly4uf968eYxmJLES84IgEQlfeR6EFn3NqB7NGTVv1b4gDvtz8fvudeqpbu/5lClw111w4olw441uh4uqQz1POXKROCrPg9Cir1mw6j/4i/zLOmguPjUVbrgBvvoKrr0WO2kSu49tyLcPTwK/H/EuBXKROCrPg9DCBUfGGJrXPsx9biDFwNnNjjwwrVJU9erk3PEgvftPYs2hR1F/+FB+b9MOFi8+4LKoH3EnMaMcuUiclSdHPmPxFkbNW4Xf2n3plZ27c0O+x1OLNjJh4Xr8fsv5az/g/k//ziE7trnq0HHjyPlfesXsfZewaB+5SCXVNjODwZ2PCytQ7tydi9/afemVnbtzw7rHvn8J+AwLTjyDDR8uheHDXdvc449nz0Pj8f+RG/u97xIVCuQiHhRpv/SiPWJaN68H48a5XuennEKnZx7krReGcOrm5RWz910iotSKiEdF0i+9VNbC/Pn8ceNNHLTlW3Z27UHGM09A/frRn4SERakVkQQTSkqmXO0BjIHzzuOg9WvhgQfI+Oh9Vx16zz2qDq2kFMhFElh5dsXsS8X8dw+MHAnr1kHPnu5EoqZNYc6cMs8OLW3Hi3bDRJ9SKyIJLpxdMaWlYtb/Yz7VR/6VGpvWu7NDJ04MWh1a2j0qrBNkglJqRSRJBQ67yN60o8xVcEmpmBmLt3DuCh8nXfgwY7oOIm9ZjqsOveUW2LUrpHuU9b3SaBVfOpXoiyS4cFbBgVTM3jz/vlRMzuadjJq3ijy/BZPCtFbncsx1V9P/X8+7Vfn06e7s0H79wOcLeo/S7h/N8ScrrchFElw4q+BgR9dlb9pBvn9/CtZnDK1aN4LJk13/lkaNYMAA6NgRFi8udg9g32o62P2jOf5kpRW5SIILdxUcCK6BgNmxQXUOSvORu9ePr+h5o23awMcfu1X57be7YN6vH23HjqVt5+NKXE2Hs6Iuzyo+2ehhp0gSiPSBJ1Ds9cXu+euvcN998PjjUKUK3HMPz7ToyiPvb8JvXR+YYWc3ZnDn42I6/kSmNrYiSSycVXCwVEbR/eol5q0ffhj693dnhw4bxtXHTWZJ+758VO/EiFbT4a7ik41y5CJygFD2npeat27c2J0dOm8eVfx5vDDjThZ+MolZ5xxdLBhrN0p0aEUuIgcIPJAsLZVRZt7aGPjLX9wB0BMm0PDBB+Gsjvxw/VDeOOcK2jU9BkC7UaJEOXIRKZew8tbffcdPNwzlT/NfY+thtXjo7GupdtnFvLL0u4jz58lEOXIRCUtZgTqsvHXduswc9jAfV2vH6Hf+xhOzH2DjhvdYcdLVrMuoE1L+PNQmYcn4UFSBXESKiUURTscG1ZnYoBU9+k2i74oFjMieyfw1N7Ki15XYUaNpXcr9QxlP4LCNfL/loLTkStXoYaeIFBOzIhxryfel8Pes81i9aCmmXz9azZlG6zPbwwsvlHh2aFnjKVx9aoHcvbEtHKpsD2kVyEWkmPJ0TSxL9qYd+wJtvt/y8c8GpkyBJUugQQO45ho4+WRYurTE8fiMO6c0o2p6sXsXPoDa53PXFA220QjAgX8dTFi4nj5TsytFMFdqRUSKCWXnSrhK3OmSlQWffAIvv+yqQ9u3d0F97FioVWvfeEb1aL4vdTJm/moaH1Vt37gC987N8+MzhgGdjmXM/NXFipqikS4K9q+DeKdwFMhFJKhoF+GU+sPB54OrroJevfZXh86Zw3dDhzO/Uy/aH3/UvnNKLcUDaNF7l5SKCXwtN4IAXBlbBiiQi0iFKfOHw2GHwSOPQP/+/HzdYOqOGUmXGpMZe871nD2kT6kBtOi9i167/v9+JdD7y28plp4JZw7BfiDFc8eMArmIxEWpga9JE14ePZnlT/+du957lmnTR7Lxuw+Z9eDDfJRbtcxgGSzYZm/agQEs7uHgzt255R570R8a8W61q0AuIhUulMDXsWENnmh6El2Pbc3Apa8x+LNZpJ7RgZYj7oCOt5f5HkWDbUbVdFJ8Br/fkp4W3ZRI9qYd/LHXf8COGQVyEUlooTwwDKyq536+lWfSL2V28y7c+cHzdL/nHpg2DR591OXUjQn6HoEVf0bVdFb98DOzc7bitxafzzCqR/OoBtqMqukE9sz4KX/aprwUyEWkwoX6wDCQEsnzW76vVpMhfxnOHwOu4/znx0Hv3nDmmTBpkjsUupDCK35/kS4kBhtRWiWYnbtz8RmXe/eZyNI25aF95CJS4cI5KajonvZ6vbvDF1+4AL5sGZxwAtx6K/z8877XFF7xF2aI3r74ksaYHoedLGqaJSKVXokPRrdvhzvvhKlT3Z7zcePgqqvI+e7nA1bkPiA1xXBRVl16t6lT6g+O8u4+Ket10djVUlLTrIgDuTHmeaAHsM1a2yKU1yiQi0hULVsGQ4ZAdjZ06ABPPEFOreP25ch37s4t9+lI0cilR+u+JQXyaKRWpgFdo3AfEZHyCVSHTpsG334LHTrQ9r7bGNz8MC7vUK/YCUcliVWPmVgfIB1xILfWfgT8FIWxiIiUn88HffvChg0wbBi8+CIcfzxMnAh794Z0i1j0mInlfQOikiM3xtQH5peWWjHGDAQGAtSrV6/t5s2bI35fEZESrVsHQ4fCwoXQvLl7ONqlS5kvi1WFZqXOkRfcvD5lBPLClCMXkQphLbzxBtxyC3zzDVxwAUyYAJmZ8R5ZucQyRy4iUjkZAz17wurVMGaMOxS6aVP38Z498R5d1CiQi0jiq1IF7r7bpVt69IDRo6FZM3jtNbdqj7FYH0QRcSA3xswEPgMaG2O2GmP6Rz4sEZEYqFcPXn0V3nsPDjnEVYeecw6sXVvuW5YVpCviIIpo7Fq5zFpb21qbZq2tY619LhoDExGJmS5dYPlyt6NlyZL91aG//BLWbUIJ0rHeeghKrYhIskpNhZtugq++gquvhscec9sVX3yxxLNDiwolSMd66yEokItIsqtZE559FhYvhvr1XVA/5RRXLVqGUIJ0OH1lyku9VkREAvx++PvfYfhw2LYN+veHBx90wb4EFXkykLYfioiUJVAdun69qw6dNg0aNXLFRHl5QV/SNjMj5BYAsaJALiJS1OGHw/jxsHIltG/vKkRbt4ZFi+I9sqAUyEVEStK0Kbz9NsydC7/95na7XHwxbNkS75EdQIFcRKQ0xsD558OaNXDvvfDmm9CkCdx/P/zvf/EeHaBALiISmipVYNQoVx3avburFG3WDObNq5Dq0NIokIuIhCMzE2bPhnffdcG9Vy/o1s0F+DhRIBcRKY8zznDVoY8/7k4matkSbrst7OrQaFAgFxEpr7Q0t6Nlwwa3bXHCBGjcGF56KeTq0GhQIBcRiVStWu4A6MWLXeqlb1/o1Alycirk7RXIRUSipV07+PRTeOEF+Ppr9/nAgbB9e0zfVoFcRCSafD7Xr2XDBncy0QsvuGZcTzxRYnVoxG8Zk7uKiCS7ww93OfOVKyEry3VabN0atm6N+lspkIuIxFLTpu4A6LlzoWFDqF076m+RGvU7iojIgQLVoeefH5Pba0UuIuJxCuQiIh6nQC4i4nEK5CIiHqdALiLicQrkIiIep0AuIuJxCuQiIh5nbBxOtjDGbAc2h3BpDeDHGA+nskrWuSfrvCF5556s84bw555pra1Z9ItxCeShMsYss9ZmxXsc8ZCsc0/WeUPyzj1Z5w3Rm7tSKyIiHqdALiLicZU9kE+J9wDiKFnnnqzzhuSde7LOG6I090qdIxcRkbJV9hW5iIiUQYFcRMTjKkUgN8Z0NcasN8ZsNMbcEeT7xhgzqeD7K40xbeIxzmgLYd59Cua70hjzqTHmxHiMMxbKmnuh69oZY/KNMRdW5PhiJZR5G2NON8YsN8asNsZ8WNFjjJUQ/rwfbox50xizomDu/eIxzmgzxjxvjNlmjFlVwvcjj2/W2rj+AlKAr4EGQDqwAmhW5JruwALAAB2BxfEedwXN+2Qgo+Djbokw71DnXui694G3gAvjPe4K+j0/AlgD1Cv4vFa8x12Bcx8JPFTwcU3gJyA93mOPwtxPA9oAq0r4fsTxrTKsyNsDG621m6y1ucArQM8i1/QEXrJONnCEMSb6B99VrDLnba391Fq7s+DTbKBOBY8xVkL5PQcYAswBtlXk4GIolHlfDsy11m4BsNYm09wtUM0YY4BDcYE8NsfOVyBr7Ue4uZQk4vhWGQL5McB3hT7fWvC1cK/xmnDn1B/3UzsRlDl3Y8wxwPnA5AocV6yF8nt+PJBhjPnAGJNjjLmqwkYXW6HM/UmgKfAD8CUw1Frrr5jhxVXE8a0yHL5sgnyt6J7IUK7xmpDnZIzpjAvknWI6oooTytwfB4Zba/PdAi0hhDLvVKAtcAZQBfjMGJNtrd0Q68HFWChzPwdYDnQBGgLvGGP+ba39JcZji7eI41tlCORbgbqFPq+D+4kc7jVeE9KcjDEnAFOBbtbaHRU0tlgLZe5ZwCsFQbwG0N0Yk2etfb1CRhgbof5Z/9Fa+zvwuzHmI+BEwOuBPJS59wPGWZc43miM+QZoAiypmCHGTcTxrTKkVpYCjYwxxxpj0oFLgTeKXPMGcFXB092OwM/W2v9U9ECjrMx5G2PqAXOBKxNgRVZYmXO31h5rra1vra0PzAZu8HgQh9D+rM8DTjXGpBpjqgIdgLUVPM5YCGXuW3D/EsEYcyTQGNhUoaOMj4jjW9xX5NbaPGPMjcDbuCfbz1trVxtjri/4/mTcroXuwEZgN+4nt6eFOO9RQHXg6YKVaZ5NgC5xIc494YQyb2vtWmPMv4CVgB+Yaq0Num3NS0L8Pb8PmGaM+RKXbhhurfV8e1tjzEzgdKCGMWYrMBpIg+jFN5Xoi4h4XGVIrYiISAQUyEVEPE6BXETE4xTIRUQ8ToFcRMTjFMhFRDxOgVxExOMUyEUAY8wiY8xZBR/fb4yZFO8xiYQq7pWdIpXEaGCMMaYW0Br4S5zHIxIyVXaKFCg4jedQ4HRr7a/xHo9IqJRaEQGMMS2B2sAfCuLiNQrkkvQKTmOZjjup5XdjzDlxHpJIWBTIJakVtIqdC9xqrV2L68B3T1wHJRIm5chFRDxOK3IREY9TIBcR8TgFchERj1MgFxHxOAVyERGPUyAXEfE4BXIREY/7f7CgY/j3JOx3AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "w0 = np.ones(npar)\n",
    "w = CGLS(w0, X, y)\n",
    "evalw(w[-1], x, y)\n",
    "print(\"w = {}, l(w) = {:f}\".format(w[-1], l(w[-1])))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "CGLS erreicht bereits im zweiten Schritt wieder (eine) exakte Lösung der Normalgleichungen. Je nach Startwert erhält man unterschiedliche Parameter $w$, die aber die selbe Modellfunktion parametrieren."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Pseudoinverse und Singulärwertzerlegung (SVD)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Um diese Mehrdeutigkeiten zu beseitigen suchen wir jetzt eine Lösung $w^+$ der Normalgleichungen, die selbst möglichst kleine Norm $\\|w\\|_2$ hat, also\n",
    "\\begin{equation*} \n",
    "\\|X w -y \\|_2^2 \\to \\min\n",
    "\\quad\\text{und}\\quad\n",
    "\\|w \\|_2^2 \\to \\min.\n",
    "\\end{equation*}\n",
    "Dieses Problem hat immer eine eindeutige Lösung $w^+$. Sie hängt linear von $y$ ab,\n",
    "d.h. zu $X\\in\\mathbb{R}^{m\\times n}$ gibt es eine eindeutige Matrix $X^+\\in\\mathbb{R}^{n\\times m}$,\n",
    "die **Pseudoinverse**, mit\n",
    "\\begin{equation*} \n",
    "w^+ = X^+ y.\n",
    "\\end{equation*}\n",
    "Es gibt direkte Verfahren (über QR-Zerlegungen) um $X^+$ zu berechnen, in der Praxis\n",
    "wird $X^+$ in der Regel über die **Singulärwertzerlegung** bestimmt.\n",
    "\n",
    "Jede Matrix $X\\in\\mathbb{R}^{m\\times n}$ lässt sich zerlegen\n",
    "in\n",
    "\\begin{equation*} \n",
    "X = U\\Sigma V^T, \n",
    "\\end{equation*}\n",
    "wobei\n",
    "$U\\in\\mathbb{R}^{m\\times m}$, \n",
    "$V\\in\\mathbb{R}^{n\\times n}$ orthonormal sind und\n",
    "\\begin{equation*} \n",
    "\\Sigma = \\text{diag}\\big(\\sigma_1, \\ldots, \\sigma_r, 0, \\ldots,0 \\big)\\in\\mathbb{R}^{m\\times n},\n",
    "\\quad r \\le \\min(m,n).\n",
    "\\end{equation*}\n",
    "Dabei sind\n",
    "\\begin{equation*} \n",
    "\\sigma_1 \\ge \\ldots \\ge \\sigma_r > 0\n",
    "\\end{equation*}\n",
    "die **Singulärwerte** und $r$ der Rang von $X$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Damit können wir jetzt das Normalgleichungssystem mit Zusatzbedingung (also $w^+ = X^+ y$) \n",
    "über das dazu äquivalente Optimierungsproblem \n",
    "\\begin{equation*} \n",
    "\\|X w -y \\|_2^2 \\to \\min\n",
    "\\quad\\text{und}\\quad\n",
    "\\|w \\|_2^2 \\to \\min\n",
    "\\end{equation*}\n",
    "ganz einfach lösen. \n",
    "Für orthonormale Matrizen $Q$ gilt $\\|Qw \\|_2 = \\|w \\|_2$, $Q^{-1}=Q^T$, und somit\n",
    "\\begin{align*} \n",
    "\\|X w -y \\|_2^2 \n",
    "&= \\|U\\Sigma V^T w -y \\|_2^2\\\\\n",
    "&= \\|\\Sigma V^T w - U^Ty \\|_2^2\\\\\n",
    "&= \\|\\Sigma \\tilde{w} - \\tilde{y} \\|_2^2\n",
    "\\end{align*}\n",
    "mit\n",
    "\\begin{equation*} \n",
    "\\tilde{w} = V^T w, \\quad \\|\\tilde{w}\\|_2^2 = \\|w\\|_2^2\n",
    "\\end{equation*}\n",
    "und $\\tilde{y} = U^Ty$. Wir müssen jetzt also die Lösung $\\tilde{w}^+$ von\n",
    "\\begin{equation*} \n",
    "\\|\\Sigma \\tilde{w} - \\tilde{y} \\|_2^2 \\to \\min\n",
    "\\quad\\text{und}\\quad\n",
    "\\|\\tilde{w} \\|_2^2 \\to \\min\n",
    "\\end{equation*}\n",
    "bestimmen und erhalten $w^+ =V\\tilde{w}^+$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Nun ist aber\n",
    "\\begin{equation*} \n",
    "\\|\\Sigma \\tilde{w} - \\tilde{y} \\|_2^2 = \\sum_{i=1}^r (\\sigma_i \\tilde{w}_i - \\tilde{y}_i\\big)^2 + \\sum_{i=r+1}^m\\tilde{y}_i^2,\n",
    "\\end{equation*}\n",
    "so dass $\\|\\Sigma \\tilde{w} - \\tilde{y} \\|_2^2$ minimal wird für\n",
    "\\begin{equation*} \n",
    "\\tilde{w}^+_i = \\frac{\\tilde{y}_i}{\\sigma_i}, \\quad i=1,\\ldots,r.\n",
    "\\end{equation*}\n",
    "Damit sind die ersten $r$ Komponenten von $\\tilde{w}^+$ bestimmt.\n",
    "Wegen\n",
    "\\begin{equation*} \n",
    "\\|\\tilde{w}\\|_2^2 = \\sum_{i=1}^n \\tilde{w}_i^2,\n",
    "\\end{equation*}\n",
    "wird $\\|\\tilde{w}\\|_2^2$ minimal für\n",
    "\\begin{equation*} \n",
    "\\tilde{w}^+_i = 0, \\quad i=r+1,\\ldots,n.\n",
    "\\end{equation*}\n",
    "Somit ist\n",
    "\\begin{equation*} \n",
    "\\tilde{w}^+ = \\Big(\\frac{\\tilde{y}_1}{\\sigma_1}, \\ldots, \\frac{\\tilde{y}_r}{\\sigma_r}, 0, \\ldots, 0 \\Big)^T\n",
    "\\end{equation*}\n",
    "und $w^+ =V\\tilde{w}^+$. Wegen $\\tilde{y} = U^Ty$ kann man $w^+$ jetzt schreiben als\n",
    "\\begin{equation*} \n",
    "w^+ = V\\Sigma^+ U^T y, \n",
    "\\quad\n",
    "\\Sigma^+ = \\text{diag}\\Big(\\frac{1}{\\sigma_1}, \\ldots, \\frac{1}{\\sigma_r}, 0, \\ldots, 0 \\Big)\\in\\mathbb{R}^{n\\times m}.\n",
    "\\end{equation*}\n",
    "Ist also \n",
    "\\begin{equation*} \n",
    "X = U\\Sigma V^T\n",
    "\\end{equation*}\n",
    "eine Singulärwertzerlegung von $X$, dann ist die Pseudoinverse\n",
    "\\begin{equation*} \n",
    "X^+ = V\\Sigma^+ U^T\n",
    "\\end{equation*}\n",
    "und $\\Sigma^+$ ist die Pseudoinverse von $\\Sigma$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wie berechnet man nun numerisch eine Singulärwertzerlegung?\n",
    "Einen ersten Hinweis bekommt man durch folgende Beziehungen\n",
    "\\begin{equation*} \n",
    "X = U \\Sigma V^T\n",
    "\\quad\\Rightarrow\\quad\n",
    "X^T X = V \\Sigma^T \\Sigma V^T,\n",
    "\\quad\n",
    "X X^T = U \\Sigma \\Sigma^T U^T.\n",
    "\\end{equation*}\n",
    "Da $U,V$ orthonormal und quadratisch und $\\Sigma^T \\Sigma$, $\\Sigma \\Sigma^T$  Diagonalmatrizen sind,\n",
    "haben wir damit eine Eigenwert-/Eigenvektorzerlegung der beiden quadratischen\n",
    "symmetrischen Matrizen $X^T X$, $X X^T$,\n",
    "d.h.\n",
    "\n",
    "- $\\sigma_i^2$, $i = 1,\\ldots,r$, sind die positiven Eigenwerte von $X^T X$, $X X^T$, alle weiteren Eigenwerte sind $0$\n",
    "\n",
    "- die Spalten von $V$ sind die Eigenvektoren von $X^T X$\n",
    "\n",
    "- die Spalten von $U$ sind die Eigenvektoren von $X X^T$\n",
    "\n",
    "Damit könnte man numerisch wie folgt vorgehen:\n",
    "\n",
    "- berechne (z.B. mit QR-Iteration mit Shift) alle Eigenwerte $\\lambda_i$ und Eigenvektoren $v_i$ von $X^T X$,\n",
    "  \\begin{equation*} \n",
    "  \\lambda_1 \\ge \\ldots \\ge \\lambda_r >0, \n",
    "  \\quad \n",
    "  \\lambda_{r+1} = \\ldots = \\lambda_n = 0\n",
    "  \\end{equation*}\n",
    "\n",
    "- $\\sigma_i = \\sqrt{\\lambda_i},\\quad i = 1,\\ldots,r$\n",
    "\n",
    "- $V = \\big(v_1,\\ldots,v_n \\big)$\n",
    "\n",
    "- aus $U \\Sigma = XV$ folgt für die $i$-te Spalte der Matrizen \n",
    "  \\begin{equation*} \n",
    "  (U \\Sigma)_i = u_i \\sigma_i,\n",
    "  \\quad \n",
    "  (XV)_i = Xv_i\n",
    "  \\end{equation*}\n",
    "  somit\n",
    "  \\begin{equation*} \n",
    "  u_i  \n",
    "  = \\frac{1}{\\sigma_i}Xv_i,\n",
    "  \\quad i = 1,\\ldots,r\n",
    "  \\end{equation*}\n",
    "- ergänze $u_1,\\ldots,u_r$ zu einer Orthonormalbasis $u_1,\\ldots,u_m$ (z.B. mit modifiziertem Gram-Schmidt)\n",
    "  und setze\n",
    "  \\begin{equation*} \n",
    "  U = \\big(u_1,\\ldots,u_m \\big)\n",
    "  \\end{equation*}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alternativ könnte man natürlich auch über die Eigenwerte/Eigenvektoren von $XX^T$ die SVD berechnen.\n",
    "Beide Varianten sind nicht optimal (Aufwand, Kondition). \n",
    "\n",
    "Der heute gebräuchlichste Algorithmus vermeidet\n",
    "die explizite Berechnung von $X^TX$. Er basiert zunächst auf der Beobachtung, dass die Singulärwerte\n",
    "von $X$ identisch sind mit den Singulärwerten von\n",
    "\\begin{equation*} \n",
    "TXS, \\quad T,S \\text{ orthonormal.}\n",
    "\\end{equation*}\n",
    "\n",
    "Im ersten Schritt wird mit einer Folge von Householder-Transformationen von links und rechts $T,S$ so bestimmt, dass\n",
    "\\begin{equation*} \n",
    "TXS = R, \\quad R \\text{ bidiagonal,}\n",
    "\\end{equation*}\n",
    "was das Analogon zur Hessenbergtransformation bei der Eigenwertberechnung ist.\n",
    "\n",
    "Der zweite Schritt beruht auf einer angepassten QR-Iteration zur Berechnung der\n",
    "Singulärwerte (Wurzel der Eigenwerte von $R^TR$), ohne $R^TR$ explizit zu berechnen."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wir wenden nun die Pseudoinverse an, um unser unterbestimmtes Regressionsproblem zu lösen."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-24T09:28:14.476297Z",
     "iopub.status.busy": "2021-09-24T09:28:14.475884Z",
     "iopub.status.idle": "2021-09-24T09:28:14.555342Z",
     "shell.execute_reply": "2021-09-24T09:28:14.555668Z"
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "w = [-1.0151 -2.532  -1.5169], l(w) = 0.055920\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAEGCAYAAAB4lx7eAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAmZElEQVR4nO3dd3iUVdrH8e+ZFAVsWYqikCBIB6VEQEFXsIHiithlVRAEWUQsqyz4AooFVBDFVVlExQL4SnFxWQsWlLWEEgSkiyiI7LsgxgprSOa8f5xMCMkkmWRmMnlmfp/r4jLlmWfOEbhzuJ/7PsdYaxEREe/yxXoAIiISHgVyERGPUyAXEfE4BXIREY9TIBcR8bjkWLxpnTp1bKNGjWLx1iIinpWdnf2dtbZu8a/HJJA3atSIlStXxuKtRUQ8yxizPdjXIxLIjTFfAz8D+UCetTYzEvcVEZHyRXJF3t1a+10E7yciIiHQw04REY+LVCC3wGJjTLYxZnCwC4wxg40xK40xK/fs2ROhtxURkUgF8q7W2g5AL2CYMebM4hdYa6dbazOttZl165Z46CoiIpUUkUBurd1V8N/dwGtAp0jcV0REyhd2IDfG1DLGHBn4GDgPWBfufYPJ3p7Dk0u2kr09Jxq3FxHxpEhUrRwLvGaMCdxvtrX2rQjc9xDZ23PoNyOL3Dw/qck+Zg3qQseMtEi/jYiI54QdyK2124BTIjCWMmVt20tunh+/hQN5frK27VUgFxHBQ+WHXRrXJjXZR5KBlGQfXRrXPuT7SruISKKKSYt+ZXTMSGPWoC5kbdtLl8a1D1mNK+0iIonMM4EcXDAvHsCztu3l2x/2K+0iIgnLU4G8qKKr8OQkH8k+Q77fBk27iIjEM88G8qIPP/Pz/VzVKZ3jj6lRIu0iIhLvPBvIAw8/D+T5SUn20bdDgxIBPJB6UXAXkXjm2UBe1sNP0ANQEUkcng3kUPLhZ1HF687nr9qp1bmIxCVPB/KyFE29JPkM87J3kpfvx2cM4y9uwzWd02M9RBGRiPBMQ1BFBVIvt5/XnMszG5KX71bneX7L2IXr1DgkInEjbgM5uGA+rPtJ9O3QAJ/bCwYAv9+StW1vDEcmIhI5cR3IAzpmpDH+4jYk+ww+IDVFteYiEj/iNkde3DWd02l+3JF64CkicSduV+TaREtEEkVcrsiD1ZADqisXkbgUl4E82N7lgDbWEpG45K1A/tNPcOSRUKQCJZji7fuBB5vBviYi4nXeCeTWQu/ecPjh8Pjj0LJlqZeW1r4/tndr3lz3b3q1qV/4Ne3HIiJe551A7vfD5ZfD2LFw8slwyy0wbhwcdVTQy4PtXT5+0Xpy8/ys+Pp7mh93JKC8uYh4n3eqVpKSYPhw2LIF+veHKVOgWTOYOdMF+XIEy5uXlksXEfES7wTygLp14ZlnYPlyOPFEGDAAunaFlSvLfFmwMz/TaqbiM65JSHlzEfEq7wXygMxM+PhjeOEF+Oor6NQJBg2C3buDXl5075VAOeL4RevxW4vPZxjbu7XSKiLiSd4N5AA+H1x3nUu33H67C+rNmsHUqZCXV+LywN4rHTPSDkmrWGvJ2ZcbgwmIiITP24E84KijYNIkWLvWrcxHjID27WHJklJfEizVIiLiRfERyANatoS334bXXoNffoEePeCKK2DHjhKXBlItV3VKp2+HBjEYrIhIZMRXIAfXLNSnD2zYAOPHw6JF0KIF3H8//Pe/JS6fv2onryzfQb8ZWdqXRUQ8Kf4CeUCNGjBmDGzc6BqJxoyBVq1g4ULXXETprfwiIl4Sv4E8ICOD7Ef+xt8nvcj+lMPcar1nT9i0iS6Na5PsMxggyWdK5Mm1g6KIeIF3Ojsr6eBOiL+j5qUP82bKOho+/hC0bcsJA4ZQ6+ju/JB0eIn9W4LtoKjyRBGpjuJ+RV40fbLf+nj9rMtdueL113PsjKd4++kbuWTde/jz8g5JrSjtIiJeEfeBPGiZYb16MGMGm/7+DruOqcej/5zC3Fkj6fHLjrJfJyJSDRlb8OCvKmVmZtqV5bTUR1JZOxxmf7WXH6bN4MznJpOy9zsYOBAefBDq1i3xOu2UKCKxZIzJttZmlvh6IgTykPz4oytXnDoVatVyH//pT5DsHiMoZy4isVZaII/71ErIjj4aJk8utTtUOXMRqa4UyIsLdIcuWHBId+gZh+0PmjMvWqKockURiQWlVsqyfz888ghMmADG8O2fbuMf517DqS2Op2NGGrOX7WDswnXk+y0pyT6wljy/JdlnuDyzIX07NFD6RUQiRjnycGzfDnfcAfPnuz3Qp0wh+5QzuHJ6Fnn+g///DGCLfHxYinLpIhI5Uc+RG2OSjDGfGWMWReqe1UZGBsybB+++61r/+/ShzhV9yPjum8JLknyGlCTXJQouoCuXLiJVIZI58hHAxgjer/o5+2xYvRqmTKHBptW89ewwRi95jmMO7Oe+i9swZ/BpXN05XfXnIlKlIpJaMcY0AF4AHgBut9b2Lut6z6VWgtm9m+9uuYPar84ir049UiY/Av36gc+nenMRiYpop1YeA+4CSj0F2Rgz2Biz0hizcs+ePRF62xiqV486r7yEycoi5cQMd1JRt26QnX3ISUQiItEWdiA3xvQGdltrs8u6zlo73Vqbaa3NrFu3brhvW3106gSffgrPPw9ffgmnngqDB0M8/LASEU+IxIq8K/AHY8zXwCtAD2PMyxG4r3f4fNC/v9uM69ZbXVBv1gyeeILsL/eotlxEoirsQG6tHWWtbWCtbQRcBbxvrf1j2CPzoqOPhkcfhTVrIDMTbrmFI07vxEfPzC33BCI1E4lIZcX9fuQx0aoVLF7MG/c9TdtHxzNnzmj+2eIM1rV5gI4ZZ5S4vOg+LmomEpGKimiLvrX2g/IqVhKGMRx7/dX0vmkaU7r14+yty7i2//lBzw4tuo9Lbr5l9jKdISoiodNeK1HUMSON54b+ntTx9/DFByvwXXhB0LNDA3ufq5lIRCpDgTzKAqWIbbue7LpD33mnsDuUXr1g82Y6ZqQxa1AXNROJSKVor5VYOHAAnnwSxo1zG3PdeqtbqR95pJqJRKRU2jSrOvrPf2DUKFeueNxx8PDD8Mc/ljgIWkQEdLBE9XTssfDcc7BsGTRseLA7dNWqWI9MRDxEgbw66NQJsrLg2Wdh61ZXgz5kCHz3XaxHJiIeoEBeXfh8cMMNrH5nGav7Xo999lnXHfrkk5CXB6hpSESCU0NQNZK9PYd+czeS2/QyWg7qwpx1czjq5pth+nQ2j36AfuuSdPiziJSgFXk1UrQxaFNaA14a/4wrWfzhB5pfdRGPzJ9IvR+/U425iBxCgbwaCTQGFdaRN6kDl14KGzeya8RdnPtFFu/PGMLNWXM57fhasR6uiFQTKj+sZsqqI//8ozXUHH0XTf61GBo3hsceI7ttV7K++l515yIJQHXk8eSdd2DECNi4kaVNOnLv2YP5tl5D5c1F4pzqyOPJuefCmjV8NHQU7Xdu5M0Zw7jtnWfJXrc91iMTkRhQIPeqlBRqjLyTnkOf4fXWZzFk2Xz639ALXn65cDMuEUkMCuQe1jEjjam39uI/jz/NxtcWk5rREK69Vt2hIglGgdzjArsrtuxzrusOnTEDvvjCdYfedBPsVZmiSLxTII8j2d/8yJONf8/qd5fDLbe4oN60KTz1VGF3qIjEHwXyOBE4Lm7y4s1cNXcj2beNg9WroV07GDbMrdCXLo31MEUkChTI40TRrtDCzs82beC992DuXMjJgd//Hq65BnbujPVwRSSCFMjjRImu0MDpQsbAZZfBxo0wdiwsWAAtWsCECfDbb7EdtIhEhBqC4khIpwt99RXccQe89ho0aQKPPQa9e+tkIhEPUGenHOqdd9g/dBg1vvyCb0/vzg3truGLo+prZ0WRakydnXKI7GaZdLpiEg90v4GjV2bxj2lDuXPJTFL2/aqdFUU8RvuRJ5hACuXbH/bzq03imU59WdiqOyOXzmTosnn03bCEXzMmwFlNdHaoiEdoRZ5AipYozsveSbLPkGTgp7Ta/PbMc8z766sccWI6jUcMhjPOgM8+i/WQRSQEWpEnkKIlivn5fq7qlM7xx9Q4+ICzczoMvRSefx5GjXK154MHw333QZ06sR6+iJRCK/IEUrxEsW+HBgzrftKhDzZ9Phg4ELZsgZtvhmeecWeHqjtUpNpS1UqCKavMMOj31q1z7f5LlsApp8DUqXDmmTEYuYio/FDKFMifBz3c2Vp3dugdd8A338DVV8PDD0ODBrEdtEiCUfmhlCloi3+AMXD55bBpE4wZo+5QkWpGgVyA4C3+2dtzeHLJVrK357iLataE8eNdu/+558Lo0W4/l3/+M7aDF0lwqlqRQn07NMAU/BcoPdVy4omuxX/xYpc/790bLrwQpkxx2+aKSJXSilwK8+OvLN/B/FVuZ8QyUy0B550Ha9fCpElui9w2bVzZ4i+/VPEMRBKbArkEDdql7qZYXGqqewi6ebN7CDpxIjRvDrNn6+xQkSqiQC5Bg3bHjDRmDerC7ec1D20Trfr1YeZM+OQT93G/fm7/8zVrqmQOIolM5YcChLgFbqjy8w92h37/PQwZ4rpDa5eyqheRkESt/NAYc7gxZrkxZo0xZr0x5t5w7ylVL3CIc0S2r01KgkGDXHfosGEwfbrrDp02zQV5EYmoSKRWfgN6WGtPAdoBPY0xXSJwX/G6tDTXCfrZZ3DyyTB0qNu/5aOPYj0ykbgSdiC3TqBMIaXgl55yyUFt28L778Orr8LevW5nxT/+Eb79NtYjE4kLEXnYaYxJMsasBnYD71hrlwW5ZrAxZqUxZuWePXsi8bbiJYHu0I0bXXfovHmuuuWhh9QdKhKmiARya22+tbYd0ADoZIxpE+Sa6dbaTGttZt26dSPxtuJFtWq57tANG+Ccc+Avf3Er9jfeiPXIRDwrouWH1tofgA+AnpG8r3hLidb+YBo3hr//Hd58022de+GFrkN069YqG6dIvIhE1UpdY8wxBR/XAM4BNoV7X/GmoqcQ9ZuRVXYwB+jZ03WHPvIIfPghtG7t9nBRd6hIyMoN5MaYtsaYj4t83sEY836RS+oDS4wxa4EVuBz5osgPVbwgpNb+4lJT4c9/duWKV13ldlVs0QLmzFF3qEgIQlmRrweaGGOSCj6fDPw58E1r7VprbXtr7cnW2jbW2vHRGKh4Q6BL1GfAGENazdTQX1y/PrzwAnz8MRx7LFxzDZx1lrpDRcpRbiC31vpxwby1MeZSYIe1dlXURyae1DEjjbG9W+Mzhny/Zfyi9eWnV4o7/XRYvtw1Em3YAB06uMai77+PzqBFPC7UHHkW0BW4BxgdtdFIXMjZl4vfWiwVSK8Ul5QEN954sDt02jTXHfq3v6k7VKSYigTy+4HXrLXq4pAyhbxzYigC3aGrV7ttcm+6CU491aVfRAQIPZBvwrXiPxTFsUicqPDOiaFo29YdAP3KK7BnD3TrBtdeC7t2hX9vEY8LNZCPAEZZa3+N5mAkfkR0E64AY+DKK93ZoXff7Vr+mzd3B0GrO1QSWJmB3BjTxBizCahhrX2hisYkUrZateD++92D0B49YORIt2J/881Yj0wkJsoM5NbaL621Lay1A6tqQCIha9IEFi50AdwYuOACuOgidYdKwtEJQVIthNTWX5qePeHzz1136AcfqDtUEo4CucRchdv6gynaHXrlleoOlYSiQC4xlb09h8fe3cJvByrY1l+a+vXhxRfd4RX16qk7VBKCArnETGAl/vHW77C4P4xh150HdO0KK1a4BqL169UdKnFNgVxipugGWz4DXZvWiVzdObju0MGDXbpl6FB1h0rcUiCXmCnaAZqa7OPWc5oFDeJhPQgF+N3v4K9/dWeHtm7tukM7dYJPPglzBiLVg7ExeBCUmZlpV65cWeXvK9VP9vYcsrbtpUvj2qUG8X4zssjN85Oa7Au6Yi/vHoewFv73f92D0W+/dWeHPvQQHH98JKclEhXGmGxrbWbxryfHYjAiAR0z0soMvsH2Nw9cn709hwWrdjJ35Tfk+W2pgf4Qxrg9zy+6CB58ECZNcicVjR0LI0a46hcRj1FqRaq10jbgCqzUZy/bQW6+rXjFS61a8MADrju0e3e46y7XHfrWW1GcjUh0KJBLtVbaBlyBlXogMWioZMVLkybw+uvu8GdroVcvuPhi+PLLiM5DJJoUyKXaC7YBV/EHpVd3TmfWoC4AlXsw2quX6w6dOBHee889FP2f/4FftU+cVH962CmeVfwhZygPRkOya5dLtcyaBQ0auDz6FVe4/LpIDJX2sFMrcvGs4iv1UA9+Lrec8fjj4eWX4V//gjp13MPRHj3cil2kGlIgl7gRyslEFdrXpVs3WLnSNRKtXQvt2sHw4eoOlWpHgVziRignE4W6ai+UlARDhsAXX7hGoqeect2h06erO1SqDQVyiSvlnUxU6fNEf/c7ePJJWLUKWrVywV3doVJN6GGnJJzAQ9K0mqnk7MsNrSO0qOLdoddeCw89RHbu4aF3mIpUQmkPOxXIJSFFpMLll1/cvueTJpGfmsrkzlfybIfemMMOi+zmXyIFVLUiUkSFc+UFDql4OeII1x26fj072pzKXe89yxvPDqfLlhXh7akuUkEK5JKQiufK02qmlttIVGrFy0kn8f0r8xl81b34rJ+Zr46j34QRsG1bFc1GEp0CuSSkQIXLVZ3SOaNpXe75x/pySxLLWsV3zEhjyMThvDVnMTtHjuOYTz50D0XHjIF9+6pqWpKgFMgloc1ftZN3N/wnpDRLeRUvHTPSGHp+axpMvMcdZnHZZXD//e7s0Fdf1dmhEjUK5JKwKrrxVih16oWKdofWru0OhD77bFi3LuLzEFEgl4R1yMZbSYZrCjbeKitAl1enXkKgO/Tpp90B0O3awS23QE4lTzsSCULlh5LQKnS6ULj27nUHWEyb5hqMJkyAAQNc96hICFR+KBJEhVfY4ahd23WHZmdDy5Zw443QuTN8+mn031vimgK5SDnCPvy5uHbt4MMPYfZs+L//g9NPh+uvdx+LVIICuUgZKrRbYkUYA1dfDZs2wahR8MorbjOuSZMgNzcy7yEJQ4FcpAyV7QAN2RFHuEOg162DM8+EO++Ek0+Gt9+O7PtIXFMgFylF9vYcdv2wn2SfqfhuiUXuEVJapmlTWLTI/crPh549oU8fdYdKSJLDvYExpiHwInAc4AemW2sfD/e+IrFUdFOt5CQfV3ZqyKUdGlTooWilNua68EI45xyYMsU1E7Vq5Vbpo0ZBzZphzkriVSRW5HnAHdbalkAXYJgxplUE7isSM0VTKvn5fk44pkaFK1sqnZY57DD4y19g82a49NKD3aFz56o7VIIKO5Bba/9trV1V8PHPwEbghHDvKxJLFTmAorT0SdF7JPkMu37Yz+xlO0KvgDnhBHcA9NKlru78iivUHSpBRbQhyBjTCFgKtLHW/lTse4OBwQDp6ekdt2/fHrH3FYmGUJqFSkufBF778/4DfLptL+v//RP5+RaL2wrgsJQK7oGen++Ol7v7bvjpJxg2DO69F445JlLTFQ8orSEo7Bx5kTc4ApgP3Fo8iANYa6cD08F1dkbqfUWipWNGWrmBtrT0Sb8ZWfx24OA+LkVZILfg2pADeVISDB3qVuVjxsBf/+rq0CdMgBtuAJ/qFhJZRH73jTEpuCA+y1q7IBL3FPGCYCmY4ptxBeMzpsIVMIDrDn3qKbd/S4sWB7tDs7IqPQfxvrADuTHGAM8CG621j4Y/JBHvCLYjYiC4B/5y+QykJLkSRgMk+wzjL24T3rYA7du73PmsWbBrF5x2GvTvr+7QBBV2jtwY0w34F/A5rvwQYLS19o3SXqNNsyTeFT/gOa1mKut2/YgB+lawjLFcv/zijpybPBkOPxzGjXM7LKakRO49pFrQ4csiMRKRg55D8cUXcOut8MYbLu0ydSqce27k30diRrsfisRI1Nv8A5o2hX/+03WHHjgA550HffvCV19F5/2k2lAgF4myitSkR8SFF8L69W4Pl7ffdt2h48bp7NA4ptSKSBWoigMssrfnMH/VzkPz8Dt3wl13wZw5kJ7u8uiXXup2XxTPUY5cpJqqTJAv/prZy3Yw5u+fk1/w1zk12cecG4vk4pcuheHDYe1a6NHD5c9bt47SjCRaot4QJCIVV5kHocVfM7Z3a8YuXFcYxOFgLr7wXmee6U4m+tvfXEPRKafAzTfDPfeoOzQOKEcuEkOVeRBa/DVvrvs3/mL/sg6ai09Odq39W7bAoEHYqVPZd2ITvn54Kvj9iHcpkIvEUGUehBZtODLG0Lr+Ue5zA0kGzmt17KFpleLq1CF71AT6DpzKhiOOo9HIEfza4VRYtuyQyyJ+xJ1EjXLkIjFWmRz57GU7GLtwHX5rC9MrOftyQ77Hk0u2MnnxZvx+S98NH3Dfpy9Ra+9u1x06cSLZ/02tmtp3qRDVkYtUUx0z0hjW/aQKBcqcfbn4rS1Mr+Tsy63QPQr/JeAzvNHubLZ8uAJGjnQt/82asf+hSfh/y41+7btEhAK5iAeFu1968T1i2rdOh4kT3V7nXbvS7ekHeeP54ZyxfXXV1L5LWJRaEfGocPZLL5O1sGgRv918C4ft+Jqcnr1Je/oJaNQo8pOQClFqRSTOhJKSqdT2AMbARRdx2OaN8MADpC19H1q2dKWK6g6tlhTIReJYZapiClMx/9kPo0fDpk1w8cXuRKKWLWH+/HLPDi2r4kXVMJGn1IpInKtIVUxZqZjN/7uI2qP/TJ1tm93ZoY8/HrQ7tKx7VNlOkHFKqRWRBBU47CJr295yV8GlpWJmL9vBhWt8nHbZw4zvOZS8ldmuO/S22+CHH0K6R3nfK4tW8WVTi75InKvIKjiQijmQ5y9MxWRvz2HswnXk+S2YJGa2u5AThvRn4FvPuVX5rFnu7NABA8DnC3qPsu4fyfEnKq3IReJcRVbBwY6uy9q2l3z/wRSszxjatW8K06a5s0ObNoVBg6BLF1i2rMQ9gMLVdLD7R3L8iUorcpE4V9FVcCC4BgJml8a1OSzFR+4BP77i54126AAffeRW5Xfd5YL5gAF0nDCBjt1PKnU1XZEVdWVW8YlGDztFEkC4DzyBEq8vcc+ff4b77oPHHoMaNeCee3i6TU8eeX8bfuv2gbn9vOYM635SVMcfz7SNrUgCq8gqOFgqo3i9eql564cfhoED3dmht99O/5OmsbzT9SxNPyWs1XRFV/GJRjlyETlEKLXnZeatmzd3B0AvXEgNfx7Pz76bxR9PZe75x5cIxqpGiQytyEXkEIEHkmWlMsrNWxsDf/iDOwB68mSaPPggnNuFXTeN4PXz/8ipLU8AUDVKhChHLiKVUqG89Tff8P2fRvC7Ra+x86h6PHTejRx59RW8suKbsPPniUQ5chGpkPICdYXy1g0bMuf2h/noyFMZ987feGLeA2zd8h5rTuvPprQGIeXPQ90kLBEfiiqQi0gJ0WjC6dK4No83bkfvAVO5fs2bjMqaw6INN7Omz7XYseNoX8b9QxlP4LCNfL/lsJTEStXoYaeIlBC1Jhxryfcl8VLmRaxfsgIzYADt5s+k/Tmd4PnnSz07tLzxFO0+tUDugeg2DlW3h7QK5CJSQmV2TSxP1ra9hYE232/56EcD06fD8uXQuDHccAOcfjqsWFHqeHzGnVOaVjO1xL2LHkDt87lrigfbSATgwL8OJi/eTL8ZWdUimCu1IiIlhFK5UlGlVrpkZsLHH8PLL7vu0E6dXFCfMAHq1Sscz9jerQtTJ+MXraf5cUcWjitw79w8Pz5jGNTtRMYvWl+iqSkS6aJg/zqIdQpHgVxEgop0E06ZPxx8PrjuOujT52B36Pz5fDNiJIu69aFTs+MKzym1lAygxe9dWiom8LXcMAJwddwyQIFcRKpMuT8cjjoKHnkEBg7kxyHDaDh+ND3qTGPC+Tdx3vB+ZQbQ4vcufu3m//uZwN5ffkuJ9ExF5hDsB1IsK2YUyEUkJsoMfC1a8PK4aax+6iX+571nmDlrNFu/+ZC5Dz7M0tya5QbLYME2a9teDGBxDwdz9uVWeuzFf2jEeqtdBXIRqXKhBL4uTerwRMvT6HliewaveI1hn84l+ezOtB31F+hyV7nvUTzYptVMJcln8PstqSmRTYlkbdvLbwf8h1TMKJCLSFwL5YFhYFW9YNVOnk69inmte3D3B89xwT33wMyZ8OijLqduTND3CKz402qmsm7Xj8zL3onfWnw+w9jerSMaaNNqphKomfFT+bRNZSmQi0iVC/WBYSAlkue3fHtkXYb/YSS/DRrCJc9NhL594ZxzYOpUdyh0EUVX/P5iu5AYbFhplWBy9uXiMy737jPhpW0qQ3XkIlLlKnJSUPGa9vS+F8Bnn7kAvnIlnHwy3HEH/Phj4WuKrviLMkSuLr60MabGoJJFm2aJSLVX6oPRPXvg7rthxgxXcz5xIlx3Hdnf/HjIitwHJCcZLs9sSN8ODcr8wVHZ6pPyXheJqpbSNs0KO5AbY54DegO7rbVtQnmNArmIRNTKlTB8OGRlQefO8MQTZNc7qTBHnrMvt9KnI0Uilx6p+5YWyCORWpkJ9IzAfUREKifQHTpzJnz9NXTuTMf77mRY66O4pnN6iROOShOtPWaifYB02IHcWrsU+D4CYxERqTyfD66/HrZsgdtvhxdegGbN4PHH4cCBkG4RjT1monnfgIjkyI0xjYBFZaVWjDGDgcEA6enpHbdv3x72+4qIlGrTJhgxAhYvhtat3cPRHj3KfVm0OjSrdY684OaNKCeQF6UcuYhUCWvh9dfhttvgq6/g0kth8mTIyIj1yColmjlyEZHqyRi4+GJYvx7Gj3eHQrds6T7evz/Wo4sYBXIRiX81asCYMS7d0rs3jBsHrVrBa6+5VXuURfsgirADuTFmDvAp0NwYs9MYMzD8YYmIREF6Orz6Krz3HtSq5bpDzz8fNm6s9C3LC9JVcRBFJKpWrrbW1rfWplhrG1hrn43EwEREoqZHD1i92lW0LF9+sDv0p58qdJtQgnS0Sw9BqRURSVTJyXDLLfDFF9C/P0yZ4soVX3ih1LNDiwslSEe79BAUyEUk0dWtC888A8uWQaNGLqh37eq6RcsRSpCuyL4ylaW9VkREAvx+eOklGDkSdu+GgQPhwQddsC9FVZ4MpPJDEZHyBLpDN2923aEzZ0LTpq6ZKC8v6Es6ZqSFvAVAtCiQi4gUd/TRMGkSrF0LnTq5DtH27WHJkliPLCgFchGR0rRsCW+/DQsWwC+/uGqXK66AHTtiPbJDKJCLiJTFGLjkEtiwAe69F/7xD2jRAu6/H/7731iPDlAgFxEJTY0aMHas6w694ALXKdqqFSxcWCXdoWVRIBcRqYiMDJg3D9591wX3Pn2gVy8X4GNEgVxEpDLOPtt1hz72mDuZqG1buPPOCneHRoICuYhIZaWkuIqWLVtc2eLkydC8Obz4YsjdoZGgQC4iEq569dwB0MuWudTL9ddDt26QnV0lb69ALiISKaeeCp98As8/D19+6T4fPBj27Inq2yqQi4hEks/n9mvZssWdTPT8824zrieeKLU7NOy3jMpdRUQS3dFHu5z52rWQmel2WmzfHnbujPhbKZCLiERTy5buAOgFC6BJE6hfP+JvkRzxO4qIyKEC3aGXXBKV22tFLiLicQrkIiIep0AuIuJxCuQiIh6nQC4i4nEK5CIiHqdALiLicQrkIiIeZ2wMTrYwxuwBtodwaR3guygPp7pK1Lkn6rwhceeeqPOGis89w1pbt/gXYxLIQ2WMWWmtzYz1OGIhUeeeqPOGxJ17os4bIjd3pVZERDxOgVxExOOqeyCfHusBxFCizj1R5w2JO/dEnTdEaO7VOkcuIiLlq+4rchERKYcCuYiIx1WLQG6M6WmM2WyM2WqM+UuQ7xtjzNSC7681xnSIxTgjLYR59yuY71pjzCfGmFNiMc5oKG/uRa471RiTb4y5rCrHFy2hzNsYc5YxZrUxZr0x5sOqHmO0hPDn/WhjzD+MMWsK5j4gFuOMNGPMc8aY3caYdaV8P/z4Zq2N6S8gCfgSaAykAmuAVsWuuQB4EzBAF2BZrMddRfM+HUgr+LhXPMw71LkXue594A3gsliPu4p+z48BNgDpBZ/Xi/W4q3Duo4GHCj6uC3wPpMZ67BGY+5lAB2BdKd8PO75VhxV5J2CrtXabtTYXeAW4uNg1FwMvWicLOMYYE/mD76pWufO21n5irc0p+DQLaFDFY4yWUH7PAYYD84HdVTm4KApl3tcAC6y1OwCstYk0dwscaYwxwBG4QB6dY+erkLV2KW4upQk7vlWHQH4C8E2Rz3cWfK2i13hNRec0EPdTOx6UO3djzAnAJcC0KhxXtIXye94MSDPGfGCMyTbGXFdlo4uuUOb+V6AlsAv4HBhhrfVXzfBiKuz4Vh0OXzZBvla8JjKUa7wm5DkZY7rjAnm3qI6o6oQy98eAkdbafLdAiwuhzDsZ6AicDdQAPjXGZFlrt0R7cFEWytzPB1YDPYAmwDvGmH9Za3+K8thiLez4Vh0C+U6gYZHPG+B+Ilf0Gq8JaU7GmJOBGUAva+3eKhpbtIUy90zglYIgXge4wBiTZ639e5WMMDpC/bP+nbX2V+BXY8xS4BTA64E8lLkPACZalzjeaoz5CmgBLK+aIcZM2PGtOqRWVgBNjTEnGmNSgauA14td8zpwXcHT3S7Aj9baf1f1QCOs3HkbY9KBBcC1cbAiK6rcuVtrT7TWNrLWNgLmAX/yeBCH0P6sLwTOMMYkG2NqAp2BjVU8zmgIZe47cP8SwRhzLNAc2Falo4yNsONbzFfk1to8Y8zNwNu4J9vPWWvXG2NuKvj+NFzVwgXAVmAf7ie3p4U477FAbeCpgpVpno2DXeJCnHvcCWXe1tqNxpi3gLWAH5hhrQ1atuYlIf6e3wfMNMZ8jks3jLTWen57W2PMHOAsoI4xZicwDkiByMU3teiLiHhcdUitiIhIGBTIRUQ8ToFcRMTjFMhFRDxOgVxExOMUyEVEPE6BXETE4xTIRQBjzBJjzLkFH99vjJka6zGJhCrmnZ0i1cQ4YLwxph7QHvhDjMcjEjJ1dooUKDiN5wjgLGvtz7Eej0iolFoRAYwxbYH6wG8K4uI1CuSS8ApOY5mFO6nlV2PM+TEekkiFKJBLQivYKnYBcIe1diNuB757YjookQpSjlxExOO0IhcR8TgFchERj1MgFxHxOAVyERGPUyAXEfE4BXIREY9TIBcR8bj/B0x6ZAGqK0MHAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "w = np.linalg.pinv(X).dot(y)\n",
    "evalw(w, x, y)\n",
    "print(\"w = {}, l(w) = {:f}\".format(w, l(w)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Abgeschnittene Singulärwertzerlegung (TSVD)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Jede Matrix $X\\in\\mathbb{R}^{m\\times n}$ kann\n",
    "in\n",
    "\\begin{equation*} \n",
    "X = U\\Sigma V^T, \n",
    "\\end{equation*}\n",
    "$U\\in\\mathbb{R}^{m\\times m}$, \n",
    "$V\\in\\mathbb{R}^{n\\times n}$, \n",
    "\\begin{equation*} \n",
    "\\Sigma = \\text{diag}\\big(\\sigma_1, \\ldots, \\sigma_r, 0, \\ldots,0 \\big)\\in\\mathbb{R}^{m\\times n},\n",
    "\\quad\n",
    "r \\le \\min(m,n)\n",
    "\\end{equation*}\n",
    "zerlegt werden.\n",
    "\n",
    "Da in $\\Sigma$ nur die ersten $r$ Diagonalelemente ungleich $0$ sind\n",
    "folgt\n",
    "\\begin{equation*} \n",
    "X \n",
    "= U\\Sigma V^T\n",
    "= \\sum_{i=1}^r u_i \\sigma_i v_i^T\n",
    "= U_r\\Sigma_r V_r^T\n",
    "\\end{equation*}\n",
    "mit\n",
    "\\begin{align*} \n",
    "U_r \n",
    "&= \\big(u_1,\\ldots,u_r \\big) \\in \\mathbb{R}^{m\\times r}, \\\\\n",
    "\\Sigma_r \n",
    "&= \\text{diag}\\big(\\sigma_1,\\ldots,\\sigma_r \\big) \\in \\mathbb{R}^{r\\times r}, \\\\\n",
    "V_r \n",
    "&= \\big(v_1,\\ldots,v_r \\big) \\in \\mathbb{R}^{n\\times r}\n",
    "\\end{align*}\n",
    "sowie\n",
    "\\begin{equation*} \n",
    "X^+ = V_r \\Sigma_r^{-1} U_r^T.\n",
    "\\end{equation*}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ist $r \\ll \\min(m,n)$, so sind Matrixprodukte über $U_r\\Sigma_r V_r^T$,\n",
    "$V_r \\Sigma_r^{-1} U_r^T$ deutlich günstiger zu Berechnen als Produkte die direkt $X$, $X^+$ benutzen."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Nehmen wir nun an, dass $X\\in\\mathbb{R}^{m\\times n}$ Singulärwerte \n",
    "$\\sigma_1 \\ge \\ldots \\ge \\sigma_k \\ge \\sigma_{k+1} \\ge \\ldots \\ge \\sigma_r > 0$ mit\n",
    "\\begin{equation*} \n",
    "\\sigma_1  \\gg \\sigma_{k+1}\n",
    "\\end{equation*}\n",
    "hat. Damit erhalten wir\n",
    "\\begin{equation*} \n",
    "X \n",
    "= U\\Sigma V^T\n",
    "= \\sum_{i=1}^r u_i \\sigma_i v_i^T\n",
    "\\approx \\sum_{i=1}^k u_i \\sigma_i v_i^T\n",
    "= U_k\\Sigma_k V_k^T\n",
    "= X_k\n",
    "\\end{equation*}\n",
    "$X_k$ ist eine Rang-$k$-Approximation der Rang-$r$-Matrix $X$ (abgeschnittene Singulärwertzerlegung, TSVD), also\n",
    "eine vereinfachte Version unseres ursprünglichen linearen\n",
    "Modells, das durch $X$ repräsentiert wird.\n",
    "Wegen\n",
    "\\begin{equation*} \n",
    "X_k = U_k\\Sigma_k V_k^T\n",
    "\\end{equation*}\n",
    "ist $X_k$ für $k\\ll r$ numerisch viel effizienter zu bearbeiten als $X$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wir wenden nun die SVD auf unser Regressionsproblem von oben an und erhalten Singulärwerte"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-24T09:28:14.558357Z",
     "iopub.status.busy": "2021-09-24T09:28:14.557777Z",
     "iopub.status.idle": "2021-09-24T09:28:14.560343Z",
     "shell.execute_reply": "2021-09-24T09:28:14.559955Z"
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1.8869e+01, 2.7241e+00, 9.2795e-16])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "U,s,VT = np.linalg.svd(X)\n",
    "V = VT.T\n",
    "\n",
    "s"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Offensichtlich können wir hier $k = 2$, wählen d.h.  zwei Parameter reichen aus.\n",
    "Betrachten wir die zugehörigen Singulärbasen $V$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-24T09:28:14.562803Z",
     "iopub.status.busy": "2021-09-24T09:28:14.562420Z",
     "iopub.status.idle": "2021-09-24T09:28:14.564391Z",
     "shell.execute_reply": "2021-09-24T09:28:14.564055Z"
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-0.5222,  0.6277, -0.5774],\n",
       "       [ 0.2825,  0.7661,  0.5774],\n",
       "       [ 0.8047,  0.1383, -0.5774]])"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "V"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "dann sehen wir, dass Parameterkombinationen $w = s(-1,1,-1)^T$, $s\\in\\mathbb{R}$, vernachlässigbar sind.\n",
    "Durch Einschränkung auf die ersten beiden Singulärbasen kann man das Modell dann vereinfachen und wir erhalten"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-24T09:28:14.574908Z",
     "iopub.status.busy": "2021-09-24T09:28:14.566554Z",
     "iopub.status.idle": "2021-09-24T09:28:14.638459Z",
     "shell.execute_reply": "2021-09-24T09:28:14.638787Z"
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "w = [-1.0151 -2.532  -1.5169], l(w) = 0.055920\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAEGCAYAAAB4lx7eAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAmn0lEQVR4nO3deXxU1fnH8c+ZLAq4pSyKAkGQHZQlBlS0ghsoKmJdqYKCIILgUkWxgkUFXBDFqhRRcQGsLIpSraDS4hYgoYDsIsoi/ioirlBDMuf3x8mEkEySSWYmkzvzfb9evprl5t5zij45PPd5zjHWWkRExLt8sR6AiIiER4FcRMTjFMhFRDxOgVxExOMUyEVEPC45Fg+tU6eObdy4cSweLSLiWTk5Od9Za+sW/3pMAnnjxo3Jzs6OxaNFRDzLGLM12NcjEsiNMV8BPwP5QJ61NiMS9xURkfJFckXezVr7XQTvJyIiIdDLThERj4tUILfAQmNMjjFmULALjDGDjDHZxpjsXbt2ReixIiISqUB+mrW2I9ATGGqMOaP4BdbaqdbaDGttRt26JV66iohIJUUkkFtrdxb877fA60BmJO4rIiLlCzuQG2NqGWMOD3wMnAusCfe+weRs3cNTizeTs3VPNG4vIuJJkahaORp43RgTuN9Ma+0/I3Dfg+Rs3UPfaVnk5vlJTfYxY2AXOqWnRfoxIiKeE3Ygt9ZuAU6KwFjKlLVlN7l5fvwW9uf5ydqyW4FcRAQPlR92aVKb1GQfSQZSkn10aVL7oO8r7SIiiSomLfqV0Sk9jRkDu5C1ZTddmtQ+aDWutIuIJDLPBHJwwbx4AM/aspuvf9intIuIJCxPBfKiiq7Ck5N8JPsM+X4bNO0iIhLPPBvIi778zM/3c2VmI449qkaJtIuISLzzbCAPvPzcn+cnJdlHn44NSgTwQOpFwV1E4plnA3lZLz9BL0BFJHF4NpBDyZefRRWvO5+7YodW5yISlzwdyMtSNPWS5DPMydlBXr4fnzGMvbgtV3duFOshiohEhGcagioqkHq57dwWXJbRkLx8tzrP81tGz1+jxiERiRtxG8jBBfOh3U6gT8cG+NxeMAD4/ZasLbtjODIRkciJ60Ae0Ck9jbEXtyXZZ/ABqSmqNReR+BG3OfLiru7ciBbHHK4XniISd+J2Ra5NtEQkUcTlijxYDTmgunIRiUtxGciD7V0OaGMtEYlL3grkP/0Ehx8ORSpQginevh94sRnsayIiXuedQG4t9OoFhx4KTzwBrVqVemlp7fuje7XhnTXf0LNt/cKvaT8WEfE67wRyvx8uuwxGj4YTT4Thw2HMGDjiiKCXB9u7fOyCteTm+Vn+1fe0OOZwQHlzEfE+71StJCXBzTfDpk3Qvz9MmgTNm8P06S7IlyNY3ry0XLqIiJd4J5AH1K0Lzz4Ly5bB8cfDddfBqafC8uVl/liwMz/TaqbiM65JSHlzEfEq7wXygIwM+PhjtyL/6ivo3BkGDoRvvw16edG9VwLliGMXrMVvLT6fYXSvNkqriIgneTeQA/h80K+fS7fcdhu8+KJLtzzxBOTllbg8sPdKp/S0g9Iq1lr27M2NwQRERMLn7UAecMQR8OijsHo1ZGbCLbdA+/aweHGpPxIs1SIi4kXxEcgDWrWCd9+F11+HX3+F7t3h8sth27YSlwZSLVdmNqJPxwYxGKyISGTEVyAH1yzUuzesWwdjx8KCBdCyJdx/P/zvfyUun7tiB68u20bfaVnal0VEPCn+AnlAjRpw772wfr1rJBo9Glq3hjfecM1FlN7KLyLiJfEbyAPS08l55G+88ehL7Es5BC65BHr0gA0b6NKkNsk+gwGSfKZEnlw7KIqIF3ins7OSDuyE+DtqXvow76SsoeETD0G7dhx33WBqHdmNH5IOLbF/S7AdFFWeKCLVUdyvyIumT/ZZH2+eeZkrV+zXj6OnPc27z9zAJWvex5+Xd1BqRWkXEfGKuA/kQcsM69WDadPY8MYidh5Vj8f+MYnZM0bS/ZdtZf+ciEg1ZGzBi7+qlJGRYbOzs6vseWXtcJjz5W5+mDKNM56fSMru72DAABg3DurWLfFz2ilRRGLJGJNjrc0o8fVECOQh+fFHV644eTLUquU+vukmSHavEZQzF5FYKy2Qx31qJWRHHgkTJx7oDh0xAjp0KOwOVc5cRKorBfLiAt2h8+bBL78Udoeefsi+oDnzoiWKKlcUkVhQaqUs+/bBI4/A+PFgDF/fdCtvnXM1J7c8lk7pacxcuo3R89eQ77ekJPvAWvL8lmSf4bKMhvTp2EDpFxGJGOXIw7F1K9x+O8yd6/ZAnzSJnJNO54qpWeT5D/z/ZwBb5ONDUpRLF5HIiXqO3BiTZIz5jzFmQaTuWW2kp8OcOfDee671v3dv6lzem/TvthdekuQzpCS5LlFwAV25dBGpCpHMkY8A1kfwftXPWWfBypUwaRINNqzkn88NZdTi5zlq/z7uv7gtswadwlWdG6n+XESqVERSK8aYBsCLwIPAbdbaXmVd77nUSjDffst3w2+n9mszyKtTj5SJj0DfvuDzqd5cRKIi2qmVx4E7gVJPQTbGDDLGZBtjsnft2hWhx8ZQvXrUefVlTFYWKcenw7XXQteukJNz0ElEIiLRFnYgN8b0Ar611uaUdZ21dqq1NsNam1G3bt1wH1t9ZGbCp5/CCy/AF1/AySfDoEEQD7+sRMQTIrEiPw24yBjzFfAq0N0Y80oE7usdPh/07+8247rlFhfUmzeHJ58k54tdqi0XkagKO5Bba++21jaw1jYGrgQ+sNb+MeyRedGRR8Jjj8GqVZCRAcOHc9ipmXz07OxyTyBSM5GIVFbc70ceE61bw8KFvH3/M7R7bCyzZo3iHy1PZ03bB+mUfnqJy4vu46JmIhGpqIi26Ftr/1VexUrCMIaj+11FrxunMKlrX87avJRr+p8HDzxQ4uzQovu45OZbZi7VGaIiEjrttRJFndLTeH7I70kdex+f/2s5vgvOd+eItm4N8+cXnh0a2PtczUQiUhkK5FEWKEVsd9qJrjt00aLC7lB69oSNG+mUnsaMgV3UTCQilaK9VmJh/3546ikYM8ZtzHXLLW6lfvjhaiYSkVJp06zq6L//hbvvduWKxxwDDz8Mf/xjiYOgRURAB0tUT0cfDc8/D0uXQsOGB7pDV6yI9chExEMUyKuDzEzIyoLnnoPNm10N+uDB8N13sR6ZiHiAAnl14fPB9dezctFSVvbph33uOdcd+tRTkJcHqGlIRIJTQ1A1krN1D31nrye32R9oNbALs9bM4ohhw2DqVDaOepC+a5J0+LOIlKAVeTVStDFoQ1oDXh77rCtZ/OEHWlx5IY/MnUC9H79TjbmIHESBvBoJNAYV1pE3rQOXXgrr17NzxJ2c83kWH0wbzLCs2ZxybK1YD1dEqgmVH1YzZdWRf/bRKmqOupOmHy6EJk3g8cfJaXcaWV9+r7pzkQSgOvJ4smgRjBgB69ezpGkn/nLWIL6u11B5c5E4pzryeHLOObBqFR8NuZsOO9bzzrSh3LroOXLWbI31yEQkBhTIvSolhRoj76DHkGd5s82ZDF46l/7X94RXXincjEtEEoMCuYd1Sk9j8i09+e8Tz7D+9YWkpjeEa65Rd6hIglEg97jA7oqtep/jukOnTYPPP3fdoTfeCLtVpigS7xTI40jO9h95qsnvWfneMhg+3AX1Zs3g6acLu0NFJP4okMeJwHFxExdu5MrZ68m5dQysXAnt28PQoW6FvmRJrIcpIlGgQB4ninaFFnZ+tm0L778Ps2fDnj3w+9/D1VfDjh2xHq6IRJACeZwo0RUaOF3IGPjDH2D9ehg9GubNg5YtYfx4+O232A5aRCJCDUFxJKTThb78Em6/HV5/HZo2hccfh169dDKRiAeos1MOtmgR+4YMpcYXn/P1qd24vv3VfH5Efe2sKFKNqbNTDpLTPIPMyx/lwW7Xc2R2Fm9NGcIdi6eTsvdX7awo4jHajzzBBFIoX/+wj19tEs9m9mF+627cueRFhiydQ591i/k1fTyc2VRnh4p4hFbkCaRoieKcnB0k+wxJBn5Kq03us88x56+vcdjxjWgyYhCccYYrXxSRak8r8gRStEQxP9/PlZmNOPaoGgdecHZuBEMuhRdegLvvhk6dYNAguP9+qFMn1sMXkVJoRZ5Aipco9unYgKHdTjj4xabPBwMGwKZNMGwYPPusOztU3aEi1ZaqVhJMWWWGQb+3Zo1r91+8GE46CSZPdmkXEalyKj+UMgXy50EPd7bWnR16++2wfTtcdRU8/DA0aBDbQYskGJUfSpmCtvgHGAOXXQYbNsC996o7VKSaUSAXIHiLf87WPTy1eDM5W/e4i2rWhLFjXbv/OefAqFFuP5d//CO2gxdJcKpakUJ9OjbAFPwvUHqq5fjjXYv/woUuf96rF1xwAUya5LbNFZEqpRW5FObHX122jbkr3M6IZaZaAs49F1avhkcfdVvktm3ryhZ/+aWKZyCS2BTIJWjQLnU3xeJSU91L0I0b3UvQCROgRQuYOVNnh4pUEQVyCRq0O6WnMWNgF247t0Vom2jVrw/Tp8Mnn7iP+/Z1+5+vWlUlcxBJZCo/FCDELXBDlZ9/oDv0++9h8GDXHVq7lFW9iIQkauWHxphDjTHLjDGrjDFrjTF/CfeeUvUChzhHZPvapCQYONB1hw4dClOnuu7QKVNckBeRiIpEauU3oLu19iSgPdDDGNMlAvcVr0tLc52g//kPnHgiDBnizg796KNYj0wkroQdyK0TKFNIKfhHb7nkgHbt4IMP4LXXYPduOP10+OMf4euvYz0ykbgQkZedxpgkY8xK4FtgkbV2aZBrBhljso0x2bt27YrEY8VLAt2h69e77tA5c1x1y0MPqTtUJEwRCeTW2nxrbXugAZBpjGkb5Jqp1toMa21G3bp1I/FY8aJatVx36Lp1cPbZcNddbsX+9tuxHpmIZ0W0/NBa+wPwL6BHJO8r3lKitT+YJk3gjTfgnXfc1rkXXOA6RDdvrrJxisSLSFSt1DXGHFXwcQ3gbGBDuPcVbyp6ClHfaVllB3OAHj1cd+gjj8C//w1t2rg9XNQdKhKycgO5MaadMebjIp93NMZ8UOSS+sBiY8xqYDkuR74g8kMVLwiptb+41FT4059cueKVV7pdFVu2hFmz1B0qEoJQVuRrgabGmKSCzycCfwp801q72lrbwVp7orW2rbV2bDQGKt4Q6BL1GTDGkFYzNfQfrl8fXnwRPv4Yjj4arr4azjxT3aEi5Sg3kFtr/bhg3sYYcymwzVq7IuojE0/qlJ7G6F5t8BlDvt8ydsHa8tMrxZ16Kixb5hqJ1q2Djh1dY9H330dn0CIeF2qOPAs4DbgPGBW10Uhc2LM3F7+1WCqQXikuKQluuOFAd+iUKa479G9/U3eoSDEVCeQPAK9ba9XFIWUKeefEUAS6Q1eudNvk3ngjnHyyS7+ICBB6IN+Aa8V/KIpjkThR4Z0TQ9GunTsA+tVXYdcu6NoVrrkGdu4M/94iHhdqIB8B3G2t/TWag5H4EdFNuAKMgSuucGeH3nOPa/lv0cIdBJ2bG7nniHhMmYHcGNPUGLMBqGGtfbGKxiRStlq14IEH3IvQ7t1h5Ei3Yn/nnViPTCQmygzk1tovrLUtrbUDqmpAIiFr2hTmzz8QwM8/Hy66CL74IrbjEqliOiFIqoWQ2vpL06MHfPaZ6w5dvBhat3apl1+VCZTEoEAuMVfhtv5ginaHXnEFjBvnukNffVXdoRL3FMglpnK27uHx9zbx2/4KtvWXpn59eOkld3hF3bruQOgzz3T7uYjEKQVyiZnASvzjzd9hcf8yhl13HnDaabB8uWsgWrsWOnSAYcPUHSpxSYFcYqboBls+A6c1qxO5unNw3aGDBrl0y5Ah8Mwz6g6VuKRALjFTtAM0NdnHLWc3DxrEw3oRCvC738Ff/+rODm3TxnWHZmbCJ5+EOQOR6sHYGLwIysjIsNnZ2VX+XKl+crbuIWvLbro0qV1qEO87LYvcPD+pyb6gK/by7nEQa+Hvf3cvRr/+2p0d+tBDcOyxkZyWSFQYY3KstRnFv54ci8GIBHRKTysz+Abb3zxwfc7WPcxbsYPZ2dvJ89tSA/1BjHF7nl94oatsefRRd1LR6NEwYoSrfhHxGKVWpForbQOuwEp95tJt5Obbile81KoFDz7oukO7dYM773Tdof/8ZxRnIxIdCuRSrZW2AVdgpR5IDBoqWfHStCm8+aY7/Nla6NkTLr5Y3aHiKQrkUu0F24Cr+IvSqzo3YsbALgCVezHas6frDp0wAd5/370U/fOf1R0qnqCXneJZxV9yhvJiNCQ7d7pUy4wZ0KCBy6NffrnLr4vEUGkvO7UiF88qvlIP9eDncssZjz0WXnkFPvwQ6tRxL0e7d3crdpFqSIFc4kYoJxNVaF+Xrl0hO9sdM7d6NbRvDzffrO5QqXYUyCVuhHIyUair9kJJSTB4MHz+uWskevpp1x06daq6Q6XaUCCXuFLeyUSVPk/0d7+Dp56CFSvcNrmDB6s7VKoNveyUhBN4SZpWM5U9e3ND6wgtylq3Pe4dd7ju0GuugYceIif30NA7TEUqobSXnQrkkpAiUuHyyy+uO3TiRPJTUpnY5XKe63gh5pBDIrv5l0gBVa2IFFHhXHmBgypeDjvMBfK1a9neLoM733+et58bRpdNy8PbU12kghTIJSEVz5Wn1Uwtt5Go1IqXE05g96vzGHzFfSRZP9NfG0PfccNhy5Yqmo0kOgVySUiBCpcrMxtxerO63PfW2nJLEstaxXdKT2PQQ8N5Z9Yidowcw1GfLnEvRe+9V92hEnUK5JLQ5q7YwXvr/htSmqW8ipdO6WkMOa8NDSbcBxs3wqWXwgMPQKtW8NprOjtUokaBXBJWRTfeCqVOvdBxx7kW/w8/hNq13YHQ6g6VKFEgl4R10MZbSYarCzbeKitAl1enXkKgO/Tpp113aIcOMHw47KnkaUciQaj8UBJahU4XCtfu3S5n/re/uQajcePg+utd96hICFR+KBJEhVfY4ahd263Mc3KgZUt3MHTnzvDpp9F/tsQ1BXKRcoR9+HNx7dvDkiUwcyZ88w2ceir06wf/93+Rub8kHAVykTJUaLfEijAGrrrKVbfcdZdr+W/e3O19npsbmWdIwlAgFylDZTtAQ3bYYTB+PKxZA2ec4fZvOfFEWLgwss+RuKZALlKKnK172PnDPpJ9puK7JRa5R0hpmWbNYMEC909+Ppx3HvTure5QCUlyuDcwxjQEXgKOAfzAVGvtE+HeVySWim6qlZzk44rMhlzasUGFXopWamOuCy6As8+GSZNcM1Hr1u7Yubvugpo1w5yVxKtIrMjzgNutta2ALsBQY0zrCNxXJGaKplTy8/0cd1SNCle2VDotc8ghLnAHukPvv99Vucyere5QCSrsQG6t/cZau6Lg45+B9cBx4d5XJJYqcgBFaemTovdI8hl2/rCPmUu3hV4BE+gOXbLE1Z1ffjmcdZbLp4sUEdGGIGNMY2AJ0NZa+1Ox7w0CBgE0atSo09atWyP2XJFoCKVZqLT0SeBnf963n0+37GbtNz+Rn2+xuK0ADkmp4B7o+fnueLl77oGffoKhQ+Evf4GjjorUdMUDSmsICjtHXuQBhwFzgVuKB3EAa+1UYCq4zs5IPVckWjqlp5UbaEtLn/SdlsVv+w/s41KUBXILrg05kCclwZAhblV+773w17+6OvTx4113qE91C4ksIn/6xpgUXBCfYa2dF4l7inhBsBRM8c24gvEZU+EKGOBAd2h2tsub33CD6w7Nyqr0HMT7wg7kxhgDPAest9Y+Fv6QRLwj2I6IgeAe+I/LZyAlyZUwGiDZZxh7cdvwtgXo0MHlzmfMgJ074ZRToH9/dYcmqLBz5MaYrsCHwGe48kOAUdbat0v7GW2aJfGu+AHPaTVTWbPzRwzQp4JljOX6+Wd48EF47DE49FC47z64+WZISYncM6Ra0OHLIjESkYOeQ/H553DLLfD22y7tMnkynHNO5J8jMaPdD0ViJOpt/gHNmsE//gFvvQX798O550KfPvDll9F5nlQbCuQiUVaRmvSI6NUL1q51+52/+67rDh0zBvbuje5zJWaUWhGpAlVxgEXO1j3MXbHj4Dz8jh2uxX/WLGjUCCZOdN2ixkRlDBJdypGLVFOVCfLFf2bm0m3c+8Zn5Bf855ya7GPWDUVy8UuWuBegq1e7s0MnT4Y2baI0I4mWqDcEiUjFVeZFaPGfGd2rDaPnrykM4nAgF194rzPOcCcTTZ0Kf/4znHQSDBvmKlzUHep5ypGLxFBlXoQW/5l31nyDv9jfrIPm4pOT4aabXHXLDTdgJ09m7/FN+erhyeD3I96lQC4SQ5V5EVq04cgYQ5v6R7jPDSQZOLf10QenVYqrXZucu8bRZ8Bk1h12DI1HjuDXjifD0qUHXRbxI+4kapQjF4mxyuTIZy7dxuj5a/BbW5he2bM3N+R7PLV4MxMXbsTvt1yy/l888MnL1Nr9resOnTCBnP+lVk3tu1SI6shFqqlO6WkM7XZChQLlnr25+K0tTK/s2ZtboXsU/k3AZ3jnpLPY9O/lMHKka/lv3px9Dz2K/7fc6Ne+S0QokIt4ULj7pRffI6ZDm0YwYYLb6/y00+j6zDjefuFmTt+6smpq3yUsSq2IeFQ4+6WXyVpYsIDfhg3nkG1fsadHL9KeeRIaN478JKRClFoRiTOhpGQqtT2AMXDhhRyycT08+CBpSz6AVq1cqaK6Q6slBXKROFaZqpjCVMx/98GoUbBhA1x8sTuRqFUrmDu33LNDy6p4UTVM5Cm1IhLnKlIVU1YqZuPfF1B71J+os2WjOzv0iSeCdoeWdY8q2wkyTim1IpKgAoddZG3ZXe4quLRUzMyl27hglY9T/vAwY3sMIS87x3WH3nor/PBDSPco73tl0Sq+bGrRF4lzFVkFB1Ix+/P8hamYnK17GD1/DXl+CyaJ6e0v4LjB/Rnwz+fdqnzGDHd26HXXgc8X9B5l3T+S409UWpGLxLmKrIKDHV2XtWU3+f4DKVifMbTv0AymTHFnhzZrBgMHQpcusHRpiXsAhavpYPeP5PgTlVbkInGuoqvgQHANBMwuTWpzSIqP3P1+fMXPG+3YET76yK3K77zTBfPrrqPT+PF06nZCqavpiqyoK7OKTzR62SmSAMJ94QmU+PkS9/z5Z7j/fnj8cahRA+67j2fa9uCRD7bgt24fmNvObcHQbidEdfzxTNvYiiSwiqyCg6Uyiterl5q3fvhhGDDAnR162230P2EKyzL7saTRSWGtpiu6ik80ypGLyEFCqT0vM2/dooU7AHr+fGr483hh5j0s/Hgys887tkQwVjVKZGhFLiIHCbyQLCuVUW7e2hi46CJ3APTEiTQdNw7O6cLOG0fw5nl/5ORWxwGoGiVClCMXkUqpUN56+3a+v2kEv1vwOjuOqMdD597A4VddzqvLt4edP08kypGLSIWUF6grlLdu2JBZtz3MR4efzJhFf+PJOQ+yedP7rDqlPxvSGoSUPw91k7BEfCmqQC4iJUSjCadLk9o80aQ9va6bTL9V73B31iwWrBvGqt7XYEePoUMZ9w9lPIHDNvL9lkNSEitVo5edIlJC1JpwrCXfl8TLGReydvFyzHXX0X7udDqcnQkvvFDq2aHljado96kFcvdHt3Gour2kVSAXkRIqs2tiebK27C4MtPl+y0c/Gpg6FZYtgyZN4Prr4dRTYfnyUsfjM+6c0rSaqSXuXfQAap/PXVM82EYiAAf+djBx4Ub6TsuqFsFcqRURKSGUypWKKrXSJSMDPv4YXnnFdYdmZrqgPn481KtXOJ7RvdoUpk7GLlhLi2MOLxxX4N65eX58xjCw6/GMXbC2RFNTJNJFwf52EOsUjgK5iAQV6SacMn85+Hxw7bXQu/eB7tC5c9k+YiQLuvYms/kxheeUWkoG0OL3Li0VE/habhgBuDpuGaBALiJVptxfDkccAY88AgMG8OPgoTQcO4rudaYw/rwbOffmvmUG0OL3Ln7txv/7mcDeX35LifRMReYQ7BdSLCtmFMhFJCbKDHwtW/LKmCmsfPpl/vz+s0yfMYrN2//N7HEPsyS3ZrnBMliwzdqyGwNY3MvBPXtzKz324r80Yr3VrgK5iFS5UAJfl6Z1eLLVKfQ4vgODlr/O0E9nk3xWZ9rdfRd0ubPcZxQPtmk1U0nyGfx+S2pKZFMiWVt289t+/0EVMwrkIhLXQnlhGFhVz1uxg2dSr2ROm+7c86/nOf+++2D6dHjsMZdTNyboMwIr/rSaqazZ+SNzcnbgtxafzzC6V5uIBtq0mqkEamb8VD5tU1kK5CJS5UJ9YRhIieT5LV8fXpebLxrJbwMHc8nzE6BPHzj7bJg82R0KXUTRFb+/2C4kBhtWWiWYPXtz8RmXe/eZ8NI2laE6chGpchU5Kah4TXujPufDf/7jAnh2Npx4Itx+O/z4Y+HPFF3xF2WIXF18aWNMjUElizbNEpFqr9QXo7t2wT33wLRpruZ8wgS49lpytv940IrcByQnGS7LaEifjg3K/MVR2eqT8n4uElUtpW2aFXYgN8Y8D/QCvrXWtg3lZxTIRSSisrPh5pshKws6d4YnnySn3gmFOfI9e3MrfTpSJHLpkbpvaYE8EqmV6UCPCNxHRKRyAt2h06fDV19B5850uv8OhrY5gqs7NypxwlFporXHTLQPkA47kFtrlwDfR2AsIiKV5/NBv36waRPcdhu8+CI0bw5PPAH794d0i2jsMRPN+wZEJEdujGkMLCgrtWKMGQQMAmjUqFGnrVu3hv1cEZFSbdgAI0bAwoXQpo17Odq9e7k/Fq0OzWqdIy+4eWPKCeRFKUcuIlXCWnjzTbj1VvjyS7j0Upg4EdLTYz2ySolmjlxEpHoyBi6+GNauhbFj3aHQrVq5j/fti/XoIkaBXETiX40acO+9Lt3SqxeMGQOtW8Prr7tVe5RF+yCKsAO5MWYW8CnQwhizwxgzIPxhiYhEQaNG8Npr8P77UKuW6w497zxYv77StywvSFfFQRSRqFq5ylpb31qbYq1tYK19LhIDExGJmu7dYeVKV9GybNmB7tCffqrQbUIJ0tEuPQSlVkQkUSUnw/Dh8Pnn0L8/TJrkyhVffLHUs0OLCyVIR7v0EBTIRSTR1a0Lzz4LS5dC48YuqJ92musWLUcoQboi+8pUlvZaEREJ8Pvh5Zdh5Ej49lsYMADGjXPBvhRVeTKQyg9FRMoT6A7duNF1h06fDs2auWaivLygP9IpPS3kLQCiRYFcRKS4I4+ERx+F1ashM9N1iHboAIsXx3pkQSmQi4iUplUrePddmDcPfvnFVbtcfjls2xbrkR1EgVxEpCzGwCWXwLp18Je/wFtvQcuW8MAD8L//xXp0gAK5iEhoatSA0aNdd+j557tO0datYf78KukOLYsCuYhIRaSnw5w58N57Lrj37g09e7oAHyMK5CIilXHWWa479PHH3clE7drBHXdUuDs0EhTIRUQqKyXFVbRs2uTKFidOhBYt4KWXQu4OjQQFchGRcNWr5w6AXrrUpV769YOuXSEnp0oer0AuIhIpJ58Mn3wCL7wAX3zhPh80CHbtiupjFchFRCLJ53P7tWza5E4meuEFtxnXk0+W2h0a9iOjclcRkUR35JEuZ756NWRkuJ0WO3SAHTsi/igFchGRaGrVyh0APW8eNG0K9etH/BHJEb+jiIgcLNAdesklUbm9VuQiIh6nQC4i4nEK5CIiHqdALiLicQrkIiIep0AuIuJxCuQiIh6nQC4i4nHGxuBkC2PMLmBrCJfWAb6L8nCqq0Sde6LOGxJ37ok6b6j43NOttXWLfzEmgTxUxphsa21GrMcRC4k690SdNyTu3BN13hC5uSu1IiLicQrkIiIeV90D+dRYDyCGEnXuiTpvSNy5J+q8IUJzr9Y5chERKV91X5GLiEg5FMhFRDyuWgRyY0wPY8xGY8xmY8xdQb5vjDGTC76/2hjTMRbjjLQQ5t23YL6rjTGfGGNOisU4o6G8uRe57mRjTL4x5g9VOb5oCWXexpgzjTErjTFrjTH/ruoxRksI/74faYx5yxizqmDu18VinJFmjHneGPOtMWZNKd8PP75Za2P6D5AEfAE0AVKBVUDrYtecD7wDGKALsDTW466ieZ8KpBV83DMe5h3q3Itc9wHwNvCHWI+7iv7MjwLWAY0KPq8X63FX4dxHAQ8VfFwX+B5IjfXYIzD3M4COwJpSvh92fKsOK/JMYLO1dou1Nhd4Fbi42DUXAy9ZJws4yhgT+YPvqla587bWfmKt3VPwaRbQoIrHGC2h/JkD3AzMBb6tysFFUSjzvhqYZ63dBmCtTaS5W+BwY4wBDsMF8ugcO1+FrLVLcHMpTdjxrToE8uOA7UU+31HwtYpe4zUVndMA3G/teFDu3I0xxwGXAFOqcFzRFsqfeXMgzRjzL2NMjjHm2iobXXSFMve/Aq2AncBnwAhrrb9qhhdTYce36nD4sgnyteI1kaFc4zUhz8kY0w0XyLtGdURVJ5S5Pw6MtNbmuwVaXAhl3slAJ+AsoAbwqTEmy1q7KdqDi7JQ5n4esBLoDjQFFhljPrTW/hTlscVa2PGtOgTyHUDDIp83wP1Grug1XhPSnIwxJwLTgJ7W2t1VNLZoC2XuGcCrBUG8DnC+MSbPWvtGlYwwOkL9d/07a+2vwK/GmCXASYDXA3koc78OmGBd4nizMeZLoCWwrGqGGDNhx7fqkFpZDjQzxhxvjEkFrgTeLHbNm8C1BW93uwA/Wmu/qeqBRli58zbGNALmAdfEwYqsqHLnbq093lrb2FrbGJgD3OTxIA6h/bs+HzjdGJNsjKkJdAbWV/E4oyGUuW/D/U0EY8zRQAtgS5WOMjbCjm8xX5Fba/OMMcOAd3Fvtp+31q41xtxY8P0puKqF84HNwF7cb25PC3Heo4HawNMFK9M8Gwe7xIU497gTyrytteuNMf8EVgN+YJq1NmjZmpeE+Gd+PzDdGPMZLt0w0lrr+e1tjTGzgDOBOsaYHcAYIAUiF9/Uoi8i4nHVIbUiIiJhUCAXEfE4BXIREY9TIBcR8TgFchERj1MgFxHxOAVyERGPUyAXAYwxi40x5xR8/IAxZnKsxyQSqph3dopUE2OAscaYekAH4KIYj0ckZOrsFClQcBrPYcCZ1tqfYz0ekVAptSICGGPaAfWB3xTExWsUyCXhFZzGMgN3UsuvxpjzYjwkkQpRIJeEVrBV7DzgdmvtetwOfPfFdFAiFaQcuYiIx2lFLiLicQrkIiIep0AuIuJxCuQiIh6nQC4i4nEK5CIiHqdALiLicf8PLkJkBfndDEAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "k = 2\n",
    "Xk = X.dot(V[:,:k])\n",
    "\n",
    "wk = np.linalg.pinv(Xk).dot(y)\n",
    "\n",
    "w = V[:,:k].dot(wk)\n",
    "\n",
    "evalw(w, x, y)\n",
    "print(\"w = {}, l(w) = {:f}\".format(w, l(w)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Jetzt betrachten wir zu den selben Daten $x_i,y_i$ das Regressionsproblem mit Modellfunktion\n",
    "\\begin{equation*} \n",
    "g(x,w) = w_1 + w_2 \\, x + w_3 \\, x^2 + w_4 \\, x^3\n",
    "\\end{equation*}\n",
    "\n",
    "Einfache Regression liefert das folgende Ergebnis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-24T09:28:14.643317Z",
     "iopub.status.busy": "2021-09-24T09:28:14.641736Z",
     "iopub.status.idle": "2021-09-24T09:28:14.722969Z",
     "shell.execute_reply": "2021-09-24T09:28:14.723295Z"
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "w = [ 4.9521 -6.5425  2.0585  0.5842], l(w) = 0.010212\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEGCAYAAABmXi5tAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAsp0lEQVR4nO3deXxU1fnH8c8zSaBsSlhklUBkkYIbiRJQFFRUEIFaVBCFai2CtGLVatWKiApaRShC4YdoFUVcQQVBxIpU1KAJsopoDAYRCohxhZrt/P64E8yeCZlkMpnv+/XKy5m5J/ee+1KfOXnuOc8x5xwiIlL7+ULdARERqR4K+CIiEUIBX0QkQijgi4hECAV8EZEIER3qDpSmWbNmrn379qHuhohIWElNTf3aOde8pGM1NuC3b9+elJSUUHdDRCSsmFlGaceCktIxsy/MbLOZbTCzYlHaPDPNLM3MNplZj2BcV0REAhfMEX4/59zXpRwbAHTy//QE5vj/KSIi1aS6HtoOARY4TzLQ2MxaVdO1RUSE4AV8B7xhZqlmNqaE422ALwu83+X/rBAzG2NmKWaWsn///iB1TUREIHgB/3TnXA+81M14MzuzyHEr4XeKFfFxzs1zziU65xKbNy/xIbOIiByhoAR859xu/z/3AUuA04o02QUcW+B9W2B3MK4tIiKBqXTAN7MGZtYo/zVwHrClSLNXgVH+2TpJwHfOuT2VvXZpUjMymb06jdSMzKq6hIhI2AnGLJ0WwBIzyz/fM865181sLIBzbi6wHBgIpAEHgauCcN0SpWZkMnJ+Mlk5edSJ9rHwmiQS4mKr6nIiImGj0gHfOZcOnFTC53MLvHbA+MpeKxDJ6QfIyskjz0F2Th7J6QcU8EVEqIW1dJI6NOGiT98lxuUSE+0jKb5pqLskIlIj1NjSCkcq4YtNJCyeyrjzfsNPc+YVG92nZmSSnH6ApPimGvmLSESpdQGfvn1h8mSOnzgR7m0M8+eDz/tDRvl9EYlktS/gA9x5J+Tmwt13Q1QUqbffT/IXmXz17SHl90UkYtXOgA9w112QkwP33cen63czrf84oqOjiPYZuXlO+X0RiTi1N+CbwT33kJr+NSMW/R/Zzrj7vLEM7xlH68b1lMMXkYhTewM+eEF/yhTmf/E117z/EkRH0W3sv0ho36RQMz3IFZFIULsDPnjB/Zl5bLjlaEa98Dj84x54+GHvy4DiD3InDupG5sEsBX8RqXVqfcAHf9B/bj60OQpmzPA+9Af9ggu1srLzmPjKFvKc0yweEal1IiLgA96I/uGHvdcFgn5SfFPqRPvIzsnDzHug64AszeIRkVomcgI+FA/6zpEwfToLr0kiOf0APxzKZu5/0gHIcxBbv07o+ioiEmSRFfChxJF+wvTpJMTFMnt1GoZXqN8HZB7MClEnRUSCL/ICPpSc3pk+naT4ptSN8dI7mqcvIrVNZAZ8KDm9M2PG4fSOZumISG0TuQEfwIzUCXcS9eW3nDxzJuTmkvDIIwr0IlIrRXTAT83IZORj68jq+Fv+lvQ9V8+eDbm5pN5yL8lfZGqULyK1StDq4ZtZlJl9ZGbLSjjW18y+M7MN/p+JwbpuZRyeg49x31lXkTp8DMydy+cXj+ThldsYOT9Z2ySKSK0RzBH+BGAbcFQpx99xzg0K4vUqreAc/JjoKJg6lZSoKC5dOAdyc7ljwJ80F19Eao2gBHwzawtcCNwH3BiMc1aHhLjYYg9pU++9l1kZ3/LHtYuIwdHu2mdD3U0RkaAI1gh/BnAL0KiMNr3MbCOwG7jZObe1aAMzGwOMAWjXrl2Qula2hLjYwiN4M2aedSVZ+Lhx7UIO3HQdvPAMqV/9oNk7IhLWKh3wzWwQsM85l2pmfUtpth6Ic879aGYDgZeBTkUbOefmAfMAEhMTXWX7diSS0w+Qk5vHzNNHkOeL4uaXF/DN0CxGn/h7DjqfauyISNgKxkPb04HBZvYF8Cxwtpk9XbCBc+5759yP/tfLgRgzaxaEawddfl4/ymD+mcPZdfvdNHntFaa9OIXo7OzDO2WJiISbSo/wnXO3AbeBNxsHL11zRcE2ZtYS2Oucc2Z2Gt4XTY2MmkXz+m3jBpDyP8f5D0/i/5bcx4RL/6YVuCISlqpsHr6ZjQVwzs0FhgHjzCwHOAQMd86FJGUTiIJ5/dSMTK6ofxoXX/BHprw+i5X/fohWd/YPcQ9FRCouqAHfOfc28Lb/9dwCn88CZgXzWtUlf67+MyddQI4vigdWzIQLL4SlS6Fhw1B3T0QkYEFbeFVbFczpv9rjfHbM+D945x04/3z47rtQd09EJGARXVohEAlxsUwc1I0VW/YwoHsr4nsOgNaxMGIEnHsurFwJTZqUfyIRkRBTwC9HakYmk5dtJSsnjw+/+IYuLRuRMGwY1K0Lw4ZB376wahW0aBHqroqIlEkpnXIU3PO20JTMiy6CZcsgLQ3OOgt27QptR0VEyqGA75eakcns1WnFiqXl5/B9gJkV3vawf39YuZLcr77iu9N6sXnthmrts4hIRSjg4y+TPD+ZaW9sL1YhMz+H7/MZec4xednWQsdT23Vn+CX3wDeZNBvYny2rPwzFLYiIlEsBnzLSNn6ZB7PIc67E48npB0ht0Ynhl0+lTm428b8dCJs3V/ctiIiUSwGfwlMvS9rLtqzj+cc+bRHPlaMeILpOjPcg90ON9EWkZrGauuA1MTHRpaSkVNv1UjMyy6yGWdbxQsfyvoVzzoH9+73FWX37Vs8NiIgAZpbqnEss8ZgCfvBtWreVtpcNpfF/d+Fb/BIMHBjqLolIhCgr4CulE2SpGZlcumwn/YfczcdN2uKGDIHnnw91t0REFPCDLf8B8IF6RzPysvvY0/Vkb1Xu44+HumsiEuEU8IOs4APenxs24r/PvwznnQe//z1rr7uD1IzMUuf8i4hUJZVWCLKi9fQdMPGq++j1xQ8MmDOFWZt3MLPPSHIcRPuMSxKP5eIebbWDlohUOT20rUL5C7p+zs7D8nKZsnI2wze9wZM9BjHp3DE482FA3RhtmygiwVEtD23NLMrMPjKzZSUcMzObaWZpZrbJzHoE67o1WX4+3wF5vihuu+BPzE/6LaPXL2P6soeJzs3BUfJiLxGRYAtmDn8CsK2UYwPwNi3vBIwB5gTxujVWwXx+nWgfI5LiOGXRPHbdMpGhH7/NvJenUD/n5xIXe4mIBFtQcvhm1ha4ELgPuLGEJkOABf5tDZPNrLGZtXLO7QnG9Wuqovn8wymbB+6G+Nb0GzeOVavuZ9/TL3CK0jkiUsWC9dB2BnAL0KiU422ALwu83+X/rFDAN7MxeH8B0K5duyB1LbQK7o9byLXXYo0b0+aKK2gz+jewYoVq6otIlap0SsfMBgH7nHOpZTUr4bNiT4udc/Occ4nOucTmzZtXtms132WXwauvwiefwBlnwI4doe6RiNRiwcjhnw4MNrMvgGeBs83s6SJtdgHHFnjfFtgdhGuHvwED+OTpJfxv736yknqr0qaIVJlKB3zn3G3OubbOufbAcOAt59wVRZq9Cozyz9ZJAr6r7fn7QKVmZDL0ozyGXDqFbw5lk3NGH1i7NtTdEpFaqMpW2prZWDMb63+7HEgH0oBHgeuq6rrhJn/q5vZmcVxyxYP8cFQTbyetZcVmtwKl78wlIlKeoK60dc69Dbztfz23wOcOGB/Ma9UW+VM3s3Py2N+0JTtfWUnstZfD0KFe/Z1Row63LbiQK8pnTB7Snct71o6H2yJS9VRaIcSKTt08KS4W3noLLr4YRo+GvXvh5pvBjOT0A/yc7S3kyslzTHxlC11aNtIKXREJiAJ+DVBs6majRl5KZ/RouOUW2LMHHnqIpPimRPmMnDxvglOecySnH1DAF5GAqFpmTVW3LjzzDEyYANOnw8iRJLSsz+Qh3Yn2GT7/6l2t0BWRQGmEX5P5fF6wb90abr0V9u/n8sWL6dKyV5nbMYqIlEQBv6Yz89I6LVvC1VdD374krFhBQr+Ooe6ZiIQZpXTCxahR3qbo27dD797w2Weh7pGIhBkF/HAyYACsXg3ff+8F/XXrQt0jEQkjCvjh5rTT4L33+F+DRmSf1Ze0+c8Ua6LFWSJSEgX8MJRapxlnX3wfHzc5lg5jriTj3mm/HPMvzpr2xnZGzk9W0BeRwxTww1By+gH+W/cohg+fytvxCcTdeTPcfjv45+Vn5eSR57STlogUpoAfhvLLMWTV/RXXXzaR/SNGw9SpMHo0vdo2OrzLlnbSEpGCNC0zDBUtx9C83YXw645w55302L2bRdMf472vszVPX0QKMa+uWc2TmJjoUlJSQt2N8PLkk3DNNdClCyxfDgV2DUvNyNRiLZEIYGapzrnEko5phF+bjB4Nbdt6hdeSkuC11+CUUw4/yM3KyaNOtI+F1yQp6ItEIOXwa5tzzoF334XoaOjTB5Yv14NcEQEU8Gun7t0hOdlL7QwezKD3l+pBrogEZRPzX5nZB2a20cy2mtndJbTpa2bfmdkG/8/Eyl5XytG6NaxZA+efT9wdN/L21yu58dxOSueIRLBg5PB/Bs52zv1oZjHAWjNb4ZxLLtLuHefcoCBcTwLVsCG88gpcfz0t5/yD8ft3wYIFoe6ViIRIpQO+f/vCH/1vY/w/NXPqTySKjobZs6FTJ7jpJvjyS+9LoGXLUPdMRKpZUHL4ZhZlZhuAfcAq51xJVb16+dM+K8ysWynnGWNmKWaWsn///mB0TcArsfznP8OSJeRu3sL3Jyey9Y13Q90rEalmQQn4zrlc59zJQFvgNDPrXqTJeiDOOXcS8AjwcinnmeecS3TOJTZv3jwYXZMCUk8+k0svn8qhnw7RbvB5fPrUS6HukohUo6DO0nHOfQu8DVxQ5PPvnXM/+l8vB2LMrFkwry3lS04/wEfNj2PoldPYddQxdPzdZTB3bqi7JSLVJBizdJqbWWP/63rAucAnRdq0NDPzvz7Nf11NBq9m+TV49h3dnCt+9xA/nHU2jBsHN9wAOTmh7p6IVLFgzNJpBTxpZlF4gfx559wyMxsL4JybCwwDxplZDnAIGO5qak2HWqxoDZ6j2w6Fm2+GGTPg009h0SI4+uhCv1OwJAOg8gwiYUy1dATmzYPx46FzZ1i2DDp0AChUkiHaZ2BGTq7KM4jUZGXV0tFKW4ExY2DlSti929tRa+1agMIlGXId2SrPIBLWFPDFc/bZ3h65sbHknXMOb972ILH16/xSkiHKiFF5BpGwpmqZ8ovOndnwwuscGnox595/C/PXrOOuhx7km59zlcMXqQUU8KWQd7/J4x+X3M2db87jmvdf4oubDtB+5Stw1FEACvQiYUwpHSkkKb4pvrp1mHT+dUwaMJ64lLVebf20tFB3TUQqSQFfCsmfunnjeV24aM492BtvwN693sPct94CvNk7s1enkZqRGeLeikhFKKUjxSTExf6SuonrBx98AIMHw3nnsfOuqYz8uRtZuU7TM0XCjEb4Ur7jjoP334eBA2k38RbuWjaT6OxsTc8UCTMK+BKYo46CJUvYM/5GRmxcyaJnb6f1oW9Jim+qFI9ImFBKRwIXFUWrWdP4/NfdOfGm8fz72b/web8W/CYlWxuki4QBjfClQlIzMnm9ax8+W7ySOvXq0umSCxn00SqtwBUJAxrhS8AK1tZ5JNrHs0veJH781Tz02nS6/zeNh877g1bgitRgGuFLwArV1snJ491Mx1Fr/s3eq8fyu9SlvP/WAyTUyw51N0WkFAr4ErD8evqF6ulER9PisTmwYAGNNqZCQoI3jVNEahwFfAlYwUVZxR7OXnklvPeet2l6nz4wf37oOioiJSo34JvZCWb2boH3PczsrQLvf2VmH/g3KN9qZneXcA4zs5lmlmZmm8ysR/BuQapTQlws4/t1LHkmzimnQEoKnHUW/OEPcO218PPP1d9JESlRICP8rcBx/h2tAKYBNxc4/jNwtn+D8pOBC8wsqcg5BgCd/D9jgDmV6bTUYE2bwooV8Ne/ehurnHkm7NoV6l6JCAEEfOdcHl7Q72ZmvwV2OufWFzju8jcoB2L8P0W30RoCLPC3TQYam1mroNyB1DxRUTB1Krz4Inz8MfTocbgOj4iETqA5/GTgdGAScHvRg2YWZWYbgH3AKufcuiJN2gBfFni/y/+Z1Ga//a33ALdpU+jfH+6/H/LyQt0rkYhVkYB/L7DEOfdV0YPOuVzn3MlAW+A0M+tepImVcM5im+ma2RgzSzGzlP379wfYNanRunb1gv4ll8Btt8FvfgPffhvqXolEpEAD/id4ufoHymrknPsWeBu4oMihXcCxBd63BXaX8PvznHOJzrnE5s2bB9g1qfEaNYJFi2DGDFi+HBIT+fj1taq/I1LNAg34E4DbnHM/FT1gZs3NrLH/dT3gXLwviIJeBUb5Z+skAd855/Ycebcl7JjBhAnw9ttk/XiQDhedy45p/2Tk/GQFfZFqUmbAN7PjzOwToJ5z7slSmrUCVpvZJuBDvBz+MjMba2Zj/W2WA+lAGvAocF1wui9h5/TTeXr2Yj5q3YWHXpvO3a/O4MNtxbKEIlIFzLliqfQaITEx0aWkpIS6G1IFUjMyGTXvXcavforr3n+ePXGdOfCvp+ne79RQd00k7JlZqnMusaRjWmkr1S4hLpYFY07nq7/8jWsuu5tf7d1D+wvOIn3246HumkitpoAvIZEQF0vrxvV4q0MCA6+ayafN44j/4+/hT3/S6lyRKqKALyGTX4xt39HNGTXq7+y95jqYNQvOOAPS00PdPZFaR/XwJWTyi7Elpx8gKb4pLeIuggv7w1VXeXV55s/35u+LSFBohC8hVawY29Ch8NFH3oKtSy+F666D//0vpH0UqS0U8KXmad8e3nkH/vIXmDMHevaE7dtD3SuRsKeALzVTTAz8/e/eytzdu72NVZ56KtS9EglrCvhSsw0YABs2eAF/1Cjv54cfQt0rkbCkgC81X5s2XnnlSZNg4UKv3PKHH4a6VyJhRwFfwkNUFNx1F588t4wfvv8J17u3l/JRuWWRgCngS1hIzcjkjiWbGfyRo8/l03mjY0+49VY4/3zYozp8IoFQwJcaLzUjk5Hzk3lm3U6ych3f1m3IdUP+yuob74V334UTT4SlS0PdTZEaTwFfarzk9ANk5eQd3jHHgJiYKI66/jpITfVy/IMHw7hxcPBgKLsqUqMp4EuNl1+CIcqgTrSPET3bsfCaJG+xVteusG4d3HQTzJ1L5vEnsG3Z6lB3WaRGUmkFqfGKlmA4vCo3X926pP7pDh75b1OmvjqNjkP6s+vmO2g7ZaL3sFdEAI3wJUwUK8FQRHL6Af5z7IlccNUsVnVKou3fJ8M558DOnYfbpGZkaltFiWiVHuGb2bHAAqAlkAfMc879o0ibvsArwA7/R4udc5Mre22RfPlpnx/rN+LGYbfTNfYLOky6FU44gR2THmB+XG9eSN1FTp6jTrTvl5SQSAQJRkonB7jJObfezBoBqWa2yjn3cZF27zjnBgXheiLFFE37dIgbCL+5gB+Gj6TDjeM4o3NvVpw/nm/qH012Th7J6QcU8CXiVDql45zb45xb73/9A7ANaFPZ84pUVLG0T3w8T015gqn9ruLszz9g5ePjOTdtHTHRPpLim4a2syIhENQcvpm1B04B1pVwuJeZbTSzFWbWrZTfH2NmKWaWsn///mB2TSJUz07H8OTpl/Cb0dM50CCW+S/dw9rPnsH3w/fK50vECdom5mbWEFgD3OecW1zk2FFAnnPuRzMbCPzDOdeprPNpE3MJltSMTJLTD9CrTUN6PDET98ADfNWoObcMnMD6405WPl9qlSrfxNzMYoCXgIVFgz2Ac+5759yP/tfLgRgzaxaMa4uUJz/V06NzS5gyhcUzniHH5+OZRbdz+/J/krL1yxJ/T7N6pLYJxiwdAx4DtjnnHi6lTUtgr3POmdlpeF80Byp7bZEj0X7weQz9ajYT/v0Eo1OWkj1uCzz1JJx55uE2+eUcsnLyNKtHao1gjPBPB64EzjazDf6fgWY21szG+tsMA7aY2UZgJjDcBSuXJFJBCXGxPDauLwf/Po3PnltK3Wgf9O0LN9xwuDRDfjmHPMfhWT0i4a7SI3zn3Fq88iZltZkFzKrstUSCJSEu1j9i7wgX9oW//hX+8Q947TV4/HGS4rtTJ9pHdk6eZvVIraGVtiINGsAjj3ibrOTmwplnkvDgnSwa3o0bz+sS1HSOngtIKKmWjki+fv1g82a44w6YOZNTli6lwb0Ps4qTACod9PVcQEJNI3yRgho0gBkzYO1aDsXUpfOoYbSYMI5xj6yq9KhczwUk1BTwRUrSuzdPzl7C7F6XMnTLWyybO5Z9Tywk9YtvAkrJlJS6KVjmWc8FJBSCtvAq2LTwSkItPwXTaddn3P/6TLr993Pe7JzExHPH8k2TY0pNyZSVuslfBFZimWeRIKjyhVcitVF+QbYLRg1k04tvsGjYHzk9/SNWzh/HZeteJTmt5PIfZaVuyivzLFKV9NBWpAz5gXnk/GR+Pu4C/vn7k5mycjZ3r5rLjwdS4cnHoXv3Qr+Tn7rRlE6paTTCFylHwT11v2rcknl3/JMd0+fScOcO6NHDm9Vz6NDh9vl/GQR7SqdIZSngi5Sj0J66MT5u6N+FDjdcC9u2wYgRMGWKN8p//fXDv6PUjdRECvgi5Sh1xN6sGTz5pLdgq04dGDAALr0UvvoqtB0WKYUCvkgAyhyx9+sHGzfCfffB0qXQtSvMnAk5OdXfUZEyKOCLBEOdOnD77bB1K5x+OkyYAKeeCu+9F+qeiRymgC8SJKkZmczOyCN1zkJ44QX4+msv+F91FezbF+ruiSjgiwRD/mKraW9sZ+Rj60g99Rzvoe5f/woLF0Lnzl6BNqV5JIQU8EUqKTUjkxlvfsrP2UUWWzVsCFOnegXZevaE66+HhARYuzbUXZYIpYAvUgn5I/t3077G4f0PVWyxVZcu3pTNl16CzEzo0wcuvxy+/PLwOVQyWapDpQO+mR1rZqvNbJuZbTWzCSW0MTObaWZpZrbJzHpU9roiNUHBMgo+g9M7NSt5sZUZXHyxl+aZOBGWLIEuXdh9421cPWeNlwqan6ygL1UqGCP8HOAm51xXIAkYb2a/LtJmANDJ/zMGmBOE64qEXKFFWdE+bji3c9mLrRo0gLvv9gL/oEG0nn4/y+eM4YJta8nOzlXJZKlSwdjicA+wx//6BzPbBrQBPi7QbAiwwL+PbbKZNTazVv7fFQlb+YuyyquAWaxKZvv28PzzbH9uGXkTJvDPV+7ng/Xd+bDZ33imfh0yD2apoqYEXVDLI5tZe+A/QHfn3PcFPl8G3O/f/xYz+zdwq3Mupcjvj8H7C4B27dolZGRkBK1vIqFSWrnk/C+BJnWjyH70US58bjaxh35gSfd+PNhnFN80OYZFf1AtHqmYssojB61appk1BF4CbigY7PMPl/Arxb5pnHPzgHng1cMPVt9EQqm0csn5XwLRPiO31Rk8eO3JXPf+C1yd8goDP3mXR08dyrJfNyEhrmeI70Bqi6DM0jGzGLxgv9A5t7iEJruAYwu8bwvsDsa1RWq6kna6KvQlkOvIzXP8ULcBD/T9Hef8YS6rOvXk+vef48/jB8H8+d7m6iKVFIxZOgY8Bmxzzj1cSrNXgVH+2TpJwHfK30ukKKn4WqEvgSijTrQPn0GUz9jTuAUTBt/CsNHT8B0XD3/4A5x0EixbBjV0hzoJD5XO4ZvZGcA7wGYgz//x7UA7AOfcXP+XwizgAuAgcFXR/H1R2uJQaruCD3KBEl8ntGsMixfDbbfBZ595c/gfeAB69Qphz6UmKyuHrz1tRWqQUve8zc6Gxx6DSZNg715vTv+UKd6iLpECtKetSBgoVI+n6CKsmBgYOxbS0mDyZHjjDejWDcaMObxiV6Q8CvgiNURZm58f1rAh3HknfP45XHcdPPEEdOoEf/6zKnJKuRTwRWqIkmbzlOqYY7xNVj77DEaO9F7Hx3v762aqPIOUTDl8kRqk1Bx+OcfYvh3uugueew4aN4abb/aqczZqVOI1Xlq/CwMu7tFWC7tqGT20FQlzpa3WLWbjRvjb32DZMnIax/LhsKv51Q3Xc0q3dgA8s24nd768mVz///Z1on1azVvL6KGtSJgLKL8P3nz9pUvZ9sqbrG3WkV7zp9Hh1O58dctEPtq6k4mvbDkc7KGcc0mto4AvEgYqlN8H3moUx9XD7mLIldP4qHUX2jx4D8f3Pplr33uOhj8fPNwukHOpXn/toZSOSJgoM4dfQtuR85PJys7D5zNmd8wh6alHOPrtN/mubgOeTBzMjsuv5ooLE8o8V3mppIr0SapHtRRPE5GqlRAXG3BQTYiLZeKgbkx8ZQt5zjFhRwwLn3ie+ps+ou4D93P9u4tgw6uwYxzceCO0alXieUpKJeX3IeDnClJjKKUjUktlHswiz7lCwbrrRWcTv/YNb5/dIUPg4YehQwcYPx527Ch2jrJSSQE/V5AaQwFfpJYqM+/fvTssXOhN57zySnj0UVzHjnx69iA+fm3N4WYlFX4L6PxSIymHL1KLBZpj35S8hQ//PIlL1y+nUdYhvu/Tl6Mm3gHnnOPtx1vJ80v10Tx8ESnT7NVpTHtjOw0P/cjIja/zx02v0eCb/XDKKXDTTXDJJVCnTqi7KQHQPHwRKVN+euaneg351xmXsv39Td7GK4cOwRVXeHn+qVPhQPXl6TUdNPg0whcRoJT0TF4erFwJ06fDqlVQrx6MGgU33EBqvRaVTueUlhLSDKAjV6UjfDN73Mz2mdmWUo73NbPvzGyD/2diZa8pItXE54MBA7xyzJs3w+WXexU6u3bl4Nn9+eifT3HlvHcLjcIDHZmXVQ5aM4CqRjDm4T+Bt5vVgjLavOOcGxSEa4lIFQhoRN29u5fmmTKFdbfeR+cXn2b+i5PZeXQLdn09mg3jxvD8joO8mLqLnNzyR+ZlzfHPTzFl5+QFPANID5DLV+mA75z7j5m1D0JfRCREygq+xRxzDNGTJnFOq3Ppu+09Rq9fRu95f+d/j8/g5K5nsqnHILa07FjueYoG9dj6dZi9Ou1wwF54TVLFVxYrBVSm6lpp28vMNgK7gZudc1tLamRmY4AxAO3ataumrolIRUfUCXGxPHntGSSnd8UXfwuL3nwPN2sWQ7eu5tLNb7KpZUde7DGA3lecUOY58oN6bP06TF62tVjADjRoV+gLK4JVxyyd9UCcc+4k4BHg5dIaOufmOecSnXOJzZs3r4auiQiUvcCqrN9Jim9KcvoBXPcTmDzoenqPf5K7zxvHMXWNycsf4ZSev/a2YSxlAkZCXCzj+3Uk82BWpXL2WgQWmCof4Tvnvi/wermZ/dPMmjnnvq7qa4tI4CoyoobiaZSJg7qReTCLpBvOo2W72bBuHV8//AhHL3iKmEcf9eb0X3219+C3SZNC5zqSnH3RvlckBRSpqjzgm1lLYK9zzpnZaXh/VeiRu0iYK5pGyTyYxfh+HQ8fT23VhZFdrqDuuCFc/Mka/rLzHer/6U/eQq4hQ+B3v4PzzoPo6KAE7Ip+YUWiSgd8M1sE9AWamdku4C4gBsA5NxcYBowzsxzgEDDc1dTJ/yISsPJG5flfCP+r24AFJw+k2S1/ZnyTn+Bf//Lq+Lzwglelc9QouPJKErp1U8CuYlp4JSJHrLw9eEfOTz78hVDo2UBWFrz2mhf8ly+H3Fxvt64rroARI6BNm4CuIcWplo6IhERAwXrfPm/z9aefhg8+wJnx1clJ5IwYwbfnX8TwF7ZpumUFKOCLSFjY8nYKb0+czqAtq2mfuYec6GjWxJ3CsuP78FannowZ3KPQc4Ki9NeAdrwSkSoUzCC7xjXm4dMv56FeIzh5z6dct3893d97g3M+/5CsqGgObjkXvhoJF10ERx9drB+BLr6K1C8GBXwROWLBXuGaFN+U6CgfWS6PDa278Mdjj2fSXfdRJ/UD+mx4mxavv+pt2BIT49XqHzoUBg+GVq0CWnyVmpHJ4vW7eCHlS3LyXJWniWraF4vKI4vIEQt2kbOEuFiGJbQlf8uV3DxH5qFsho2/hBaPzoaMDHjvPZgwAT77DMaOhdatoVcvhqx8mi7f7SYKV+Ksofwvp2fW7SQr11V5YbayisOFigK+iByxqljh+tsebakbU8o5fT7o1QsefNAL+Js3wz33QFYWbR+4mxVzxrD+qfG88ukLfPXcy6zfvufwr+Z/OeU/tTQgymfs/vZQsWAcjFr8NbHipx7aikilVEXa4ojOuXMnLF3Kdy+9Qt131vCrnCwOxtQl66yzaTxsCJu79eSSlXvIzskjKsrHWZ2bs+bT/cUqewYrTVXmtNQqpIe2IlJlqmKFa0XOWejLYfx4nv71+cx+bSOnZWzmnPQUhmzeAG+u4ARgQ1wH0k5Mot6FA3grtgX/zi2e8y84Ms+qRCG2mljuQQFfRMJWSaPxpPimPFKvPu90PJXk43vy69/3JOHQXli1il+tWkX31Uth6SLifT5ObdWZ99qdwPr2J9K7xYkAxNavQ54/8ZHnvPdHqugXV6gf4irgi0iNVlaQLClPPr5fxxJG1k2ga1e4/nrIzobkZGzVKjq+9jonfLCEqPdfgOcnQUICx3c4kbNz25DSpis//qohmQezgnYfoa7Zr4AvIjVWeUGyrE1USl2gFRMDffpAnz40nDwZfvrJm/mzZg2sWcPJi5/k8exs8jDSmrcjNuNM+KIf9O4NnTuDWcnnLUdy+gF+zvYeGmdlh6ZmvwK+iNRY5c2tD2QTlXI1aAD9+0P//qRmZJLy8S46pm+hQeoHdN2xhaNXLoNnn/LaNmkCPXtCYiIkJHj/bN06oC+B2Pp1Ds8QyqNyqaIjpYAvIjVWIHXy8/Pks1enHdGuV/mLsfb98HOBWTuNWXjXRI6Oi4W8PNi+Hd5/3/tL4IMPvE3dc3O9E7Rs6QX/hAQ48UTvJz4eoqIKXSfzYBY+854L+IygpYoqQgFfRGqsisx0OdKNz0c86qWMCir0heHzefn/rl29DVwADh6EjRu9nbxSU71/rljhfTkA1K8P3bp5wf+EE6BrV/o0bMWsKMjKDd2uXJqHLyK1RkVnwcxencZDK7dTMAoaUDfmCB6qHjwIH3/sLQbbtOmXn69/2dwvt159vmkTR1TXrjRJOBE6doQOHby/CFq0OOLnAwVpHr6IRISKrglIim9KTLTv8Ag/2geXndqOi3u0LfM8JX6x1K/v5fQTC8Ra52DvXi8l9MknRH3yCc0/+QQ2r4dlS7zj+erVg/h4vm11LJuOP5UGN/856A91g7Hj1ePAIGCfc657CccN+AcwEDgI/M45t76y1xURqayEuFgW/SGJxet34fDKOpQXZCs0vdLMy/G3bAlnnVX42KFD8MUXkJ4OO3ZAejrfbtnO3o0fs/OAj3sbJwd96mYwRvhPALOABaUcHwB08v/0BOb4/ykiEnIV/asgkKqcAalX75dnA34LV6cx7Y3t5DmIqsy5S1Hp4mnOuf8A35TRZAiwwHmSgcZm1qqy1xURCYWqKBhXHeeG6snhtwG+LPB+l/+zPUUbmtkYYAxAu3btqqFrIiIVU5U1cqq6/k51BPySHjuXODXIOTcPmAfeLJ2q7JSIyJGqioJx1XHu6qiHvws4tsD7tsDuariuiIgUUB0B/1VglHmSgO+cc8XSOSIikS4YG6+UJRjTMhcBfYFmZrYLuAuIAXDOzQWW403JTMOblnlVZa8pIhJOAlkQVh3VNCsd8J1zI8o57oDxlb2OiEg4CjSQB226Zxm0p62ISBUKdG/bqp6SCSqtICJSpQIt6lYdWyKqeJqISBWrzq0NVTxNRCSEqnJufUUohy8iEiEU8EVEIoQCvohIhFDAFxGJEAr4IiIRQgFfRCRC1Nh5+Ga2H8gIsHkz4OtyW9U+kXrfELn3Hqn3Dbr3QO89zjnXvKQDNTbgV4SZpZS20KA2i9T7hsi990i9b9C9B+PeldIREYkQCvgiIhGitgT8eaHuQIhE6n1D5N57pN436N4rrVbk8EVEpHy1ZYQvIiLlUMAXEYkQYRPwzewCM9tuZmlm9tcSjpuZzfQf32RmPULRz6oQwL2P9N/zJjN7z8xOCkU/g628+y7Q7lQzyzWzYdXZv6oUyL2bWV8z22BmW81sTXX3saoE8N/70Wa21Mw2+u+9VuyTbWaPm9k+M9tSyvHKxzjnXI3/AaKAz4F4oA6wEfh1kTYDgRWAAUnAulD3uxrvvTcQ6389oDbceyD3XaDdW8ByYFio+12N/84bAx8D7fzvjwl1v6vx3m8HHvC/bg58A9QJdd+DcO9nAj2ALaUcr3SMC5cR/mlAmnMu3TmXBTwLDCnSZgiwwHmSgcZm1qq6O1oFyr1359x7zrlM/9tkoG0197EqBPLvHOBPwEvAvursXBUL5N4vBxY753YCOOdqy/0Hcu8OaGRmBjTEC/g51dvN4HPO/QfvXkpT6RgXLgG/DfBlgfe7/J9VtE04quh9/R5vFBDuyr1vM2sD/AaYW439qg6B/DvvDMSa2dtmlmpmo6qtd1UrkHufBXQFdgObgQnOubzq6V5IVTrGhcsWh1bCZ0XnkwbSJhwFfF9m1g8v4J9RpT2qHoHc9wzgVudcrjfYqzUCufdoIAE4B6gHvG9myc65T6u6c1UskHs/H9gAnA0cB6wys3ecc99Xcd9CrdIxLlwC/i7g2ALv2+J9u1e0TTgK6L7M7ERgPjDAOXegmvpWlQK570TgWX+wbwYMNLMc59zL1dLDqhPof+9fO+d+An4ys/8AJwHhHvADufergPudl9hOM7MdwPHAB9XTxZCpdIwLl5TOh0AnM+tgZnWA4cCrRdq8CozyP8lOAr5zzu2p7o5WgXLv3czaAYuBK2vBCC9fufftnOvgnGvvnGsPvAhcVwuCPQT23/srQB8zizaz+kBPYFs197MqBHLvO/H+ssHMWgBdgPRq7WVoVDrGhcUI3zmXY2Z/BFbiPcV/3Dm31czG+o/PxZulMRBIAw7ijQLCXoD3PhFoCvzTP9rNcWFeVTDA+66VArl359w2M3sd2ATkAfOdcyVO5wsnAf57vwd4wsw246U5bnXOhX3ZZDNbBPQFmpnZLuAuIAaCF+NUWkFEJEKES0pHREQqSQFfRCRCKOCLiEQIBXwRkQihgC8iEiEU8EVEIoQCvohIhFDAF6kAM1ttZv39r+81s5mh7pNIoMJipa1IDXIXMNnMjgFOAQaHuD8iAdNKW5EK8u8u1RDo65z7IdT9EQmUUjoiFWBmJwCtgJ8V7CXcKOCLBMi/u9BCvJ2HfjKz80PcJZEKUcAXCYC/BPFi4Cbn3Da8io2TQtopkQpSDl9EJEJohC8iEiEU8EVEIoQCvohIhFDAFxGJEAr4IiIRQgFfRCRCKOCLiESI/wer6+u5aigodwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "npar = 4\n",
    "\n",
    "def g(x, w):\n",
    "    return w[0] + w[1] * x + w[2] * x**2 + w[3] * x**3\n",
    "\n",
    "\n",
    "wX = np.eye(npar)\n",
    "X = np.array([g(x, wX[k]) for k in range(npar)]).T\n",
    "\n",
    "w = np.linalg.pinv(X).dot(y)\n",
    "\n",
    "evalw(w, x, y)\n",
    "print(\"w = {}, l(w) = {:f}\".format(w, l(w)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Als Singulärwerte von $X$ erhalten wir"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-24T09:28:14.726559Z",
     "iopub.status.busy": "2021-09-24T09:28:14.726191Z",
     "iopub.status.idle": "2021-09-24T09:28:14.728802Z",
     "shell.execute_reply": "2021-09-24T09:28:14.729118Z"
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([12.3789,  4.0561,  0.8277,  0.0902])"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "wX = np.eye(npar)\n",
    "X = np.array([g(x, wX[k]) for k in range(npar)]).T\n",
    "\n",
    "U,s,VT = np.linalg.svd(X)\n",
    "V = VT.T\n",
    "\n",
    "s"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Hier würden wir bei $k=3$ abschneiden. Die Regression auf den reduzierten Daten liefert dann"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-24T09:28:14.732253Z",
     "iopub.status.busy": "2021-09-24T09:28:14.731845Z",
     "iopub.status.idle": "2021-09-24T09:28:14.816614Z",
     "shell.execute_reply": "2021-09-24T09:28:14.816941Z"
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "w = [ 4.853  -5.3505 -0.8073  2.4439], l(w) = 0.011277\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEGCAYAAABmXi5tAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAArnElEQVR4nO3deXgUVdbH8e9JCJuiRBZB2VcVkCUsEVxABVEQFVEQXEYHGUb0xXXcRhRHx21wQVBEREURBwTFAcQdQTQgAQRRwcgaUFmMKIKQkPv+UZ3YCZ2kQzrp7vTv8zz9JJ2qVJ0anNOVU/eea845RESk/IsLdwAiIlI2lPBFRGKEEr6ISIxQwhcRiRFK+CIiMaJCuAMoSM2aNV2jRo3CHYaISFRJTU3d6ZyrFWhbxCb8Ro0asWzZsnCHISISVcxsU0HbQlLSMbONZrbazFaa2SFZ2jxjzSzNzFaZWYdQnFdERIIXyjv8Hs65nQVsOxdo7nt1AZ71fRURkTJSVg9tLwCmOE8KUN3M6pbRuUVEhNAlfAe8Z2apZjYswPbjgS1+79N9P8vDzIaZ2TIzW7Zjx44QhSYiIhC6hN/NOdcBr3QzwsxOz7fdAvzOIU18nHMTnXMdnXMda9UK+JBZREQOU0gSvnNum+/rduBNoHO+XdKB+n7v6wHbQnFuEREJTokTvpkdYWbVcr4HegFf5dvtbeBK32idZGC3c+6Hkp67IKmbMhj/cRqpmzJK6xQiIlEnFKN0jgXeNLOc473mnJtvZsMBnHMTgHnAeUAasBe4OgTnDSh1UwZDJqVwICubihXimDo0maSGiaV1OhGRqFHihO+cWw+0DfDzCX7fO2BESc8VjJT1uziQlU22g8ysbFLW71LCFxGhHPbSSW5Sg4oV4og3SKgQR3KTGuEOSUQkIkRsa4XDldQwkalDk0lZv4vkJjUOubtP3ZRR4DYRkfKs3CV88JJ+oGSu+r6IxLJymfDzy7mr3/rLPtX3RSRmlb+E7xxMnw4DBkB8fJ67+grxcVSIMw5mO9X3RSTmlL+Ev2ABDBoEffrAtGl5Ru0cPJjNoM4NOK56FdXwRSTmlLtROvToAc8+C/Pnw6mnclqlfXlG7fTvUI8RPZrlSfaaqCUisaD83eEDDB8OTZrAJZdwcv+evPncq3x0VKMCR+34P8gd1bcVGXsP6C8AESl3yt8dfo5eveDzz6FqVU4cdD4jdq0MmMD9Sz4HMrMZNfsrxry3liGTUnTHLyLlSvlN+AAnnQRLlkCHDnDJJfDvf3sPdf34T9SK8z3QzXZwwDeKR0SkvCjfCR+gVi348EMYPBjuvhv+8hfYvz93c85ErZt7tWToqY1zezZnO0isWjEsIYuIlIbyn/ABKleGV1+F0aNhyhSv3LPrz7v3pIaJjOjRjGpVEnIb98cBGXsPhCVcEZHSEBsJH8AMRo2C116DlBRIToZ16/LsktykBpUSvPJOxQSN0xeR8sWcO2ThqYjQsWNHt2zZstI5+OLFcOGFcPAgvPkmnHFG7ib12hGRaGZmqc65joG2xc4dvr9u3byHuXXqkN2zJx/e/kjuiJyc8o6SvYiUN7GZ8AGaNGHl6/P4vF4rznr0DpYNHk7qBq+ur4lYIlIehSzhm1m8ma0wszkBtnU3s91mttL3GhWq85bE4l1ZXD3gPl5rew5/+2w6R199BcvXbmPIpBSNxReRcieUd/gjgW8K2b7IOdfO97o/hOc9bMlNahBXqSL39L6eR8/6K00Xzue4i87jqN278nTUFBEpD0KS8M2sHtAHmBSK45WV3DH455zAWS88hs2cSe0N65g55VZa7NhEfJxppI6IlBuhusN/EvgHkF3IPqeY2Zdm9o6ZtQq0g5kNM7NlZrZsx44dIQqtcHke0l50EWunz6FS1gHeePU2um5Ykbuf6voiEu1KnPDNrC+w3TmXWshuy4GGzrm2wNPAW4F2cs5NdM51dM51rFWrVklDOywfHdmAi656nK1H12bS66P47alxuQ3WVNcXkWgWijv8bkA/M9sIvA6caWav+u/gnPvVObfH9/08IMHMaobg3CGX3KQGu445loGXP8qnTZPo/sQo4m67jczMLNX1RSSqlTjhO+fudM7Vc841AgYBHznnLvffx8zqmJn5vu/sO29EZs2cuv7fzm9Ptflz4frraT/jBZ598yGOOPCHVsoSkahVav3wzWw4gHNuAjAA+LuZZQH7gEEuUqf4kncR9NRb72f+ukzueP95Xp92B99Nek2TskQkKsVma4ViGP9xGmPeW0v375by9NuPkp2YSLX358PJJ4c7NBGRQ6i1Qgnk9Mv/pHlnLr/qMSoZcOqp8M474Q5NRKRYlPCLkNQwkVF9W9G1WU0uGdqPiqlfQLNm0LcvPPNMuMMTEQmaEn4RUjdlcP+cNSxO28n9c9aQmlUVFi6EPn1gxAi46Sav66aISIRTwi+C/5q3uUMyjzzSa6s8ciQ8+ST07w979oQ7VBGRQinh+xQ0kzanhh8HmNmfyx7Gx3vJftw43Jw57GjfhVVL1pR53CIiwVLCh0Jn0ubU8OPijGznvLKO3/bUvoP526X3UmXzBmr3PIOv5y0MxyWIiBRJCZ8CyjZ+MvYeINu5gNtT1u/ig0ZJXDLkERxG84vPhblzy/oSRESKpITPn2WbeCPgTNrCtudsW3dsEwZe8wQHmjWHfv3g6afL+jJERAqliVc+Ra1lW9j2PNtqVoTBg+Htt+GGG+CJJ7x6v4hIGShs4pUSfilIXb+T+Nv/Qbs3XvTG60+b5o3sEREpZZppW4ZSN2Uw5MUv6N/sYu47dwRu3jw47TRITw93aCIS45TwQ8z/AfArbc9lzoMTIS0NunSBFSuKPoCISClRwg+x/A94j7usPyxezAGMzK7dSHthmlbPEpGwUA2/FPg/xAWYtTydBQu+ZML00bT68XseOnsoLyT1o0J8HJd0rE//DvXUcllEQkIPbcMkZ0LX/sxsHFDlwB88MXcMvdd9zpT2fRh99jCy4+KplBDH1KHJSvoiUmJl8tDWzOLNbIWZzQmwzcxsrJmlmdkqM+sQqvNGspx6fs5H6h8VKzNywN1M6tKfK1fMZdLM+zli/14tmygiZSKUNfyRwDcFbDsXaO57DQOeDeF5I5Z/Pb9ihTgu69KA14Z1pf1/J/Hm8FGctmEFb0z9Bw1+36llE0Wk1IWkpGNm9YCXgQeBm51zffNtfw5Y4Jyb5nu/FujunPuhoGOWh5IOFD5ha92rs2g4/Criqh5Bwtz/QadOYYpSRMqLsijpPAn8A8guYPvxwBa/9+m+n+VhZsPMbJmZLduxY0eIQguvpIaJjOjRLGB9vsXl/am0JIWEqpXhjDNg1qwwRCgisaLECd/M+gLbnXOphe0W4GeH/GnhnJvonOvonOtYq1atkoYWHVq1giVLoG1buPhiePRRiNAH6SIS3UJxh98N6GdmG4HXgTPN7NV8+6QD9f3e1wO2heDc5ULqHxWZcO/z/NznQrj9dhg2DDIzwx2WiJQzJU74zrk7nXP1nHONgEHAR865y/Pt9jZwpW+0TjKwu7D6fSzJGbr56Ceb6NZuKD9cfwtMmgS9e0OGJmaJSOiU2kxbMxtuZsN9b+cB64E04HngutI6b7Txb8Vw4CDM6j8cXnoJFi2Crl3h++/z7K9ZuiJyuCqE8mDOuQXAAt/3E/x+7oARoTxXeZEzdDMzK/vPXvs9roJGjby1cpOT4a23oFu3PBO54uOM+y9ozeAuDcJ9CSISJUKa8KX4khomMnVo8qFDN884A1JSoE8fOPNMmDyZlOO65M7azcp2jJr9FS3rVNMMXREJipqnRYACh242b+4l/VNOgcsv56K3JhLvN94p2znN0BWRoCnhR7pjjoH33oO//IXjxj7GB8snUjU7kzjf7F3N0BWRYKmkEw0qVoTJk6FlSxrdeSdLd/7A9Lufpm1SC5VzRCRousOPFmZwxx0wYwZHfr2aa24ZRNJvW8MdlYhEESX8aDNgACxcCH/84Q3bnD8/3BGJSJRQwo9GnTrB0qXQpIk3imfcuHBHJCJRQAk/SqVmH8nEf7/ML2f2ghtu8F5ZWd42Tc4SkQD00DYK5UzAOpCVzZOdR/BR46bUGTcO0tJY8dgEhvz3aw5kZVOxglbSEpE/6Q4/Cvm3Y9ifbcy87EaYOBE++IAG5/ek9s5tZDu0kpaI5KGEH4X8V9LKbcdw7bXw7rsc/fN23ppyM523fv3nNhERtIh51CpwJa116/ij93kkpG9m8yNjaXzT8IIPIiLlTmErXqmGH6WSGiYGrs23aEHlZUthwAAa3/x32LEZHniA1C27C1xqUURigxJ+eXTMMfDuuzBiBDz0EBkrVjP05GvYHVdRD3JFYphq+OVVQgI89xw8/jjV353LlJdvo9avO/UgVySGKeGXZ2Zw002kvTCNJhnbeHvKzbTfnqYHuSIxKhSLmFc2s6Vm9qWZrTGz0QH26W5mu81spe81qqTnleA1v3ogG//3PlWOrMr0qXeQtOT9cIckImEQihr+fuBM59weM0sAPjWzd5xzKfn2W+Sc6xuC88lhaNWrG3yZ6q2iNXAgfP013Huv91eAiMSEUCxi7pxze3xvE3yvyBzrGetq14YPP4SrroLRo2HQINi3L9xRiUgZCUkN38zizWwlsB143zm3JMBup/jKPu+YWasCjjPMzJaZ2bIdO3aEIjTJJ/XHvYy/8m7S77wPZsyA00+HrWqzLBILQjrxysyqA28CNzjnvvL7+VFAtq/scx7wlHOueWHH0sSr0PPvwVOxQhxzGmbQ7Ka/QbVqMHu214VTRKJaYROvQjpKxzn3C7AA6J3v57/mlH2cc/OABDOrGcpzS9H8e/BkZmXzbtPO8NlnUKmSd6c/bVq4QxSRUhSKUTq1fHf2mFkV4Gzg23z71DHzng6aWWffeTUYvIwF7MHTpo3XW79TJxg8GO65B7Kzwx2qiJSCUIzSqQu8bGbxeIl8unNujpkNB3DOTQAGAH83syxgHzDIRWoTn3IsqWEiU4cmH9pioVYt+OADuO46eOABbwTPlClwxBF5evYAas8gEsXUPE3+5ByMHQs33wxt2rB6/BQumb+VA1nZVIgzMCProPrsi0SyMqvhS5Qzg5EjYe5c2LiRpn17cPKG1V7N/6Aj06/+r/YMItFHCV8OkXpiF6Y+NZ2so6rz6rS7uWzVuyTEGwn56/8iElXULVPy8B+6+dQlDzH70/E89M7T3HTsfrb88wFSNu9WDV8kSinhSx7+Qzd3JRzBmw88x3XvPE/txx+n9pbvSZo+HY5RsheJRirpSB75h252aX4sjBkDkyfDokXQuTOsWRPuMEXkMGiUjhyiwOUTP/sM+vfn4J7fmX/Hf6hzxaUq7YhEGI3SkWJJapjIiB7NDk3mXbuy6s33+eaoOvS5ZziLrhpJ6safwxOkiBSbEr4Uy6I/qnDJ4Ed486Tu3PjJKxx91RDYs6foXxSRsFPCl2JJblIDV6UKt55/C4+e9VeafvoeezslM2XaAlI3ZYQ7PBEphBK+FEtOe4abzzmBs154jLTJr5O1YSN9/3oBY//5nJK+SARTwpdiyf9A970G7bnwqsfZVfVoXph6N3sfG+O1aBCRiKNx+BK0/P30pw5NJrlJDZ6uXZ8BV45hzNzHOXv8g/DbFpgwAapUCXfIIuJHd/gStPz99FPW78ot8Qw7vz2J8+fAffd5nTZPPx22bAl3yCLiRwlfghawnz5+wzgb1/AWRp89G9auhY4dvclaIhIRlPAlaLkPbHu1LLw9cr9+3qIq1avDmWfCM8+ori8SAYpM+GbWxswW+73vYGYf+b2vbGZLfQuUrzGz0QGOYWY21szSzGyVmXUI3SVIWSpwUlZ+J5zgJf3evWHECLjmGvjjj7IJUkQCCuYOfw3Q1LeiFcAY4Fa/7fuBM51zbYF2QG8zS853jHOB5r7XMODZkgQtUeLoo73yzr33wksvwamnwubN4Y5KJGYVmfCdc9l4Sb+VmV0MbHbOLffb7nIWKAcSfK/8f79fAEzx7ZsCVDezuiG5AolscXHeg9y334bvvoOkJPjooyJ/TURCL9gafgrQDbgPuCv/RjOLN7OVwHbgfefckny7HA/4D9lI9/1MYsX558MXX0Dt2tCzJzz+uOr6ImWsOAn/AeBN59zW/Budcwedc+2AekBnM2udbxcLcMxD/t9uZsPMbJmZLduxY0eQoUnUaNECUlLgoovglltg8GD4/fdwRyUSM4JN+N/i1eofKWwn59wvwAKgd75N6UB9v/f1gG0Bfn+ic66jc65jrVq1ggxNokq1ajBjBun/GEX29OnsS+oE69aFOyqRmBBswh8J3OmcO+R2zMxqmVl13/dVgLPxPiD8vQ1c6Rutkwzsds79cPhhSzRL3fwLZ1dM5qpL72fflq0c7NjRe7grIqWq0IRvZk3N7FuginPu5QJ2qwt8bGargC/wavhzzGy4mQ337TMPWA+kAc8D14UmfIlGOTN2FzVsR7+rnmTncY3hwgvhrrvg4MFwhydSbmnFKylzOT15MrOyiY+PY1CbWtwwexy1pr0MZ58Nr70GKumJHBateCURJWfG7qDODcA5Xl35E6c1HcjGR8d6rRiSkmBJ/oFeIlJSSvgSFkkNEzmuehWysl1uM7a5Hc+FxYshPh5OOw3Gj9fQTZEQUsKXsAnYjC0pCVJToVcvuP56GKIlFEVCRTV8Cav8C6rkys6Ghx+Ge+7x+vLMnOl9FZFCqYYvEavAZmxxcd6onffegx07oFMnmD49PEGKlBNK+BLZzjoLVqyAk0+GgQPhhhtg//5wRyUSlZTwJfIdfzwsWAA33QTjxnldNzdsCHdUIlFHCV+iQ0KC13Bt1iyv62aHDl4HThEJmhK+RJeLLoLly6FJE7jgArjtNsjMDHdUIlFBCV+iSuqmDMZvymb5a3PguuvgP/+B7t21YLpIECqEOwCRYKRuymDW8nRmLNtCVrajYoU4pv7jAZJOPx2GDoV27WDKFOjTJ9yhikQs3eFLxMvpvfPaks0cOPjnzNyU9bu8kTupqdCgAfTtC7feCgcOhDtkkYikhC8RL6e7Zs4UQcNvZi54C6t8/rlX4hkzBk4/HTZuDFO0IpFLCV8inn8LhooV4risSwOmDk3OO1mrcmVS//EA8+8dy8Gvv/ZKPLNmhS1mkUik1goSFQpsweC3fcikFA5kZdPkt5+Y/el4jvhyOYwY4T3YrVw5DFGLlL3CWivooa1EhaSGiQETfY6csk+2gw3VjmXKo6/w9/mT4IknvJbLr79OatU6hX5oiJR3JS7pmFl9M/vYzL4xszVmNjLAPt3NbLeZrfS9RpX0vCL+8nfe7NyyrjdRa+5cMrdu40D7Dsy87j7GvPstQyalkLopI9whi5S5UNTws4BbnHMnAsnACDM7KcB+i5xz7Xyv+0NwXpFcOYuq3NyrZZ76fmqrUzjz8idZWqcl/543lrGzH6Xynl+9ET4iMabEJR3fYuQ/+L7/zcy+AY4Hvi7psUWKI1DZJ2X9LrZWqc4VA//F8CUzuWXhK7T7YR17TnsZaBaeQEXCJKSjdMysEdAeCLQ+3Slm9qWZvWNmrQr4/WFmtszMlu3YsSOUoUmMyin1xMXF8cKpA3n+3y9Tq1olWl7ahyXX3Ejq+p3hDlGkzIRslI6ZHQl8AjzonJuVb9tRQLZzbo+ZnQc85ZxrXtjxNEpHQiX/CJ8VqzeRftlfOH/NApbVb0Wl16bS5tS24Q5TJCRKfQEUM0sAZgJT8yd7AOfcr865Pb7v5wEJZlYzFOcWKUr+RVY+25nJyPNvZWTfW2j503pa9D4Vpk075PdSN2Uw/uM0PeCVcqPENXwzM+AF4Bvn3OMF7FMH+Mk558ysM94HjZ6aSVjklHnmtO7BVw1bMTtlApUGD4Z587yF0486Ks+4/ooV4g6d6CUShUIxDr8bcAWw2sxW+n52F9AAwDk3ARgA/N3MsoB9wCAXqTO+pNzLGdHjlXm6cuTxV8CDD8L998PixTBlCimZdXLH9ef07VHCl2inmbYiOT77DK64AjZu5IfhI+l1dHf2ungSdIcvUUSLmIsEo2tXWLkSrrmGus88wZLZ/+SB5qFN9nouIOGkhC/ir1o1eP55mD2bqjt+ZODfL2LvY2NI3VDyR045zwXGvLdWs30lLJTwRQLp148v5y3ik4ZtOW38g+w/82xWfb66RIf07/eT289fpAwp4YsU4NPf4vlr/3u4vfcNtN36LS17dmXDmGcY/9F3Rd6dByrd5O/3k9vPX6SMqFumSAGSm9SgYkI8b7Q7h9Sm7Xht8XM0vnUELZp1YWTfG3jqxvOKbNXsP6Qz7+ggdeyUsqeEL1IA/wSdWLUNt7Y9kRavT+a2T17m7eeu44uj/gWj/++Q3wtUuslJ7kW1eRYpTUr4IoXISc5DJqWwPzObhZ0u5JMmSYyZ9wTn3D8Svl3sTdaq+efE8ZzSTWZWtko3ElFUwxcpgv+aunFAneT2ZC781Jus9eab0KoVzJyZu39BrZpFwk0JX6QIedbUTYjjxrNbkNS0Ftx1FyxbBvXqwYABcOmlsH07cGj/HpFIoJm2IkEodE3dzEx47DEYPRqOOgrGjfOSv1l4gpWYppm2IiVU6B17QoJ3t798OTRuDIMGwcUXw48/ln2gIoVQwhcJlVatvH48jzzidd486SR48UWI0L+iJfYo4YuESOqmDMYv2kjqwGu9njytWsE110DPnvD99+EOT0QJXyQUDumTU+VY+OQTePZZWLoU2rSB//wHsrLCHarEMCV8kRJK3ZTBkx+sY39mvj45cXEwfDh88413l3/bbdClC6xYEe6QJUYp4YuUQM6d/eK0nbnj9A+ZbHX88fDWWzBjBmzdCp06wa23wp49ucdQy2QpCyVO+GZW38w+NrNvzGyNmY0MsI+Z2VgzSzOzVWbWoaTnFYkE/m0U4gy6Na8ZeLKVmTdW/+uvvbr+mDFw0kmkPT9VLZOlzITiDj8LuMU5dyKQDIwws5Py7XMu0Nz3GgY8G4LzioRdnklZFXyTsgqbbHXMMTBxInz6KRx9NM2GXc7Y/97Psbt3qGWylLqQT7wys9nAOOfc+34/ew5Y4Jyb5nu/FujunPuhoONo4pVEi0InZRW2T2Ym6fc8SI0xD5NtxtjTL2fv8Os4sX4NMvYeUEdNOSyFTbwKafM0M2sEtAeW5Nt0PLDF732672d5Er6ZDcP7C4AGDRqEMjSRUlNUB8yC2iWnbttDyjmXU6/L2TQefQd3fjiJb1d9wKiew1lavzUVK8Qx7Vr14pHQCVnCN7MjgZnAjc65X/NvDvArh/xp4ZybCEwE7w4/VLGJhFNBK13lfAhUiDMOnnsnZzf+jHs+ep7pr93BrFY9eKj71cxanq6ELyETkoRvZgl4yX6qc25WgF3Sgfp+7+sB20JxbpFIF6hdcp4PgYMOB7zbsiufNOnAiM9nMGzpTHp+l8Ine6+H8/8NFdTJXEouFKN0DHgB+MY593gBu70NXOkbrZMM7C6sfi9SngRql5xnucN4o2KFOOIMMitV4ckzrqD3NeNZUe8k+r74GLRvDwsWhPsypBwo8UNbMzsVWASsBrJ9P74LaADgnJvg+1AYB/QG9gJXO+cKfSKrh7ZS3vk/yAUO/b7xMSStXAg33gibNnkN2R57zGvQJlKAwh7aqj2ySAQJOJpn3z5v3P5DD8HBg96krTvugCOPDG+wEpHUHlkkChzSjydnElaVKvDPf8Latd7krQcfhJYt4dVXITu78IOK+FHCF4kQBY3myVWvnpfkFy+G446DK66AU07xJnGJBEEJXyRC5HmQW9ji5127wpIl8NJLkJ4Op53m1fe/+65M45Xooxq+SAQpbNZuwG1798Ljj8PDD8P+/XDddXDPPVCzZqHnmLk8HQP6d6incf7ljB7aikS5gmbr5vrxR7jvPnj+eahWDe66i+UXXM7n2/bm+YB4bclm7nlrNQd9/7fXbN7yRw9tRaJckfX9OnVgwgRYtQq6dYPbb+e4Tiez5bGxXDlxMambMkjdlMGo2V/lJnso4FhSbinhi0SBoOv7rVrB3Lm8+cSrbD2qFg+/8zT/mzCcHS++Ssr3O8nO9xd9ocfyUb/+8kPztUWiQM5s3aK6cuZocNF5DNmZyGnfpHDbwin0Hv1//N6mHctbD+DjBm0xM8468Vj+dkbTw2r85r892Jgk/JTwRaJEUV058+876vzWjMqGj5t15OJvF3J/6nRemPZPtrXpyG93jaLloIBl3jwClZJyYijyuYJEHJV0RMqpjL0HyHaOLIvnjZN6MPmFd2DcOI7btY2Wl/WDHj1g4cJCj1FYKanI5woScZTwRcqp/Mm68wnHwYgR8P338NRT8O23cMYZcPbZsHhxwFp9oMZvBR2/qGcBEn4alilSjhVaY9+3zxvZ8/DDsH07nzdqy7jkS0lt2o6p154SVHlGNfzIo3H4IlKw33/n09sepPkrz3Hsnp9ZflxLto24mb53Xustvi5RRePwRaRgRxxBldtvo+f1k/nnOddR+/df6Hv337w+/NOnex06w0DDQUNPCV9ESGqYyIvDT6fu7Tfx07JV8PLLXquGgQPhhBPg2We9Ng5+QpGQCzpGgZ1DpURCseLVZDPbbmZfFbC9u5ntNrOVvteokp5TREpRQgJceSV89RW88QbUqOH16GnY0GvfsGNHoQk52A+Cwo6hEUClIxTj8F/CW81qSiH7LHLO9Q3BuUSkFBQ4pv7ii6F/f68l82OPwejR8MgjVOp5IcfX6c73x9TLk5BnLk/njdR0sg4WPTa/sDH+gdYBDuYa9AC5cCVO+M65hWbWKASxiEiYFJZ8MYNTT/Vea9fC449z0ksv8+GB11nYuD2vde5HYuVWDJmUwv7MbHKGgRxynHzyJ/XEqhUZ/3FabsIuzsxiTQILTlnNtD3FzL4EtgG3OufWBNrJzIYBwwAaNGhQRqGJSNB31C1bwnPPEfevf7H10afo8NILnP7f0exe9CLrT+jF9DZn82vlIzGKHpvvn9QTq1bk/jlrDknYwSbtQj+wJFdIhmX67vDnOOdaB9h2FJDtnNtjZucBTznnmhd1TA3LFClbh1MSSU3bzs5XXqf97Fep/eUX7E2oxOzWZ7JryF84ZUDPoI8z/uM0xry3lmwH8QY392rJiB7NihX7kEkpuR9YsXyHX+rj8AtL+AH23Qh0dM7tLGw/JXyRyJa/jPJEc0ejaZNp+fFc4vb/AR07suniIbzbpjtJrRsG1aStJAlbNXxPYQm/1Es6ZlYH+Mk558ysM97IID1yF4ly+cso6+u35Nx33oCMDJg6lX3jnqHhnTcxuGIV5rbqTtXRt3Ji3x4BJ3MVt2YfSHFKQLGqxAnfzKYB3YGaZpYO3AskADjnJgADgL+bWRawDxjkInV6r4gErcC6f2IiXH89k086hw9feptBK+fTb/VHVOn3DrRu7Q35HDLEW4jdjxJ26VNrBRE5bEWtwZtTpknM2sfMxC00nDMDPv8c4uKgZ08v+V94IVSteljnkEOpl46IhEXAZP3ddzBlCrzyCmza5K3Be+GFcOml0KsXqT/8nvs7gIZbFpMSvohEnuxsrx//K6/ArFnwyy9kHXU0sxt35n8tT2VZ0/b069SI15duDnr0jv4aCPNDWxEp3w47ycbFQffu3uvZZ+H990kb+wK9PnmXi798n4zK1fi64xnsrtuOBQ3acbBK1ULH9Rdn8lWsfjAo4YvIYQvZDNeKFaFPH35v3ZVTnl1I8nep9P12IWd+8THj988hq2Il9pzeg+qNtkHfvnDssYccIpjJV6mbMpi1PJ0Zy7aQle1KvUwUaR8sSvgicthCPcM1qWEi/bo0YRoV+KB5FypnZ/JI7d1csDmV6rNnw9Ch3rDOzp3hnHO8V+fOUKFCkbOFcz6citP+oSQisd2D2iOLyGErjWUOL+5Qj0oJ3jGpVIl6l/SDsWNh40ZYsQLuvdfb8YEHoFs3qFkTLr6YpPnTmXFOXW7u2YJRfVuRsn5XwA6cOcnegPg4Y9sv+wK2Zy5p6+dI7Piph7YiUiKlUbYI6pg//wwffQTvvuu9tmwB4MBxxzP/mBak1GvFikZteOD2ASQ1OibPMNH4+DjOaFGLT9btOKSzZ6juzMPV7kGjdESk3ErdlEHK9zs5gwxaf/MF3818h6OXfkbt3727873Vj6Hqmd2hSxfWNjiRT6rVJ6l1Q1LW7wrYv8e/r0+cwS3F7OtzSGxlXMPXKB0RKZf878afrhDH1KGDoe9gzn/+c47fkU7y1q+5udJPVF2xBGbNoiXQ0gxOPJGGJ7Zl1+/V+apmYzbUaZRbjkqsWpFs331wtvPeH678s4fD/RBXCV9EIlphSTJQnXxEj2ZMvfYU3+9cQo2c39m5E774IvdVc9GHjNq+/c+D/bc+tG7NCUfXo/9v1UirUY8t1euSsfdAyK4j3A9xlfBFJGIVlSQLW0TlkDJMzZpw7rneC8A5SE+HVau85RxXr4bVq2n3wQd0yMzM/bWsl6tDi+bQtCk0awaNGnl9gHJeNWp4cwqKkLJ+V+4IoQOZ4enZr4QvIhGrqGGfwSyiUiAzqF/fe/XpA3gfMEvX/Uj9XVuJS0ujzR87qP/zD5CWBkuXwowZcPBg3uMkJEDdulCnjtc4LjERqlf/82v16lC5Mu02/0qvdT9wID6BA/EJtFi3D+K3QVaWd0z/r3XrQpcuIf/fUwlfRCJWMCtx5dTJx3+cdlhzAnImY23/bX++UTsDqZ//9zMzYdu2wK+ffvJaQ2/Y4H3NyPCSt0833yvXfwsJqn9/mDmzyNiLSwlfRCJWcfrkH+7C55c975WM/BX4gZGQAA0beq+iOAd798Lu3fDHH6zZsJ27/ptK3P79VOUg957TnBZ1joL4eKhQIe/XxNIp9Sjhi0hEC7ZP/uEsopKyfheZ+ZJ9MOvxBsUMjjjCewGtmjRhVLOWufG10CgdEZHDV9xFVJKb1CChQlzuHX6FOBjYqQH9O9QrcknGwxleGUx8pTl0MxQrXk0G+gLbC1jE3ICngPOAvcBfnHPLS3peEZGSSmqYyLRrk5m1PB2H19YhmIRcWsMrS3voZiju8F8CxgFTCth+LtDc9+oCPOv7KiISdsX9qyDUDePK6tgQguZpzrmFwM+F7HIBMMV5UoDqZla3pOcVEQmH0mgYVxbHhrKp4R8PbPF7n+772Q/5dzSzYcAwgAYNGpRBaCIixXM4D4cj4dhQNgnfAvwsYMc259xEYCJ4zdNKMygRkcNV3DJQpBy7LPrhpwP1/d7XA7aVwXlFRMRPWST8t4ErzZMM7HbOHVLOERGJdaFYeKUwoRiWOQ3oDtQ0s3TgXiABwDk3AZiHNyQzDW9Y5tUlPaeISDQJZmx9WXTTLHHCd85dVsR2B4wo6XlERKJRsIm8tIdkgta0FREpVcGubVvaQzJBrRVEREpVsE3dSntIJmhNWxGRUleWSxtqTVsRkTAqzbH1xaEavohIjFDCFxGJEUr4IiIxQglfRCRGKOGLiMQIJXwRkRgRsePwzWwHsCnI3WsCO0sxnEgVq9cNsXvtsXrdoGsP9tobOudqBdoQsQm/OMxsWUETDcqzWL1uiN1rj9XrBl17KK5dJR0RkRihhC8iEiPKS8KfGO4AwiRWrxti99pj9bpB115i5aKGLyIiRSsvd/giIlIEJXwRkRgRNQnfzHqb2VozSzOzOwJsNzMb69u+ysw6hCPO0hDEtQ/xXfMqM/vMzNqGI85QK+q6/fbrZGYHzWxAWcZXmoK5djPrbmYrzWyNmX1S1jGWliD+ez/azP5nZl/6rr1crJNtZpPNbLuZfVXA9pLnOOdcxL+AeOB7oAlQEfgSOCnfPucB7wAGJANLwh13GV57VyDR9/255eHag7luv/0+AuYBA8Iddxn+m1cHvgYa+N7XDnfcZXjtdwGP+L6vBfwMVAx37CG49tOBDsBXBWwvcY6Lljv8zkCac269c+4A8DpwQb59LgCmOE8KUN3M6pZ1oKWgyGt3zn3mnMvwvU0B6pVxjKUhmH9zgBuAmcD2sgyulAVz7YOBWc65zQDOufJy/cFcuwOqmZkBR+Il/KyyDTP0nHML8a6lICXOcdGS8I8Htvi9T/f9rLj7RKPiXtdf8e4Col2R121mxwMXARPKMK6yEMy/eQsg0cwWmFmqmV1ZZtGVrmCufRxwIrANWA2MdM5ll014YVXiHBctSxxagJ/lH08azD7RKOjrMrMeeAn/1FKNqGwEc91PArc75w56N3vlRjDXXgFIAs4CqgCfm1mKc25daQdXyoK59nOAlcCZQFPgfTNb5Jz7tZRjC7cS57hoSfjpQH2/9/XwPt2Lu080Cuq6zOxkYBJwrnNuVxnFVpqCue6OwOu+ZF8TOM/Mspxzb5VJhKUn2P/edzrnfgd+N7OFQFsg2hN+MNd+NfCw8wrbaWa2ATgBWFo2IYZNiXNctJR0vgCam1ljM6sIDALezrfP28CVvifZycBu59wPZR1oKSjy2s2sATALuKIc3OHlKPK6nXONnXONnHONgDeA68pBsofg/nufDZxmZhXMrCrQBfimjOMsDcFc+2a8v2wws2OBlsD6Mo0yPEqc46LiDt85l2Vm1wPv4j3Fn+ycW2Nmw33bJ+CN0jgPSAP24t0FRL0gr30UUAN4xne3m+WivKtgkNddLgVz7c65b8xsPrAKyAYmOecCDueLJkH+u/8LeMnMVuOVOW53zkV922QzmwZ0B2qaWTpwL5AAoctxaq0gIhIjoqWkIyIiJaSELyISI5TwRURihBK+iEiMUMIXEYkRSvgiIjFCCV9EJEYo4YsUg5l9bGY9fd8/YGZjwx2TSLCiYqatSAS5F7jfzGoD7YF+YY5HJGiaaStSTL7VpY4Eujvnfgt3PCLBUklHpBjMrA1QF9ivZC/RRglfJEi+1YWm4q089LuZnRPmkESKRQlfJAi+FsSzgFucc9/gdWy8L6xBiRSTavgiIjFCd/giIjFCCV9EJEYo4YuIxAglfBGRGKGELyISI5TwRURihBK+iEiM+H9m5D/CFDREVQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "k = 3\n",
    "Xk = X.dot(V[:,:k])\n",
    "\n",
    "wk = np.linalg.pinv(Xk).dot(y)\n",
    "\n",
    "w = V[:,:k].dot(wk)\n",
    "\n",
    "evalw(w, x, y)\n",
    "print(\"w = {}, l(w) = {:f}\".format(w, l(w)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dimensionsreduktion über Principle Component Analysis (PCA)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Motivation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Oft sind die Daten in Form hochdimensionaler Vektoren\n",
    "$x_j\\in \\mathbb{R}^m$, $j=1,\\ldots,n$, also\n",
    "\\begin{equation*} \n",
    "X = (x_1,\\ldots,x_n) \\in \\mathbb{R}^{m \\times n}\n",
    "\\end{equation*}\n",
    "mit $m$ groß gegeben. Ein typische Beispiel ist die Verarbeitung\n",
    "von Bildern ($m$ ist die Anzahl der Pixel multipliziert mit der Anzahl der Farbkanäle).\n",
    "\n",
    "Die direkte Anwendung von numerischen Algorithmen ist dann meistens mit großem Rechenaufwand verbunden.\n",
    "Deshalb versucht man, die Dimension der Daten zunächst zu reduzieren\n",
    "und zwar so, dass der dadurch entstehende Informationsverlust möglichst gering ist.\n",
    "\n",
    "Wir betrachten den folgenden zweidimensionalen Beispieldatensatz"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-24T09:28:14.820363Z",
     "iopub.status.busy": "2021-09-24T09:28:14.819978Z",
     "iopub.status.idle": "2021-09-24T09:28:15.123816Z",
     "shell.execute_reply": "2021-09-24T09:28:15.123517Z"
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD4CAYAAADvsV2wAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAYzElEQVR4nO3df4xd5X3n8ff3ztgIEF1ck4SAf4ALQQlUlfDIcZZoUzaUEouEDQkKSaSipqyFFFRF/ScgS25FFanZ/JPdLSviRZW6EoQ2Ba9p4hRwRYUUdRI8Fkns8KPOiIHB2QLOQFmZZTxzv/vH3Du+Ht87c6/vz7nn/ZKsmXvP4ZxnjobPfeZ7nvM8kZlIkoZfqd8NkCT1hoEvSQVh4EtSQRj4klQQBr4kFcRovxuwnIsuuigvu+yyfjdDklaNiYmJNzPzffW2DXTgX3bZZRw8eLDfzZCkVSMiphpts6QjSQVh4EtSQRj4klQQBr4kFYSBL0kFYeBLUkEY+JJaNjE1w/1PH2ViaqbfTVELBnocvqTBMzE1w5cfHGd2rsza0RIP3bmdrZvX9btZaoI9fEktGZ88zuxcmXLCybky45PH+90kNcnAl9SS7VvWs3a0xEjAmtES27es73eT1CRLOtKQmpiaYXzyONu3rO9oyWXr5nU8dOf2rhxb3WXgS0Oo23X2rZvXGfSrkCUdaQhZZ1c9Br40hKyzqx5LOtIQss6uegx8aUhZZ9dSlnQkqSAMfEkqCANfKiDnwikma/hSwTgXTnHZw5cKxjH6xWXgSwXjGP3isqQjFYxj9IvLwJcKyDH6xWRJR5IKwsCXtCyHcA4PSzqSGnII53Cxhy+pIYdwDhcDX1JDDuEcLpZ0JDXkEM7hYuBLWpZDOIdHWyWdiLgtIo5ERDkixpbZ76aIeDEijkbEPe2cU5J0dtqt4R8GbgWeabRDRIwA9wOfAj4CfDEiPtLmeSVJLWqrpJOZzwNExHK7bQOOZuZkZd9HgFuAX7RzbklSa3oxSudS4NWa19OV9yRJPbRiDz8iDgAX19m0KzP3NXGOet3/XOZ8O4GdAJs2bWri8JKkZqwY+Jl5Q5vnmAY21rzeABxb5nx7gD0AY2NjDT8YJEmt6UVJ51ngyoi4PCLWArcDj/fgvJKkGu0Oy/xsREwDHwN+EBFPVN6/JCL2A2TmHHA38ATwPPC3mXmkvWZLklrV7iidvcDeOu8fA3bUvN4P7G/nXJKk9jiXjiQVhIEvSQVh4EtSQRj4ks7gKlfDydkyJZ3GVa6Glz18SadxlavhZeBLOo2rXA0vSzrSMiamZgq32pOrXA0vA19qYBhr2c1+gLnK1XAy8KUG6tWyV3MIDuMHmFpjDV9qoFO17EEZ4ujNWNnDlxroRC17kHrV1Q+wk3Nlb8YWlIEvLaPdWvYglYW8GSsDX+qiQetVezO22Ax8qYvsVWuQGPhSl9mr1qBwlI4kFYSBL0kFYeBLUkEY+JJUEAa+JBWEgS8NiUGZwkGDy2GZ0hAYpCkcNLjs4UtDwInR1AwDXxoCrlKlZljSkYaAUzioGQa+NCScwkErsaQjFYSjeGQPX+qTXi6Q7igeQZuBHxG3AX8GfBjYlpkHG+z3MvAOMA/MZeZYO+eVVrteB/AgLcSi/mm3h38YuBX4ThP7Xp+Zb7Z5PmlVq/bqj731bt0A7lavf9AWYlF/tBX4mfk8QER0pjXSEKvt1Y+WgtGREvPzpwK4m71+R/EIelfDT+DJiEjgO5m5p9GOEbET2AmwadOmHjVP6r5HD03z3skyCcyXky9s28ilF567GMC79v58cXszZZdW/xpwFI9WDPyIOABcXGfTrszc1+R5rsvMYxHxfuCpiHghM5+pt2Plw2APwNjYWDZ5fGmgTUzN8HcT01R/oUdKweeu3bAYwBNTM3zv4Kunto8sX3bxJqzOxoqBn5k3tHuSzDxW+fp6ROwFtgF1A18aRO3W1scnjzM3XwYggNvGNp52nPHJ48yVc3H757duWPY83oTV2eh6SScizgdKmflO5fsbgfu6fV6pUzrRm1560/TWazecdvxjb73LaCmYLydrRkt8rmZ77X7VD511562lFAGkN2HVtHaHZX4W+O/A+4AfRMRzmfn7EXEJ8GBm7gA+AOyt3NgdBR7OzH9os91Sz3SiN93opunDP36F3fsOU85kdKTEF7ZtPK3UU3XaDd+REmQyX05GSsHum6+2d6+mtDtKZy+wt877x4Adle8ngd9p5zxSP3VqSOPSm6YTUzPs3nd4sZQzN1/m0gvPrRveSz90YGEkRGYyc2L2rNqj4vFJW2kF3RrSOD55nPnyqXEJpYiGHya1HzojNT18yzlqhYEvNaGZIY0r3dhdun37lvWcs6bE7MkypVJw3y3XNDzH0g8dwDH1allkDu7Ix7GxsTx4sO5sDdJAWenGbu32UgR3fvxyLjh3DevOW8vMiVmDWx0TERONpq+xhy91wEo3dmu3lzN54JlJSoFj6NVTTo8sdcBKK05t37K+MozyFJcjVK/Zw5c6YKUbu1s3r+POj1/OA89MLr4XuByhesvAlzpkpRu7F5y7hoVHpRbC/uNXXsTXbviQ5Rz1jCUdqUeqo3JGAs5ZUzot7F2NSr1gD1/qkUZlHydCU68Y+FIP1Sv7OBGaesWSjtRnjUb4WOZRp9nDlwbArdduICpfq0sdWuZRpxn4Ug80mnZh6bKHVZZ51A0GvtRljXrrE1MzfPvAS4vBPjufPPzjV3j00DS7b77aRcfVcQa+1GX1eusAX35wfHEN2+r4/Op6tjMnZl10XB1n4EtdtnRq49feepfHDk0zO7cQ9iXgtzf8O57/P+8wP3+qR++i4+o0A1/qsur4+8cOTfO9g6/yyE9eYbQUjI6UFgN+96evBpzyWN1l4Es9sHXzusWFyssJ8+XkC9s2cumF554W8Aa9usnAl7qg3qicpUsl1lu7VuomA1/qsOrC5PPl5Jw1p0bl1E6tsO68tYs3bw199YqBL3XQ0oXJZ0+ePoa++tWHqtQPBr7UgpXWrR2fPE65ZtnQUilOmyrh0UPTHHnt7cXhmD5UpV4y8KUmNTPdQbVOX127trow+cTUDF/c88/Mztd8GOACKOotA19aQbVX/9pb7y4+QPXeyTKPHpquu7JVvQemxiePc7Im7AO4zgVQ1GMGvrSM2l59RBCVR2IT+LuJ6bojbao3aKuzXW7fsp7tW9azZiQWe/hrRkuGvXrOwJeWMT55fLHeTia1y5DPzzeuv9cr/3x358d49ND04qyYwOIHgsGvXjDwVWgr3YRdd95asuZ1AqOlIDOXrb/Xmz/nq9df4SpX6isDX4XVTOjOnJhdnNgMFsL+vluuYebE7LI986UPWS39YHD6Y/WDga/CWNqbbyZ0qwuP1466+dJHN614rkY3b2uP6/TH6rW2Aj8ivgV8GpgFfgn8YWa+VWe/m4D/CowAD2bmX7RzXqlV9XrzzYTuSsG9nNoROrWv2z2udLba7eE/BdybmXMR8U3gXuDrtTtExAhwP/B7wDTwbEQ8npm/aPPcUtMa1dSbCd2znaZ4pZKR0x+r19oK/Mx8sublOPD5OrttA45m5iRARDwC3AIY+OqZRr35boaudXoNmk7W8L8C/E2d9y8FXq15PQ18tNFBImInsBNg06aVa6VSM/pRQrFOr0GzYuBHxAHg4jqbdmXmvso+u4A54KF6h6jzXtZ5b2FD5h5gD8DY2FjD/aRWtdObX24R8kYfItbpNWhWDPzMvGG57RFxB3Az8MnMrBfQ08DGmtcbgGOtNFLqpaUhvtwi5MvV6Fca4y/1WrujdG5i4SbtJzLzRIPdngWujIjLgdeA24EvtXNeqVvqhXjt07a10x0/dmi64ayXPlilQVRq87//S+AC4KmIeC4iHgCIiEsiYj9AZs4BdwNPAM8Df5uZR9o8r9QV9W601j5tW2bh6duJqRm+d/DVxfdHRk6v0dc7zkqqc+9MTM10/OeSoP1ROlc0eP8YsKPm9X5gfzvnkrptYmqG1956l5FSkPO5GOLjk8cpBZQTSrHw9G11fVpYuEn1+a2nT6LW6g1b/yJQL/ikrcSpwK2WaAKgcktqaXivO28th4+9zehIifn5U+vT1mr1hm3tXwSNpl6W2mXgq/Ampmb49oGXTs2KycIwsvlynvGA1rrz1nLf948wO1dmtBTcvm0TtzZYjLyVUUHbt6xntLQwffJyUy9L7Wi3hi+tatWe/Y+OvnmqZ89C6WbpA1pfvf4KZk7MLvbE58vJJRee25FQ3rp5HbeNbVw8f3XqZamT7OFr6LQyHLK2lFIKuO6Ki/jUNR9sOBtmNx+muvXaDTx6aNoHtdQ1Br6GSqs3P5cG+EqrUHXzYSof1FK3GfgaKs3OX1P7V0CrIdvN+XecUE3dZOBrqCxXcqmGfO2N1+pfAV+9vu4I47PmU7YaRAa+hkqjskhtqacUQTmz6VksWw1vx9RrUBn4Gjr1yiK1pR4yKZWCYPl1aeHswttpkTWoDHwVwtJSz+6br14ciQNw/9NH6/bgzya8nRZZg8rAVyE0U+qp14M/m/B2tI0GlYGvwlip1FM7yVltWJ9NeDvaRoPIwFehLe3Bv/PuSb7wnX9mvpycs+ZUj9/w1jAw8FVotT34deetZfe+w4uzYNbOfS8NA+fSUeHVzpNTrlm0rVQKb7hqqBj4UkW1vFMKGC0F991yzRlLFrpAiVYzSzpSxXI3aF2/VsPAwJdqNLpBu9x4fJ+s1WphSUdqQrXcM7Jknnw4u/VrpX6wh6+h0O2SynLlHp+s1WoRWTMqYdCMjY3lwYMH+90MDbhBKKlYw9egiIiJzByrt80evla9Xk5W1ijYfThLq4GBr1WvVyWVQfhLQmqHga9Vr1eTlTntsVY7A19DoRclFW/OarUz8LUq9eMmqdMea7Uz8LXq9LOW7s1ZrWY+eKVVxwedpLNj4GvVWe6pV0mNtVXSiYhvAZ8GZoFfAn+YmW/V2e9l4B1gHphr9FCA1Axr6dLZabeG/xRwb2bORcQ3gXuBrzfY9/rMfLPN80lA41q6T7xKjbUV+Jn5ZM3LceDz7TVHOnvVm7nvnSwzUpnP/ksf3VR3Pz8UVESdHKXzFeBvGmxL4MmISOA7mbmn0UEiYiewE2DTpjP/Z5WWqgb4sbfe5b2TZRKYKye79x3mqosvaGle+3602w8e9cqKgR8RB4CL62zalZn7KvvsAuaAhxoc5rrMPBYR7weeiogXMvOZejtWPgz2wMLkaU38DCqIegFZG+CjpaBUCuYra9KWM8+Yt/7bB15a/FDo59Oyg/TBo+JYMfAz84bltkfEHcDNwCezwdSbmXms8vX1iNgLbAPqBr5UT6OArB2iOV9O/uOHP8DTL7xOOZO1oyXWnbeW+58+yrrz1nLf948wO7cQ9iX6O8LHaRrUD+2O0rmJhZu0n8jMEw32OR8oZeY7le9vBO5r57wqntqAnD1Z5tsHXuJrN3zojOkO7vrEb3HXJ36L8cnjp4V8KYJyJuWEUsB1V1zE1274UN9C1mka1A/t1vD/EjiHhTINwHhm3hURlwAPZuYO4APA3sr2UeDhzPyHNs+rgqkG5OzJMmXgR0ff5NmXf81Dd25vOETzvr8/wv87WV54kUmpFATJmtFSX8MeHFqq/nABFK0a1Rr8j46+STlhJOBPbryKr15/xRn7ffF/LpR/qtaOBH/2mWuYOTFbN2C9gaph4QIoGgpbN6/jazd8iGdf/vWypZDxyeOcrAl7gNvGNtYdogneQFVxOLWCVpVqKeRPbryKh+7cDsD9Tx9lYmpmcZ/tW9azZvTUr/boSJBw2j61nJtHRWEPX6tO9SnbRj3zrZvX8d3/vJ3HDk3zxjvv8U8vvs4jP3mFxw5N1+29ewNVRWEPX6vWcj3zrZvX8Y3P/ja/s/FC5sq5bO996V8NlnM0rOzha9VqpmfebO/dee5VBI7S0cBbbgRNM6NrHIGjInGUjlatlUbQNNMzt/cuLbCGr4HmCBqpcwx8DTRXt5I6x5KOBlp1BM1jh6YZ3LtN0upg4GtVePTQNLNzZR47NM3um69uOEWCpMYMfLWkHyNels6UuXvf4cXpjx03LzXPwFfT+jXnTO1Y+qiZ5vhkpcfvkEupOQa+mtavRTtqpxKuznF/cq7MyEiJ7x18lbmyvX2pGQa+mtbLOWeWlo5qx9JfdfEFjE8e57W33uWRn7ziqlFSkwx8Na1Xi3Y0+7DVxNQMjx2adtIzqUkGvlrSi6dWmy0duWqU1BoDXwOnldKR0yZIzTPwNXDsuUvdYeBrINlzlzrPuXQkqSAMfA2ciamZM9apldQ+Szpq29Ix8/WmX2h2SoZ+Pc0rFYGBr7YsDejdN1/Nfd8/clpgA02HeL+e5pWKwMBXW5YG9A8P/6rugiXNhngvn+aVisbAV1uWBvSnrvkgz7786zMCu5Vx9d0Ykum6tpKLmKsDOlnD71b7vC+gonARc3XV0jHz9cbQ93NcvfcFpAVtDcuMiD+PiJ9FxHMR8WREXNJgv5si4sWIOBoR97RzTqlVrosrLWirpBMRv5GZ/1b5/o+Bj2TmXUv2GQFeAn4PmAaeBb6Ymb9Y6fiWdNQp1vBVFF0r6VTDvuJ8qLvO9DbgaGZOVhrzCHALsGLgS53iVA1SB2r4EfEN4A+At4Hr6+xyKfBqzetp4KPtnleS1JoVa/gRcSAiDtf5dwtAZu7KzI3AQ8Dd9Q5R572GdaSI2BkRByPi4BtvvNHszyFJWsGKPfzMvKHJYz0M/AD40yXvTwMba15vAI4tc749wB5YqOE3eW5J0graHaVzZc3LzwAv1NntWeDKiLg8ItYCtwOPt3NeSVLr2q3h/0VEXAWUgSngLoDK8MwHM3NHZs5FxN3AE8AI8FeZeaTN80qSWtTuKJ3PNXj/GLCj5vV+YH8755Iktcf58CWpIAx89YWLnEi951w66jknM5P6wx6+eq7eZGaSus/AV885mZnUH5Z01HPdWuRE0vIMfPWFk5lJvWdJR5IKwsCXpIIw8CWpIAx8SSoIA1+SCsLAl6SCMPDVFc6VIw0ex+Gr45wrRxpM9vDVcc6VIw0mA18d51w50mCypKOOc64caTAZ+OoK58qRBo8lHUkqCANfkgrCwJekgjDwJakgDHxJKggDX5IKIjKz321oKCLeAKY6cKiLgDc7cJxOs12tGcR2DWKbwHa1apjatTkz31dvw0AHfqdExMHMHOt3O5ayXa0ZxHYNYpvAdrWqKO2ypCNJBWHgS1JBFCXw9/S7AQ3YrtYMYrsGsU1gu1pViHYVooYvSSpOD1+SCs/Al6SCGMrAj4g/j4ifRcRzEfFkRFzSYL+bIuLFiDgaEff0oF3fiogXKm3bGxEXNtjv5Yj4eaX9BweoXb2+XrdFxJGIKEdEw6FpvbxeLbSp19fqNyPiqYj4l8rXunNT9+parfTzx4L/Vtn+s4i4tlttaaFNvxsRb1euzXMRsbvbbaqc968i4vWIONxge+euVWYO3T/gN2q+/2PggTr7jAC/BLYAa4GfAh/pcrtuBEYr338T+GaD/V4GLurh9VqxXX26Xh8GrgL+CRhbZr+eXa9m2tSna/VfgHsq39/Tz9+tZn5+YAfwQyCA7cCPB6BNvwt8vxe/R0vO+x+Aa4HDDbZ37FoNZQ8/M/+t5uX5QL0709uAo5k5mZmzwCPALV1u15OZOVd5OQ5s6Ob5mtVku/pxvZ7PzBe7eY5WNdmmnl+ryvH/uvL9XwP/qcvnW04zP/8twP/KBePAhRHxwT63qS8y8xng18vs0rFrNZSBDxAR34iIV4EvA/X+NLsUeLXm9XTlvV75Cguf2vUk8GRETETEzh62CRq3q9/Xazn9vF719ONafSAzfwVQ+fr+Bvv14lo18/P3+ho1e76PRcRPI+KHEXF1F9vTio5dq1W7xGFEHAAurrNpV2buy8xdwK6IuBe4G/jTpYeo89+2PUZ1pXZV9tkFzAEPNTjMdZl5LCLeDzwVES9UegH9bFffrlcTOnq9OtCmnl+rFg7T8d+tOpr5+btyjZbRzPkOsTAPzf+NiB3A/wau7GKbmtWxa7VqAz8zb2hy14eBH3Bm4E8DG2tebwCOdbtdEXEHcDPwyawU6Ooc41jl6+sRsZeFP0fb+p+yA+3qy/Vq8hgdvV4daFPPr1VE/GtEfDAzf1X5c//1Bsfo+O9WHc38/F25Ru20qbYUnJn7I+J/RMRFmdnvSdU6dq2GsqQTEbWfyp8BXqiz27PAlRFxeUSsBW4HHu9yu24Cvg58JjNPNNjn/Ii4oPo9CzdU696972W76MP1akY/rlcT+nGtHgfuqHx/B3DGXyI9vFbN/PyPA39QGYGyHXi7WpLqkhXbFBEXR0RUvt/GQj4e72KbmtW5a9XrO9K9+Ac8ysIv8s+Avwcurbx/CbC/Zr8dwEss3L3f1YN2HWWhFvdc5d8DS9vFwiiCn1b+HRmUdvXpen2Whd7Ne8C/Ak/0+3o106Y+Xav1wD8C/1L5+pv9vFb1fn7gLuCuyvcB3F/Z/nOWGYXVwzbdXbkuP2Vh8MK/73abKuf9LvAr4GTld+uPunWtnFpBkgpiKEs6kqQzGfiSVBAGviQVhIEvSQVh4EtSQRj4klQQBr4kFcT/B0jBYIYjyqNrAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.datasets import make_blobs\n",
    "\n",
    "X,y = make_blobs(n_features = 2, centers = 1, random_state = seed)\n",
    "\n",
    "np.random.seed(seed)\n",
    "T = np.random.rand(2,2)\n",
    "\n",
    "X = X.dot(T).T\n",
    "\n",
    "plt.scatter(*X, marker = '.')\n",
    "plt.axis('equal');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Offensichtlich gibt es eine Richtung, in der die Daten\n",
    "stark variieren. Senkrecht dazu ist die Variation eher gering.\n",
    "\n",
    "Zur Datenreduktion könnte man nun eine Gerade in die Richtung\n",
    "der starken Variationen legen und die Projektionen der Daten\n",
    "auf diese Gerade betrachten.\n",
    "\n",
    "Die Projektion für einen Datenpunkt $x_i$ ist ein Skalar, also eine\n",
    "eindimensionale Größe (\"Koordinate entlang der Geraden\")\n",
    "\n",
    "Wenn diese Projektionen für die einzelnen Datenpunkte\n",
    "sich hinreichend stark unterscheiden, dann kann man z.B. ein\n",
    "Klassifikationsverfahren statt auf den zweidimensionalen\n",
    "Originaldaten ohne große Verluste auch auf den eindimensionlen\n",
    "Projektionen der Daten durchführen.\n",
    "\n",
    "Um die weiteren Betrachtungen zu vereinfachen, zentrieren\n",
    "wir den Datensatz. Den allgemeinen Fall behandeln wir im Anschluss."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-24T09:28:15.135922Z",
     "iopub.status.busy": "2021-09-24T09:28:15.135167Z",
     "iopub.status.idle": "2021-09-24T09:28:15.201568Z",
     "shell.execute_reply": "2021-09-24T09:28:15.201261Z"
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD4CAYAAADvsV2wAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAmTUlEQVR4nO3deXxU1f3/8ddnJgkCUgiLsiZAQRFQK0kRN1xRUBTBpVq+bf21EkXqvrTWCoqtWhU3oCIialuL1roBslpBFEVJKCiLYIosEZQtIouQZc7vj0lCEmZIwiw3ybyfjwePzMy9c+9nHsA7Z8499xxzziEiIvWfz+sCREQkPhT4IiIJQoEvIpIgFPgiIglCgS8ikiCSvC7gUFq2bOk6duzodRkiInVGTk7ONudcq1DbanXgd+zYkezsbK/LEBGpM8xsfbht6tIREUkQCnwRkQShwBcRSRAKfBGRBKHAFxFJEAp8EZEEocAXEUkQCnwRqZZNuzZx55w72bV/l9elyGFS4ItItdw08ybGLR7Ht3u+9boUOUy1+k5bEakdpq6eyuurXufBcx6kS/MuXpcjh0ktfBE5pF37dzFixgh6HtWTO069w+tyJAJq4YvIIf3xvT/y9fdf89oVr5HsT/a6HImAWvgiEtanX3/K2E/HcsNPb6BP+z5elyMRUuCLSEiFxYVkTcuiTZM2PHjug16XI1GgLh0RCemJRU+w7NtlvHHlG/yowY+8LkeiQC18ETnI2vy13Df/Pi7tdimDjxvsdTkSJVEJfDObbGZbzGx5mO1nmdlOM1ta8mdkNM4rItHnnGP4O8NJ8iUxdsBYr8uRKIpWl86LwDjgb4fY5wPn3MAonU9EYuSfn/+TOf+bw9gBY2n/o/ZelyNRFJUWvnNuAbAjGscSEe9s37udW2ffysntTmZ45nCvy5Eoi2cf/ilmtszMZppZj3A7mVmWmWWbWfbWrVvjWJ6I3Dn3TvL35TPx4on4fX6vy5Eoi1fgLwHSnXMnAmOBt8Lt6Jyb6JzLdM5ltmoVcuF1EYmBeV/N44WlL3DHKXdwwtEneF2OxEBcAt85971zbnfJ4xlAspm1jMe5RaRq+4r2cd306/hx6o8ZeabGVNRXcRmHb2atgW+dc87MehP8RbM9HucWkar9ecGf+XLHl8z9xVwaJjf0uhyJkagEvplNAc4CWppZHjAKSAZwzk0ALgeGm1kR8ANwlXPORePcIhKZFVtW8PDCh/nFCb/gvM7neV2OxFBUAt85d3UV28cRHLYpIrXI4nXbuerNX9I4+UeMOX+M1+VIjOlOW5EElbM+n4snj2Lt90touPcaNmzTTCv1nQJfJEHNWrWSLb4XOKL4BBoWns2itbqsVt8p8EXqoJz1+Yyfl0vO+vzDPsa8bx4BCmlVNIKUJD99OreIXoFSK+k7nEgdk7M+n6GTFlFQFCAlycfL1/YhIz21RseYtnoa/1k/lRsy/kj3I8+hT+cWNT6G1D0KfJE6ZtHa7RQUBQg4KCwKsGjt9hqFdfklC5+48F5S/CkxrFZqEwW+SB3Tp3MLUpJ8FBYFSE7y1bgr5t5595L3fR6vXv6qwj7BKPBF6piM9FRevrYPi9Zur3FXzOKvF/P0J08zPHM4p3Q4JYZVSm2kwBepgzLSU2vc515YXMiwacO0ZGECU+CL1CM56/PDtvyfXPQky75dxutXvk7TI5p6VKF4SYEvUk8cavTOV/lfMWr+KAYdO4jB3bRkYaLSOHyReiLU6B04sGSh3+dn7ICxmJnHlYpX1MIXqSfCjd6ZsnwKs/83m6f7P02Hph08rlK8ZLV50srMzEyXnZ3tdRkidUblPvwdP+yg27hudErtxEe//kirWCUAM8txzmWG2qYWvkg9Unn0zp1z7mTHDzt495fvKuxFffgi9dX8dfOZvHQyd5x6YMnCaMzBI3WXWvgi9VDpkoWdUzuXLVkYjTl4pG5TC1+kHnrwgwdZs30NEy6aQKPkRkD4UTySOBT4IvXMyq0refjDh/m/E/6Pfj/uV/Z66Sgev3FYc/BI3acuHZF6JOACZE3LokmDJjx+/uMVtkUyB4/UDwp8kXrkuZznWLhxIS8MeoFWjVsdtP1w5uCR+iMqXTpmNtnMtpjZ8jDbzcyeNrNcM/vMzHpF47wicsDmXZv53bu/4+yOZ/OrE3/ldTlSC0WrD/9FoP8htg8Aupb8yQKeidJ5RaTEzbNuZl/RPp4d+KymT5CQohL4zrkFwI5D7DII+JsLWgQ0M7M20Ti3iASXLHxt5Wvc2/deurbo6nU5UkvFa5ROO2Bjued5Ja+JSIR2F+xmxIwR9GjVgztPu9PrcqQWi9dF21DfL0NO4mNmWQS7fUhLS4tlTSL1wr3v3cvG7zey8NcLtWShHFK8Wvh5QPlp+toDm0Lt6Jyb6JzLdM5ltmp18CgDETlg8deLefrT4JKFp3Y41etypJaLV+BPBX5ZMlqnD7DTObc5TucWqZeKAkVkTc/i6MZH89C5D3ldjtQBUenSMbMpwFlASzPLA0YByQDOuQnADOBCIBfYC/y/aJxXJJE9uehJln6zlH9f8e8qlyw81NKHkjiiEvjOuaur2O6AEdE4l4gElywcOW8klxx7CUOOG3LIfTVpmpTSXDoidYxzjhtm3IDf52fcgHFVjrnXpGlSSlMriNQxryx/hVm5s3iq/1PVWrIw3NKHkni0xKFINXzxxmzyZ84ldUA/ug25wLM6Spcs7NisIx//5uNqr2KlPvzEoSUORSLwxRuzSb9iIMcEitj/wtN88a9pnoX+XXPvYscPO5j7i7kVwr6qQNekaQLqwxepUv7MuTQIFOEDkouLyJ85t8r3xGIpwfnr5vP8f5/n9lNu58TWJ1Y419BJixgzZzVDJy3S8oUSlgJfpAqpA/pR5PNTDBQmJdPijD5wwQWwdGnI/WMRwKVLFnZq1olRZ42qsE0XZaW61KUjUoVuQy7gi9feOdCHn94SvvgCAoGQ+4cK4Ei7Ux764CHWbF/D7P+bXbZkYSldlJXqUuCLVEO3i8+BvhnQsmXwhdxcSE4OPh4zBrp0gUGDgOgH8MqtK3now4cYevxQzv/x+Qdt10pWUl0apSNSHaNHw6hRUFgISeXaSYWF0KcP/OQn8PzzZS9Ha1RMwAXo+0JfVm1bxaoRqziq8VERfAhJBBqlIxKp/v0hNfXgbpzkZFi0CPbtCz5fuxZmzybjuuui0tKetGQSCzcuZPIlkxX2EjFdtBWpjt694cYbISXE9MPJydCkSfDx5Mlw113wzTcRn3Lzrs3cNfcuzup4Ftf85JqIjyeiwBepjsJC2Lz5QEs+nAcegJwcaNs2+PzVV6t+Txi3zL5FSxZKVCnwRarjo4+CIf7RR4fezwyOOSb4+LPP4KqrYMKEGp9u+prp/GvFv/hj3z9yTIvg8WIxtl8Si/rwRaqjWzd45pkDYV4dJ5wA8+cHL+oCrFoFrVodGOkTRumShd1bdeeu0+4CNOOlRIcCX6Q6jj4arr++5u8788zgT+dg6NDgN4Ds7ODPMEbOG8mGnRsqLFkYi7H9kngU+CLV4Rxs3AiNG0OLwxhXbwYvvQQ7dgQfBwLB46WnV9gte1M2T33yFNdnXF9hyULdXCXRoD58keoIBILhPHbs4R/j+OMPtPgnTgx2E61YUba5KFDEsGnDgksWnldxycLSm6tuO/9YdefIYVMLX6Q6/H544YXgDVbRcPHFsGULdO8efL5rF099PpGl3yzltSteo9kRzQ56S7gZLzX1sVSXAl+kuq65JqK3VwzmdjByZHBDfj7Fx3Sl95H5XPnH/lx23GU1OqYu5kp1KfBFqmvjxuB4/M6da/S2nPX5vL4kj3/n5FFUfCCYIXgx9pSWSbRze+iVF+CJPvfVaMy9LuZKTSjwRarr5z8P3lX73nvVfktpC3x/YYDSWasKiwK8sSSP15fkUVAU4ILVz/LM9n3Mv3UwZ/U4uUYl6WKu1ERUAt/M+gNPAX5gknPu4UrbzwLeBr4qeekN59zoaJxbJG7uuy/Yl18Di9ZurxD2BiQn+XBAQVGAxns3c9+c6Szt0JAz/vLKQe+vzkpWmilTqiviwDczPzAe6AfkAYvNbKpzbmWlXT9wzg2M9Hwinjn33Bq/JbVRCuXno+3X/WiuO/PHrP5mFwEHdyy4jxZ7HdMee4SfJFecp6e6/fNavlCqKxrDMnsDuc65tc65AuAVYFAUjitSqyzLXs2/nn2rRlMb5O8twFfSJe8zOLFDMwBmLt/MSXlv86ulXzOuzzHsP7Y/UHH6hDeW5LG/UCtZSfREo0unHbCx3PM8IFRH5ClmtgzYBNzhnFsRYh/MLAvIAkhLS4tCeSKRy1mfz+JbRnHtR6/R8w/TeHnYKdVqVVfuY9/1QyE/e/Zj/AW7eXvWC6xt5mdy3/t5oXOLCi36JL+PQOBAV5Dfr/55iVw0Aj/UkILKq6osAdKdc7vN7ELgLaBrqIM55yYCEyG4AEoU6hOJ2KK123n7uDP5pE03ioqKqz0apnwfe2qjFEa+vZyigGPYJw/QbXsRv7vhOl4YfgEZ6amMn5dbYcRNKQMuz2ivbhuJWDS6dPKADuWetyfYii/jnPveObe75PEMINnMDj2DlEgt0qdzCza07siCLj8lKTnpoNZ2uJksy190zd9bQHHAkb79U279aDmv9DyKK+96qCzIS78N+C14YTfZb/gNGiT7uKxX+7h9Vqm/otHCXwx0NbNOwNfAVcDPy+9gZq2Bb51zzsx6E/xFow5JqTMy0lOZcmV31s5fROez+3BSudZ2uIur5V83MzLSmpHsD/Dg7EfZ1QD2/enlCq32yiNuAI2+kaiKOPCdc0Vm9ltgNsFhmZOdcyvM7PqS7ROAy4HhZlYE/ABc5WrzYroiIZy0eTUn3XQVfPghcGDSs3A3P5V/Hef4dF0+Qz5/gtM2/sCbtw/lmkHnHXSOyiNuFPQSTVEZh1/STTOj0msTyj0eB4yLxrlEPJORAbNnQ48eFV4Od/NTn84tgnfNlrRtmu9ey8j/zGNRlyZc+sjf4l6+iO60FamuFi3g/PMPejnczU8Z6amc0+0o5q78FoA/zBvNEUXw/ZjJmE8T1Ur8KfBFqquoKNidk54OnTpV2BTu5qfrz/wx76/eQuaX/+DylduYcvmpXH3J5ZrhUjxhtbkrPTMz02VnZ3tdhkjQvn3QsCE8+CDcfXe137Zw2RekndWTH47w03Htdj7fUqgZLiVmzCzHOZcZapta+CLVdcQR8J//1GxdW6Dw/qF0+K6Yz/49lpSGR7Jobe5BF3lBI3Ik9hT4IjVxzjk12n3VrH9wxltLeH9Ad868bDgQnF/HZwY4kpN8pDZKUYtf4kKBL1ITH38MPh+cXPU0xkUF+wgMG8bWI334R0+pOC9+wOEzuOaUjuTvLdCc9hIXCnyRmrj5ZmjeHGbNqnLX6SMGc2nePn4z+FI+eOtrcHkUFruyeUcCDiZ9+BWjB/XUnPYSFwp8kZp4/vnghdsq5H32If1emsU7XZvxbtdfY0UBgp04FQWcI39vgea0l7hQ4IvUxPHHV7mLCwTY/MshNDN48IJ7MQuOuXcEp0j2GTgM5xwpJS16zWkv8aDAF6mJ5cvhyy9h8OCwu7x933AuXbaVt7Iu4sxz+jHlkw3BsAdO69KSW84LjvJRi17iTYEvUhMvvQTjx8PevaE3z1xI/8efI7ttA+5uOYx72zalQbKPgsIAPp8xoGebCnfiisST7u8WqYlbboEwNwPmrM/H3fMzWux13H3BTewvTiJ/bwEjB/bA5zMCzjF6+ooarZglEk0KfJGaaNeOnMZtQs59/9HfH+ea/37N+JO78uVRZ+LzGamNUnh18QaKAk5LFYrn1KUjUg2lc9+03b2dj557jTmdf8pTjZswJSu41OH+Pd9z4di/sLaZnwmnjiLJZ1x7eifum7qcguIDY3P8PtOwS/GMAl+kCuUXMjnny0+YNG0MX/zyCT4/4kheX5JHRnoqH/32Es7eUsjbY/7Ab086mdRGKcxcvrlC2ANckdlBfffiGQW+SBUWrd3O/sLgguIfdzieM7MmsrlJKyC43uz/Fk7n1L+/z8LT0xl0258r/IIoLyXJR4+2TRk/L1ejc8QTCnyREuGmLE5tlFJ2w9SeBo3Y06AREAzwS09sze4Bp7M3xej6t+lAxRWwfAbHt2tKz3ZN6dG2KaOnr9CcOeIZBb4krPIBD4SdwCx/b0HZXbINCvcz6IsFtOvXl9MHn83eSbfw0zXf88HIazijU0/g4BWwRl7cg4z0VMbPO3iWTAW+xJMCXxJS5YXHL+vVPmwY9+ncIjiWvihAQ1fMIzOegnPT2FK0kZTH/s5/uzXl9FHPlx07Iz2VkQN7MHP55grj7sMthSgSLwp8SUiVFx53EDaMKyxh2Kk53PQVHHUUuf260avI0fTFVyosWZizPr+s62bxuh0c27pJ2dQJmjNHvBSVwDez/sBTgB+Y5Jx7uNJ2K9l+IbAXuMY5tyQa5xY5HJVb25f1as9lvdqHDeOKc900Z/Fz93PqRxuZ/+tzOOvk/hX2rfzLpPy3Bc2ZI16KOPDNzA+MB/oBecBiM5vqnFtZbrcBQNeSPycDz5T8FPHEoRYeD6e0z/+8T6fS6U+jyW2dQsM7Xzxo1I26bqS2ikYLvzeQ65xbC2BmrwCDgPKBPwj4mwsuoLvIzJqZWRvn3OYonF/ksFSntV0a8qmNUsq6aa58+m5a7g3w1n0jefhvnxMomfWy/IXeIb3aYyU/1aKX2iIagd8O2FjueR4Ht95D7dMOOCjwzSwLyAJIS0uLQnkih6f8hV2fGcUBxz6+5IO0Aop/lMbD20+iKBAcsFlQbsqE8heDh/RqH/bY6suXeItG4FuI1yqv81CdfYIvOjcRmAiQmZkZch+RWMtZn8+T764pu+HKOYdZMTtSxnHlz1JJ3/cwxYED/zx9Fpwy4VD99+WPrTVsxQvRCPw8oEO55+2BTYexj0itUBrI+woP3CnrgKYt3+Wr3f+j5f7fE+BHwYVMXHB+nNGDepaFdlX99+V/KewvDJRNzyASa9GYLXMx0NXMOplZCnAVMLXSPlOBX1pQH2Cn+u+lNirfsi+v2L5l5Z7naBz4KUe603AE16QtDfufnxzsfiy9GHzb+ceGbbn36dyCJF/wS68D/p2TpymTJS4iDnznXBHwW2A2sAr4l3NuhZldb2bXl+w2A1gL5ALPATdEel6RmspZnx9yWuPy24dOWsTC3G04DvRDmjnyUyaQ5PfzypWTOL1Lq7JtrmRN2vIy0lMZcXaXsK32jPRUrsjsUHaM4mJNmSzxEZVx+M65GQRDvfxrE8o9dsCIaJxL5HBU1W9e2rIvPwfOaV1aMqBnG/6z/i0mr1rME+c+wcAeJ9DmyHwWr9sR0bDLIb3a8/qSPA3dlLjSnbaSEEJdTC19vXTIZekFWp8F++FvOe8YOh8Ft33wIBltMrix941A+DH8UP3RN7rrVrygwJeEUPlmqNRGKRWGXAacO2ih8Yz0VLKmZbFt7zZmDp3J0o3fVwjoSEff6K5biTcFviSEyi3q8i1+nMPnMwxHcknLPiM9lQ/Wf8BzS57jjlPuIFDQscowr86QTBEvKfAlYVRuUVeYwnhgD/L3FpDaKCUY3MX7yZqZRXrTdO476z5eXLi5yjDXlApS2ynwJSGF6kMv3yXzffIUtvu/YHjPSXyxuaBaYa5+eantLDiApnbKzMx02dnZXpchCWL8vFzGzFnNfjayqcGNNA6cRquCO0n2G1OyTgFQmEutZ2Y5zrnMUNvUwhcp0adzC5KTYLNvHD4akFpwLQ4oKHa8viSPBwcfr6CXOi0ad9qK1AsZ6alc2TeXfb4VnNbqVvwcCPdQk0GJ1DUKfJES3+7+lnFL7qNvel8ev+hWUpJ8GFSY9bKqu3VFajN16YiUuGX2Lewt3MuzA5+lW8vmTBkW/qJu5aGZmu5Y6gIFvtR71QnjmV/O5JXlr3D/WffTrWU34OBhnOHG2Wu6Y6krFPhSr1UnjPcU7GH4O8Pp1rIbvzvtd2GPFW5opm64krpCgS/1WnXCeNT8UazfuZ4F1yygQVKDsMcqHWf/+pK8ChdxdcOV1BUKfKnXqgrjJZuX8MSiJ8jqlcUZ6WdU65hvLMmjoCi4cEnpNwbdcCV1gQJf6rVDhXFRoIisaVm0atSKh897uFrHC/eNQROhSV2gwJd6qfKF2lBhPPaTseRszuHVy18ltWH1wlrdN1KXKfCl3qnOhdr1363n3nn3clHXi7ii+xXVPra6b6QuU+BLvVPVhVrnHCNmjMDhGH/heMxqdh+tum+krlLgS71TVbfLaytf450v3+Hx8x8nvVm6R1WKxJ8CX+qdcN0uOevzeW/1Wv7y3xvp1aYXN558o8eVisRXRIFvZs2BV4GOwDrgSufcQZOMmNk6YBdQDBSFm7pTJFoqd7vkrM/n6okfs9k3lt3+bTzV71WSfEll2+LVJ68pGMRLkbbwfw/8xzn3sJn9vuR5uFsVz3bObYvwfCJhhQrT0teWbfyO793n7E6axY8KB7NifQvoBf/8ZAMj315OccDRIDm20yJoCgbxWqSBPwg4q+TxS8B8wge+SMyEClOg7DUoZEfKePyBo2haNJRtu/bzhzc/59XFGykOBBcBKiiM7bQImoJBvBZp4B/tnNsM4JzbbGZHhdnPAXPMzAHPOucmhjugmWUBWQBpaWkRlieJonyYFhQGePLdNaQ1b1T22s6kf1Po28hR+0fRwN+Q+au3UFjsKL/em89nMR1XrzH84rUqA9/M3gVah9h0Tw3Oc5pzblPJL4S5ZvaFc25BqB1LfhlMhOAShzU4hySw0jAtKAwQABbmbuMTn5Hk9/FDYAM7k17llNaX0LPhQD7L+45vvt9f9l4D/D5j9KCeZbNfxqKfXWP4xWtVBr5z7rxw28zsWzNrU9K6bwNsCXOMTSU/t5jZm0BvIGTgixyO0jB98t01LMzdRsBBccBx5U/b8e8Nozjyh8b84riR/GXGN2XvMSDZb1yR2YEhvdrHZapjjeEXL0W64tVU4Fclj38FvF15BzNrbGZNSh8D5wPLIzyvyEEy0lO55bxjSEny4Tfw+318lv8WuTsX8/gFj/HRl0UV9m/aKJn7LunJn8utVRuqn12kvog08B8G+pnZl0C/kueYWVszm1Gyz9HAh2a2DPgUeMc5NyvC84qEVNrSv6p3GkVuB7M2PkZD15MTmw9mQM82Ffb9bm8ho6evqLBcYWnXkN9QP7vUOxFdtHXObQfODfH6JuDCksdrgRMjOY9ITWSkp7Jo7Xa2+J8jwD5aFIzg06/yGXF2FwAmLvgf67bvBQ4eLaN+dqnPdKet1HmhLrIGUv7LHv/7pBb+nMb+9LKW+s9PTuPY1k0YOmlR2NEy6meX+sqcq70DYTIzM112drbXZUgtFuoia7c2KfR8pie4ZG4+/k3O6NL2oADXHa9SX5lZTrjZDNTClzot1EXWV1Y/y7rv1vH+Ne/TN71HyPepFS+JSIEvdVr5m5n8fh85m5fwt9wnGNZrGH3T+3pdnkitEukoHRFPlR+V41wRL6++BwJN+GmqZsIUqUyBL3GTsz6f8fNyKwyDjIaM9FTaNmvIdqZR4MsltTCLv8zYGPXziNR16tKRuIj1Hawdj/qB75L/TsPiTBoVn0HAHG8sydOFWZFyFPgSF7GYKbJ0pM3JnZrzzLK7SUkyWu29Ab8F59B5LXsjRQGnqYhFSijwJS6iPVNk+W8M+5MXssn/DmPOH8OZbS9l0drtfP3dD7zy6QZNRSxSjgJf4iLad7CWfmMocrv51jeBDkf24KaTbyLJl1Q2CdobS/I0FbFIOQp8iZtojn0v/cawjRcpZicPnf1G2ZKFpefSFAkiFSnwpU7KSE/lzovh2pmzGNrjBob2OivkPgp6kQM0LFPqpP1F+xmz+HbSmqYx4ZK/eF2OSJ2gFr54pvx8NkCF7peq5rp5ZOEjrNq2iuE9n2P15kIy0uNdvUjdo8AXT5QfZZPkMzCjqDg4Rn/kwB6Mnr4i7Jj91dtW88CCP9Ek0JdZ2W14b+kiDbsUqQZ16YgnKozLL3YUlhujP3P55rCrTjnnuG76dST5jqBZwbCIVqaK1Z2/IrWVAl88UWFlKb+RXG6VqQE924RddeqFpS/w/vr3ua33/TRKanHYK1OVfsMYM2c1QyctUuhLQlCXjnii8rBJqNiHf2zrJgf14W/Zs4U75tzBGWlnMPq8mxh8zM7DHnYZizt/RWo7Bb54pvKwycqPKwfwrbNvZXfBbp4d+Cw+80U07DLad/6K1AUKfKkTZuXO4p+f/5NRZ47iuFbHRXw83ZgliSiiPnwzu8LMVphZwMxCLqlVsl9/M1ttZrlm9vtIzimJZ0/BHoa/M5xjWxzL3affHbXjZqSnMuLsLgp7SRiRXrRdDgwBFoTbwcz8wHhgANAduNrMukd4Xkkg979/P+u+W8fEiyfSIKmB1+WI1FkRdek451YBmNmhdusN5Drn1pbs+wowCFgZybklMSz9ZimPf/w41550rZYsFIlQPIZltgM2lnueV/KayCEVB4oZNm0YLRu15JF+j3hdjkidV2UL38zeBVqH2HSPc+7tapwjVPPfHeJ8WUAWQFpaWjUOL/XVuE/Hkb0pmymXTSG1ofrZRSJVZeA7586L8Bx5QIdyz9sDmw5xvonARIDMzMywvxikftuwcwP3vHcPA7oM4Gc9fuZ1OSL1Qjy6dBYDXc2sk5mlAFcBU+NwXqmjnHOMmDECh+OvF/2VJRu+0xQIIlEQ0UVbMxsMjAVaAe+Y2VLn3AVm1haY5Jy70DlXZGa/BWYDfmCyc25FxJVLvfX6qteZvmY6j/V7jO07m8Z08XORRBLpKJ03gTdDvL4JuLDc8xnAjEjOJYnhu33fcePMGzmp9Unc3Odmnn1/naZAEIkS3Wkrtcrd797Nlj1bmH71dJJ8SZoCQSSKFPhSayzcsJAJORO4tc+tZLTNADQFgkg0KfClVigoLiBrehZpTdMYffboCtu0Nq1IdCjwpVZ4ZOEjrNy6kulXT+fIlCO9LkekXtICKOK5NdvX8KcFf+LKHldy0TEXeV2OSL2lwBdPlS5ZeETSETzV/ymvyxGp1xT44qkXl77I/HXzeaTfI3y9vYFusBKJIfXhi2e27NnC7XNu5/S00zmpxWW6wUokxtTCF8/cNvs2dhfsZuLAiXz6Vf5BN1iJSHSphS+emJ07m5c/f5mRfUdyXKvj2Ns5XzdYicSYOVd7J6TMzMx02dnZXpchUba3cC89/9qTFH8KS69fyhFJRwCQsz5fN1iJRMjMcpxzIZecVQtf4u7++ffz1XdfMf9X88vCHnSDlUisqQ9f4mrpN0sZ8/EYfnPSbziz45lelyOSUBT4EjfFgWKypmXRolELLVko4gF16UjcjF88nsWbFvPPIf+kecPmXpcjknDUwpe42LhzI/e8dw/9u/Tnqp5XeV2OSEJS4EvMlS5ZGHAB/nrhXzELta69iMSaunQk5t5Y9QbT1kzj0X6P0im1k9fliCQstfAlpnbu28mNM2/kJ61/wi19bvG6HJGEpha+xNTd/7mbb/d8y9Srp5Lk0z83ES+phS8xs3DDQp7Jfoabet9EZtuQN/6JSBxFFPhmdoWZrTCzgJmF/R9tZuvM7HMzW2pmmishAZRfsvCBcx7wuhwRIfIuneXAEODZaux7tnNuW4Tnkzri0YWPsnLrSqZdPU1LForUEhEFvnNuFaBhdlLBmu1reGDBA1zR/QoGHjPQ63JEpES8+vAdMMfMcsws61A7mlmWmWWbWfbWrVvjVJ5Ei3OO66dfryULRWqhKlv4ZvYu0DrEpnucc29X8zynOec2mdlRwFwz+8I5tyDUjs65icBECE6PXM3jSy3x0rKXmLduHhMumkCbJm28LkdEyqky8J1z50V6EufcppKfW8zsTaA3EDLwpe7aumcrt8+5ndM6nMawjGFelyMilcS8S8fMGptZk9LHwPkEL/ZKPXPbnNvYtX8XEy+eiM804lektol0WOZgM8sDTgHeMbPZJa+3NbMZJbsdDXxoZsuAT4F3nHOzIjmv1D5z/jeHf3z2D35/+u/p3qq71+WISAha4lAiVrpkYbI/mWXXL6uwipWIxJeWOJSYGv3+6JBLFopI7aKOVonIsm+W8dhHj/Hrn/xaSxaK1HIKfDlsxYFihk0bRvOGzXn0/Ee9LkdEqqAuHTlsf138VxZvWszLQ17WkoUidYBa+HJYNu7cyB/e+wMX/PgCru55tdfliEg1KPDlsBQGCjk97XSeuegZzaUkUkeoS0cOS+fUzswcOtPrMkSkBtTCFxFJEAp8EZEEocAXEUkQCnwRkQShwBcRSRAKfBGRBKHAFxFJEAp8EZEEUavnwzezrcB6r+uIgpbANq+LiJNE+az6nPVPffms6c65VqE21OrAry/MLDvcggT1TaJ8Vn3O+icRPqu6dEREEoQCX0QkQSjw42Oi1wXEUaJ8Vn3O+qfef1b14YuIJAi18EVEEoQCX0QkQSjw48TMHjWzL8zsMzN708yaeV1TLJjZFWa2wswCZlbvhriZWX8zW21muWb2e6/riRUzm2xmW8xsude1xJKZdTCzeWa2quTf7c1e1xRLCvz4mQv0dM6dAKwB7va4nlhZDgwBFnhdSLSZmR8YDwwAugNXm1l3b6uKmReB/l4XEQdFwO3OueOAPsCIevx3qsCPF+fcHOdcUcnTRUB7L+uJFefcKufcaq/riJHeQK5zbq1zrgB4BRjkcU0x4ZxbAOzwuo5Yc85tds4tKXm8C1gFtPO2qthR4Hvj14AWhK172gEbyz3Pox6HQ6Ixs47AScAnHpcSM1rEPIrM7F2gdYhN9zjn3i7Z5x6CXyNfjmdt0VSdz1lPWYjXNK65HjCzI4HXgVucc997XU+sKPCjyDl33qG2m9mvgIHAua4O3wBR1eesx/KADuWetwc2eVSLRImZJRMM+5edc294XU8sqUsnTsysP/A74BLn3F6v65HDshjoamadzCwFuAqY6nFNEgEzM+B5YJVz7nGv64k1BX78jAOaAHPNbKmZTfC6oFgws8FmlgecArxjZrO9rilaSi66/xaYTfDi3r+ccyu8rSo2zGwK8DFwrJnlmdlvvK4pRk4DfgGcU/L/cqmZXeh1UbGiqRVERBKEWvgiIglCgS8ikiAU+CIiCUKBLyKSIBT4IiIJQoEvIpIgFPgiIgni/wPYzSHFGdGr/wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "X = X - X.mean(axis=1).reshape(-1,1)\n",
    "\n",
    "U, s, VT = np.linalg.svd(X)\n",
    "v = U[: ,0]\n",
    "\n",
    "plt.scatter(*X, marker = '.')\n",
    "plt.axis('equal');\n",
    "\n",
    "plt.plot(*(np.c_[-v, v] * 2), 'g')\n",
    "\n",
    "Xi = X[:,45]\n",
    "plt.plot(*Xi, 'r.')\n",
    "\n",
    "pXi = Xi.dot(v) * v\n",
    "eXi = Xi - pXi\n",
    "\n",
    "plt.plot(*np.c_[0*pXi, pXi], 'r');\n",
    "plt.plot(*np.c_[pXi, Xi], 'r:');\n",
    "plt.plot(*np.c_[0*pXi, Xi], 'r:');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Mathematische Formulierung"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Gegeben ist also ein zentrierter Datensatz\n",
    "\\begin{equation*} \n",
    "x_j \\in \\mathbb{R}^m, \\quad j = 1,\\ldots,n,\n",
    "\\quad\n",
    "\\bar{X} = \\frac{1}{n} \\sum_{j=1}^n x_j = 0.\n",
    "\\end{equation*}\n",
    "Wir suchen eine Gerade\n",
    "\\begin{equation*} \n",
    "g(s) = s \\, u, \n",
    "\\quad s\\in \\mathbb{R}, \\quad u \\in \\mathbb{R}^m \\quad\\text{mit}\\quad \\|u\\|_2 = 1,\n",
    "\\end{equation*}\n",
    "so dass die Varianz \n",
    "\\begin{align*} \n",
    "V_1(u) \n",
    "&= \\frac{1}{n-1}\\sum_{j=1}^n (x_j,u)_2^2\\\\\n",
    "&= \\frac{1}{n-1}(X^T u, X^T u)_2\\\\\n",
    "%&= \\frac{1}{n-1}u^T X X^T u\\\\\n",
    "&= \\frac{1}{n-1} \\|X^T u\\|_2^2\n",
    "\\end{align*}\n",
    "der Projektionen von $x_i$ auf $u$ maximal wird.\n",
    "Mit der Singulärwertzerlegung $X = U\\Sigma V^T$ erhalten wir\n",
    "\\begin{align*} \n",
    "V_1(u) \n",
    "&= \\frac{1}{n-1} \\|V\\Sigma^T U^T u\\|_2^2\\\\\n",
    "&= \\frac{1}{n-1} \\|\\Sigma^T z\\|_2^2\\\\\n",
    "&= \\frac{1}{n-1} \\sum_{i=1}^r (\\sigma_i z)^2\\\\\n",
    "\\end{align*}\n",
    "mit\n",
    "\\begin{equation*} \n",
    "\\quad z =  U^T u,\n",
    "\\quad \\|z\\|_2 =  \\|u\\|_2.\n",
    "\\end{equation*}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wir müssen also das Problem\n",
    "\\begin{equation*} \n",
    "\\max_{z} f(z), \\quad g(z)= 0\n",
    "\\end{equation*}\n",
    "für\n",
    "\\begin{align*} \n",
    "f(z) \n",
    "&= \\sum_{i=1}^r (\\sigma_i z_i)^2 = \\|\\Sigma^T z\\|_2^2 = (\\Sigma^T z, \\Sigma^T z)_2,\n",
    "\\\\\n",
    "g(z) \n",
    "&= \\| z\\|_2^2 - 1 = (z, z)_2 - 1\n",
    "\\end{align*}\n",
    "lösen. \n",
    "Damit neben der Zielfunktion $f$ auch die Nebenbedingung $g$ differenzierbar ist benutzen wir\n",
    "nicht $\\|z\\|_2 = 1$ sondern äquivalent $\\|z\\|^2_2 = 1$.\n",
    "Beide Funktionen $f$ und $g$ sind auch konvex.\n",
    "\n",
    "Wir stellen nun die Lagrangefunktion \n",
    "\\begin{equation*} \n",
    "L(z,\\lambda) = f(z) - \\lambda g(z)\n",
    "\\end{equation*} \n",
    "auf und betrachten deren stationäre Punkte.\n",
    "Für die Ableitungen folgt nach der Produktregel\n",
    "\\begin{align*} \n",
    "\\partial_z L(z, \\lambda)(w)\n",
    "& = (\\Sigma^T w, \\Sigma^T z)_2 + (\\Sigma^T z, \\Sigma^T w)_2- \\lambda \\big((w, z)_2 + (z, w)_2\\big)\\\\\n",
    "& = 2 \\big((\\Sigma^T w, \\Sigma^T z)_2 - \\lambda (w, z)_2 \\big)\\\\\n",
    "& = 2 \\big(w^T\\Sigma \\Sigma^T z - \\lambda w^T z \\big)\n",
    "\\end{align*}\n",
    "also\n",
    "\\begin{equation*} \n",
    "\\partial_z L(z, \\lambda)\n",
    "= 2 \\big(\\Sigma \\Sigma^T z - \\lambda z \\big)\n",
    "\\end{equation*}\n",
    "bzw.\n",
    "\\begin{equation*} \n",
    "\\partial_\\lambda L(z, \\lambda)  = \\|z\\|_2^2 - 1\n",
    "\\end{equation*}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Als notwendige Bedingung für eine Maximalstelle\n",
    "$\\hat{z}$ muss es ein $\\hat{\\lambda}\\in\\mathbb{R}$ geben mit\n",
    "\\begin{equation*} \n",
    "\\partial_z L(\\hat{z}, \\hat{\\lambda}) = 0,\n",
    "\\quad\n",
    "\\partial_\\lambda L(\\hat{z}, \\hat{\\lambda}) = 0.\n",
    "\\end{equation*}\n",
    "Für die Maximalstelle $\\hat{z}\\in\\mathbb{R}^m$ muss also\n",
    "\\begin{equation*} \n",
    "\\|\\hat{z}\\|_2 = 1,\n",
    "\\quad\n",
    "\\Sigma \\Sigma^T \\hat{z} = \\hat{\\lambda} \\hat{z}\n",
    "\\end{equation*}\n",
    "gelten, d.h. $\\hat{\\lambda}$, $\\hat{z}$ sind Eigenwert und Eigenvektor\n",
    "der Diagonalmatrix\n",
    "\\begin{equation*} \n",
    "\\Sigma \\Sigma^T = \n",
    "\\begin{pmatrix}\n",
    "\\sigma_1^2 \\\\\n",
    "& \\ddots\\\\\n",
    "&& \\sigma_r^2\\\\\n",
    "&&& 0\\\\\n",
    "&&&& \\ddots\n",
    "\\end{pmatrix},\n",
    "\\end{equation*}\n",
    "also\n",
    "\\begin{equation*} \n",
    "\\hat{z} = e_i,\n",
    "\\quad\n",
    "\\hat{\\lambda}_i = \n",
    "\\begin{cases}\n",
    "\\sigma_i^2 & i = 1,\\ldots,r \\\\\n",
    "0 & i>r\n",
    "\\end{cases},\n",
    "\\end{equation*}\n",
    "wobei $e_i$ der $i$-te Einheitsvektor ist.\n",
    "Als Wert unserer Zielfunktion $f$ erhalten wir\n",
    "\\begin{equation*} \n",
    "f(\\hat{z}) \n",
    "= \\|\\Sigma^T e_i\\|_2^2\n",
    "= \\sigma_i^2\n",
    "\\end{equation*}\n",
    "so dass das gesuchte Maximum für $i=1$ angenommen wird.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Für unser Ausgangsproblem erhalten wir somit\n",
    "\\begin{equation*} \n",
    "\\hat{u} = U e_1= u_1,\n",
    "\\quad\n",
    "V_1(\\hat{u}) \n",
    "= \\frac{\\sigma_1^2}{n-1},\n",
    "\\quad\n",
    "X = U\\Sigma V^T,\n",
    "\\end{equation*}\n",
    "d.h. die Gerade mit maximaler Varianz der Projektionen wird vom ersten\n",
    "linken Singulärvektor $u_1$ von $X$ aufgespannt und die Varianz der Projektion\n",
    "ist durch den zugehörigen Singulärwert $\\sigma_i$ gegeben,\n",
    "oder anders ausgedrückt, $u_1$ ist\n",
    "der Eigenvektor von $C = XX^T$\n",
    "zum größten Eigenwert \n",
    "$\\lambda_1 = \\sigma_1^2$ und\n",
    "\\begin{equation*} \n",
    "V_1(u_1) \n",
    "= \\frac{\\lambda_1}{n-1}\n",
    ".\n",
    "\\end{equation*}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Projiziert man die Daten $X$ jetzt auf $u_1^\\bot$, so reduziert sich die Dimension auf $(m-1)\\times n$. Wendet man die selben Überlegungen erneut an, so erhält man \n",
    "als zweite Richtung mit der zweitgrößten Varianz\n",
    "\\begin{equation*} \n",
    "\\hat{u} = u_2,\n",
    "\\quad\n",
    "V_2(u_2) \n",
    "= \\frac{\\sigma_2^2}{n-1}.\n",
    "\\end{equation*}\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Setzt man dies nun per Induktion weiter fort, so erhält nach $r$ Schritten \n",
    "\\begin{align*} \n",
    "\\mathbb{R}^{m\\times n} \\ni X \n",
    "&= U\\Sigma V^T \n",
    "= U_r \\Sigma_r V_r^T \n",
    "= U_r \\tilde{X}, \n",
    "\\\\ \n",
    "\\tilde{X} &= \\Sigma_r V_r^T = U_r^T X\n",
    "\\in \\mathbb{R}^{r\\times n}\n",
    ".\n",
    "\\end{align*}\n",
    "Die $u_j$ heißen **Hauptkomponenten**\n",
    "oder auch **Karhunen-Loeve-Richtungen**\n",
    "zum Datensatz $X$.\n",
    "\n",
    "Die Daten $X\\in \\mathbb{R}^{m\\times n}$ werden in ein neues Orthonormalsystem der Dimension\n",
    "$r \\le m$ transformiert (Spalten von $U_r$). Die neuen Koordinaten sind die Projektionen\n",
    "$\\tilde{X}= U_r^T X$. Für die Varianzen der einzelnen Koordinaten gilt\n",
    "\\begin{equation*} \n",
    "V(\\tilde{X}_{i,\\cdot}) = \\frac{\\sigma_i^2}{n-1}, \\quad i = 1,\\ldots,r\n",
    "\\end{equation*}\n",
    "also\n",
    "\\begin{equation*} \n",
    "V(\\tilde{X}_{1,\\cdot}) \\ge V(\\tilde{X}_{2,\\cdot}) \\ge \\ldots \\ge V(\\tilde{X}_{r,\\cdot}) > 0.\n",
    "\\end{equation*}\n",
    "Außerdem gilt\n",
    "\\begin{equation*} \n",
    "V(X) \n",
    "= \\sum_{i=1}^m V(X_{i,\\cdot})\n",
    "= \\sum_{i=1}^r V(\\tilde{X}_{i,\\cdot})\n",
    "= V(\\tilde{X}),\n",
    "\\end{equation*}\n",
    "d.h. in $\\tilde{X}$ \"stecken alle Variationen von $X$\" drin, d.h. bezüglich Unterscheidungsmöglichkeiten\n",
    "der Daten haben wir keine Information verloren."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Damit liegt nun folgende Vorgehensweise zur Dimensionsreduktion nahe:\n",
    "\n",
    "- berechne $V(X)$\n",
    "\n",
    "- bestimme zu $\\varepsilon > 0$ ein\n",
    "  möglichst kleines $k$ mit \n",
    "  \\begin{equation*} \n",
    "  \\sum_{i=1}^k V(\\tilde{X}_{i,\\cdot}) \n",
    "   = \\frac{\\sum_{i=1}^k\\sigma_i^2}{n-1}\n",
    "  \\ge V(X) - \\varepsilon\n",
    "  \\end{equation*}\n",
    "  \n",
    "- benutze statt $X\\in \\mathbb{R}^{m\\times n}$ jetzt $\\tilde{X} = U_k^TX\\in \\mathbb{R}^{k\\times n}$\n",
    "\n",
    "Man nimmt dabei einen (geringen) Informationsverlust ($\\varepsilon > 0$) zugunsten einer\n",
    "Dimensionsreduktion (oft ist $k\\ll m$) in Kauf."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Bisher haben wir angenommen, dass die Daten $x_j$ zentriert sind.\n",
    "Die Verallgemeinerung ist recht einfach. Sind die Daten $x_j$ beliebig, so betrachtet\n",
    "man die zentrierten Größen\n",
    "\\begin{equation*} \n",
    "y_j = x_j - \\bar{X}, \\quad \\bar{X} = \\frac{1}{n} \\sum_{j=1}^n x_j. \n",
    "\\end{equation*}\n",
    "Man erhält Hauptrichtungen $u_j$ und Varianzen $V(Y_{i,\\cdot})$\n",
    "über die Eigenvektoren und Eigenwerte von\n",
    "\\begin{equation*} \n",
    "C = YY^T = (X - \\bar{X}) (X - \\bar{X})^T,\n",
    "\\end{equation*}\n",
    "der [Kovarianzmatrix](https://de.wikipedia.org/wiki/Kovarianzmatrix) der Originaldaten $X$.\n",
    "Die neuen Koordinaten $\\tilde{Y} = U_r^T(X - \\bar{X})$\n",
    "sind die Projektionen auf die Geraden\n",
    "\\begin{equation*} \n",
    "\\hat{g}_j = \\bar{X} + s \\, u_j,\n",
    "\\end{equation*}\n",
    "d.h. man projiziert auf affine Unterräume."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Beispiel"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wir betrachten den folgenden Datensatz."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "code_folding": [
     17
    ],
    "execution": {
     "iopub.execute_input": "2021-09-24T09:28:15.211670Z",
     "iopub.status.busy": "2021-09-24T09:28:15.205398Z",
     "iopub.status.idle": "2021-09-24T09:28:15.284497Z",
     "shell.execute_reply": "2021-09-24T09:28:15.284851Z"
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "/* Put everything inside the global mpl namespace */\n",
       "/* global mpl */\n",
       "window.mpl = {};\n",
       "\n",
       "mpl.get_websocket_type = function () {\n",
       "    if (typeof WebSocket !== 'undefined') {\n",
       "        return WebSocket;\n",
       "    } else if (typeof MozWebSocket !== 'undefined') {\n",
       "        return MozWebSocket;\n",
       "    } else {\n",
       "        alert(\n",
       "            'Your browser does not have WebSocket support. ' +\n",
       "                'Please try Chrome, Safari or Firefox ≥ 6. ' +\n",
       "                'Firefox 4 and 5 are also supported but you ' +\n",
       "                'have to enable WebSockets in about:config.'\n",
       "        );\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure = function (figure_id, websocket, ondownload, parent_element) {\n",
       "    this.id = figure_id;\n",
       "\n",
       "    this.ws = websocket;\n",
       "\n",
       "    this.supports_binary = this.ws.binaryType !== undefined;\n",
       "\n",
       "    if (!this.supports_binary) {\n",
       "        var warnings = document.getElementById('mpl-warnings');\n",
       "        if (warnings) {\n",
       "            warnings.style.display = 'block';\n",
       "            warnings.textContent =\n",
       "                'This browser does not support binary websocket messages. ' +\n",
       "                'Performance may be slow.';\n",
       "        }\n",
       "    }\n",
       "\n",
       "    this.imageObj = new Image();\n",
       "\n",
       "    this.context = undefined;\n",
       "    this.message = undefined;\n",
       "    this.canvas = undefined;\n",
       "    this.rubberband_canvas = undefined;\n",
       "    this.rubberband_context = undefined;\n",
       "    this.format_dropdown = undefined;\n",
       "\n",
       "    this.image_mode = 'full';\n",
       "\n",
       "    this.root = document.createElement('div');\n",
       "    this.root.setAttribute('style', 'display: inline-block');\n",
       "    this._root_extra_style(this.root);\n",
       "\n",
       "    parent_element.appendChild(this.root);\n",
       "\n",
       "    this._init_header(this);\n",
       "    this._init_canvas(this);\n",
       "    this._init_toolbar(this);\n",
       "\n",
       "    var fig = this;\n",
       "\n",
       "    this.waiting = false;\n",
       "\n",
       "    this.ws.onopen = function () {\n",
       "        fig.send_message('supports_binary', { value: fig.supports_binary });\n",
       "        fig.send_message('send_image_mode', {});\n",
       "        if (fig.ratio !== 1) {\n",
       "            fig.send_message('set_dpi_ratio', { dpi_ratio: fig.ratio });\n",
       "        }\n",
       "        fig.send_message('refresh', {});\n",
       "    };\n",
       "\n",
       "    this.imageObj.onload = function () {\n",
       "        if (fig.image_mode === 'full') {\n",
       "            // Full images could contain transparency (where diff images\n",
       "            // almost always do), so we need to clear the canvas so that\n",
       "            // there is no ghosting.\n",
       "            fig.context.clearRect(0, 0, fig.canvas.width, fig.canvas.height);\n",
       "        }\n",
       "        fig.context.drawImage(fig.imageObj, 0, 0);\n",
       "    };\n",
       "\n",
       "    this.imageObj.onunload = function () {\n",
       "        fig.ws.close();\n",
       "    };\n",
       "\n",
       "    this.ws.onmessage = this._make_on_message_function(this);\n",
       "\n",
       "    this.ondownload = ondownload;\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._init_header = function () {\n",
       "    var titlebar = document.createElement('div');\n",
       "    titlebar.classList =\n",
       "        'ui-dialog-titlebar ui-widget-header ui-corner-all ui-helper-clearfix';\n",
       "    var titletext = document.createElement('div');\n",
       "    titletext.classList = 'ui-dialog-title';\n",
       "    titletext.setAttribute(\n",
       "        'style',\n",
       "        'width: 100%; text-align: center; padding: 3px;'\n",
       "    );\n",
       "    titlebar.appendChild(titletext);\n",
       "    this.root.appendChild(titlebar);\n",
       "    this.header = titletext;\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function (_canvas_div) {};\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function (_canvas_div) {};\n",
       "\n",
       "mpl.figure.prototype._init_canvas = function () {\n",
       "    var fig = this;\n",
       "\n",
       "    var canvas_div = (this.canvas_div = document.createElement('div'));\n",
       "    canvas_div.setAttribute(\n",
       "        'style',\n",
       "        'border: 1px solid #ddd;' +\n",
       "            'box-sizing: content-box;' +\n",
       "            'clear: both;' +\n",
       "            'min-height: 1px;' +\n",
       "            'min-width: 1px;' +\n",
       "            'outline: 0;' +\n",
       "            'overflow: hidden;' +\n",
       "            'position: relative;' +\n",
       "            'resize: both;'\n",
       "    );\n",
       "\n",
       "    function on_keyboard_event_closure(name) {\n",
       "        return function (event) {\n",
       "            return fig.key_event(event, name);\n",
       "        };\n",
       "    }\n",
       "\n",
       "    canvas_div.addEventListener(\n",
       "        'keydown',\n",
       "        on_keyboard_event_closure('key_press')\n",
       "    );\n",
       "    canvas_div.addEventListener(\n",
       "        'keyup',\n",
       "        on_keyboard_event_closure('key_release')\n",
       "    );\n",
       "\n",
       "    this._canvas_extra_style(canvas_div);\n",
       "    this.root.appendChild(canvas_div);\n",
       "\n",
       "    var canvas = (this.canvas = document.createElement('canvas'));\n",
       "    canvas.classList.add('mpl-canvas');\n",
       "    canvas.setAttribute('style', 'box-sizing: content-box;');\n",
       "\n",
       "    this.context = canvas.getContext('2d');\n",
       "\n",
       "    var backingStore =\n",
       "        this.context.backingStorePixelRatio ||\n",
       "        this.context.webkitBackingStorePixelRatio ||\n",
       "        this.context.mozBackingStorePixelRatio ||\n",
       "        this.context.msBackingStorePixelRatio ||\n",
       "        this.context.oBackingStorePixelRatio ||\n",
       "        this.context.backingStorePixelRatio ||\n",
       "        1;\n",
       "\n",
       "    this.ratio = (window.devicePixelRatio || 1) / backingStore;\n",
       "\n",
       "    var rubberband_canvas = (this.rubberband_canvas = document.createElement(\n",
       "        'canvas'\n",
       "    ));\n",
       "    rubberband_canvas.setAttribute(\n",
       "        'style',\n",
       "        'box-sizing: content-box; position: absolute; left: 0; top: 0; z-index: 1;'\n",
       "    );\n",
       "\n",
       "    // Apply a ponyfill if ResizeObserver is not implemented by browser.\n",
       "    if (this.ResizeObserver === undefined) {\n",
       "        if (window.ResizeObserver !== undefined) {\n",
       "            this.ResizeObserver = window.ResizeObserver;\n",
       "        } else {\n",
       "            var obs = _JSXTOOLS_RESIZE_OBSERVER({});\n",
       "            this.ResizeObserver = obs.ResizeObserver;\n",
       "        }\n",
       "    }\n",
       "\n",
       "    this.resizeObserverInstance = new this.ResizeObserver(function (entries) {\n",
       "        var nentries = entries.length;\n",
       "        for (var i = 0; i < nentries; i++) {\n",
       "            var entry = entries[i];\n",
       "            var width, height;\n",
       "            if (entry.contentBoxSize) {\n",
       "                if (entry.contentBoxSize instanceof Array) {\n",
       "                    // Chrome 84 implements new version of spec.\n",
       "                    width = entry.contentBoxSize[0].inlineSize;\n",
       "                    height = entry.contentBoxSize[0].blockSize;\n",
       "                } else {\n",
       "                    // Firefox implements old version of spec.\n",
       "                    width = entry.contentBoxSize.inlineSize;\n",
       "                    height = entry.contentBoxSize.blockSize;\n",
       "                }\n",
       "            } else {\n",
       "                // Chrome <84 implements even older version of spec.\n",
       "                width = entry.contentRect.width;\n",
       "                height = entry.contentRect.height;\n",
       "            }\n",
       "\n",
       "            // Keep the size of the canvas and rubber band canvas in sync with\n",
       "            // the canvas container.\n",
       "            if (entry.devicePixelContentBoxSize) {\n",
       "                // Chrome 84 implements new version of spec.\n",
       "                canvas.setAttribute(\n",
       "                    'width',\n",
       "                    entry.devicePixelContentBoxSize[0].inlineSize\n",
       "                );\n",
       "                canvas.setAttribute(\n",
       "                    'height',\n",
       "                    entry.devicePixelContentBoxSize[0].blockSize\n",
       "                );\n",
       "            } else {\n",
       "                canvas.setAttribute('width', width * fig.ratio);\n",
       "                canvas.setAttribute('height', height * fig.ratio);\n",
       "            }\n",
       "            canvas.setAttribute(\n",
       "                'style',\n",
       "                'width: ' + width + 'px; height: ' + height + 'px;'\n",
       "            );\n",
       "\n",
       "            rubberband_canvas.setAttribute('width', width);\n",
       "            rubberband_canvas.setAttribute('height', height);\n",
       "\n",
       "            // And update the size in Python. We ignore the initial 0/0 size\n",
       "            // that occurs as the element is placed into the DOM, which should\n",
       "            // otherwise not happen due to the minimum size styling.\n",
       "            if (fig.ws.readyState == 1 && width != 0 && height != 0) {\n",
       "                fig.request_resize(width, height);\n",
       "            }\n",
       "        }\n",
       "    });\n",
       "    this.resizeObserverInstance.observe(canvas_div);\n",
       "\n",
       "    function on_mouse_event_closure(name) {\n",
       "        return function (event) {\n",
       "            return fig.mouse_event(event, name);\n",
       "        };\n",
       "    }\n",
       "\n",
       "    rubberband_canvas.addEventListener(\n",
       "        'mousedown',\n",
       "        on_mouse_event_closure('button_press')\n",
       "    );\n",
       "    rubberband_canvas.addEventListener(\n",
       "        'mouseup',\n",
       "        on_mouse_event_closure('button_release')\n",
       "    );\n",
       "    rubberband_canvas.addEventListener(\n",
       "        'dblclick',\n",
       "        on_mouse_event_closure('dblclick')\n",
       "    );\n",
       "    // Throttle sequential mouse events to 1 every 20ms.\n",
       "    rubberband_canvas.addEventListener(\n",
       "        'mousemove',\n",
       "        on_mouse_event_closure('motion_notify')\n",
       "    );\n",
       "\n",
       "    rubberband_canvas.addEventListener(\n",
       "        'mouseenter',\n",
       "        on_mouse_event_closure('figure_enter')\n",
       "    );\n",
       "    rubberband_canvas.addEventListener(\n",
       "        'mouseleave',\n",
       "        on_mouse_event_closure('figure_leave')\n",
       "    );\n",
       "\n",
       "    canvas_div.addEventListener('wheel', function (event) {\n",
       "        if (event.deltaY < 0) {\n",
       "            event.step = 1;\n",
       "        } else {\n",
       "            event.step = -1;\n",
       "        }\n",
       "        on_mouse_event_closure('scroll')(event);\n",
       "    });\n",
       "\n",
       "    canvas_div.appendChild(canvas);\n",
       "    canvas_div.appendChild(rubberband_canvas);\n",
       "\n",
       "    this.rubberband_context = rubberband_canvas.getContext('2d');\n",
       "    this.rubberband_context.strokeStyle = '#000000';\n",
       "\n",
       "    this._resize_canvas = function (width, height, forward) {\n",
       "        if (forward) {\n",
       "            canvas_div.style.width = width + 'px';\n",
       "            canvas_div.style.height = height + 'px';\n",
       "        }\n",
       "    };\n",
       "\n",
       "    // Disable right mouse context menu.\n",
       "    this.rubberband_canvas.addEventListener('contextmenu', function (_e) {\n",
       "        event.preventDefault();\n",
       "        return false;\n",
       "    });\n",
       "\n",
       "    function set_focus() {\n",
       "        canvas.focus();\n",
       "        canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    window.setTimeout(set_focus, 100);\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function () {\n",
       "    var fig = this;\n",
       "\n",
       "    var toolbar = document.createElement('div');\n",
       "    toolbar.classList = 'mpl-toolbar';\n",
       "    this.root.appendChild(toolbar);\n",
       "\n",
       "    function on_click_closure(name) {\n",
       "        return function (_event) {\n",
       "            return fig.toolbar_button_onclick(name);\n",
       "        };\n",
       "    }\n",
       "\n",
       "    function on_mouseover_closure(tooltip) {\n",
       "        return function (event) {\n",
       "            if (!event.currentTarget.disabled) {\n",
       "                return fig.toolbar_button_onmouseover(tooltip);\n",
       "            }\n",
       "        };\n",
       "    }\n",
       "\n",
       "    fig.buttons = {};\n",
       "    var buttonGroup = document.createElement('div');\n",
       "    buttonGroup.classList = 'mpl-button-group';\n",
       "    for (var toolbar_ind in mpl.toolbar_items) {\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) {\n",
       "            /* Instead of a spacer, we start a new button group. */\n",
       "            if (buttonGroup.hasChildNodes()) {\n",
       "                toolbar.appendChild(buttonGroup);\n",
       "            }\n",
       "            buttonGroup = document.createElement('div');\n",
       "            buttonGroup.classList = 'mpl-button-group';\n",
       "            continue;\n",
       "        }\n",
       "\n",
       "        var button = (fig.buttons[name] = document.createElement('button'));\n",
       "        button.classList = 'mpl-widget';\n",
       "        button.setAttribute('role', 'button');\n",
       "        button.setAttribute('aria-disabled', 'false');\n",
       "        button.addEventListener('click', on_click_closure(method_name));\n",
       "        button.addEventListener('mouseover', on_mouseover_closure(tooltip));\n",
       "\n",
       "        var icon_img = document.createElement('img');\n",
       "        icon_img.src = '_images/' + image + '.png';\n",
       "        icon_img.srcset = '_images/' + image + '_large.png 2x';\n",
       "        icon_img.alt = tooltip;\n",
       "        button.appendChild(icon_img);\n",
       "\n",
       "        buttonGroup.appendChild(button);\n",
       "    }\n",
       "\n",
       "    if (buttonGroup.hasChildNodes()) {\n",
       "        toolbar.appendChild(buttonGroup);\n",
       "    }\n",
       "\n",
       "    var fmt_picker = document.createElement('select');\n",
       "    fmt_picker.classList = 'mpl-widget';\n",
       "    toolbar.appendChild(fmt_picker);\n",
       "    this.format_dropdown = fmt_picker;\n",
       "\n",
       "    for (var ind in mpl.extensions) {\n",
       "        var fmt = mpl.extensions[ind];\n",
       "        var option = document.createElement('option');\n",
       "        option.selected = fmt === mpl.default_extension;\n",
       "        option.innerHTML = fmt;\n",
       "        fmt_picker.appendChild(option);\n",
       "    }\n",
       "\n",
       "    var status_bar = document.createElement('span');\n",
       "    status_bar.classList = 'mpl-message';\n",
       "    toolbar.appendChild(status_bar);\n",
       "    this.message = status_bar;\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.request_resize = function (x_pixels, y_pixels) {\n",
       "    // Request matplotlib to resize the figure. Matplotlib will then trigger a resize in the client,\n",
       "    // which will in turn request a refresh of the image.\n",
       "    this.send_message('resize', { width: x_pixels, height: y_pixels });\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.send_message = function (type, properties) {\n",
       "    properties['type'] = type;\n",
       "    properties['figure_id'] = this.id;\n",
       "    this.ws.send(JSON.stringify(properties));\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.send_draw_message = function () {\n",
       "    if (!this.waiting) {\n",
       "        this.waiting = true;\n",
       "        this.ws.send(JSON.stringify({ type: 'draw', figure_id: this.id }));\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_save = function (fig, _msg) {\n",
       "    var format_dropdown = fig.format_dropdown;\n",
       "    var format = format_dropdown.options[format_dropdown.selectedIndex].value;\n",
       "    fig.ondownload(fig, format);\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_resize = function (fig, msg) {\n",
       "    var size = msg['size'];\n",
       "    if (size[0] !== fig.canvas.width || size[1] !== fig.canvas.height) {\n",
       "        fig._resize_canvas(size[0], size[1], msg['forward']);\n",
       "        fig.send_message('refresh', {});\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_rubberband = function (fig, msg) {\n",
       "    var x0 = msg['x0'] / fig.ratio;\n",
       "    var y0 = (fig.canvas.height - msg['y0']) / fig.ratio;\n",
       "    var x1 = msg['x1'] / fig.ratio;\n",
       "    var y1 = (fig.canvas.height - msg['y1']) / fig.ratio;\n",
       "    x0 = Math.floor(x0) + 0.5;\n",
       "    y0 = Math.floor(y0) + 0.5;\n",
       "    x1 = Math.floor(x1) + 0.5;\n",
       "    y1 = Math.floor(y1) + 0.5;\n",
       "    var min_x = Math.min(x0, x1);\n",
       "    var min_y = Math.min(y0, y1);\n",
       "    var width = Math.abs(x1 - x0);\n",
       "    var height = Math.abs(y1 - y0);\n",
       "\n",
       "    fig.rubberband_context.clearRect(\n",
       "        0,\n",
       "        0,\n",
       "        fig.canvas.width / fig.ratio,\n",
       "        fig.canvas.height / fig.ratio\n",
       "    );\n",
       "\n",
       "    fig.rubberband_context.strokeRect(min_x, min_y, width, height);\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_figure_label = function (fig, msg) {\n",
       "    // Updates the figure title.\n",
       "    fig.header.textContent = msg['label'];\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_cursor = function (fig, msg) {\n",
       "    var cursor = msg['cursor'];\n",
       "    switch (cursor) {\n",
       "        case 0:\n",
       "            cursor = 'pointer';\n",
       "            break;\n",
       "        case 1:\n",
       "            cursor = 'default';\n",
       "            break;\n",
       "        case 2:\n",
       "            cursor = 'crosshair';\n",
       "            break;\n",
       "        case 3:\n",
       "            cursor = 'move';\n",
       "            break;\n",
       "    }\n",
       "    fig.rubberband_canvas.style.cursor = cursor;\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_message = function (fig, msg) {\n",
       "    fig.message.textContent = msg['message'];\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_draw = function (fig, _msg) {\n",
       "    // Request the server to send over a new figure.\n",
       "    fig.send_draw_message();\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_image_mode = function (fig, msg) {\n",
       "    fig.image_mode = msg['mode'];\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_history_buttons = function (fig, msg) {\n",
       "    for (var key in msg) {\n",
       "        if (!(key in fig.buttons)) {\n",
       "            continue;\n",
       "        }\n",
       "        fig.buttons[key].disabled = !msg[key];\n",
       "        fig.buttons[key].setAttribute('aria-disabled', !msg[key]);\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_navigate_mode = function (fig, msg) {\n",
       "    if (msg['mode'] === 'PAN') {\n",
       "        fig.buttons['Pan'].classList.add('active');\n",
       "        fig.buttons['Zoom'].classList.remove('active');\n",
       "    } else if (msg['mode'] === 'ZOOM') {\n",
       "        fig.buttons['Pan'].classList.remove('active');\n",
       "        fig.buttons['Zoom'].classList.add('active');\n",
       "    } else {\n",
       "        fig.buttons['Pan'].classList.remove('active');\n",
       "        fig.buttons['Zoom'].classList.remove('active');\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function () {\n",
       "    // Called whenever the canvas gets updated.\n",
       "    this.send_message('ack', {});\n",
       "};\n",
       "\n",
       "// A function to construct a web socket function for onmessage handling.\n",
       "// Called in the figure constructor.\n",
       "mpl.figure.prototype._make_on_message_function = function (fig) {\n",
       "    return function socket_on_message(evt) {\n",
       "        if (evt.data instanceof Blob) {\n",
       "            var img = evt.data;\n",
       "            if (img.type !== 'image/png') {\n",
       "                /* FIXME: We get \"Resource interpreted as Image but\n",
       "                 * transferred with MIME type text/plain:\" errors on\n",
       "                 * Chrome.  But how to set the MIME type?  It doesn't seem\n",
       "                 * to be part of the websocket stream */\n",
       "                img.type = 'image/png';\n",
       "            }\n",
       "\n",
       "            /* Free the memory for the previous frames */\n",
       "            if (fig.imageObj.src) {\n",
       "                (window.URL || window.webkitURL).revokeObjectURL(\n",
       "                    fig.imageObj.src\n",
       "                );\n",
       "            }\n",
       "\n",
       "            fig.imageObj.src = (window.URL || window.webkitURL).createObjectURL(\n",
       "                img\n",
       "            );\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        } else if (\n",
       "            typeof evt.data === 'string' &&\n",
       "            evt.data.slice(0, 21) === 'data:image/png;base64'\n",
       "        ) {\n",
       "            fig.imageObj.src = evt.data;\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        var msg = JSON.parse(evt.data);\n",
       "        var msg_type = msg['type'];\n",
       "\n",
       "        // Call the  \"handle_{type}\" callback, which takes\n",
       "        // the figure and JSON message as its only arguments.\n",
       "        try {\n",
       "            var callback = fig['handle_' + msg_type];\n",
       "        } catch (e) {\n",
       "            console.log(\n",
       "                \"No handler for the '\" + msg_type + \"' message type: \",\n",
       "                msg\n",
       "            );\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        if (callback) {\n",
       "            try {\n",
       "                // console.log(\"Handling '\" + msg_type + \"' message: \", msg);\n",
       "                callback(fig, msg);\n",
       "            } catch (e) {\n",
       "                console.log(\n",
       "                    \"Exception inside the 'handler_\" + msg_type + \"' callback:\",\n",
       "                    e,\n",
       "                    e.stack,\n",
       "                    msg\n",
       "                );\n",
       "            }\n",
       "        }\n",
       "    };\n",
       "};\n",
       "\n",
       "// from http://stackoverflow.com/questions/1114465/getting-mouse-location-in-canvas\n",
       "mpl.findpos = function (e) {\n",
       "    //this section is from http://www.quirksmode.org/js/events_properties.html\n",
       "    var targ;\n",
       "    if (!e) {\n",
       "        e = window.event;\n",
       "    }\n",
       "    if (e.target) {\n",
       "        targ = e.target;\n",
       "    } else if (e.srcElement) {\n",
       "        targ = e.srcElement;\n",
       "    }\n",
       "    if (targ.nodeType === 3) {\n",
       "        // defeat Safari bug\n",
       "        targ = targ.parentNode;\n",
       "    }\n",
       "\n",
       "    // pageX,Y are the mouse positions relative to the document\n",
       "    var boundingRect = targ.getBoundingClientRect();\n",
       "    var x = e.pageX - (boundingRect.left + document.body.scrollLeft);\n",
       "    var y = e.pageY - (boundingRect.top + document.body.scrollTop);\n",
       "\n",
       "    return { x: x, y: y };\n",
       "};\n",
       "\n",
       "/*\n",
       " * return a copy of an object with only non-object keys\n",
       " * we need this to avoid circular references\n",
       " * http://stackoverflow.com/a/24161582/3208463\n",
       " */\n",
       "function simpleKeys(original) {\n",
       "    return Object.keys(original).reduce(function (obj, key) {\n",
       "        if (typeof original[key] !== 'object') {\n",
       "            obj[key] = original[key];\n",
       "        }\n",
       "        return obj;\n",
       "    }, {});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.mouse_event = function (event, name) {\n",
       "    var canvas_pos = mpl.findpos(event);\n",
       "\n",
       "    if (name === 'button_press') {\n",
       "        this.canvas.focus();\n",
       "        this.canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    var x = canvas_pos.x * this.ratio;\n",
       "    var y = canvas_pos.y * this.ratio;\n",
       "\n",
       "    this.send_message(name, {\n",
       "        x: x,\n",
       "        y: y,\n",
       "        button: event.button,\n",
       "        step: event.step,\n",
       "        guiEvent: simpleKeys(event),\n",
       "    });\n",
       "\n",
       "    /* This prevents the web browser from automatically changing to\n",
       "     * the text insertion cursor when the button is pressed.  We want\n",
       "     * to control all of the cursor setting manually through the\n",
       "     * 'cursor' event from matplotlib */\n",
       "    event.preventDefault();\n",
       "    return false;\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function (_event, _name) {\n",
       "    // Handle any extra behaviour associated with a key event\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.key_event = function (event, name) {\n",
       "    // Prevent repeat events\n",
       "    if (name === 'key_press') {\n",
       "        if (event.key === this._key) {\n",
       "            return;\n",
       "        } else {\n",
       "            this._key = event.key;\n",
       "        }\n",
       "    }\n",
       "    if (name === 'key_release') {\n",
       "        this._key = null;\n",
       "    }\n",
       "\n",
       "    var value = '';\n",
       "    if (event.ctrlKey && event.key !== 'Control') {\n",
       "        value += 'ctrl+';\n",
       "    }\n",
       "    else if (event.altKey && event.key !== 'Alt') {\n",
       "        value += 'alt+';\n",
       "    }\n",
       "    else if (event.shiftKey && event.key !== 'Shift') {\n",
       "        value += 'shift+';\n",
       "    }\n",
       "\n",
       "    value += 'k' + event.key;\n",
       "\n",
       "    this._key_event_extra(event, name);\n",
       "\n",
       "    this.send_message(name, { key: value, guiEvent: simpleKeys(event) });\n",
       "    return false;\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onclick = function (name) {\n",
       "    if (name === 'download') {\n",
       "        this.handle_save(this, null);\n",
       "    } else {\n",
       "        this.send_message('toolbar_button', { name: name });\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onmouseover = function (tooltip) {\n",
       "    this.message.textContent = tooltip;\n",
       "};\n",
       "\n",
       "///////////////// REMAINING CONTENT GENERATED BY embed_js.py /////////////////\n",
       "// prettier-ignore\n",
       "var _JSXTOOLS_RESIZE_OBSERVER=function(A){var t,i=new WeakMap,n=new WeakMap,a=new WeakMap,r=new WeakMap,o=new Set;function s(e){if(!(this instanceof s))throw new TypeError(\"Constructor requires 'new' operator\");i.set(this,e)}function h(){throw new TypeError(\"Function is not a constructor\")}function c(e,t,i,n){e=0 in arguments?Number(arguments[0]):0,t=1 in arguments?Number(arguments[1]):0,i=2 in arguments?Number(arguments[2]):0,n=3 in arguments?Number(arguments[3]):0,this.right=(this.x=this.left=e)+(this.width=i),this.bottom=(this.y=this.top=t)+(this.height=n),Object.freeze(this)}function d(){t=requestAnimationFrame(d);var s=new WeakMap,p=new Set;o.forEach((function(t){r.get(t).forEach((function(i){var r=t instanceof window.SVGElement,o=a.get(t),d=r?0:parseFloat(o.paddingTop),f=r?0:parseFloat(o.paddingRight),l=r?0:parseFloat(o.paddingBottom),u=r?0:parseFloat(o.paddingLeft),g=r?0:parseFloat(o.borderTopWidth),m=r?0:parseFloat(o.borderRightWidth),w=r?0:parseFloat(o.borderBottomWidth),b=u+f,F=d+l,v=(r?0:parseFloat(o.borderLeftWidth))+m,W=g+w,y=r?0:t.offsetHeight-W-t.clientHeight,E=r?0:t.offsetWidth-v-t.clientWidth,R=b+v,z=F+W,M=r?t.width:parseFloat(o.width)-R-E,O=r?t.height:parseFloat(o.height)-z-y;if(n.has(t)){var k=n.get(t);if(k[0]===M&&k[1]===O)return}n.set(t,[M,O]);var S=Object.create(h.prototype);S.target=t,S.contentRect=new c(u,d,M,O),s.has(i)||(s.set(i,[]),p.add(i)),s.get(i).push(S)}))})),p.forEach((function(e){i.get(e).call(e,s.get(e),e)}))}return s.prototype.observe=function(i){if(i instanceof window.Element){r.has(i)||(r.set(i,new Set),o.add(i),a.set(i,window.getComputedStyle(i)));var n=r.get(i);n.has(this)||n.add(this),cancelAnimationFrame(t),t=requestAnimationFrame(d)}},s.prototype.unobserve=function(i){if(i instanceof window.Element&&r.has(i)){var n=r.get(i);n.has(this)&&(n.delete(this),n.size||(r.delete(i),o.delete(i))),n.size||r.delete(i),o.size||cancelAnimationFrame(t)}},A.DOMRectReadOnly=c,A.ResizeObserver=s,A.ResizeObserverEntry=h,A}; // eslint-disable-line\n",
       "mpl.toolbar_items = [[\"Home\", \"Reset original view\", \"fa fa-home icon-home\", \"home\"], [\"Back\", \"Back to previous view\", \"fa fa-arrow-left icon-arrow-left\", \"back\"], [\"Forward\", \"Forward to next view\", \"fa fa-arrow-right icon-arrow-right\", \"forward\"], [\"\", \"\", \"\", \"\"], [\"Pan\", \"Left button pans, Right button zooms\\nx/y fixes axis, CTRL fixes aspect\", \"fa fa-arrows icon-move\", \"pan\"], [\"Zoom\", \"Zoom to rectangle\\nx/y fixes axis, CTRL fixes aspect\", \"fa fa-square-o icon-check-empty\", \"zoom\"], [\"\", \"\", \"\", \"\"], [\"Download\", \"Download plot\", \"fa fa-floppy-o icon-save\", \"download\"]];\n",
       "\n",
       "mpl.extensions = [\"eps\", \"jpeg\", \"pgf\", \"pdf\", \"png\", \"ps\", \"raw\", \"svg\", \"tif\"];\n",
       "\n",
       "mpl.default_extension = \"png\";/* global mpl */\n",
       "\n",
       "var comm_websocket_adapter = function (comm) {\n",
       "    // Create a \"websocket\"-like object which calls the given IPython comm\n",
       "    // object with the appropriate methods. Currently this is a non binary\n",
       "    // socket, so there is still some room for performance tuning.\n",
       "    var ws = {};\n",
       "\n",
       "    ws.binaryType = comm.kernel.ws.binaryType;\n",
       "    ws.readyState = comm.kernel.ws.readyState;\n",
       "    function updateReadyState(_event) {\n",
       "        if (comm.kernel.ws) {\n",
       "            ws.readyState = comm.kernel.ws.readyState;\n",
       "        } else {\n",
       "            ws.readyState = 3; // Closed state.\n",
       "        }\n",
       "    }\n",
       "    comm.kernel.ws.addEventListener('open', updateReadyState);\n",
       "    comm.kernel.ws.addEventListener('close', updateReadyState);\n",
       "    comm.kernel.ws.addEventListener('error', updateReadyState);\n",
       "\n",
       "    ws.close = function () {\n",
       "        comm.close();\n",
       "    };\n",
       "    ws.send = function (m) {\n",
       "        //console.log('sending', m);\n",
       "        comm.send(m);\n",
       "    };\n",
       "    // Register the callback with on_msg.\n",
       "    comm.on_msg(function (msg) {\n",
       "        //console.log('receiving', msg['content']['data'], msg);\n",
       "        var data = msg['content']['data'];\n",
       "        if (data['blob'] !== undefined) {\n",
       "            data = {\n",
       "                data: new Blob(msg['buffers'], { type: data['blob'] }),\n",
       "            };\n",
       "        }\n",
       "        // Pass the mpl event to the overridden (by mpl) onmessage function.\n",
       "        ws.onmessage(data);\n",
       "    });\n",
       "    return ws;\n",
       "};\n",
       "\n",
       "mpl.mpl_figure_comm = function (comm, msg) {\n",
       "    // This is the function which gets called when the mpl process\n",
       "    // starts-up an IPython Comm through the \"matplotlib\" channel.\n",
       "\n",
       "    var id = msg.content.data.id;\n",
       "    // Get hold of the div created by the display call when the Comm\n",
       "    // socket was opened in Python.\n",
       "    var element = document.getElementById(id);\n",
       "    var ws_proxy = comm_websocket_adapter(comm);\n",
       "\n",
       "    function ondownload(figure, _format) {\n",
       "        window.open(figure.canvas.toDataURL());\n",
       "    }\n",
       "\n",
       "    var fig = new mpl.figure(id, ws_proxy, ondownload, element);\n",
       "\n",
       "    // Call onopen now - mpl needs it, as it is assuming we've passed it a real\n",
       "    // web socket which is closed, not our websocket->open comm proxy.\n",
       "    ws_proxy.onopen();\n",
       "\n",
       "    fig.parent_element = element;\n",
       "    fig.cell_info = mpl.find_output_cell(\"<div id='\" + id + \"'></div>\");\n",
       "    if (!fig.cell_info) {\n",
       "        console.error('Failed to find cell for figure', id, fig);\n",
       "        return;\n",
       "    }\n",
       "    fig.cell_info[0].output_area.element.on(\n",
       "        'cleared',\n",
       "        { fig: fig },\n",
       "        fig._remove_fig_handler\n",
       "    );\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_close = function (fig, msg) {\n",
       "    var width = fig.canvas.width / fig.ratio;\n",
       "    fig.cell_info[0].output_area.element.off(\n",
       "        'cleared',\n",
       "        fig._remove_fig_handler\n",
       "    );\n",
       "    fig.resizeObserverInstance.unobserve(fig.canvas_div);\n",
       "\n",
       "    // Update the output cell to use the data from the current canvas.\n",
       "    fig.push_to_output();\n",
       "    var dataURL = fig.canvas.toDataURL();\n",
       "    // Re-enable the keyboard manager in IPython - without this line, in FF,\n",
       "    // the notebook keyboard shortcuts fail.\n",
       "    IPython.keyboard_manager.enable();\n",
       "    fig.parent_element.innerHTML =\n",
       "        '<img src=\"' + dataURL + '\" width=\"' + width + '\">';\n",
       "    fig.close_ws(fig, msg);\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.close_ws = function (fig, msg) {\n",
       "    fig.send_message('closing', msg);\n",
       "    // fig.ws.close()\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.push_to_output = function (_remove_interactive) {\n",
       "    // Turn the data on the canvas into data in the output cell.\n",
       "    var width = this.canvas.width / this.ratio;\n",
       "    var dataURL = this.canvas.toDataURL();\n",
       "    this.cell_info[1]['text/html'] =\n",
       "        '<img src=\"' + dataURL + '\" width=\"' + width + '\">';\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function () {\n",
       "    // Tell IPython that the notebook contents must change.\n",
       "    IPython.notebook.set_dirty(true);\n",
       "    this.send_message('ack', {});\n",
       "    var fig = this;\n",
       "    // Wait a second, then push the new image to the DOM so\n",
       "    // that it is saved nicely (might be nice to debounce this).\n",
       "    setTimeout(function () {\n",
       "        fig.push_to_output();\n",
       "    }, 1000);\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function () {\n",
       "    var fig = this;\n",
       "\n",
       "    var toolbar = document.createElement('div');\n",
       "    toolbar.classList = 'btn-toolbar';\n",
       "    this.root.appendChild(toolbar);\n",
       "\n",
       "    function on_click_closure(name) {\n",
       "        return function (_event) {\n",
       "            return fig.toolbar_button_onclick(name);\n",
       "        };\n",
       "    }\n",
       "\n",
       "    function on_mouseover_closure(tooltip) {\n",
       "        return function (event) {\n",
       "            if (!event.currentTarget.disabled) {\n",
       "                return fig.toolbar_button_onmouseover(tooltip);\n",
       "            }\n",
       "        };\n",
       "    }\n",
       "\n",
       "    fig.buttons = {};\n",
       "    var buttonGroup = document.createElement('div');\n",
       "    buttonGroup.classList = 'btn-group';\n",
       "    var button;\n",
       "    for (var toolbar_ind in mpl.toolbar_items) {\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) {\n",
       "            /* Instead of a spacer, we start a new button group. */\n",
       "            if (buttonGroup.hasChildNodes()) {\n",
       "                toolbar.appendChild(buttonGroup);\n",
       "            }\n",
       "            buttonGroup = document.createElement('div');\n",
       "            buttonGroup.classList = 'btn-group';\n",
       "            continue;\n",
       "        }\n",
       "\n",
       "        button = fig.buttons[name] = document.createElement('button');\n",
       "        button.classList = 'btn btn-default';\n",
       "        button.href = '#';\n",
       "        button.title = name;\n",
       "        button.innerHTML = '<i class=\"fa ' + image + ' fa-lg\"></i>';\n",
       "        button.addEventListener('click', on_click_closure(method_name));\n",
       "        button.addEventListener('mouseover', on_mouseover_closure(tooltip));\n",
       "        buttonGroup.appendChild(button);\n",
       "    }\n",
       "\n",
       "    if (buttonGroup.hasChildNodes()) {\n",
       "        toolbar.appendChild(buttonGroup);\n",
       "    }\n",
       "\n",
       "    // Add the status bar.\n",
       "    var status_bar = document.createElement('span');\n",
       "    status_bar.classList = 'mpl-message pull-right';\n",
       "    toolbar.appendChild(status_bar);\n",
       "    this.message = status_bar;\n",
       "\n",
       "    // Add the close button to the window.\n",
       "    var buttongrp = document.createElement('div');\n",
       "    buttongrp.classList = 'btn-group inline pull-right';\n",
       "    button = document.createElement('button');\n",
       "    button.classList = 'btn btn-mini btn-primary';\n",
       "    button.href = '#';\n",
       "    button.title = 'Stop Interaction';\n",
       "    button.innerHTML = '<i class=\"fa fa-power-off icon-remove icon-large\"></i>';\n",
       "    button.addEventListener('click', function (_evt) {\n",
       "        fig.handle_close(fig, {});\n",
       "    });\n",
       "    button.addEventListener(\n",
       "        'mouseover',\n",
       "        on_mouseover_closure('Stop Interaction')\n",
       "    );\n",
       "    buttongrp.appendChild(button);\n",
       "    var titlebar = this.root.querySelector('.ui-dialog-titlebar');\n",
       "    titlebar.insertBefore(buttongrp, titlebar.firstChild);\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._remove_fig_handler = function (event) {\n",
       "    var fig = event.data.fig;\n",
       "    if (event.target !== this) {\n",
       "        // Ignore bubbled events from children.\n",
       "        return;\n",
       "    }\n",
       "    fig.close_ws(fig, {});\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function (el) {\n",
       "    el.style.boxSizing = 'content-box'; // override notebook setting of border-box.\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function (el) {\n",
       "    // this is important to make the div 'focusable\n",
       "    el.setAttribute('tabindex', 0);\n",
       "    // reach out to IPython and tell the keyboard manager to turn it's self\n",
       "    // off when our div gets focus\n",
       "\n",
       "    // location in version 3\n",
       "    if (IPython.notebook.keyboard_manager) {\n",
       "        IPython.notebook.keyboard_manager.register_events(el);\n",
       "    } else {\n",
       "        // location in version 2\n",
       "        IPython.keyboard_manager.register_events(el);\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function (event, _name) {\n",
       "    var manager = IPython.notebook.keyboard_manager;\n",
       "    if (!manager) {\n",
       "        manager = IPython.keyboard_manager;\n",
       "    }\n",
       "\n",
       "    // Check for shift+enter\n",
       "    if (event.shiftKey && event.which === 13) {\n",
       "        this.canvas_div.blur();\n",
       "        // select the cell after this one\n",
       "        var index = IPython.notebook.find_cell_index(this.cell_info[0]);\n",
       "        IPython.notebook.select(index + 1);\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_save = function (fig, _msg) {\n",
       "    fig.ondownload(fig, null);\n",
       "};\n",
       "\n",
       "mpl.find_output_cell = function (html_output) {\n",
       "    // Return the cell and output element which can be found *uniquely* in the notebook.\n",
       "    // Note - this is a bit hacky, but it is done because the \"notebook_saving.Notebook\"\n",
       "    // IPython event is triggered only after the cells have been serialised, which for\n",
       "    // our purposes (turning an active figure into a static one), is too late.\n",
       "    var cells = IPython.notebook.get_cells();\n",
       "    var ncells = cells.length;\n",
       "    for (var i = 0; i < ncells; i++) {\n",
       "        var cell = cells[i];\n",
       "        if (cell.cell_type === 'code') {\n",
       "            for (var j = 0; j < cell.output_area.outputs.length; j++) {\n",
       "                var data = cell.output_area.outputs[j];\n",
       "                if (data.data) {\n",
       "                    // IPython >= 3 moved mimebundle to data attribute of output\n",
       "                    data = data.data;\n",
       "                }\n",
       "                if (data['text/html'] === html_output) {\n",
       "                    return [cell, data, j];\n",
       "                }\n",
       "            }\n",
       "        }\n",
       "    }\n",
       "};\n",
       "\n",
       "// Register the function which deals with the matplotlib target/channel.\n",
       "// The kernel may be null if the page has been refreshed.\n",
       "if (IPython.notebook.kernel !== null) {\n",
       "    IPython.notebook.kernel.comm_manager.register_target(\n",
       "        'matplotlib',\n",
       "        mpl.mpl_figure_comm\n",
       "    );\n",
       "}\n"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<img src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAoAAAAHgCAYAAAA10dzkAAAgAElEQVR4nOzdeXSj9X0vfvf2Ns1pWifN8qOQ085AWpbBBAIpNCENSRq4DdlIwyXJ7SHcm+2SewpJmpCZhBCHrQmEJWwhLIGBpAEGCo+sxdoXy7JlS5Zt2dZmSZZlWbYsW5bkdYYZv39/GAnZlmxZeuxHtt6vcz7nMJb0+DtmmOfN9/t8P98GEBEREVFdaZB6AERERES0uxgAiYiIiOoMAyARERFRnWEAJCIiIqozDIBEREREdYYBkIiIiKjOMAASERER1RkGQCIiIqI6wwBIREREVGcYAImIiIjqDAMgERERUZ1hACQiIiKqMwyARERERHWGAZCIiIiozjAAEhEREdUZBkAiIiKiOsMASERERFRnGACJiIiI6gwDIBEREVGdYQAkIiIiqjMMgERERER1hgGQiIiIqM4wABIRERHVGQZAIiIiojrDAEhERERUZxgAiYiIiOoMAyARERFRnWEAJCIiIqozDIBEREREdYYBkIiIiKjOMAASERER1RkGQCIiIqI6wwBIREREVGcYAImIiIjqDAMgERERUZ1hACQiIiKqMwyARERERHWGAZCIiIiozjAAEhEREdUZBkAiIiKiOsMASERERFRnGACJiIiI6gwDIBEREVGdYQAkIiIiqjMMgERERER1hgGQiIiIqM4wABIRERHVGQZAIiIiojrDAEhERERUZxgAiYiIiOoMAyARERFRnWEAJCIiIqozDIBEREREdYYBkIiIiKjOMAASERER1RkGQCIiIqI6wwBIREREVGcYAImIiIjqDAMgERERUZ1hACQiIiKqMwyARERERHWGAZCIiIiozjAAEhEREdUZBkAiIiKiOsMASERERFRnGACJiIiI6gwDIBEREVGdYQAkIiIiqjMMgERERER1hgGQiIiIqM4wABIRERHVGQZAIiIiojrDAEhERERUZxgAiYiIiOoMAyARERFRnWEAJCIiIqozDIBEREREdYYBkIiIiKjOMAASERER1RkGQCIiIqI6wwBIREREVGcYAImIiIjqDAMgERERUZ1hACQiIiKqMwyARERERHWGAZCIiIiozjAAEhEREdUZBkAiIiKiOsMASERERFRnGACJiIiI6gwDIBEREVGdYQAkIiIiqjMMgERERER1hgGQiIiIqM4wABIRERHVGQZAIiIiojrDAEhERERUZxgAiYiIiOoMAyARERFRnWEAJCIiIqozDIBEREREdYYBkIiIiKjOMAASERER1RkGQCIiIqI6wwBIREREVGcYAImIiIjqDAMgERERUZ1hACQiojVWVlawsrKCkydP4vXXX8eJEyewvLyMpaUlLCwsYG5uDtlsFktLSzh16pTUwyWiCjAAEhHVkZWVFZw6dSof7I4fP47l5WUsLi5ifn4e2WwW6XQas7OzCIVCCAQCmJmZQSqVwszMTP6fU6kU5ubmsLi4iNdffx0rKytS/9aIaBsYAImI9oFcsMvN2uWCXeGsXSaTwezsLGZnZ4sGulzNzs4inU7D5XKhu7sbmUwG2Wx2Qy0uLubr+PHjnA0k2kMYAImIalix5djjx49vWI7NzdoVBrlcwMuFvFz4S6fTyGQy+SoW7rLZLFwuFxwOR8nXl5aW8iFzcXERS0tLOHnyJGcDifYABkAiIokUztoVW46dm5vLB7vCWbtis3e5YFcY7koFt3Krp6cHTqez5OuLi4tYXl5eEwJzs4EMgUS1jQGQiEhkWy3HTk9PI5lMFl2OXb8kW2zWrtpgt50A2NPTU1YALAyCCwsLWF5e5pIwUQ1jACQi2gYxlmO7u7vR09OzreVYKcrpdG4aAHNLwKVCIDeIENUuBkAiImy+HJsLd6WWY9f/81bLsQ6HAy6XS/KAt1VtNc5SAbDYkjBnA4lqCwMgEe1r65dji/W0W787ttQOWbGWY/dSAOzt7a0oAK4PgtwgQlRbGACJaM8q1tMuF+zW97Qrd3fsbjxrt1WwqpXq7u6uOgBygwhRbWIAJKKaU2o5drNZu9HRUdhstm0vx9ZisKqV6urqQl9fX9UBcP2zgTxBhEh6DIBEtGvKOWIsk8lsmLUrZzk2EolArVZLHpr2UwC02+3o7+8XLQAW/rseHx/H/Pw8ZwOJJMIASESiKLUcW+yIsfUnT4ixHDs6OsoAuAcCYC4EarVajI2NsV0MkUQYAIloU5UsxxYGut1ajo1GowyAIldnZyfcbrfoAXB5eRlarRaxWCy/JMwNIkS7iwGQqE6Vsxw7OTmJeDxedDl2fbCTqllxYQBsbW2VPDSVU1s9W1cr1dHRgYGBgR0JgBqNBvF4nBtEiCTCAEi0DxVbji08Yiyb3bgcW+yIMYfDAYfDUXObKBgAdy8ADg4Olny90vC3vLwMtVqNiYmJohtETp48KfV/QkT7HgMg0R6y1RFjYi/HulyuTc+CraUaGxtjABS5bDbbjgXA1tZWTE5OFt0gsri4iBMnTnA2kGgHMQAS1YBylmOz2eKzdpv1tKt2xq63txcOh0PyIFJOMQCKX+3t7RgaGkI6nUYikUA0GsXw8DAGBwfR09OD4eHhipeBVSoVEonEpj0DuUGEaOcwABLtsO0sx64/YqzYs3a7uRzb29uL7u5uyYNIOTU2NgaVSiX5OMqpWgmAmUwGMzMziMfjGBkZgc/nQ39/P7q7u2G1WiGXyyGXyyEIAmQyGVpbW2E0GmGz2eB0OqFWq9HR0YFsdvuzgUqlEslkcst2MUtLSzxPmGgHMAASVaiS5Viv1wu3270nmhVns1n09fWhq6tL8nGUU3spAG7VXkWMymQySCaTiMViCIVC8Hg86O3tRWdnJ8xmM7RaLVpaWiAIAuRyObRaLSwWC+x2O3p7e+HxeGAwGOByuZBMJov+mcxkMujo6IBarUY8Ht9WAJTL5Zient6yXQw3iBDtDAZAonWKLccWHjFWajl2qyPGMpkM+vr6YLfbJQ8g5VZ/f/+eGW8sFqubADg7O4vJyUmMjo4iEAhgYGAATqcTNpsNRqMRra2tkMlkEAQBSqUSer0eVqsV3d3d6O/vh8/nw8jICCYmJpBKpUr+D0dbWxu8Xm/JcSwtLWFpaQl+vx9yuRyDg4NlLwm3tLRsGQC5QYRo5zAAUl0p1dMutxw7Nze3o8uxbrcbnZ2dkgeQcmsvjTcWi0GpVEo+jnKqVADMLcmOj48jHA7D6/XmZ2Hb2tqg0+mgUCjyS7JqtRomkwkdHR3o6enB4OAghoeHEY1GkUgkkE6nqxqnxWKBz+cr+Xph2JuamoJOp4PFYkE6nd4y1MlkMqRSqbJnDAtnA7lBhKh6DIC0L42Pj0MQhLJ2x251xJiYy7GDg4P5Z6b2QjEAilfpdBpTU1MYGxuD2WxGW1sbXC4XOjs7YTKZoFar80uyCoUCOp0ObW1t+ecFvV4vwuEwxsfHMT09vSuPCVgsFvj9/pKvr5/tm5+fh8PhgFKpRDQa3TTQCYKA2dnZbS0bc4MIkXgYAGlf0mg0OPPMM7fcHbvbz9oNDQ2hvb1d8jBSbg0MDOyZwDo+Pi5ZAEylUpiYmEAkEoHf74fb7YbD4UB7ezsMBgNUKhUEQYAgCFCpVFAqldBqtXA4HHC73fD7/YhEIvklWal/lrkym83bCoC5CofDkMvl6O3tzYe19UFOEISyZgq5QYRoZzAA0r5kNptxxhln1Mxmilx5PB5YrVbJx1FuDQ4OwmazST6OcmonAmAmk8H09DTGx8fXbKSw2+2wWCzQarX5XbItLS3QaDQwm83o7OyEy+XC0NAQgsEgxsbGMDU1lV+S3eqItVopk8mEQCBQ8vXNQloqlYLRaITBYNjwrN/i4iIEQUAmk6koABbOBuaeyWUIJNoeBkDalzo6OvCe97yn5gKgz+eDxWKRfBzl1l6asRwfH4dCoSj7/aV623V0dOSXZMvZSBGPxzEzM7OtP2t7JQAajUYMDw+XfH2rkLa4uIi+vj7I5XKEQqH81xcWFiAIQlnX2KpGRkZgNBq5QYRomxgAaV9yOp1obGysuQDo9/thNpslH0e55fF49lwA3Kq3nV6vh1KpLNnbbmBgAIFAAKOjo5icnMTs7KzoY91LATAYDJZ8vdyQlmvR093djfn5eczPz0MQBMzNzYkSAM1mMzeIEG0TAyDtS4ODg3jrW99acwEwEAjAZDJJPo5yy+v11tSSdbHedrmNFAaDIb8Uu1lvu1AohFgsVrK33W7UXgmABoNBlAC4vLyMdDoNq9UKrVab36Q1Pz9fdQAMh8OwWCz5GceFhQVuECEqAwMg7UvDw8P4oz/6o6rbYIhdwWAQRqNR8nGUW16vF21tbbvyvbbb285gMKC9vT2/kaK/vx8tLS01t5GiWHV2dmJgYEDycWxVer0eoVCo5OuVPLc3NDSUD+liBMBgMIi2trY134MbRIi2xgBI+1I0GkVDQwOmp6clv4kWVigUgl6vl3wc5ZYYzywWbqSoprfd2NjYpr3t4vE45HK55D+zcqqjo2PPBMBwOFzy9UpDWzQahSAI+ccLqgmAw8PDsFqtG4ImTxAh2hwDIO1LU1NTaGhoQCwWk/wmWljhcBg6nU7ycZRbWz2zWNjbLhgMYmhoCC6Xa81Git3qbTcxMcEAKHLpdLodCYDpdBqCIKCzsxOtra0YHx+v+FqBQAA2m63kjGPhCSIMgkRvYgCkfSmTyaChoWHTm5cUNTIyAq1WK/k4yqlUKoWBgQHodLqyetsZDIb8Rgq3273jGynWFwOg+KXVajEyMlL0tWo2cMzOzuYbtQcCAcjlcgwMDBTtGbhV+Xw+dHR0bLrszA0iRBsxANK+tLy8jIaGhk2PsZKiRkdHoVarJR1Dbkm2cCNFqd52MpkMLS0tJXvbJZPJmnnOcmJiAi0tLZKPo5zq6OjA4OCg5OPYqrRaLSKRSNHXqgmAqVQKMpks/+tkMgm9Xg+z2bzt00G8Xi86Ozu3fB83iBCtxQBI+9LJkyfxR3/0R0XPW5WyotEoWltbd+z6s7OzJXvbFdtIsVVvu0AgsGc2reylAGiz2fZEANRoNDsSAKenpyGXy9d8bWFhAU6nEwqFApFIpOxreTwedHV1lfVebhAhehMDIO1Lp06dwlvf+lY4HA7Jb6KFFYvFoFKptv25anrb5TZSBAIBRKNRJBKJspdkg8EgDAaD5D+3cmpycpIBUORSq9UYHR0t+lo1ATCZTG4IgLmKRCJQKBTo6enJz9htVkNDQ+ju7i77e3ODCNEqBkDal1ZWVtDY2FhzTYyLnVZR2NuucCNFZ2cnzGYzNBqNZL3t9tKuZQZA8au1tRXRaLToa9UGQKVSuekSsclkgl6v33CM3PoaGBiAw+HY9hhyQZAbRKheMQDSvrSysoL3vOc9ks9ere9t53Q6IZPJ1vS2W7+RorC3nd/vRyQSkay3XTgc3lMBUCaTST6OcmovBcCxsbGir1UTABOJBFQq1abvWVxcRH9/P+RyOYLBIJaWloq+z+12o6enp6JxcDaQ6hkDIO1LKysreO9731vRcms5VWlvO6vVCkEQ1mykmJqaqpmNFOtrZGRkz7StSSQSeyoADg0NST6OrUqlUu1IAJycnERra2tZ7809NmG324t+z/7+frhcrorHkguC3CBC9YYBkPallZUVnHXWWRUFgnQ6jUQike9tV7iRotrednsppGSze6ttzV762ba3t++JAKhUKkv20qwmAE5MTECtVpf9/kwmA6vVCo1Gg8nJyTWv9fb2ore3t6oAWBgCw+EwlpaWOBtI+x4DIO265uZmNDQ0rKnTTjtN9O9z7rnn4tixY2tuWqlUChMTE4hEIlv2tivcSOF0OjEwMFB1b7upqSkIglBzZxSXqkgkAo1GI/k4yqlEIgFBECQfRzm1lwLg+Ph40deqCYDxeBwajWbbAc3r9aKlpQVerze/JOxyudDf3191AMx9D5lMhpmZGRw/fpyzgbSvMQDSrmtubsb555+PiYmJfE1NTVV93UwmA6fTCUEQ8Nhjj+HAgQO44oorcMkll+DYsWP53nYtLS3QaDQbetsVbqTYqSXZ6elpCIJQs0u+66sW+haWWwyA4pdCodiRADg+Pg6tVlvRZycnJ6HRaNDe3o5MJoOenh643W7RAqAgCJidneUGEdr3GABp1zU3N+PCCy8U/bpHjx7Fu9/9blx44YW4+uqrccYZZ+Daa6/F3XffDbfbne9tJ+Xs28zMDARBkGRDRyW1030Lxazc7KrU4yin9koAlMvliMfjRV+rJgDGYjHodLqKPz83Nwe73Q6VSgWr1YqBgQFRAuDCwkL+zxA3iNB+xwBIu665uRl/9md/htNPPx0HDx7El770JYRCoaqvu/4v6I985CP49a9/LflNtLBSqRQEQcDMzIzkYymnxsbGGAB3oNrb2+HxeCQfx1Yll8sxMTFR9LX5+fmKg9bY2Bj0en3Vs3XBYBAymQwWi6WiY+TW1/z8PARBWBNuuUGE9isGQNp1KpUKr7zyCtxuN3Q6Ha644gqcdtppmJ6eFvX7fOITn8CDDz4o+U20sHJnoE5PT0s+lnJqbGxsx3ZSi1176flKq9W6JwJgS0vLjgTAaDQKg8Egyqxde3s7VCoVTCYTUqlUVdfKZrMQBGFDA+pcCFxcXOQJIrRvMACS5Obn53Haaafh/vvvF/W6n/rUp3DPPfdIfhMtrEwmA0EQkEwmJR9LORWLxaBUKiUfRzmVTCYZAEUumUyGycnJoq9VEwBHR0dhNBpFCYCdnZ355ukKhQIjIyMVXyv332ex2cT1S8KcDaS9jgGQasInP/lJ3HjjjaJe8/Of/zzuvPNOyW+i60sQBCQSCcnHUU4VO7mkVisXAPfCBpu9FABL/VmtJgBGIhGYTCZRAmBHRwd8Pl8+WCoUCjidzrKOkVtfuRn6Uk2nC4MgN4jQXscASJJbXl7Ge9/7Xtx+++2iXve6667DbbfdJvlNtNhNtdSsSq1VPB6HXC6XfBzlFAOg+LXZ/6xUEwBHRkZgNptFCYA2mw2BQGBNiDObzdDr9Ugmk9u6ViqVgkwm2/J93CBC+wEDIO2673//+zCbzQiHw7Db7fjMZz6Dv/iLv0AkEhH1+1x//fX44Q9/KPlNdH1t9mB9rdXExMSeCYB7qcWO1WqF1+uVfBybVW45dGpqqujr1QTAcDgMi8UiSgC0Wq0YHh5e87XFxUUMDAxALpcjEAhsOqNXWNPT05DL5WV/79yzgUtLS1wSpj2HAZB23Ze+9CWcfvrp+JM/+ROcccYZ+Jd/+RcMDQ2J/n2+8Y1v4Lvf/a7kN9L1tVlvtVqriYkJtLS0SD6OcmovBcC2traaC4CBWBJ6dxSB2Orzqel0etPnVasJgMFgEG1tbaIEwLa2NgSDwaKvjY+Po7W1FZ2dnWW1rUkmk1Aqldv6/oUbRE6cOMHZQNozGABp3/p//+//4dvf/rbkN9b1tdnxWrVWk5OTe+Z4tVwArOSElt2uXQ2AmQzm4oFN33O0zY8zjyhw4LACZx5R4GibPx8AS+1YrzYAWq1WUQKgxWJBOBwu+Xo2u3r2slqtxsTExKbXSiQSUKlU2x5D4ZLw8jLbxdDewABI+9a///u/42tf+5rkN/v11drairGxMcnHUU7tpdM1GACL12LPS1i54z1Y1t5V9PVALJkPf7k684gCvtHJHQuAgUAANptNlABoNpu33Pm7tLQEn88HuVyOoaGhkkvC2z2juNRsIDeI0F7AAEj71pEjR3D99ddLfrNfX2q1GqOjo5KPo5zaS731cqesMAAWVGoKpx5oApobsaz6SdH36N3RNeEvV9q+kU2bllcTAP1+v2gB0Gg0IhKJlPXeRCIBrVYLq9WKdDq94fVKzijebDaQG0SoljEA0r7V3NyM6667TvKb/frSaDSIRCKSj6Oc2ku99fbSMXsWiwU+n2/Hv8+y9i6guREn7j0bRpcv/3xfYZWaAfRGJvIBcGhkHC12H+z9Png8HrhcLnR3d2NqaqqikOTz+dDZ2SlKADQYDIhGo2W/f35+Ht3d3VCpVBgbG1vzWrVH1G02G0hUaxgAad+6++67cc0110h+s19fOp0OIyMjko+jnNpLrVUYANfW3LgPK3eeBjQ34js//tGa5/ty78lkMpiZmcFvtO41zwD++HdG/PoVPY6+JODIky04eFiOA4cVOHhYgTv/0wCHwwGHwwG5XF5yA8Zm5fV6YbfbRQlaer1+Q5Arp0KhEORyOfr6+vLP7olxRN36ENjW1oZwOMwNIlRzGABp37rvvvtw9dVXS36zX196vR6hUEjycZRTe+m5utw5ywyAq3Xiha8CzY1w3PZBHHgjwOUC3mtqE7RaLVpaWiAIAhQKBV6Sa/HEa0bc+jsTDr7x3oOH5fl/Lvx8IJbEwsJC/qSY7TZe9ng86OrqEiVkabVaxGKxij47PT0Ng8EAo9GImZkZUU8oyZXRaMTIyAg3iFDNYQCkfeuRRx7BP/3TP0l+s19fBoMBwWBQ8nGUU3tpVq3eAuDs7CwmJiYQiUTg8/nQ39+Prq4utLW1ofu/HgGaG7HS/HZcfeSRDc/3vWRyYWRkBBMTE2t+XsWWg4uVYSCaD3yzs7MwmUwwGo1ln8U7NDSE7u5uUQKWRqPB+Ph4xZ9fXFxEb28vFAoFent7RTuhJFe5JerCJWGeJ0y1gAGQ9q0nn3wSH/nIRyS/2a8vk8mE4eFhycdRTuVCVamNALVUe2msFosFfr+/5Ouzs7OYnJzE6Ogo/H4/3G43HA4HrFYr9Ho9FAoFBEFAS0sLNBoNzGYz7HY7ent74fUMYfmhy4DmRqT+8+tFn+8r9ixgNlt6Q0ipGcDCEOVyuaBUKstajh0cHITD4RAlYJXT3qWcikajkMvlUKlUVW1wWV86nS4/Q8kNIlRLGABp33ruuedw6aWXSn6zX19ms3nTm38tVe5s1L0QqvZKAEyn0zAYDOjt7UUgEMDAwAAcDgfa29thMBigVCohCAJkMhnUajVMJhM6Ozvhcrng8XgQCoUwPj6O6enpoptzlmxPrM7+3f1ezE2Gi/b4KzW2YjOAB9+o9Z8vtuQbDochl8sxMDCw6ekbAwMDcDqdogSs1tZWTE5OinItr9cLhUIBnU5X8QaX9aXRaBCPx9d8jRtEqBYwANK+9dJLL+EDH/iApDf7Yjfo3doBKkblAmCpXnC1VLUQVtPpNKampjA2Nobh4WEMDg7C6XTCZrPBYDBApVJBEIT8c3cmkwkdHR1wuVwYGhpCMBhELBZDMpmsbOd1MoZT95wFNDdiyXhf/uuBWBKGgWjJmb/CKgyMBw/LcbTNX/TzxQJgJJFGa08QL7Zo0N7ejmw2WzQUud1u9PT0iBKwlEolEomEKNcaHh6G1WrF4OAg5HI5/H5/2cfIbRZQi81QFs4GcoMISYEBkPatV199FYcOHZIsDMz09mL4c59H0mRe8/W9cA5sYaDZ7DiwWqqdDoCZTAbJZBKxWAzBYBBDQ0Po6elBR0cHjEYjWltbIZPJIAgCVCoVjEYjbDYbenp6MDg4iGAwiLGxMUxNTcFkMu3ILHDqle8BzY04/sBFyM5WHtoDsSSUjgCOHit9Csz6APg7W2jNTOPdL5igVquLhrP+/n64XC5RQptCoUAymRTlWn6/Hx0dHVheXu0J2Nraio6ODmSzxYNstQGVJ4iQlBgAad9SKpV43/veJ1kgGbnx2/Cccy4855yLkZtuRuqN5/7a29sxNDQkeWAqN/QIgoCpqSnJx7JVVTNbmclkMD09jfHxcYRCoXyvu87OTphMq0EmF+6USiUMBgPa29vhdDoxMDCAQCCAaDSKRCJRVsucnXgM4NVWDU789C+B5kbc8KO7N13qXV/rzwHOZlePAcydA13s9cIAGEmkiz5r2OYcgFwuRyAQWDOT1tfXh97eXlFCm1wux/T0tCjXWt+fMJvNoqOjA2q1esMyrpgBlRtESAoMgLRvGQwG/PVf/7VkTYzTY2MYvfUn8Jx3CJ5zzoX3og8g9sgjsJnNGBwclDwwlVuCICCRSEg+ji1/3iXOrs31uhsfH0c4HIbX60Vvby/sdjvMZjM0Gs2adih6vR5WqxUOhwNutxuBQACjo6OYnJwUrR2O2WxGILD5+bzbqcDYFEy3/SPQ3AjdT67YcrNHYZV6RnBiYgItLS0lXy8MgGZvvOiGEYs3jng8DpVKhe7u7vxnent70dfXJ0pok8lkmJmZEeVaxdrTLC0twe/3Qy6XY3BwcNtLwi0tLWWNjxtEaLcxANK+1d7ejtNOO03yUyymHU4Er/tSfjZw4IorMPD885IHpnJLJpPVdABMpVKIx+MIhUIQBAE9PT3o6uqCxWLZ0OtOp9PBarWiu7sb/f398Pv9iEQiooa7cspkMokaAHs1z68u/f70L3HFkafWtGspNnuXza7O6h2zB0v2+YvH43j+FXnJXcQLCwvoDiZwj3IQv9J4ir4vklg9bi2dTsNisUCv12NmZgYulwv9/f2ihLZc6x8xrrVZe5qpqSnodDpYLJaix8iVCnW5PprljoEbRGi3MADSvtXd3Y13vOMdkgfAbHZ1FmrixZfg+/Dl+SAY/ta3kNoDzwLKZDJMTk5K8r0363Wn0+kgl8shCALkcjm0Wi0EQUBHRwf6+vrg8/mK9rqrhRI1AKaSOH7/BUBzIx6/9V/XBLCHNENFZ+8KZ/VK9fmLx+N44PelX//OHxwbvn6w4Hv9zhZaE2wWFxfR19cHhUIBq9UKt9tddWDLBaxyA9lWtdXu5Pn5eTgcDiiVyrKOn1tcXMwfpbjd3xc3iNBOYwCkfcvtduNtb3tbTQTAXKXjcfR997sYOnT+ahC84P0Y++V9SNfwM3YtLS2YmJgQ/bpV9brzehEOhxGPxzEzM4NMJrOnNqyIGQCX9b8AmhuxcPeZaDryckH4Gyw6K2fzxTYNf2ceUeCYPQhdTwC3P6vYMEN44LACR17uKfn5Z9uG8aojkrU/mAYAACAASURBVJ/9W1+RSAQymQxmszm/+aHSqjRglar+/v6ydifn2t309vZu+nuYn5+HIAir/54q/P3lls25QYTExgBI+5bf78d//+//vaYCYDabhcPhQJ9CgdD1X83PBvo/9nEkZLKaG2s2m4VcLkc8Ht/WZ9LpNBKJBKLRaFm97sxm87Z63RWr3IaVegqAc/FhrNx1OtDciMXO365p11KqqfOjOk/J8FYs7JX7tfWvFZsFzFVHRwdUKhXa2tqqCm/VBqz1tZ3NKTMzMzAajflj5Iq9J5tdfYa2msbS3CBCO4UBkPatkZERNDQ01Nw5tk6nEy6XC5lMBpOCAP8VH8sHwdBXb8CM2y35GAtLoVBgfHw8/+tye93JZDK0trZu6HUXCoWq63VXovZSADQajaKcBnPi2NeB5kacfPyjyGbW7j6+q6W/6OyezRcrGtyue9y65Skg263C5wALq6urCwMDA+jq6irZJ6+cmpubqzpgFZbL5drW5pTcsrZcLkcotDHs5v5MVjvTuX6DCJEYGABp35qYmEBDQ8OOLF9WUz09PXA6nW8GqqkpjN37S3gueP9qEDx0PqJ33YW0BOMu1uuupaUFFotlW73uxA535Y59r7SsESMAzvstQHMj0NyIef/aXpMPaQaLBrJvPNuJ56x+0YPeZmXxvtk+JZJIw+yNQ2mywev1rtlh6/P5tr3DNptdnWEr1pS6kurp6ano2cSxsTEolUp0d3evCaO51kTVNpMuDJzLy8tS/9VK+wQDIO1bqVQKDQ0NiEQikt/wC8vlcsHhcGz4esrjRfib38rPBvou/wgmXzomWpCqtNddS0sLnE7ntnvdSVF1EwAzaZz8zRVAcyNOHPv6mtcCseSmS7S7WYUzgIWNog8eVuDOlzth9sYRSaQxMTGB1tZW2O32bc3m5Z77rHaGLVcOhwMDAwMVfTadTqOtrQ1arTbf+HlmZgYtLS2ijI0BkMTGAEj71sLCAhoaGkRttyFG9fX1oaurq+TridZWBD55ZT4IBr/8ZUwXzBgWq+30ulMqlSV73RULdyqVCmNjY5L/3MqpvdKzsNoAuNj5zOp5v3edjrn42uuUevZvd6oFB350LP/rx/S+/MxfqY0nuWcFs9nVU3J0Ol3ZjZ3FnmHr7u7G0NBQxZ9fWlrKz5r7fD5MTU1BoVAwAFJNYgCkfevkyZNoaGjAwMCA5Df8wurv74fdbt/0PZnpacQeehjeCy9aDYLnHULw8GGE3W54vd58iNyNXnetra2IRqOS/9zKqb0UAIPBYGWfn47j1L1/CzQ3Yln/8w2vSzYD+KNXcO4j1+K8x/8ZB468hkffCH/Ly6UbRa+fKVxaWoLb7YZcLsfIyMiWgSiVSkEmk4kWsOx2O7xeb9XXmZiYyG9uEjsA8hlAEgsDIO1bp06dwlve8ha4XC7Jb/iF5Xa70dnZmf/1Zr3ujMeOoetLBU2kL7kEXXfdja7Ozl3rdadWqzE6Oir5z62cqvWm1bkyGAwVB8BlxZHVps/3XwBDb7DoaR+lngHcsaXe257BoSc+hqajTTj/mYvwvtsfze8AjiTSeNUR2TKUFj4rGI1GoVAotmyzMj09LeoSa2dnJ3w+nyjXmpubQ1tbGwRBwPj4OAMg1RwGQNq3Tp06hT//8z9fE7Z2u4r1ujMajVCpVEV73VkslqK97ia0WgQ+dXU+CA5/4V8wvUu/LwZA8avSADg32o+VO94NNDfi6z++PT97Vuzc34c0Q7sS/s6685c4/7eXoOloEw49dTnObP5Nflw/+a++fHPog4c3bx/zqN6X3yQSSaSRSqVgMBhgNptLNnpOJpOizrDZbDYEAgHRrjc+Pg6lUgm5XI6BgQFR+h4yAJJYGABp31pZWcE73/lOmM3mqm/YxarSXncWiwU6nW7bve4yqRTGn3gS3osvyQfByC23YHaHN7loNJqa20hTqqQ8tWQ7VWkAfP25LwLNjTDf9hEcOCxfs4S6fibwIc1QWUvBl92tqzD8yfF39/47zn/2AjQdbcK5v/4MDv74hS2Xel9zRnDTk9oNrx184/Xc+35nC2FhYQEOhwMqlaroLNrU1BSUSqVogc1qtWJ4eFi068ViMeh0OiSTyfwxcts5Fm59LSwsMACSaBgASTKPPfYYDh48iD/90z/FxRdfjLa2NlGvv7KygtNPPx0ajWbbN1qxet0VC3cejwdWq7Xi8DA7MoLID27Jh0DvJR/E+JNPIbNDS8BarRYjIyOSh6Zyaj8HwAV3C9DciFM/eyc+ceSJDQGqsAH0XbK+nZ35O/JfOOfhr6DpaBOajjbh7Ae/gQOHhbI+a/HG8firhq2XlQt2EAeDQcjlcng8njUbPiYnJ6FSqUQLbG1tbUX7+VVa0WgUBoMhH96cTieUSiVGR0crDoAnTpwQ9e9Jql8MgCSJF198EX/yJ3+Cp556Ch6PB9/5znfwtre9DaOjo6J9j5WVFRw4cAByuXzNjbRYr7uenh50dHTsSq87n88Hi8VSdYhI2mwYvuaaN5eFP/0ZJE3iz3bqdDqEw2HJQ1M51dLSsmcCYCgUKv8zszM4+dDFQHMjUi9/d8OO2oOHFThyrPTxbGLWwZ88h/N+88nV5/2evRB/+/Oflv3ZXKh7tdW46XF0hWExF34SiQQ0Gg06OjowNzeH5eU3N1uIFdjMZjPC4bBo14tEIjCZTGu+NjIyAoVCAZfLte3+hQyAJCYGQJLEpZdeihtvvHHN184991wcOXKkquuurKwgkUjA6XTitddew8GDB3HNNdfgyiuvhFqtXtPrTqVS5XvdOZ1ODAwMYHh4eMd73fn9ftGWpTOzs4gffQ6+Sy/LB8GRm25GSoRTJnKl1+u3F1YkrJ06t1js2u7PdMl0/+rs3y/ORDAUQvOrvZLs9D3r9gdx/tOXrj7v9/Q/4KyfPbqtzz/2xnN+v35Fj5+/5ljTF3D976fYKSLZbBY2mw0ajQZTU1OIx+PQaDSiBTaTyYRIJCLa9cLhMMxm84avp1IpmEwmGAyGslveMACS2BgAadcdP34cf/zHf4xXX311zddvvvlmfPSjH63q2k8++SQaGhrwrne9CxdeeCHe9a534aqrrsLNN9+M3t7ekr3udrMCgQBMJpOo10yPjWH01p/Ac96h1WXhiz6A2COPIDM9XfW1tz1bJWHtxwA4NxnGyt3vBZobYXvhnk1D086VHH97z+H8837nPf7POHjrf27rGo/qfWuaQZ95ZPVrFm8c3cEEbpe5yzpHONdrTy6Xo7+/H1qtVrTAZjAYEI1GRbteKBRCW1tb0dcWFxfR398PuVyOYDBYVi/DxcVFvP7661X9HUmUwwBIu258fBwNDQ2w2Wxrvn733Xfj7LPPruraCwsLWFhYyP/6kksuwdGjRyW/4RdWMBiE0WjckWtPO5wIXvdm25jAlVdhSr39ZyALq5qWJbtdcrl83wXA46/cCDQ3YumRD+F9R1p2fdbvwJHXcM6vrs8/73fOr76KA0de29Y1uoOJos2gzzyyOitYGArvkLk3zPwV7g7OfS0Wi0Eul0OhUIh2FJxer8fY2JhoAXB4eBjt7e2bvicWi0GlUsFut+eXthkAaTcwANKuywXAjo6ONV+/6667cM4554j6vT70oQ/hiSeekPyGX1ihUAh6vX7Hrp/JZDDx4kvwffjyfBAMf+tbSHm9FV1PjHNrd6v2WwCcH7ZhpfntQHMjHMbyNlmIWQdv/U+c9/g/v/G83/vxt784gsLdx+XWa85IyWbQB4uEwsKgt37WsHBmMBgMoqWlBUajEalUqurAptVqRevZt7y8DL/fj46Oji3fl8lkYLVaodFoMDk5yQBIu4IBkHbdTi4Br/exj30MDz/8sOQ3/MIKh8PQ6XQ7/n3S8Tiid9wJz6HzV4PgBe/H2C/vQ3qbZ+WaTKaaO06vVMnlcsTjccnHsVXp9fqtN9ZkMnj9iX9aPe/3hesRiCXL2jghVp31s0dx6Ol/WA1/T1+Ks27/VcXXes0ZKToDWGoJO7f5o9SsYS4gjo6OwmAwwOVyQalUVj17p9FoEI/Hq7pGYXm9Xtjt9rLeu7S0BI/Hg5aWFni93qJLwgsLCwyAJBoGQJLEpZdeim9/+9trvnbeeedVvQlkvauuugr33Xef5Df8wopEItBqtbv2/Wb6+xG6/qv52UD/xz6OhExW9g5ms9kMv39jo+FarL0SAMvZWb3Y9fzqeb93noa5cR+y2SyOtvk3fQbw4BEFfvZab9Xh729/fhvOf/bC1ef9fvNJHPzJ81VdLxfYij0DuFnAKzVrmAuIIyMj+V224XA433C50rOBW1tbN52B224NDQ2hq6trW5+ZnJyERqNBe3s7MpkMAyDtGAZAkkSuDcxvf/tbeDwefPe738Xb3vY2RCIRUb/PZz/7Wdx9992S3/ALa3R0FGq1ele/ZyaTwaQgwH/Fx/JBMPTVGzDjdm/52b0UABUKxf4IgDOTOPXLs1fP+9Xeuea1QCwJw0AUx+zBouHIMBDFN57trDCsCTj7wa+/+bzfw1/BgSOvVhX+Cs8Ezs3qPfSCCn3+SNFQWLjEu9UMYDgchsViyb8/mUxCq9Xm+2xuN7CpVCokEgnRAmCuOfx2Pzc3Nwe73Q6VSoVYLJb/OpeASUwMgCSZxx57DAcOHMBb3vIWXHzxxbBYLKJ/j2uvvRbNzc2S3/ALKxqNorW1VZLvnZ6awti9v4TngvevBsFD5yN6111Ib/LcnMVigc/nk/znVk7tlwC43HrbatuX+89Hdqb40XbFloTPPKKAzReraKn44I9fwLm//swbz/tdgL+79/uo5Hm/wtnIx9aFv8KZtomJiTVBz7Juk0euNguIxXbZzs3NobOzE2q1etthTqFQIJlMihYA3W43enp6Kvrs0tJSvgF2f38/FhcXsbi4iJMnT4r+9yTVJwZA2tf+9V//FT/60Y8kv+EXVm7Xn5RjSHm8CH/zW/nZQN/lH8HkS8eKLgu3tbXBW+EGkt0uhUKB8fFxycexVel0upKnq8yNDWLljvcAzY1Y7Hlx0+sULgnnzgTWu6PbDmtnNv8Gh5768Gr4++0lOOvO+6peRn7NWbqfXrGZtmI7fbcKiMPDw7BarUXDk9frhVwuRyAQKHtJWC6Xb6sv31bV19eH3t7eqq4xPT0NvV4Pk8mEmZkZBkASDQMg7Wtf+9rX8P3vf1/yG35hjY+PQ6FQSD6ObDaLRGsrAp+8Mh8Eg1/+MqadzjXvsVqt8Hg8ko+1nFIqlXs+AJ74/ZeA5ka8/ttPIVvGc5q5JeHcWcA2X2xb/QHfd/cdOP+Zi1af93vi4zh427NVh78Dh0vP/kUSaTzwnwoMhN5c2txslm+zCgQCsNlsJV+Px+NQqVTo7u4uq1WMTCYTZTdxrlwuF/r7+6u+zsLCAnp6emA2mxkASTQMgLSv3Xjjjfi3f/s3yW/4hRWPxzccTydlZaanEXvoYXgvvGg1CJ53CKM/uQ3psTFks1m0t7djaGhI8nGWU3slAJY6X3lhsHV148fP/hLz4e5tX7dwRnDrkuHs+2/MP+937iPX4sCPXhEl/OWC3DNtw3jVESm5CeR3ttCWz/ltVuW0WUmn07BYLNDr9ZiZmSn5vqWlJQiCgHR66+9bbuVOGBLjWktLS8hkMgyAJBoGQNrXbr75Znzzm9+U/IZfWBMTE2hpaZF8HOsrFRjGyL/d9Oay8GX/gPhzz8Nmte6pABiLxSQfx1ZVNACmUzj5yN8DzY04/trN275msWcCS84E/vglnPvoNfnw93f33YQDh3euyfTBw2sbPhcGvVcdkaKfKTwHuFR5vV50dnZu+b7FxUX09fVBoVBgdHS05HsEQdiw87aa6u7uxuDgoGgBcGFhgQGQRMMASPvaLbfcghtuuEHyG35hJRIJyGQyycdRqqaMRgQ+dXU+CLr/xz9j8NVXJR9XObWXA+CS5aHVjR8/P4BsYnTb1yz32b8zf/o0Dj350dXn/Z65GO+76+c7FvzWhMASM5OvOSMVzwB6PJ6y++wtLy8jEolAoVDkN1UUvjY/Pw9BELY8jWM7Zbfb4fF4RAuAi4uLOHXqlNR/rdI+wQBI+9ptt92Gr3zlK5Lf8AtramoqP9Mg9VhKVSaVwvgTT8J78SX5IBj5wS2YjUQkH9tmlWubIfU4tiqtVotI4c8yMYqVn//16pFvbY9WdM1yGkW/766f4/xnLkbT0SYcevKjOPOnT+9K+Cs1I5kLepU+Azg0NITu7u5tBancpoq2trY1s31zc3MQBAHz8/OiBcDOzk74fD4GQKpJDIC0r91555344he/KPkNv7Cmp6fzzxpJPZatanZkBH3f+EY+BHov+SDGn3wKmVRK8rEVq70aAI+/ehPQ3IiTj1yKbLr0z3Z2dhaTk5OIRCLw+Xzo6+uD3W6HxWKBRqPB/3mo1DJuC/7uvpvefN7vsWtw4Mcv7Wr4W9/4uVjPv1KtYEpVpX325ufn0dXVtaYdTSaTgSAIG2YGqymbzYZAIMAASDWJAZD2tXvvvRef+cxnJL/hF9bMzAwEQcDs7KzkYymn7HY73C+/jOFrrskHwcDVn8aUyST52NaXSqXC2BubV2q5NBpNPgDOh+xY+dk7gOZGpPtW29iEQiEMDQ2hp6cHNpsNBoMBKpUKgiBAJpNBo9HAYrGgq6sLfX198Pl8iEQi6B+Obnzu70ev4NxHrs2Hv7PvvxEHDst2NfwdOPxmQ+iRyVn88jkZ/NHqGy673W44nc6KA5Xf74dcLofP58Ps7CwEQaj4FJFiZbVaEQwGGQCpJjEA0r728MMP46qrrpL8hl9YqVQKgiBgZmZG8rGUU3a7Hf39/cjMziJ+9Dn4Lr0sHwRHbroZqeFhyceYq9bW1poNgOl0GolEAqOjo1AqlbDb7eju6sLsg5cDzY0Yf/CTEAQBSqUSer0e7e3tcDqdGBwcRDAYRCwWQzKZ3PLRgTXHxd32LM574uNvPO93Ed73H7fv+LJuqcpt6hBzs0V/fz9cLldV15icnIRarUZ7ezsEQRAt/C0vL8NisSAcDjMAUk1iAKR97fHHH8cVV1wh+c2/sHIzDdPT05KPpZzKzTLlg8zYGEZv/Qk85x1aXRa+6AOIPfwIMjXw+5EyAKZSKcTjcYTDYXg8HrhcLnR0dMBoNKK1tTU/e6dWq9HS0rIaDhQPrm78uOM9SAScos0K23wx3CL/PS46+verz/s9dTnObP7Njizr3i5zl/Xe7uDqjN/CwgIEQUA2u/2j2tZXb29v1Y2Wl5eXkc1mYTKZ8v9dihUATSYTRkZGRA2AKysrUv+1SvsEAyDta8888ww+9KEPSR5MCiv3rFEymZR8LOVUd3c3ent7N3x92uFE8LovvbksfOWVmFJrJB1ra2srotHojvw7m5qaQjQaRSAQgNvthsPhgNVqhU6ng1wuhyAIkMvl0Ol0sFqtcDgcGBgYQCAQwNjYGKampvKzdxqNBtGQH6fuP7R63m/rT0Ub67MWH86+9/s4/9kLVp/3+/WncfDHL2wazr76lG3b4e8e1SAiiXTRPn6bzQCKudvW5XKhr69PlIA1NTUFmUwGuVwuWmgzGo0l284wAJLUGABpX/vDH/6Aiy++WPIQtb4EQUAiUfyM11orh8MBl8tVMhhNvPgSfB++PB8Ew9/8FlIeaY6OqzQAzs7OYmJiAiMjI/B6vejt7UVnZyfMZjPUajVkMhkEQUBraytMJhM6Ojrgcrng8XgQDocRj8eR2sbGGLVajVnhyOrs333nljzvd7vlHo3i3Ie/8ubzfg9+HQePCFuGs7ta+rd1esiBwwr88NibS6/f+UPPljOFuc0dYu627enpEeWkjeXlZSQSCahUKkSjUSgUCvT29la9IUSv12NsbIwBkGoSAyDta6+88gqampokD1HrSyaTYXJyUvJxlFNOpxM9PT2bvicdjyN6x53wHDp/NQhe8H6M/fI+pKemdnWsarV6QwDMZDJIJpOIxWIIBoMYHByE0+lEe3s79Ho9FAoFBEFAS0sLtFot2tra0N3dDbfbDb/fj9HRUSQSCVF3bZtlv8epO/6/1fN+u38vyjV9Ez5c9dJnV5/3e/ZC/O3Pb8sHsJftIbxsD2075G32jN/Bw4qSM4AHj7z5uYOHFfjO4/L8TFg2u/o/QOUczbZViXnSxuTkJFpbW7G8vIxUKgWDwQCz2VzVySBarRbj4+OijG9xcZEBkETFAEj7mlwux9lnny15iFpfcrkcExMTko+jnOrp6dkyAOZqpr8foeu/mp8N9F/xMSQE2Y72PCxsjaJQKGC329e0RmlpaYEgCFCpVDAajbDZbOjp6cHQ0BBCoRDGx8cxMzOzq30Z4w//8+p5v09dWdZ5v1uVMWjE5X+4fDX8Pf0POOtnj66ZfcudE7y9o+LKW9Y1e+NFXzt8zJVv/nzmEQV+9JQcQ0ND+WdgxWi3IuZJG/F4HBqNJv/rhYUFOBwOqFSqikOcWq3Ot5lhAKRawwBI+5pOp8PBgwdrrumyQqHYE2fWZrOrAdDpdJb9/kwmg0lBgP+Kj+WDYOirN2DG7d72985kMpieni67NUpLSwva2trWtEaZnJysqZY7Cx7tG+f9vgPzwc6qrpXJZPCU6ylc+NyFaDrahGuFa/Gw0bam197RNv+azwRiSTyq84iyAaTUDOCZR4o3fX5RpobNZhOt3UpXVxeGhoZECVjj4+PQarUbvh4MBiGXy+HxeLY95tbWVkxOTooWAJeWlhgASTQMgLSvWSwWnH766TUXAPfKkWXZbBYulwsOh2Pbn0tPTWHs3l/Cc8H7V4PgofMRvfMupAtmPgtbo/j9fvT396O7uxttbW3QarX52btyW6Oo1WqMjm7/GLVdq/QsTj76D0BzI+Ze+HpV10rOJvFD4w/zz/v9wPgDJGdXZ/oCsSQMA9H8zN/6KufUkNzybbEl4IOH1zZxXn+SR6mdwYaBaH637ezsbNWhyG63w+v1ihKwxsbGoNfri76WSCSg0WjQ0dGxrc0rSqUSU1NTogZAIrEwANK+Zrfb8a53vavmAmAt96tbX729veju7q7os6lUCmNOJ3w3/O/8bODgZZeh4xe/QGvB7J1arYbZbIbdbkdvby+8Xi9GRkYwMTGxrdm7wgbLtVhL7b8Gmhtx4o7TEfP3VXydUCKE/yn7n2g62oT3H30/nux5ctt/xguXg888osBDmkE0v9abD3xnHlHg5mdMRYPcy/bQhtmwwpM8Ss0KRhJpJJPJfKiPx+NVhSIxj1qLRqMwGAwlX89ms7DZbNBoNGWHOrlcLlpbGQZAEhsDIO1rvb29+PM///OaC4A1P1NVUH19fejq6trw9e22RrE/9msMfvSKN58P/OK1iNtsov67qekAOBXFqV8cBJob4Xnmporb1dhGbPjoCx9F09EmfPgPH4YhaKh4TMVmCnNfU1vs0LY7iga5QCy55XJoqfN9p6enIZfLMTw8DLlcXtVRaTabDX6/X5SAFYlEYDKZNn3P0tIShoaGIJfLyzrhQyaTIZVKMQBSTWIApH3N4/HgLW95S80FwJoOKm9UrjVKZ2cnTCaTKK1RMtPTiD30MLwXXrQaBM87hNGf3Ia0SLOh68/YraU6Lvve6nm/D38QaqW8ogD4u/7f4aLnLkLT0SZ8/tXPwzuxc+12cv0f188U5p4pLOd5uGLn+yaTSSgUCiwvr268UKlU6OnpqWhTSHt7O4aHh0UJWOFwGBaLpaz3xmIxKJVKOJ3OkruZl5aW8md+ixUAl5eXpf4rlfYRBkDa10KhEBoaGkRt4SFG6XQ6jIyMSPb9t9MaRalUQqVSidoaJRUYxsi/3ZSfDfRd9g+IP/c8MlVu1qjVADg/4sTKz/4SaG7Eglux7UcAZtIzuM1yW/55v5t0N2EytbNthApPgCk2U1jpJo6pqSkolcr8r2dnZ2E0GmGxWJDNbu90EDHP2g2FQmhrayv7/bOzszCZTDAajUVn+cQ88YQBkHYCAyDta+Pj42hoaKi5Uzf0ej1CodCOXb+wNYrP50NfX1/FrVHcbjc6O6vbrVqqpoxGBD51dT4IDl/zBSQ7Oiq+nlarlTRYF61MBq8/85nVZ/+e/5/IZrNQqVRlB8DIVAT/Kv9XNB1twgVHL8DD3Q/vyoz2+iMA11elATDXcHl9WLLb7VCr1dvaNGGxWBAKhSoax/oaHh5Ge3v7tkOZy+WCUqnc0PBZzBNPct/r+PHjUv+VSvsIAyDta9PT02hoaKi5DRcGgwHBYLCiz263NYrFYsnfzCtpjTIwMICOKkLZlr+fVArjTzwJ78WX5INg5Ae3YLaCmbxaDICLrpdX277c8W7MRd3IZssPgN2j3fjES59A09EmXPr7S6Hyq3Zt3Ha7Hf39/SVfrzQAFjZcLqzC5+sikUhZ1zKbzaId2xYIBGCz2Sr6bDgchlwux8DAQP7nks2K1/CaAZB2AgMg7WvZbBYNDQ07OttWSZlMJgwPDxd9TezWKNXW4OAgbDbbjv9MZkdGEPnBLfkQ6L3kgxh/8ilktnHEmtRL6xsqNYVTD1ywet6v8sf5r5cTAF8efBkXP38xmo424epXroZ73L2rY+/s7IR7k96NlQbAiYkJqNXqkq/njmIrDFOlymQylR0Wtyqfz4fOzs6KP59MJqHVamG1WvP/HYvV75ABkHYCAyDtaydOnEBDQwM8Ho/0YeCNyh0z5XQ64fF44HK50NHRAaPRiNbW1rJao2zn3Nlqa2hoaFcCYK6SNhuGr7kmHwQDV38aUyZTWZ+ttQC4rPuP1fN+f/l3yE6/2f9wswCYSqdwt+3u/PN+32z9Jsand79p+GYBMJPJVBxk1p+4Uaymp6eh1WrR0dGx6ZnBRqMxf8RcteX1emG326u6xvz8PDo7O6FWqxGJRCCTyUQZGwMg7QQGQNrXTp06hT/+4z/e9FkmMavc1ii53bNWqxUOhwMDj4+hvgAAIABJREFUAwMIBAIYGxvD1NRUTe1aHhoaQnt7+65+z8zsLOJHn4Pv0svyQXDkppuRKjFrmiudTodwOCz5zyybzWJu3I+Vu/5q9bxf+7NrXlOpVEUbgcemY/g/qv+TD3/3dt6L2Yw0p5h0dHRgYGCg5OuVBplSJ26sr2w2C6vVCr1eX7KVil6v3/DsXaU1NDSE7u7uqq+ztLQEr9eLlpYWyGQy0WYAFxYWGABJVAyAtK+dOnUKf/Znf1a0j10llWuNMjIyAq/XW3FrFKvVCq9351p4iFkejye/rLXblR4bw+itP4HnvEOry8IXfQCxhx9BZnq66Pv1en3NBMATL/7v1bYvv/n4hvN+i50E0zvWi6tevgpNR5vwwd99EIJHkHT8NpsNg4ODJV+vNMjEYjHodLqyZ716e3vzP6/1r+t0uqJfr6QGBgbgcDhEudby8uqmEkEQ0N3dLcpzgAsLCzhx4oTUf6XSPsIASDXlwIEDaGhoWFOHDx+u+HorKyt4xzveUVaA2U5rFK1Wi7a2topbo7S3t2NoaEjykFJOeb1etLW1STqGaYcTweu+9Oay8JVXYkqt2fC+WgmA8z4T0NwINDdiPrDxz976ANjibcHf/+7v0XS0CVceuxI90R7Jfw82m23TP6OVBpnNjlwrVaFQCHK5HD6fb82Mmlarxfj4uCiBze12o6enR7QAmEgkoFQqYbFYoNfrMTMzwwBINYUBkGrKgQMHcMcdd2BiYiJfc3NzFV9vZWUFp512GvR6PSYnJ0VrjSLGzXWz2ZVaKp/Pl+/RJmVlMhlMvPgSfB++PB8Ew9/8FlKeN2dSd7q9TlmVSePkr/9xte3LsW8iEEtC717bQy8XANOZNO63359f8r1BeQOiycpOCBG7tvqflEqDzFZHrpWq3O5hh8OR74mnVqsxMTEhSmDr6+uDy+USLQDmNrssLi6ir68PCoWiqucVGQBJbAyAVFMOHDiABx98sKpreDwe/OpXv8L3vvc9fPGLX0RjYyPe/va347/9t/+GV155RZTWKNXWVs9X1VL5/f6aCIC5SsfjiN5xJzyHzl8Nghe8H2O/vA/pqamaCICLHU+ttn25+wy8qLMVPUVDqVTCH/HjRvWN+fB3h/UOpNK7t7lnq7JarZtunqo0yIyOjsJoNFb02XQ6DZPJBJPJhHQ6jdbWVkxOTooS2Hp7e9HX1ydaAFz/rGMkEoFCoUB/f39Fp54sLi7i9ddfF+lvWiIGQKoxBw4cwF/91V/hne98Jy688ELcdddd237wWSaT4fOf/zxuuukm3Hffffibv/kb3HPPPRgcHKyZE0G26rFWS+X3+2E2myUfx/qa6e9H6Pqvvnm28BUfg+2++yvur1hOFZvNW1PJcZy6531AcyOmVP9R8hzdo7KjuPrlq9F0tAkXP38xXhx4UfKf5/ra6jnVSoNROWfubjUT1t3djdbWVigUCiQSCVECW09PD/r7+0ULgMWWuqenp6HX62GxWLa9k5oBkMTGAEg15YEHHoDZbEZ/fz+eeuopvPvd78bXv/71qq556NAhvPhibd1gtzploZYqEAjAVGYblt2uTCaDSUGA/4qPvdk/8Mtfwcwm/euK/h63CnbZbMkzcQvruPyW1bYvv7oIhr7QmvCXq0fa/wsffO6DaDrahI+/+HHYI3bJf47Fqq2tDT6fr+hr1ZxuEQ6HYTabqwpXuZ22giBgaGhIlMDmdDoxMDAgWgAsNdM5Pz+Prq4utLa2bmv5emFhgQGQRMUASDuuubl5w8aO9eVwOIp+9pVXXkFDQwOmp6cr/v4XXXQRnn/+eclvqIXlcDjQ29sr+TjKqeHhYRiNRsnHsVmlp6Ywdu8vMXR+02oQPHQ+onfehfTExJafLSfYBWLJkrN5uffMRfqwcvu7Vs/77XutyGfkOPveH+CCoxeg6WgTrnvtOoxM1U7PwvVlsVh2JACGQiFYLBZRQlZLSwvkcjn6+vqqbrfS3d0tWphcXl7GyMhIyZnOpaUl+P3+ohtbGABptzAA0o5LJpPwer2b1tLSUtHPxmIxNDQ0wG63V/z9L7vsMjz99NOS31ALy+l0wuVyST6OcioYDMJgMEg+jrJCy0vH4P3qDfnZQN/lH4Hv6H9C3z8Kmy+2YZavnGCXzWZxzB4sOptnGHhzw8brR68Bmhvx+tFr8l/Lh8sjr+Lch/9X/nm/r73wNYyM1W74y2azMJvN8Ps3huFstroAGAwGYbVaRQuAY2Nj0Ol0aG9vr2pcXV1d8Hg8ogXAcDi8ZdCdnJyEWq2G3W7ftOE1AyDtBAZAqmlyuRwNDQ0YHR2t+Bof/ehH8dhjj0l+Qy2snp4eOJ1OycdRToVCoT0TAI1GI4LBIBKtrQh88sp8EDz24U/h4//3iQ2zfHp3dMtgd7TNj4NF3nPgsAIPaVZ3yS70vba68eP2d2EusnZpv314AP/jpc+h6WgTLnzuQjzX9xwUCgXi8bjkP6/Nymw2IxAIFH2tmqA1PDyM9vZ2UUKWIAhIpVKYm5uDzWaDVqvF9PR0Rdfq7OyEz+cTLQCWG3Sz2Wy+UfxmY19cXMTJkydF/NuV6h0DINWMjo4OPPDAA+jt7UU4HMZLL72EM844A5/73Oequu4nP/lJPPDAA5LfUAvL5XLB4XBIPo5yKhQKQa/XSz6OrSoQS+Kxl3Xo6FvdueoPj+OHX/geXIcugOecczFwznn4xT9/Ded/71h+lq/UDGButtDmi214fcNs4eg4Tv3qIqC5Ecflt6wZkylkwuV/uBxNR5vwjy/8I9rCq/0U90IANJlMOxIAA4EAbDZb1QFraWkJgiAgnU7nf+12u6FQKCo6HcRms8Hv94sWALfz+8yNXS6XIxwOMwDSrmAApJrR09ODyy67DG9/+9vx1re+Feeccw6am5uxsLBQ1XU//elP4xe/+IXkN9TC6uvrE+10kp2ucDgMnU4n+Tg2q2LP8eWWbS+56fd4/BNfzs8Gdl5wCf7vl34CQ9/qEuxNv+9eE+o+97Alf61SM3+F1f7srasbP+55H4LhMPTuKPxjU3ja9TQufO5CNB1twheFL2J48s1j7ORyec0HQKPRiOESR+9VEwD9fj86OjqqDliLi4sQBGHDbtqRkRHI5XJ4PJ5tPRfY3t6O4eFh0QKgz+dDZ2fntj4TjUahUCjgcrnWtIpZWlrCwsICAyCJigGQ9r0vfOELuP322yW/oRZWf38/7Pba3P25vkZGRqDVaiUfR6kqNYv3pMm75muf/dqvoPrgFfkg6PncNfDrzJvO8G1VHzz8e2R/ehrQ3IjfP3bHG8/7vYZzfnVD/nm/fzf8O5Kza58p3CsBsFRLne22MKk2GBWr+fl5CMLqcXnrX0skElCr1ejq6ir7GLa2tjYEg0HRAqDH44Hdbt/251KpFAwGA8xm85rZTQZAEhsDIO17X/7yl3HrrbdKfkMtLLfbjc7OTsnHUU5FIpGaDoClnuN72R7aMIN31i0yfOfaw3BfdHE+CP7qyuvx/u++UFEAfOnWzwLNjei77SIcPNyCg7f+Aec9/ik0HW3C+c9egL+754d41rJxJ22tB8B0Og2dToe+vr41Z16bTCaoVCoIgrCtcFVYXq+3omC0vubm5iAIQsnNE5lMBmazGUajMR+kNiuLxVJy+bWSGhoaQnd3d0WfXVhYgMPhgEqlwvj4OAMg7QgGQNr3brjhBvzgBz+Q/KZaWIODg+jo6JB8HOXU6OgoNJqN5+7WSm22k7dwA8fBwwo0v9aHQCyJ2ZERRH5wSz4Edp9/EW669jDOvEVWdvj77JGH8uf9fuHIAzjzZ4/h0NMfWg1/v/17nHXHgyV3FcvlckyU0aJmp2p2dhaTk5MYGRlZE/CMRmM+4OWORbRYLPkzrwOBAMbGxjAxMQGTyZRvEL7dmbGurq6qA1Y2m4UgCJuG0MXFRTidTqhUqi177plMJkQiEdEC4MDAAJxOZ1XXCAaDkMvlGBoawsLCAk6dOiX1X6e0jzAA0r73rW99CzfffLPkQaWwhoaG0N7eLvk4yqnR0VGo1WrJx7FZbdbLz+aL4VGdBzZfbMPnkrb/n70zD2+rvPK/2+fpDFAwDAwzHbokYc1ikhYolAIFylZKf7SlndKZFuhGaSnQwgBOSIgTCC1bA5QChbCEpRASApKudsmSJcuyrNWyLdmSV1myZFuLJVmyDEn8/f0h7o0ka9eVJSfv93m+Txvr6t5XstD9+Jz3nNOBtiu+w4Cg4OtX4I57/1EEAFIwPnw+0NKI/Vu+izP/3IJ1ryf3+635x1VYueXNnFXF0Wj1AXBmZgY+ny8N8LRabRrg8Xg8SKVSqNXqRYDn9/shlUoxMjKS9fyzs7OIx+PQ6XSQSCQlVd5WEhlLdSQSAYfDKThWLbXnXr4Ur0KhqGhWb6atVisrs4WnpqYgkUhgMBgIABKxKgKAREe87rrrLvzud7+rOaSk2m63o729vebrKMbj4+MQiUQ1X0chOz1+vLRfDo25j/lZPjB0evzYqxvCGc08/O7mLeg893Ba+MVv/xTn3/NOTgD846aNQEsjwltPxYadtzD7/c75281YsXH/ouN3cK3YqxvCXt0QnB4/eDxeRQCYCXhms7kowBscHGQALxKJ5L2GTCbD6Gj2XoV0EUgikYDVaoVAIMDExERRQNPX1weDwVAxGM3MzIDD4RRd6DExMQGBQLCowIK2XC4vq3o4l9mcLRyJRDA5OUkAkIhVEQAkOuJ1//3345e//GXNASXVAwMDUKvVNV9HMXa73RAKhTVfRzFObV5cKDWc+di6e/fiie/8Cr3nrIH9nNUwrV2P+394H868n5N23JrmD+DbugKB7Sfh2y9ezMDfWU/eixXNvILRw5XNfPzxJR74emfO0XOpgGe32wsCnsFgQG9vb0mAV8hSqbQgANKmU5XDw8MFYYaN1Oj8fLJYgsvllvwcuVzO/LeX+phMJoPH42ENANmcLZxIJDA3N0cAkIhVEQAkOuK1adMm/OxnP6s5nKTa4XAw+6fq3R6PBwKBoObrKMapAJi7OGQob+XvlXe8jH0XH04LS8+7FDf94mnm8b9v/jlsj52CK3et+3S/3/k4/dEnSywgoRggfYarSwM8gUCQBnjt7e1pgOfxeBAIBCoGvEKWSCQYGxvL+li2wouJiQnw+Xz09vbmjcr19PTAZDJVDEXBYBA8Hq/k58ViMXR2dkIikcDv9zM/l0gk8Hq9rAEg/TsjAEhUryIASHTEa/v27fjxj39cczhJtdPphFKprPk6ijGdOqv1OoqxSqXKGwFc2czHnylrYUB7kMIv/nc7NOu/zoDga5f/CD+8+6/gPPFFnP/6WjTtbsLal6/AqodfL6uCODUqKVbplhzwClkikcDlcmV9LFflbSAQgEQigU6ny1mcwdbeOL/fD4qiygaqvr4+UBTF7PsTi8UFC0VKMZuzhQkAElVDBACJjnj95S9/wY033lhzOEn10NAQFApFzddRjOnITq3XUYxVKhUGBg63XXlO0pdWBVxMY+e0dO99H+DRG+5Az+q1sJ+zGj1rV2P7r9bgvF3rsPrvN2HFpn0Fo3zFeJ9uuObvXabFYnHJADg/n9yvlq9C2GKxwGKxVAxF09PTEAgEFZ1jbGwMFEWhr68PAoEAU1NTrAGgTqdDf38/qwC4sLBQ669ToiNIBACJjnjt3LkT119/fc1vqKleLuPVotEovF4vKIqq+TqKcSoAZtvnV64v/cML2Hf1BiYaqD5/Pf7nlsew4sHiIa/QvsDUApV6sEgkwvj4eNbH8gHg/Px83gphs9nMSnHE1NQUhEJhxefx+/2QSCTgcrmYnJxkDQC1Wi1ro+UIABJVQwQAiY54vfDCC/j2t79d8xtqqpfDeDXaPp8PPB6v5usoxjQAZkv/lp2iffg1rH3lW2h6Yx1+37wG+vPWMSD4zqU34pI7K0sBp0JgtlY1tbJQKITb7c76WCEApKGlp6dnUYWw2WxmpTjC5/NBJBKxAljRaLKnoFQqRSgUYuWcbI6WIwBIVA0RACQ64vXqq6/i0ksvrfkNNdX1Pl0j1ZOTk3UDgE6PH/Ke8ZzVs2q1Gv39/djT4WQFys549Amse+08NO1uwrW7VkO34ys4///ewpYb70L3mqZkWnj1Wjxywx1Yfd8HR1QkUCgUwuPJDqTFACDtzApho9GInp6eiqHI6/VCLBazAljz8/PgcDjQ6/UltbTJZzZHy83NzREAJGJdBACJjni9/fbb+PrXv17zG2qql0NzZdpTU1Pgcrk1X0dmT79/SHswPDwMm80Gk8kEjUYDiqKwaRevpP132c3DWU/9kWnxcutLZyO0/SRseuhe5piL7noTr13+IyYa2L7hQvzifx+pOC2cbXJILSwQCFgBwPn5wxXCPT090Ov1rFTHTkxMQCqVshZh43A4mJmZweDgICiKgtPpLLrHYDa3tbWxNlqOACBRNUQAkOiI1969e7Fhw4aa31BTvVyaK0ejSQDkcDhLdr1IJIJgMIiJiQkG8CTqrkUFHKua+djLl0Gj0cBoNMJms2GfQF5yoccib/oA5zz/Ewb+bnvmUnzS0oi+h8/Fqix9/n5821OQnXcpA4J7v3k9rrzj5QLXobBtvzHnWjMnh9TCfD4fExMTWR8rFQDn5w9XCIvFYlYA0O12QyaTsQZYHA4HkUgE8/PJ9LJQKITRaCw4aSSX2RwtNzc3h0QiUeuvUqIjTAQAiY54cTgcrF69uuY31FR7PJ5l01x5enqauTmycb5MwOvr64PRaIRGo4FcLgdFUeBwOODz+ZDL5dBoNNgt0hYFSi9/pCwa9O55R784BbtlN9a8/O1kf7/XN+Cyv9yFQ1tPBFoa8eONT+c815n3c/DAD++Fee25sJ+zGr3nrMHj3/kV1t27N+vxNz7Bw9TUFDoGPIvBtk4igHw+H16vN+tj5QDg/HyyQlggEDDR70qgaHx8HHK5nBXAisfjzB859M9mZmagUCigUqkYMCzFra2tGB8fJwBIVLciAEh0xEskEuH000+veV+1VC+n3np+vx8cDgfhcLgkwPN4PBgaGkoDPJlMtgjwOjo6YDKZYLPZMDw8DK/Xi1AolHbOfFM9Uo/jSFVZgaoYIDz9kb9i3WsXJPv77fomVrW8AN3DFwItjeBtubaoc5x/9zt46ds/ZaKBneeejztu3oJVD6ZHDlc2U+gZShZY5BtXV0tTFMU6AM7Pz6OzsxOtra0lzxDOtMvlgkKhYAWwZmdnweFwFr0uuppZLBZjenq6pHOyOVmEACBRNUQAkOiIl1KpxJe+9KW6AsDl1FolEAgw+6NowAsEAosAr729fRHgtba2pgHeyMhIVsArxsWA0qP/bE0HrY187OBZC6SFKZz1xANY98a5aNrdhDUvfhcrN7+LP2zaDLQ0Ym7rv+MbzW+WlEb+f796FsILLmdAkHPR1bj29hfSjnlGYGUA1unxo7U3d3FLLZxvXnGlANjf34+enh4mzVzOecbGxqBUKlkBrGg0WQWcLd2bSCRgt9tBURRGR0eLPqdEImGlmIQGwPn5+Vp/lRIdYSIASHTES6vV4j/+4z/qCgDrubVKJuBZrVZwOByo1eo0wBMIBGmAZ7fbKwK8YpwPlJwef1bQyxYBvG6nIvn/N36Ic577GbPf7+xnfokVGzk4p3k/PFtXAS2N+OtDvyprL+HpD3Bxz4+bYVj3VQYEn73mFqz/03ugi1Tqqeo30zweD5OTk1kfqwQAU/vj0RXC5VTLjoyMoK2tjRXACofD4HA4eYs+xsfHmUKWYopDRCIRa5NFCAASVUMEAImOeBmNRpx44ol1BYC1rKzNBLze3l4YDAYmgsfj8dIAT6PRgMPhoK+vr+qAV4lzzf7N5ZWb38aaf1yT3O/3xnqc+fhmBsye3fwLoKUR7q2n4+zmD8sCQNob/vgenr3mFgYC9eu+irt/3IxVD3CZY1L7/xVqdbNUphsjZ3ss15i3YtzR0QGn08n8O7VCuJSq2+HhYajValYAKxQKgcvlFjwuEAhAKpWio6OjIASzOVmEACBRNUQAkOiIV19fH4499ti6AkC2CytyAd7g4GAa4Eml0kWAp9VqYTabc0bwQqEQOBwOgsFgzd+3fM4VAcwandv+HNa+elFyv9+rF+H0bX9jHvtm8xtIbD0FaGnE7zY9XBH8pUUdb38BnIuuZkBQ8PUr8L1fPZsWCayn/YAcDgdTU1NZH6sEALM1SC5mhnCmh4aG0N7ezgpgBYNB8Hi8oo6dnZ1l/lgKBoM5j+Pz+fD7/awB4Mcff1zrr1KiI0wEAImOeDmdTnz2s5+tKwCk99UVW1iRDfDcbnca4KnV6kWAp1Ao0gBvdHQUPp+P2c9XjGdmZsDhcBAIBGr+vuWz0+PHL1+UFgAxCmc+/hDWvbE+ud/vpeuwcvM7acfwt1wNtDRCu+UbqLyfYEY6+kEefnfzFnSeex4Dgi9++6c4/553sGrj4lnFtaoIjkQi4HA4mJ6eZh0A29vbs6Z8o9Eo2tracs4QzrTT6YRGo2EFsPx+P/h8fklA1t3dzfRKzHYMj8fLC4gEAIlqLQKAREe8XC4XGhoa6iqCFQwG0worMm++fr+fATy6eW4m4AmFwjTA6+/vLwvwCpneH1XPAFjU3N+NHJz97C+Y/X7nPPczrNiYnt796aYngJZGHNx6Iq7b+CKr8JfqdffuxRPf+RV6z1kD+zmrYVq7Hvf/8D6ceT9n0bG16AlIA6Dfnx0+KwFAtVrNTAXJdL4Zwpl2OBzQarWsANb09DQEAkHJzxseHgZFURgYGEhLX6c2liYASFSvIgBIdMRramoKDQ0NOVta1OLm6vP5wOFwmGrIQoBnsViqBniVwkCtXWju78qNfKzc/C7WvPjdT/f7nYszn3hwUXTv9GYu7A+vA1oa8ebmH1UN/lJ95R0vY9/F32GigbLzLsWPfvFUzSOANPRXAwBVKlXeCRn0DOFCFcIDAwPo7OxkBbAmJyfLnitMP1ev1zPvC91YOhwOEwAkqlsRACQ64hUOh9HQ0ICxsbElAyY6gud0OhnAU6lUkEgkDOBxOBy0trais7OzpoBXzOvJlw6stfMVf6zayMfNb/wTa3ddnIS/1y7A6Y/szHrs5of+CLQ0Ymbrf2FD83tLAoArmvlY8SCFX/zvdpgu+AYDgq9d/iNcfPebNdsDWCjtXwkAKpXKotqpFKoQ7u/vh06nYwWwfD5fRXOFw+EwlEollEolwuEwYrHYosbSlTgejxMAJGJdBACJjnglEgk0NDTA4WDnZkoD3vj4OAN4XV1dDOBxuVwmgqdUKhcB3uTkJFNYUa9RtUznKwiotbNFAFc287FPN4wn1a9h3esbkvv9/nEVVm7ZjRXNfOzgdqcdv755D0JbTwNaGvHwQ3cvHfyl2OEYw/gjj8K+dl0SBJvWw/3kUwjXALxpAMy1baJSACx2RFq+CmGbzQa9Xs8KYLExVzgej0Ov10MkEsHj8WRtLF3JuT/55JNaf5USHWEiAEhUc+3YsQMXX3wxjj32WJx44olZj3G5XPje976H4447Dqeccgruvvvuov8iPnjwID7zmc+gp6enaMCbnp5mAM9qtWYFPJFItAjwxsbGMDk5WVQEr56hKtP5WoLUgzOrZ19T2bBNve3wfr+/3YwVm/YzsHXnW11p8PXG5h8DLY3of3gdTm/mLhn00W75yMK8lqDViuFbbmWigY7Lr8AUh7ukRUz0Hyi52v1UAoAKhQIul6vo43NVCPf19cFgMLACWG63m5WxcolEAgMDA0yUv9w5wgQAiZZCBACJaq6tW7di586duO+++7IC4MGDB9HU1IQrr7wSZrMZMpkMp512Gu66666izn/o0CEcc8wxMBqNiwDP4XAUDXgDAwMM4JVTvZvpfI126835pkIU42J621Xa/87p8eNVngo8jQq38G853Nz5qT9hRTMvJ3xds/ElHNh6EtDSiJ9uemLJ4S/bPr9IJIJJDgeOy69gQHD4llsRtFqX5PdNFylVAwDlcnnJM3Kj0WSFsFKpZOby9vb2wmg0sgJY4+PjaG1tZeVc8/PJ4hAOh4Pu7m5WIJAAIFE1RACQqG70xhtvZAVAoVCIz372s5iYmGB+9t577+Ff//VfEYlEsp7L7/dDpVLhzTffxPbt23Hqqadiw4YNOOuss7ICXnd3N+uAV8gURVUEVUvpStZaTG87tvrfva94H1e8ewWadjfhwncuBH+Aj5YPu/MAGAXNlouBlkYIt1xVE/jL91rD09NwP/kU7OeuT4Lg2nUYf3QHwlUuaMpXpR6NVgaAMpkMbre7LAhKrRC2Wq0wmUysABubY+Xm55N9BblcLmQyGdrb2zE7O1vR+ebm5nDgwIGqffcSHZ0iAEhUN8oFgA8//DDWr1+f9rNQKISGhgYoFIqs5/rb3/6GL3/5y7jssstwyy234NRTT8W9996Ld999d8kAr5DpKsdar6MYUxRVVhV1tv15mRGvYo4pxvtt+3Hem+ehaXcTrnn/OrzTpYHT48/aIJr+9283bQVaGjG/9RRcsvH1JYQ/Cu9pBop+jSF7P0Zu/y0TDRz45iXw7Xm/amnhQn0qKwFAqVSas3deIScSCfT29oLP5zOReTaAbXR0lLWxcvPzyb6CFEUhFouho6MDUqm0YFsbAoBESy0CgER1o1wAePvtt+Oaa65Z9PN/+Zd/wbvvvlvwvAsLCzjttNMgFotrDlKpppvI1nodxTgVVktJ1eaq0E3tbVfMMfk8E5nBX7R/YVK+1719M1Y9tDctwnb3O/q0c9/9jh6PfqDD+MNnAC2NeG7zrUsa+fvFs9yy+ipOiURwXn0NA4JDN9+MgMHI+u/b7/dXDQAlEkne9i7FeGhoCFwul7VJIGyOlZufn8fU1BTTVzC1rU2pqW/a8XicACAR6yIASFQVtbS0oKGhIa8NBkPac/IB4LXXXrvo55+6s6YXAAAgAElEQVT73Ofw3nvvFVzLwsICVq1aBR6PV3OQSrVIJILb7a75OooxDaulpmqrHQH0BDz4jeg3DPz94cP/w8qM/X65Jmy4PtgCtDRiYutKrG7+YGnTvu+XP1ovEgjA89zf0L/hq0kQXLMWrs1bEGbxs0QDYK4IYyUAKBaL4fP5KoYsrVYLHo9X8gzhXEDJFkzOz2fvKzg6OgqKomC320teLwFAomqIACBRVeT3+9Hf35/XiUQi7TlspoAztXr1auzbt6/mIJVqsVgMl8tV83UUY6FQCKnJUdaosmzQmBlFLGcPoNVjxXf2fQdNu5tw/tvnY79tP16l1EVB2EXNb+LgI/8JtDTinoceWtLo34pmPu5+gVvxZJqQcxCjd919OC184UXwvvkWIiz0kKRnVed6vJLCBpFIhMnJyYohy2g0wmQylTxDOJudTic6OjpYA0Cv1wuJRLLo59PT0xCLxejq6ippvQQAiaohAoBEdaNCRSBer5f52Z49e/IWgWRq/fr1+Oc//1lzkEq1RCJZsubUldjp8ePnz+SGmdRULQ12HQOeNMBzevxo7U3+OxfspR5TaE38AT4ufOdCNO1uwlXvXwXDuAHRaBQCZUdWSM38GXfLd4CWRhx45Wp09LsXPV6sf/matuAx2c9N4RlRDyu/n2mFAs7rv8uA4OAPfgB/R0dF55yamgKXy60KAAqFQkxNTVUMWQaDAX19fYhGF1cIl+qBgQHWxsrNz8/D4/FAJpNlfSwSiUClUkGhUBQ9Km5ubg4HDx5k7buWiAggAEhUB3K5XLBYLNi+fTuOP/54WCwWWCwWzM7OAjjcBuaqq66C2WyGXC7Hl770paLbwADABRdcgNdff73mMJVqmUyG0dHRmq8jn3erHQUBp2PAwxybDb5So3mVFnyEI2E80/UMk/K9lX8rXP7DUVSdToenPuxYBJip0Pnfm54GWhqx0HIiXN3KvJNECnmfbignPK7amHx8r24oJxiyNeYtEgph4uVX0H/e+QwIjt3/AGbK/HxNTk5WDQAFAgGmp6crhiy9Xg+bzcZEyLq6uoqaIZzNbE4VmZ9PtpXJ11dwbm4ORqMRQqGwqHQ4AUCiaogAIFHNddttt2XdI6hUKpljXC4XbrjhBhx77LE4+eSTcdddd2F+fr7oa1xyySV46aWXag5UqZbL5RgeHq75OnI5W/VsrghgvmNTAS8XbO3TDRVcjy/ow52SOxn429a+DaFwep86nU4Hq9WaNZro9PjR2jOKqScvAFoa8e7m72PVRj5+84auLPhbtTEJvy0fWrI+lhrZzPXeFFvoUqxnRkcxdv8DDAT2n3c+Jl5+BZEc/fzyAWDmntlwOIypqSmMjY3B5XKVve+Oz+fD7/dXDFk6nQ79/f3Mv1MrhEutMrbZbOjq6mINAMfGxqBQKPIek0gk4HQ6QVEUBgcH8x4Xj8cJABKxLgKAREeFrrzySjz33HM1h6pUt7a2YmioMPgsNfTt/TRqlStylemNe434w9tdeY+hQadjwJMVhvLt+3N6/HhPr8P1+76Hpt1N+NpbX8O7Pe9mPbarqwvd3d25wUb+d6ClEZGtX8B5zf8sO/K3sjlZSZwaaWz5qBsdA56saeznJLacYFxpA+xs9nd0YPAHP2BA0PndGzCtVOZ9TiQSQSAQgNvtRm9vL7hcLrq6utDW1gaxWAwulwsulwuJRAKBQACj0VhWJJCiqIpaotDu7OzEwMDAop8PDw/nnSGczWxOFZmfn8fIyEjRbWUmJiYgEAhgNpuzvp8EAImqJQKAREeFvvOd7+Cpp56qOWClWqlUYnBwsObroJ2Zwi13X1yuVGdqGrZQpDB1TWc+shPrXrsATbubcNHbl0E7ps35GvIC4LQbicdWAC2NeOShOwuuOd86eaaRklPZz0n6mPOubKbwmrKftQbYWYFuZgbe3W9i4MKLGBAc+cMf4LFaMTw8DJvNBqPRiPb2dshkMmZ8mVAohFwuB5fLZRqku1wuTE9PM1XBoVAIcrkcWq225OILHo+HYDBYMWRptVo4HI68UGW1WouKVPb09LA2VYSG0FLayoRCIbS2tkKtViMajRIAJFoSEQAkOir0/e9/Hzt27Kg5ZKW6ra0NDgd7N/xinCvaVGy6t1xnA6ZsTp2J63BP46wnH8S6N85F0+4mrHnxuzh9y7t5ISsfAPa9+jugpRGDD6/GGc2cvIC3g2tNgbXDQEhDWrm9C50ePySWEex+n4P+MR8rDbBTHQ6HMTk5ibGxMfT398NisaBTKoXh9t/CtnoN7OesRl/TudA9+CA0SiVMJhPsdjtGRkbg8/mYyR9erxd8Pj/nNWKxGGZnZ6FSqdDW1rYIWvKZy+UiFApVDFkajQZOpzPn44FAAFKptKgKYavVCrPZzBoADg4OQqPRlPScWCyGzs5OSCSStBR5IpHA3NwcDh06VOuvUaIjTAQAiY4K/eQnP8HWrVtrDn2pVqlUGBgYWLLr5Ys2VVIIwabpaOF0aBq3ce86PM/32V9gxUZOQcjS6/VZAXC0txOfbP03oKURt2x6LO16manc5yS2rGC2TzeUVtVcLrzRY9YkltGSITI1Tet0Opk51nSalsPhMGlalUoFvV6P3t5eDA4OwtXWhsH//snhtPA112BKtLg5ejgchtvtBp/PRygUQjAYRDAYRCAQYBwOhzE3N4d4PI7Ozk7IZLKiKloTiQQzYq5SyGpvby+Y5o1Gi6sQpgvP2AJAh8NRVlVxIpFAX18fKIrC2NgYAUCiqooAINFRoVtuuQXNzc01h75Ut7e3o7+/f0muVQhYqh0BLMX7TEbc9NFNaNrdhHVvrMeZj2/GimaKATa66jgXAFoslvSfRyIIvHg90NII6ZYr0q+lG2ZeP713r9joXrnpWxoAc0UA+8d88Hq9GB4eRl9fX840beYc68w0bS6A9L73Hga+eQkDgsO/+Q2menrSAM/tdoPL5WJoaAjBYBChUAgzMzOYmZlBOBxGJBLB7Ows5ubmMDc3B7PZDJFIVLC4gwbAcDhcMWSp1WoMDw8XPK6YCmGz2Qyr1coaAPb396Ozs7Ps57tcLlAUhb6+PuY9JgBIxLYIABIdFfr1r3+Ne++9t+bQl2qNRgO73b4k1yoGarK1cVlqn7H9b/jmPy9F0+4mXPrupXhU8mFWSMoFW9kAMG7Z/+m835PxrY2vFozYlRLdK6V3IW16ysbIyAie4XYdhshmPh56lQ8OhwOKoiCXy9HR0ZEzTZvP4XAY4XAYMzMzTBQvFfCmXS6MbtsG+9p1SRA8dz1cjz+BkM/HjH9zOp3g8XgYHh5GPB5PM50Cnp2dRTwex9zcHGw2GzMyMBfYzM3NMRNGKoWstrY2jIyMFB1Zy1chbDQa0dvbyxoAslFV7Pf7IZFIoNUm97wSACRiWwQAiY4K/f73v8edd95Zc+hLdUdHB9PIttrOBjUrmpPpzrQ1DXhw666OGsAfhbMe34z1uzegaXcTbvroJjgmHcyaip1AYjAY0gEw5MehZ84FWhrR8/pdRUfsKinOiEQi8Pv9edO0HA4HEokEarUaorZO/FNmQGd3PzweD4LBYN4oXi7Ay4Q82nQEj47i0RG8aDSKUE8PRm69jYkGOi6/An4ehVgshng8zkSi+vv7F0FgPB5nIDAWiyGRSGBoaCgtfZnpWCzGTBipFLKUSmXO6+QyXSGc2XaFbirNFgD29vayUlUcjUahVqthMpkIABKxLgKAREeF7r33XvzmN7+pOfSlWqvVore3d8mu95ykbxF4pYJUtseXwis3cvD99w7v9/uT/E+YDk0z6y6l4CITAOflfwFaGnHoybMQDXhLitjlOzYUCpWcpnU4HHC5XPB4PMw+uEKQR6dds0XxCkFeNBpNi9LlciwWwzSPB8flVxyuFr71NoR7+xCPx9MqamkwzAeB4+PjoCgqa4UuDYCzs7MVw5FCoYDL5Sr5eV6vd1GFcFdXF+x2O2sA2NPTA5PJxMq54vE4olESASRiXwQAiY4KNTc349Zbb6059KWablq8VNfL14Q5W5+6JYG/ze9izYvf/XS/37m4k/fEouhXKSlZg8EAs9mchB/vIBZ2/BfQ0oi5ztKmwGRW05rNZmi1WrS2tkIgEFSUpg0EAuBwOAgGg3nTtJmAl7oXLzWKVwjwivVsIICJp5+G/dz1SRBctw6eHY9hdmoKU1NTEIlEMBgMWSEwFoshGo0iHo8jkUhgcnIyaxuWaDQKDoeDWCxWMRjJ5XKMj4+X9Vy6QrizsxPxeHxRU+lK3d3dzVpRCV0EsrCwUOuvUaIjTAQAiY4KtbS04Oabb6459KW6UNNitp0rDVyrfX+rWl7E2l3fTMLfaxfg9Ef+ihXN2cGu2JSs0WhkAPCT938FtDTi4EvfQjQSTjsuX5pWJBKlVdOq1WqmmnZoaKjsNC0NeT6fDxwOBx6PJy/glRLFY9NhhwOjv73jcFr4kkswtXcfAoEAZDIZsyctFwTSkcBAIACJRAK9Xs80OI5EIuBwOBWNkqMtk8ngdrvLfn40mqzEVyqV0Gg0OXsKlmM2i0oIABJVSwQAiY4KPfbYY/jhD39Yc+hL9aL9akvgeij0WNHMxxl/3oZ1ryf3+615+dtYuWU389j/vNyRE2ALpW9pAIw52oCWRqClER4dJy1NK5VKC6ZpC1XTpkJeKWla2jqdDmKxGFNTU0sOeMU6IJHAefU1h6uFb/4ppru6oFQqoVKpEA6HC0JgOBxGa2srNBoNYrEYwuEwOBxO2WPkUi2VSvMWnBTjubk5dHV1gcfjoaenhzUANBqNrJ2PACBRtUQAkOio0NNPP40bbrih5tCXDVaW8ppOjx9/pqw1hD8uzt75W2a/3znP/wQrNu1fdFy+Vi+p8JWZppVIJBDwKcw8+TWgpRHjf7thUZp2dHS0omrafMUWxaZpY7EYjEYjxGIx/H5/zWEvl2MzM/A+/zz6N3w1CYJr1sK1eQs0IjHkcjlCoVBBCIzFYlCr1VAoFJiammINAMViMbxeLyuAJZFIwOPxSp4hnMt6vZ61ohK6DQwBQCK2RQCQ6KjQ888/j6uvvrrm0Jdqk8kEo9G4ZNcrNIqt2l750PtY/cKNDPyd9dSfsKKZl/XYF2R2Jk1r6h/G3rZutHaa8qZpDQYDlEolHO+3AC2NWNjxX4hOOEsCvHxRvEzAK7XYIhsoWSwWCIVCTE9P1xz28jkyPIyxu+9hooEDF14E8+NPQPJp7798EEgDDB315HA4rICRSCSCz+dj5VwqlQomkylrhXA5ZrOohAAgUbVEAJDoqNDLL7+Mb33rWzWHvlSbzWYYDIaqX8fp8WOvbqimqd9VW1/B2lcu/XS/33k449En8h7/j31i8Hg8NL/MTRvJ1vKeOm+a1tLVjk/+vDLZ90/+OAN5paZpy6mmLSvCFovBarVCIBBgcnKy5qBXyMG2Ngx+9wYGBPuuvx7yF16Ez+fL+tro942GGIPBAA6Hg6mpqYrBSCgUsnKe+flkT8HR0dGsFcLluLOzEwMDA6wBYCKRqPVXKNERKAKAREeFdu/ejW984xs1h75Ud3d3Q6/XV/UatY76rWjm44wdj2Hd619D0+4mrH3lcqx6+LW8x3/vrzK8JutGq2UoR+/C9N6JqVE871u3Jws/dq5HYGqiJtW05bivrw8CgQBer7fmkFcQWsNh+HbtQv/55zMg2PXzWzBmtRaEQL/fDy6XC4qiKk63CgQCTE9PswJZqS1lgsFgWoVwOefr6OjIO6eYACBRPYgAINFRoT179uC8886rOfSl2mq1QqfTVTXyV1v44+Gspw/391v99x9ixaa9FZ93ZTMf9jHvIrgLDRlwaNvJQEsjAtp3WEnTLqXtdjszSaPWaynG0fFxuB54kIHA3g1fhXPnTsQikUXH0u/95OQk+Hw+RkZGQFFU0ZM8spmiqIKj54p1a2trWkuZaPRwhXA5U0uKmVNMAJCo1iIASHRU6MMPP8S6detqDn2p7unpQWdnZ9XOn6vv35J40z6s/vuPGPg7+693Itd+v3IssYwsStMeePMmoKURkReuhlgkQiAQqDkkleqBgQFQFAW3213ztRTrUKcOQz/4IQOC9muvRVClSoM/v98Pl8vFpLsTiQQ8Hg8zZaQcMOLxeAgGg6xAlkwmWxSRpCuE6UKdUs6nUqmKmlNcLADOz8/X+iuU6AgUAUCio0ICgQBnnnlmzaEv1X19fUxPtaWKAK5srn7fv1UPv461L1+R3O/3+ldxxmM72D3/Rj6GfcE0CEn0UsnCj+0nI+6ywGQyQSKRIBgMLikMseHBwUFQFAWXy1XztRTrmWAQY6+8AtsFX2dA0PLzn0O+531wudy0Yp2enh6mQnhqagpCoRAWi6XkPXdcLhehUIgVyMrVUiaRSKCvry/nDOFcViqVGB0dJQBIVNciAEh0VKi1tRVf+cpXiurttlS22WzQaDRVvUbqHsClgL/TH30S6147P7nfb9elWNXycsWw92eqh1n3qo18vKUZTAeQ6AwOPZds+/IJ9QDi8eTeM4PBsGwhcHh4GDweD6OjozVfSzyejOIFAgGMj4/D4XCgu7sbnZ2daG1tBZ/PB4fDgUAggJKiYP7d72BbvSYJghs2wPXMM5hNaReT2SYmFApBKpVCp9MV3SA6kUgw4/TYgCyxWJy3ophOWRdbIVzumDoCgERLKQKAREeF2tvb8Z//+Z91BYB2ux3t7e2snCsUCmFiYoKZTWswGJimx2/u4+GpN7l4dLegivDHw1lP3Xt4v98L/w8rH9pT8XkfeN+EYV8Qw74glDbPoshfPB7Hx21/TUb/nliFeMiXBhp6vR4ymSxrv7p69+joKCiKwvDw8JJcLxwOw+fzYWRkBDabDQaDAWq1GhKJZFEUz2g0wmazMaPvMptC+7SdsFz/XSYa6LzmWgSk0pwQGIlEoFQqoVarixoTNzc3Bw6HU9b+vGwWiUQFK4pLqRCWy+UVTSnJfK0ff/xxrb9CiY5AEQAkOirU1dWFk08+ua4AcGBgAGq1uqhj6abHo6OjTNPjjo6OtAgMn89nmh6bzeZFTY+rVhSycT/O+dvNh/f77bwdK5q5rJ1/ZXOWqB/t6TEsPPbFZNsX7SuLHo/FYujq6oJMJsPMzEzNoa5Uu1wuJvJU6blisRgCgQDcbndaFE+hUDDzjQUCARQKRdpkFLfbjUAgUHIBTSgUQufjj6P36xcyIDj62zsQdjiyQmAsFoNGo0FrayvC4XBeKIrH4+BwOIhGo6xAVrEVxcVWCLMxpYQAIFG1RQCQ6KiQ1WrF8ccfX1cA6HA40NbWhmj08GxaOsVGVwhnNj2WSqVM02N6Nu3ExASCwWDB62WOgVvZzMd97+org7Mtb2LNP676dL/fBpz555aqRBiz7fuLx+M48MHvgJZGHHrxEsRnF8+njceT6cvOzk7I5fKs48vq3W63GxRFwfEpOOVzJBLB5OQkE8Wjx99li+IZDIa8UTw2HI1GoZFKob/zD7CvXZcEwXPXY+LppzEbCGRtGK3X6yGRSBAIBHJCUSwWA4fDwezsLCuQRVFU3uulOhotXCHM1pQSAoBE1RQBQKKjQgMDA/jc5z5XUwCk07RDQ0Po6+uDSqUCn89Pm00rEomgVCqh0+mYCMz4+HhJs2mzjS6zjU5kLQhR9Y1hZZlRwdMfeQbrXvt6cr/fqxdj1bYXqgJ/tJU2TxpczA1rsdByItDSiIRDmRdEZmdnodVqmehSraGuVE9MTIDP58NutzNRPKfTCavVuiiKx+fz0dramhbFGx8fLyuKx4ZnZ2eh0+nQ+uabGPzZz5looOPyK+DnUUyLGLpXYCKRQE9PD9McOxeEcTicotLFxbjUimIaVHNVCItEopxrL9XxeJwAIFFVRACQ6KjQ6OgoGhoaipr/Wq6LTdO2traio6MDGo0GYrG4otm0xY4uk3aP5oSqbR+VOhuYwplPPIh1b5yLpt1NWPPS9Vi5+Z9Vhb9FEcBYDAdfuQpoacSBPbcVDSIajYaJ3NQa6nKZjuKNjo6mRfHoSDCHw0mL4vX19VU1iseGY7EYzGYzhAIBXHv2wHH5FQwIjtx6G2Z6exdBoMPhAEVRaf35aEciEXA4nKKLRgq5nIKSfBXCbDapjsfj+OSTT2r9FUp0BIoAINFRIZ/Ph4aGBkxOTpYNeOWkafv6+nKmaYeHhyGXy7NCHtujy4Z9wUURQBqqdE5v8TC28SOc89wtzH6/c569DSs2flRV+FvZzMfzMnsaUMwb3v503u9/Ym6y+P1xs7OzaG9vR1tbW80gMBaLIRgMpkXxdDodlErloiieVqtNi+KNjY2Bz+czrVRqDXalvm4amMYHBzHx9NOwn7s+CYLr1sGz4zFEfD7mc5tIJDA2NgaKohY1VQ6Hw+BwOBWNa0sFOQ6HU3DfYS7TFcKpkz9KSSkTACSqlQgAEtWFduzYgYsvvhjHHnssTjzxxKzHNDQ0LPJLL71U1PmDwSAaGhrgcrnyQl4wGExL09KVkLnStFarlbk5+/3+ktK0Q0NDkEqlDMQVC3jlji57SzPIQGBqOxVFn6c4ENv8Dta8dF1yv98b63Hm45uwopmqKvylwipTCDLjx6GnzgFaGvGx/LGSQSQajUKtVkOlUiEazb5vsFKnRvHsdjsTxZNKpeByueBwOBCLxWl/KAwPD8Pr9SIcDueFu9TeecsNAuPxOJxOJ3g8HkZGRhB2ODD62zsOp4UvuQST77/P7AtMJBJM+ttmszHAFwqFwOFwWAOsSgtK6Arh7u5uJBIJVnsUEgAkqpYIABLVhbZu3YqdO3fivvvuywuAb7zxBnw+H+O5ubmizh+LxdDQ0IC+vj6YzWbmxlwoTZtaTTs5OclE2NhI046NjYHH48HlcjGAV+3RZdnaqWSLDmb69G3PY+2r3/h0v9+FOH37c0sCftkilp+IH04WfuxsQjxSXo8/eiM/XYVd6vPpKJ7H41kUxRMKheBwOKAoioniWSwWDAwMMH8oVPp79fv9EIvFMJlMyxIC6cjewMAA4vE4AhIJnFdfw4Dg8M0/RcBgZCDQ7/dDJBLBZDIhkUggGAyCx+OxAlh0QUml+wnpCmGtVltRRDHTc3NzOHDgAJtft0REAAgAEtWZ3njjjbwA+NFHHxV9Lp1Oh23btuHWW2/FpZdeimOOOQaf+cxncNpppxWdpi2l2KKcNG1fXx9EIlHNmxVnRgf/9+WONCg847FHkvv9/nENVm55qyRw+/ZTirR//3a3ruyG1J1dnVh45NRk2xfLvopecyQSQVtbG9rb27MCWWYUz2QyZY3iqVQq6PX6tCjezMxM1cEsEAhAIpHAYDAsSwj0eDxp6ezYzAy8zz+P/g1fTYLgmrVwbd6CyMQEEokEZmZmIJPJ0NnZienpaVAUxQpg0QUl8Xjuti6lnKutrQ0cDoe1PYAEAImqJQKARHWlQgD4xS9+EaeccgouuOACvPTSSzh06FDOc+3Zswe33XYbtm3bht27d+OEE06AQCAoq9iiUJqWjvSVmqalmxW3trbWvDBh2BfEdo6VgbOVzXxs51ghsIxh414zznjsUazY+GHJ0KZzeqFzevFiaz90Ti8DnKVC4KqNfMR2/zfQ0oiDr9+AeIXQE4vFMD09DZlMBrlcju7u7qxRPLlcnhbFc7lcmJ6erklFbabpqJNOp6uL9ZTqqakpiEQiGI1GBmIjw8MYu/seJho4cOFFmHzrbcx92jKmra0NCoWCNQBkcz9hKlCWM0OYACDRUooAIFFdKR8APvroo8yN+Omnn8Zxxx2HRx99tKjzHjp0CMcffzx0Ol3ZxRaZkMdWmnZ2dhZqtRoajaamkZx8hSLFpIlzeb9+JOf1lDYPnpfZs173j+8a0qKSUu4/k4Uf2/4Nc2Omol5TNBrF1NRUWhRPo9FAJpOl7emkKApisRg9PT0YGhpasigeGw6FQgykLkcIDAQCTHPl1PUH29ow+N0bGBAc+sEPMdPVhXg8DpVKBS6Xy8oouJmZGdb2E6YCYE9PD/h8fsUTQeLxOAFAoqqIACBR1dTS0pK1cCPVBoMh7Tn5ADBTTz/9NBobG4s69tChQ/j85z8PoVBY9WKLchwOhyGTyWA2m2t2I97Gyd4Ohu6/l5omXtnMx627tEVF8fIB4Af6EXzQNQKd0wulzcP8L71HkdmzODGFQ89/PTnvl/sn5hype/EGBwfR09ODrq6urFG8jo6OnFG8cDgMuVy+CEKWi2dmZtDa2srMlq71espZv0KhgFqtTouEx8Jh+HbtQv955zMgOP7gg5iw2cDj8ViJsrG5nzAzopitQpgAIFG9iAAgUdXk9/vR39+f14lEIu05pQCgRqNhWrsU0scff4xTTz0V1157LUKhEJPirVaxRTn2+/0QCATMxvil9LAvmBXmMvvvPS+zM42jV21MtmfZrx/Ba22OrPC3sjn7BI/MFHC+cW/RaBRR2VPJnn+PfRndncpFUbzUvXj0hJRSo3gzMzOQy+Xo6upaFpG/TIfDYWae7nKEwEgkgvb2drS2ti6a3RwdH4frgQcYCLSfdz40Gzeiz2qFQCCAz+crG7ACgQBr6eT5+WSFMpfLZf7t8/nSKoRLPR9JARNVSwQAiepKpQDg888/j2OOOQbz8/NFHe/1evHFL34Rjz32WN3e4CcmJkBRFFwu15JeN1crmG0cK3NMvhRxMc/Pdx4aArXddiaK19bWBqFQCOEHb+PjR74AtDRiaM8mmM3mtCgem7ATCoUgk8mg1+vr9jOSz5FIBCqVqqZ9DisxPTVEIpHA7/cvejyg1cJ54/cZEBy84Xtw7P+Q+W+mHGCbnp6GQCBgDQCzAWWxM4RzAeDBgwcr+VolIsoqAoBEdSGXywWLxYLt27fj+OOPh8VigcViwezsLACAx+PhlVdeYaI7u3btQmNjI+65556SrqPVanHccceBy+XW/GaXy08KgpUAACAASURBVENDQ+Dz+ZicnFyya2aDsszoXS7I2/9pGjff8+m9eGNjY9ijMOdMF7+0X54WxZuYmMD8/juTbV/+/o2c837ZdDAYXNbVtdFoFO3t7VAqlXU7GaTQ+nU6Hfh8Prq7u2E2m9Mjvh9+iPatW2E7/4LD/QN/eweEb79dVqqVLkRhCwCnp6fB5/MX/ZxuPaRQKIpuEZNIJAgAElVNBACJ6kK33XZb1j2CSqUSACASifDVr34Vxx9/PI477jg0NTXh2WefLSs1smvXLvzbv/0bent7a36zy+Xe3l6IxeJFqbBqOlejaNqFCkFWfmr6+c0vcyGTyZgpKTweD3K5HDx5e9Z0c7Z08dyoHgvbTkrO++2XLdl7QUPgcu2zR4+9a21txczMTM3Xk+lwOAyfz4eRkRGm4bpKpYJYLGY+KwKBAFwuF1qtFv39/Yv3bXo8GN+yBfY1a5MguOGr0Nx3H3rN5pJSrT6fD2KxmDUApBt154rm5ZshnA0A4/E4AUCiqogAINFRp4WFBfz+979HU1MTpqaman4zzOZYLIauri4oFIol3c+VrVE07Wg0iqf53XkLPlY28/HkuzIIlVp0dnaCy+ViYGAAoVAoDaTe0gwyewlz7gGMxXDw1euSe//e/dmS/w4CgQDEYjHMZvOyhcDOzk7IZLIl/UMiHl9cnEM3ylYoFGnj7hQKBTo7O2G1WuF0OuF2uxEMBpn32+FwgMfjYXR0NOd1AgYjhn9yMxMNtF56Gcyvvlr0nOCJiQlIpVLWALAQUKbOEC5UIUwAkKiaIgBIdFTq448/xmWXXYabbrqpLgpAct3A29raoNVqlwRAYrEYQqEQMwovdS8eHcV7+i1ewapfumo4Ho9jYGAgZzp72BfEfv0I9utHsgLnvGlPsu3Lo/+BOe/SF8bE43FmAkV3d/eyhcCuri5m5CDb5/b7/XC5XBgYGGCm6mS22MnWKLuU1PTo6CgoioLD4cj5uY1Go5jauxeOSy5lQLD75p8iPDBQENg8Hg9kMhlrAFgsUBZTIUwAkKiaIgBIdNTK5/PhS1/6Eh555JG6vbnPzMxAKpWiu7ublfNFo1FMT09jbGwM/f39zE1bLpczN22hUIi2tjZ0dXWhp6cHg4ODmJiYQCgUwpA3kDcNnFk1HI8n09l0+52i1xr249Bf1yTbvki21/R3MD09DaFQiJ6enpp/HspxLBaDwWCARCIp7XcQPzwNZWRkBDabjZmNLZFIwOVyweVyIZVKodFoYDKZ0N/fj7GxMUxNTbEauaanhvT29mb9b5WGwOjkJDyPPQb72nWwn7MatnVNcD/5FOby9AscHx9Ha2srawDodrshl8uLjhbmqxCm9wDma3hPRFSuCAASHdXq7OzEcccdx4yCqkfTVYq5IiCZN8KZmRkmitfb2wu9Xp8WxePxeJDJZGXftNP6AW5M3/eXrZVLLBaDyWSCVCotej/ax9JHkoUfT69GPLy4GnSpPTU1BYFAUNf7Rgt9LkwmE0QiEaanp9N+Tkd9c/VRpGdja7VadHd3w+FwwO12IxAILOkfTpOTkxAKhTn3ZdIQGIvFELHZMHLrbYeniVx+OYJ8QVbIcrlcUCgUrAGgy+UqCShTZwhnziMmAEhUTREAJDrq9dprr+Gkk06C1bq4ZUm92OPxgMfjYXx8HLOzs5ienl6UepPL5aAoioniKZXKtCiex+NJ219ViVP3CubbN5h6c9bpdEWNvJvzObDw6H8k5/2a3qv5e58KIAKBADabreZrKcWpqVq1Wg0ejwe1Wr3o80JHfcvto7gUzjU1JPVzRkNgPB5HgKJgS0kLj952G6J2expkjY6Ooq2tjTUAHBsbg1KpLOk5uSqECQASVVMEAImOei0sLOAPf/gD1q1bt6StV3KZjuJ5vd60KJ5UKgWHw8kaxbPb7RgdHWU99cY2iLS3txdsVHzg3Z8n5/3uurbieb9se3JyEnw+H/39/TVfS6ozR94ZjUa0t7enpWolEgna29uhUCjA4/Fgt9vr+vOSy/TUk8ypIbRjsRhmZ2cRi8UwNzeHeCgE+5aH0beuKQmC65ow8ec/Yy4QYPbiqVQq1gCw3PNlqxAmAEhUTREAJCJCsijkW9/6Fr7//e8vSVFIZhTPYrFAq9VmjeLpdDomitfV1bXk7WHYdCQSgVKpzFnYkuiXfTrv9yTMjXTVfL3Z7PV6wefzl3RiS7Y/ClKbZaeOvKPnZTscDoyPjyMQCCz6TNtsNvD5fHi93pq/n+V+jtRqNRQKRdZtBTQEzs7OYm5uDolEAoPtGhhuuulw78BLLoX/g/0YHBxEe3s7awA4NDRU9vkSiQTzu3G73QQAiaoqAoBERJ9qcnISX/7yl7Ft27aK016pN+zh4WH09fVBr9en9TrL3EBfTBQvFouhs7MTSqVy2UVuaM/MzEAmky1utDwbxaEXLk4Wfnz4h5qvM58nJibA5/OL2pdZymcmEAjA7XbD4XCgu7sbWq0Wra2t4PP5OVP7ExMTZaVq+/v7wefz4fF4WHsNS2m6zY1UKs1Z3JI65jGRSMDtdqP1ySdhv+JKBgT7//QnaDQa1gDQ6XSio6Oj4igiXfk8NzeHhYWFWn89Eh2BIgBIRJSirq4ufP7zn8f+/fuLugGltsGgo3jZbtg6nQ5Wq5WVvXjRaBRKpRKdnZ11tT+rFAeDQYhEorR9lx9rXkxG//78ZcT94zVfYyF7PB5QFIXBwewzjHP97uiJKHa7HSaTCe3t7ZBKpWmpWrVaDYPBAJvNhpGREUxOTlZltJvD4QBFURgfr//3O5vp4hahUJhz+wYNgbFYDIlEIrmXk8tF77Zt6P/aeRjg8qDValkDwIGBAXR2dlZ8HrpC2G63EwAkqooIABIRZWj37t046aSTYDabMTIyAovFUjCK197enhbFq9YNm/bMzAwkEkldF64UMl1Z29/fj3hwAguPrwBaGvGx+rmar61Yu91uUBSFoaEh5mfhcHhR5DezClsul6OjoyNtrrHf769JT8qhoSFQFIWxsbGav5/lOBaLobe3N280M7U4JJFIMJNe9Eol7HY7K8BG2263Q6fTsXKuYDCIQCBAAJCoKiIASET0qd5++208+OCD+PGPf4xTTz0VxxxzDBoaGnD99denRfGyTSyohaempsDn80uKQNWbvV4vKIrCzHt3JNu+/O0CxKP1Pb+WnnLhdrvhdDqZiScSiYSJ/AoEgqyR38yJKPXikZER8Hg8jIyM1Hwt5ZqOZuabGkJvm0gkEohEIlAoFExVMVsAaLPZoNfrWTnX3NwcSQETVU0EAImIPtU999yD3//+93jqqafw/vvv47LLLsONN95Yt5NC4vE4xsfHwePx4Ha7a76Wcu21KnCo5dN5vzZRzdcTjx8u0kltmK3RaNKmXIjFYqhUKhgMBnR1dYHH48Fms5U05aKePDY2tiiaudw8OjoKHo8Hp9OZ9fHMSGAsFoNEIoFAIEAkEmEF2np7e2E0GgkAEtW9CAASEeXQ1NQUvvKVr2Dr1q11GbWh7XA4IBAI0hr8LhvHYjj4+veAlkb4nvn2klalhsNh+Hw+jIyMoK+vDwaDIS29n9pqx2w2Mw2zp6ens/5RQI8sW66p1Hg8+QcFPZ6s1msp1263G3w+H319fXkbRs/OziKRSKCnpwcSiQQSiQTBYLBiaLNarTCbzawBYCKRIABIVBURACQiyiODwYDPf/7z2LdvX81vbPnc3d0NiURS9KSNevF89wfJwo9HTsWQQQ6BQICpqSlWzk1PufB4PBgcHITVaoVOp4NCoYBAIGCmXCgUCnR2drKS3qerN10uV83f23JN72tcyjY3bLuUqSFWqxUmkwnd3d0QCoWYmpqqCNosFgssFgurAEhEVA0RACQiKqC33noLJ510EiwWS81vbLkci8Wg1WrR1tZW1ynrNEeCOLSzKdn2RbQF8XgcVqsVIpGo6Jm1mZXY2WYbi0QiqFQq6PV69PX1YXh4GF6vt2qp2uHhYVAUtazT8nSbm+U29STVfr8fEokEOp1u0X8Ts7OzTONstVrNROz6+/uZ31250GYymWC1WgkAEtW9CAASERXQwsIC/vjHP2L16tXw+Xw1v7HlcjQahUKhQFdXV12nrGl/LP9zsvDjqbMRn0mmr2OxGIxGI2QyGRPNjEQimJycxMjICGw2GwwGA9RqddqUi8xK7FJmG1fDg4ODoChq2fbYi8fjTBuS5Tr/OBqNwu12QywWQyqVQq/XQ61WZ03xu1wuBrboKO7w8HBZ0GYwGNDb20sAkKjuRQCQiKgIffLJJ7jyyivxve99r64bMIdCIYjF4rq/ac9NDmFhxxeShR/6txAKhTAxMcGkakUiESiKSkvVtra2QqvVoru7Gw6HA263G4FAoG5h1+l0gqIoTExM1Hwt5ZpOpVqt1rp8n1P/OEht00S33KGnowgEAgiFQtjt9qwp/syG0XQE1G63I5FIlARter0eNpuNNQCcn5+v9dcf0REqAoBEREVqenoaK1euxJYtW+ryZph60+bz+XVVzTk7O4tAIIDx8XE4HA4Ed/0IaGnEzNNfB/VpqlYoFKKtrY2ZciGTyaBQKGrebqcSDwwMLOuRa/F4HNPT0xCJRDCbzTX5PdB9FXONwKP/OKD3cWZruVPK1BC6Qnh6ehpCoRAWi6UkCNTpdOjv7ycASFT3IgBIRFSCjEYjjj/+eLz//vs1vzHns8vlWvLoEz3lYnR0FHa7HUajEe3t7WmpWolEgm7q5WThR8uJmDDwc6ZqI5EIc2NfrgAYjx8euVbP2wcK2e/3QywWw2g0sv67oMcm0hHgnp6eRcU6dF9F+o+DoaEheL3ekkbg0dsLRCJRzkKjzDYxoVAIUqkUOp0O8Xi8KGjTarVwOBysAeDHH39c6689oiNUBACJiErUO++8gxNPPBEmk6nmN+Z8HhgYgEAggN/vZ/VGnS8aQ6fctFotLBYLBgYGMD4+fnjKRWwWB1+8FGhpxIEP7ih4zZmZGUil0pzVnMvFNpsNAoEg57iy5eBAIJCcnqHXl/y7oJtnezweOJ1OdHd3o7OzE62traAoKi0CnFqs4/P5WC3WicVi6OnpKWpqSDyeTAfToxfVajVmZ2cLQptGo4HT6SQASFT3IgBIRFSiFhYWcN999+Hss8+u+9Se2WyGTCYr+iYai8UQCATgdrvhcDjQ3d2dc74xHY0ZHBzExMREUVMu5jtfTUb/Hvsi4tPF9csLBAIQiUR1v6+xkPv6+lhtc1MLh0IhyGQydHZ2LqqspT87dJrfYrEsqsgWi8VLNuc4nwcGBvL2bMzWMLqjowOtra0Ih8N5oU2tVmNoaIgAIFHdiwAgEVEZOnDgAK666irccMMNdV0UEovFoNFomOhFPH44VUtPuTCZTNBoNJBKpeByueBwOJBIJOzfqEM+LDxxenLer/KvJT2XHnu3nHvTxeNx9PT0QCgULs+m3fHkPjmv1wuxWAy5XM58dmQyWVqav729HUajsS4qsnOZHn9X7NSQubk5GAwGiMViBAKBnNDW1taGkZERAoBEdS8CgEREZcrv92PVqlV46KGH6i49SW+cHx4ehtVqZaog6epIugVGR0cHzGYzBgYG4HK5ck65YMOf8B9Mtn157muIR0tvWD0xMQGKopb1vFq68bBIJGItNc+2U8fg2e12mEymrHs5+Xw+xGIx7HY7XC7X4TR/HbyGYl3s1BAaAunJIQKBAD6fLyu0KZVKjI2NsQKA8XicACBR1UQAkIioAlksFhx//PHYs2fPkoNEMBiE2+2G0+mE1WpFZ2cnFAoFk6oVCARQKBTQ6XQwmUzg8/kwGo01qaqdG7diYfvJybYvvbyyzzM2NgYej4fx8fGaw0MlvzuLxcJEkmqxhmg0isnJSYyOjqb1VkztkSeXy9HR0QGLxQKHw4Hx8fG0tjuRSARtbW1QqVRLnsJl0z6fD0KhMGeVcywWY6qD6apcusXP+Pj4ImhrbW3N+vNyAfCTTz6p9dcc0REqAoBERBXq3XffRWNjI4xGI6s3ptRITH9/P8xmM5NuS91TpVKpYDAY0NfXh5GRkZwb530+X80iaAff/AHQ0oiDb95U8bnoJsv1vv8yn2OxGEwmE8RiMYLBYFWuEYlE4PP5MDw8zPTIa2trS+uRl9pbsZwxeNFoFGq1Gm1tbVWbrLIUzjc1hP590W1i6ObMY2NjoCgKg4ODadAmk8kqmiRCAJBoqUQAkIioQi0sLOD+++/HWWedVXLbFfomTTeyNRgMUKlUWacVmM1m9Pf3Y2xsrOxU7ejo6JLDU6KHmyz82H4K5tzsFHLY7XYIBIJlu5eOhgqj0QiJRFI2BKb2yOvp6UFXVxeUSuWiWcc6nY7pkTcxMVFS+5RCjkajaG9vh0KhWNYQGAqF0Nraivb29qz7FVMhMB5PVgh7vV4mhUz3CpRIJJiYmCAASFT3IgBIRMSCDhw4gGuuuQbXX3992s0jFoshFArB4/EwUy50Ot2imzTd767cSEwpttvtEAqFS5N+jM3i0HPnJef9Cjayeu7u7u6qRtCWwrFYDHq9HlKpFKFQKOvjmVNSMlP9qVXZvb29TI+8pYSx2dlZpuKXHuG3HB0Oh6FSqaBQKHK+jsyG0X6/HyKRCCaTCYlEAiKRKOf+wHKKQA4cOFDrrzeiI1QEAImOWK1YsQINDQ1pbm5uZv06n3zyCQYHB7F3716ccsopuPLKK/Gtb30Lt99+O5OqFYlETI+z3t5eDA8PL/lNOhUqjEYj5HL5kuzdmhvV48A//wfxELs98Gh4KqXNTT16dnYWWq2WKahIbb1D98gTiURQqVRMjzw61V9Pe+/oaRsymWxZQ3k0GoVWq807NSSzOGRmZobpf0m3+iEASFTvIgBIdMRqxYoVeOSRR+Dz+RjPzs6yeo2FhQU0Njbic5/7HM4++2xccsklWLVqFW677TZ89NFHddn+gr5Zt7e3o729fdlVbma+jo6ODiiVyrp8n1OBge6RNzAwwPTIS93PyePxQFEUurq6YLPZMDo6Wrefn3yvU6/XQyKR1KzAha3XUezUkHj8cMNolUoFDocDr9fLWgqYACBRtUQAkOiI1YoVK/DMM89U/ToTExM4ePAg8+89e/agsbERBoOh5jeyfI5EIpDL5VUZ77WUpm+8tYZZumjH5XJl7a+Y2iPPZDIt2s9JRwLpZsO1fl/LNQ1PYrG4blvdFPs66Kkhufb2ZkYC4/E400dzZmaGACBRXYsAINERqxUrVuALX/gCTj75ZGzYsAE7duxYkp5aCwsLePDBB3HGGWfkHDdVLw4EAhAKhbDb7TVfSyUOh8NobW1FV1dXVWE227xjtVqd1iMvtWiH7q9YbI88OqK53AsqYrEYzGYzRCLRsi7UiceLmxqS2iaGw+Ggq6uL6fVIAJCoXkUAkOiI1c6dO9HW1gar1Ypdu3bh3//93/HrX/96Sa594MABXHfddbjuuuvqPoXn9XpBURRGR0drvpZKHAqFIJFIcvZzK9aRSASTk5MYGRlheuRlVmanzjt2OBxwu91pPfIq8ezsLDQaDZRKZV3t8SvVdNNroVC4rGcgx+OHp4YMDg7mfK3RaBTRaBQcDgczMzOw2Wzg8/kVpYPn5ubSsgtERGyKACDRslJLS8uiwo5MGwyGrM/94IMP0NDQgEAgsCRrDQaDOPPMM/HAAw/UfYp1ZGQEFEXB5/PVfC2V2O/3QygUoq+vL+9x4XCY6ZHX29vL9MgTCoWLKrOtViucTic8Hk9R847ZcGp/vXr/A6KQe3t7mckZtV5LJaanhthsNsTj6X8o9PX1MS14OBwOIpEI5ufnmZ6V5U4GIQBIVE0RACRaVvL7/ejv78/rRCKR9bkejwcNDQ3Q6XRLtt6enh6ccMIJeOedd2p+Aytkm80GkUi0rCs44/E4JicnQVEUbDYbJiYmmB55me13BAIBlEoldDodenp6qtIjrxLTexvVavWyh0C73Z53L129mp6YQkNeR0cHuFwuU52d2meR/gx5vd60htHj4+OgKAoOh6Mk+KP3FBIAJKqWCAASHTWiKAoNDQ1wuVxLet29e/fihBNOgF6vr/kNLZ/pCs7W1tZlkXpM7bGYOg4vtX0Kn89HW1sb0yOvlu13ygWQtra2nM2Jl5PpvXRut7vma8l8j1MhT6/XQ6VSMRNTMqPBNpsNQqEQnZ2dRTeMnpychEAgQE9PD9MwmgAgUa1FAJDoiJRWq8XOnTthsVgwMjKC999/H6eddhpuvPHGJV/LwsICNm3ahNNPP73ubn6Znp2dhVqthkajqYsoGN0+xe12w+FwMD3y5HJ5Wo/FzHF4k5OTGBoaqkvgKNWRSARKpRIajWZZt+yJx+PMDF2Xy7Wk100t3knd15k5Fo+GvELR4FAoBLlcDo1GkxPMMxtGBwIBiMViGAwGZqYwAUCiWooAINERKZPJhIsuuggnnngijjnmGJxzzjloaWlBPB6vyXoOHjyI66+/Htdcc03dR9fC4TBkMhnMZvOSXG92dhZ+v5/pkZdt5rFEIoFarYbRaITdbi+6R57T6QSfz1/2+8/C4TAUCgU6OjqWPQQODQ2Bx+OxXnSUDfLUajVTvENRFFO8Q0/cqWRfJz01RKlU5pwaktkmhq5Wp//AKgSAc3NzOHToUE2+s4iOfBEAJCJaIoVCIZx11ln4v//7v7qIruWz3++HQCDAwMAAK+eje+SNjY0xPfLa29vTeuRJpVJoNJqsPfIquTadslvOPelo4KCjVMsdAkdHR8Hj8TAyMlL25yi1DU+2Cm02IK+Q6akh+aafZDaMpqPsdPNyAoBEtRIBQCKiJVRvby8aGxvx9ttv1/wmXMgTExMlpesyIzDZbs4ymQwdHR1Mj7zx8XEEAoGqAk0sFoPFYoFEIsk6b3c5mR45ptPp6v6PiEIeGxsDRVGLWqvkgjyJRJK3DU+1ZmcX8/kyGAxFTQ2hI4Fzc3PQ6XTMDGgCgES1EAFAIqIl1gcffIATTjgBXV1dNb8JF/Lw8DD4fD7Txy2z9UXmhvnMNFutb86pN+Curi7I5fJlUwCSyzMzM5DJZFVvel1N05DX29sLLpfLTHKhG2qn/rFQD5BXzOfLarVCIBDknRqSuicwkUjAYrFAKBRmnR1MAJCo2iIASES0xFpYWMDmzZuxatWquixQCIfD8Hq9GBoaQm9vL+RyObhcLtM+hW59kbphfil75FUCHRqN5ojorRcKhSCVSmEwGOr2Paf3dqaOxkuFPHpqilKpBPf/t3fnYVHV+x/AjxAgi6JgIZpCt9w1vaS0mEsu11uSuTxFLmldr9t1vZrpo+mkYtfdzMhyLa3UriuDwyYwsiMKCgKCgKCyb85C4lV4//7omfkJgqDAHIbzfj3P558TymeGifP2nPP9fM+eRWRkpP6KcHN9TXVVcnLyE6+aV78SeO/ePf2fuXPnDgMgGRQDIJEIHj58iLFjx2LkyJEGXxSi1WpRWlqK7Oxs3LhxA1evXkVUVBSCgoIem5EXHR2N+Ph4KJVK+Pv7N9srMPUt3ViVlrCitri4GH5+fqLu5axbpV19AU9Nz3ZWv+3/aM85OTnw9vauc4C3MVR6enq9dg3RbR1XXl6O9PR0yOVyZGRkMACSwTAAEomktLQU3bp1w7///e9GP4FrtVoUFxfrZ+RduXLlsRl5CoUCSqUSFy9exLVr15Ceno7c3Nwab5FqNBoolUpEREQYdQAsK/vzCuf58+dx8eJFo38tutEiDd3+rq7P0tOGvKfZ/1hXj87KM/afi274s27XkJreU92YGN3A6Nu3b0MulyM5OblKAKysrBT7VxW1UAyARCJKTEyEra0tfvrppwadmFNSUhAXF4fw8PAqM/J8fX0REhKCmJgYJCYm6mfkPcst0NLSUvj7++PKlSuin2AbWsXFxfD19W0Rr6WwsBA+Pj64cuXKMwen6iFP91kKCAjQhzw/Pz+EhobqV2k/S8irq/Lz86FQKBr0WppL5eTk4Ny5c4iLi6vxtdQ0MPrR1//HH38wAFKTYgAkEtmpU6dgY2ODyMjIx04S1Z+jenRGXvUTs25GXmZmZr1m5D1LFRQU4Ny5c0hJSRH9BNtYr6W2qzTGVAUFBVAoFLh69WqtX6O7Kqwbqv1oyHt03uKjIa+xRvE87Wvx9fXF5cuXjT4E6l5LdHR0re9h9YHRxcXF8Pf316/0ZgCkpsIASJLn6ekJZ2dnWFhYwMXFBSEhIQb9/vfu3cO8efPQtWtXLF26FB999BFOnDhR5WH5R2+xNdXVl/rWnTt34OXlhVu3bol+gm1o5ebmwtvbu9bntYypdIE2Li7uiTun6K4KP/oPBkOHvLqqqKgIfn5+zXqRS32ruLj4qXcNUalUCAoKQmJiIgMgNRkGQJK0Y8eOwczMDPv27UNSUhIWL14Ma2trg+wXfPjwYTg5OaFVq1awtLSEra0t+vbti08++QRKpbJZr4i8ceMGvL29a517Zkyle/YqMzNT9F7qU9X3QK4p5Hl7e9e4c0pzCnl1le5K2JOunhlL3b17F0qlEsHBwbWOIao+MFq3UIQBkJoKAyBJmqurK+bOnVvlWM+ePbFy5com/95paWlQKpW4c+cOKioqUFpaip49e2Lx4sXNMvRVr6tXr8LPz6/WbbCMqTIyMvSjOMTuRRcGqoe86ot4Ht0DOTExUR/ydM+eJSUlif46Glq6PXcjIiKMPgSq1Wr9Lfe6dg3RrRC+d+9ek/8eIuliACTJun//PkxNTXHq1KkqxxctWoShQ4eK0lNSUhJsbW1x8OBB0U9YdZVWq0VkZKR+Syux+2lopaSkVBl6bYj3r/o4nvqGvLreb92t7cbayk/MKi0t1e+fa+yfM41Gg4sXL8LX1xcFBQW1fi7UajXUajUKCwtF+T1E0sAASJKVnZ0NQRAQHh5e5fjGjRvRvXt3kboCzpw5AxsbG0RERIh+LmcPqAAAGRBJREFUwqqr1Go1goODERkZaRRXLeuqhIQEKBQKFBUVNcrf96SQ5+3tXeM4noas1H60dLP1WsKCnbt37yI4OBghISFGHwJ1u4a4u7tDLpcjNzcXoaGhOHjwIFavXg13d3e89tpraNeuHVxcXET7PUQtHwMgSZYuAEZERFQ57uHhgR49eojU1Z87hchkMnTt2rXe+/CKWaWlpfDz83viClRjKa1Wi8uXLz/1vsGP7p4SHx+vH6xdV8hr6iHguv2cU1NTRX9vG1oqlQpKpRJKpdLgw9Mb+pkqLi5GTEwMfvvtN2zYsAHTp0/HkCFDYG5uDkEQ0L59ewwaNAhTpkyBTCbDL7/8gujoaKhUKtF+D1HLxwBIktUcbwHrPHz4EB988AGGDx9uFHvX5ufnt5jVtFqtFlFRUQgMDKwSNGoLebrdUxQKhX73lISEBP1gbbHDim6RS1pamujvbUNLrVYjJCTkiYspxPrMqFQqxMfH49SpU9iyZQtmz56NESNGwMnJCSYmJrCyskK/fv0wceJErFixAvv378eaNWtgaWmJI0eOiPr7hqSJAZAkzdXVFfPmzatyrFevXgZZBFKXu3fvolevXliwYIFR3F69desWvLy8muX+xvUtXci7ceMG/Pz8oFAoat0i79GQ15zCSG0/G7lcjvT0dNF7aWip1WqEhYUhMDDQoAuQdIObU1NToVAosGvXLixcuBDvvfceunfvDnNzc5iZmaFbt2547733sHjxYnh6eiIgIABZWVmoqKiocUWvn58f1q5dK8JvGJI6BkCSNN0YmAMHDiApKQlLliyBtbU1MjMzxW4NAJCcnIx27dph//79op9461MpKSk4d+5crQ+4N4dSqVTIzc1Feno6EhISEB0djeDg4MdCXmRkJHx8fBAUFITs7OxmH/LqqqysLMjlcty8eVP0XhpaGo1GP/rmaW7V16e0Wi1u3bqFwMBA/PDDD1i+fDkmTJiAfv36wdraGq1atYKTkxNGjBiBOXPmYPv27fDy8kJKSgru37/PsS1kNBgASfI8PT3h5OQEc3NzuLi44MKFC2K3VIWXlxesra0RHh4u+om3PnXlyhXRx8OoVCrk5eUhIyMD165dw8WLF6FUKqFQKPRz8oKCghAVFYX4+HikpaUhJyfnsZBXWlqKgICAFjGQuKysDJmZmUY18/BJpdFoEB0dDX9//1rHqtRWWq0WeXl5CA8Px08//YQvv/wSkydPxqBBg9C+fXsIggAHBwcMHjwYn376KTZu3Ijff/9d/5wrQx61BAyARM1cZWUl1q9fj65duxrFiVur1SIiIgJKpbJJZ7ep1ep6hbzIyEhcvXoVN27cQHZ2NkpLS58qzOn2DW4Ji1zKyspw8+ZNyOVyo1hgVFdptVrExMTA19cXycnJj/23kpISXLp0CceOHYOHhwc+/fRTDBkyBI6OjhAEAba2thg4cCA+/vhjrFmzBkeOHEFUVBRKS0sZ8qjFYwAkMgIVFRWYMGEChg4dahS3ItVqNYKCghAdHd2gK2ePhrzExETExMTgwoUL8PHxwZkzZyCXyxEYGNjgkFdX6bZZqx4yjLXS09Mhl8uNfjs/rVaLu3fv4quvvoK9vT1WrVqFOXPmYNSoUXB2doaJiQksLS3Rt29fjB8/HsuXL8f+/fuhVCqRm5uLiooKsf/XJhINAyCRkVCpVOjduzfmz59vFLcjS0pK4Ovri4SEhCd+nVqtRn5+Pm7evFnvkHfnzh2UlJQY9H3IyclpMatpy8rKkJaWBrlc3uwX7egWX9y4cQM+Pj7YvXs3Fi1aBDc3N/Ts2RPm5uYwNTVFz549YWFhgalTp2L37t3w9/fHzZs38eDBA17NI6oBAyCREUlJSUH79u2xd+9e0U/M9am8vDx4e3sjNTX1sZAXEhICX19fnDlzBl5eXvotv65cuYLU1FRRQl5dpVvp3BJun5aVlSE1NRVyuRzZ2dmi96LVanH79m0EBwdj7969+OKLLzBp0iT0798fNjY2EAQBXbp0wTvvvIPZs2dj69atOHv2LJKTk1FeXo6KigrIZDJ06NCBO2gQ1QMDIJGR8fb2hrW1NUJDQ0U/aT9aGo0GBQUFyMzMRFJSEi5duoSQkBD9M3nVQ15KSgpu376N4uLiZhXy6ird7dPmEJoao3Rb4OXk5DT599JqtSgoKEBkZCQOHz6MtWvXYsqUKXj99ddhb28PQRDw/PPP46233sL06dPh4eGB48ePIzY2Flqttl5X8qoPdieimjEAEhmZyspKeHh4oEuXLgYf6VFTyAsNDYWfn58+5AUEBCA8PBxxcXH6kJeQkABvb28UFhaKHngao65fv45z584hPz9f9F4ao5KTk+Ht7Y3c3NwG/126xReXL1/G8ePH8fXXX+Ozzz7DsGHD0LlzZwiCgLZt28LFxQUfffQRVq9ejZ9//hmRkZEoKSnh7VoiA2EAJDJCFRUVmDRpEoYMGdLo41Y0Gg0KCwuRlZWF5ORkXL58WR/yzp49i7Nnz9YY8oqKip54JS82NhYBAQFGsYilPhUfHw8fH59G2zdY7EpKSsK5c+eQl5dX59dqtVqo1WokJibCy8sL27dvx7x58zB69Gj85S9/gampKSwsLNC7d2988MEHWLZsGfbu3Yvg4GBkZ2fXOhSZiAyHAZDISKnVavTp0wfz5s176luo9Q15YWFhiI2NxfXr13Hr1q06Q15doSEsLAwhISFNOh7GUKXVanHp0iX4+/uLOvOwMevatWvYvHkzlEqlfvFFeno6fH194enpiSVLlsDNzQ29e/dG69atYWpqipdffhljxozB/PnzsWvXLvj6+iIjI4OLL4iaOQZAIiOWmpqK9u3b48cff6wxoBQVFeHWrVu4fv06YmNjERYWBn9/f33I8/f3b9SQV1epVCoEBgbi4sWLRvXc35NCYGRkJIKCgkTf87chryE7OxtKpRL79+/H+++/j7Zt26JXr15o27YtBEFA586dMWzYMMyaNQubN2/G6dOnkZSUhHv37jHkERkpBkCiZyCTySAIQpVycHAweB8VFRU4dOgQWrdujUWLFmHatGnYunUrAgIC9CHPz88PoaGhuHz5MpKTk5GVlYXCwkLRrsIVFxfDx8cHiYmJooefxiiNRoOQkJBmfWVTq9WisLAQUVFROHLkCL766itMmzYNb7zxBp5//nkIggB7e3u88cYbmDZtGoYPH4527drh9OnT0Gg0DHlELRADINEzkMlk6NOnD3Jzc/VVUFBgsO8fFhaG3r17w8LCAqamprCzs8PLL7+MiRMn4vDhw6KHvLoqNzcXcrkcGRkZovfSGKVSqfS7joh1ZVOr1aK0tBRxcXH473//i//85z+YOXMmhg8fji5dukAQBNjY2GDAgAH48MMPsWrVKhw6dAjh4eEoKiqqEvIqKyuxfPlyTJ061WCfaSIyLAZAomcgk8nQv39/0b6/br6ebgP6iooKfPjhhxg8eLDRPI+m25fWEONHDFGlpaXw9/fHpUuXmiwE6hZfJCcnQy6XY+fOnZg/fz7GjBmDV155Bc899xzMzc3Rs2dPvP/++1i6dCn27NmDoKAg3Llz56kWX1RWVqK8vLyJP8lEJBYGQKJnIJPJYGVlBUdHRzg7O8Pd3R3p6emi9qRWq9GvXz/MmTPHaJ6vS0pKgkKhaDEraYuKiuDj44P4+PgGhTyNRoOMjAz4+/vj+++/x9KlSzFu3Dj06dMHlpaWMDExwUsvvYTRo0fjX//6F3bu3AmFQoG0tDT873//4y1bIqoTAyDRM1AoFDhx4gTi4+MREBCAYcOGwcHBAUVFRaL2dePGDdjZ2WHPnj2ih6H6hp1Lly7h/PnzLWY8TH5+Pry9vXH9+vU6X3tOTg5CQkJw4MABrFq1Ch999BFcXFzQrl07CIIAR0dHDB06FDNnzsSmTZtw8uRJXLt2jYsviKjBGACJGoFWq4WDgwO2b98udivw8fGBlZUVlEql6GGoPqXRaBAaGorQ0NBm+8zi01Z2djbmzp2LXbt2oaioCBcvXsSvv/6K9evX45NPPsFbb72FF154AYIgwM7ODq6urpg6dSrWrVuHX3/9FTExMVCpVAx5RNRkGACJGsmoUaMwd+5csdtAZWUlNm3ahE6dOiE9PV30MFSfUqlUOH/+fJM+P9dUpdVqcffuXVy9ehUnT57Epk2bMGvWLIwaNQpmZmYQBAHW1tZ49dVXMWnSJKxcuRIHDx5EaGgo8vPzGfKISBQMgESNoLy8HJ07d8a6devEbgXAn+Nh3N3d8dZbb6GkpET0kFSfKioqgkKhQFJSkui91BTyNBoNUlJScO7cOXzzzTdYsGAB3n33XXTr1g1mZmYwMzND9+7dMXbsWCxZsgTff/89VqxYARsbG0RFRTHoEVGzwgBI9AyWLVsGpVKJjIwMREVFwc3NDW3atEFmZqbYrelpNBq8+uqrmDVrltFcVcvJyYFcLjf4HsePBr2srCycP38ee/bsweeff47x48ejX79+sLKyQqtWreDs7IyRI0di7ty52LFjB+RyOVJTU3H//v0aQ97OnTuxZcsWET4BRES1YwAkegbu7u5wdHSEmZkZOnXqhIkTJyIxMVHsth6TlpYGOzs7eHp6ih7u6lsZGRmQy+XIzc1tspCXl5eHsLAwHDp0CKtXr8bHH3+MgQMH6hdfdOzYEW+//TY+++wzfP311/oFP2VlZbySR03Ow8MDb775JiwtLWFra1vj11QfRC8IAvbs2WPgTsmYMQAStXB+fn6wsrJCUFCQ6OGuvpWYmAgfHx8UFxc/c8grLi5GTEwMjh49ig0bNmDGjBl4++230bFjRwiCgHbt2mHQoEGYPHky1q5di19++QXR0dG4e/cuQx6Jau3atdixYweWLl36xAB46NChKsPo//jjDwN3SsaMAZCohausrMTWrVvh6OhoNItCtFotYmJiEBgYWOseu1qtFiqVCgkJCTh9+jS2bNmC2bNnY+TIkXB2doaJiQksLS3Rr18/TJgwAV988QX279+PCxcuIC8vDxUVFWL/aIie6NChQ08MgKdPnzZwR9SSMAASSUBFRQUmT56MN954w2gWhWg0GuzcuRMjRoxAUlISfHx88O2332LhwoUYO3YsevToAXNzczz33HPo1q0b3n33XSxatAjfffcdAgICkJmZiYcPH/JqHhmtugJg586dYW9vj4EDB2LPnj38Rw09FQZAIonQarUYMGAA/vGPfzS7RSFarRa3bt1CUFAQfvzxRyxfvhwTJkyAi4sL7O3tYW5ujq5du2LEiBGYM2cOtm3bBi8vL1y/fr3WxRdExu5JAXDDhg2IiIhAXFwctm3bBisrK2zYsMHAHZIxYwAkkpCMjAzY29tj9+7dooS8/Px8hIeH4+eff8aaNWswefJkuLq6ws7ODoIg4IUXXsDgwYMxY8YMbNy4Eb///ju8vb3RoUMH7N69W+y3j+iZyWSyGhduPFoxMTFV/syTAmB127ZtQ9u2bZuidWqhGACJJOb8+fOwsrJCYGBgk4S8kpISXLp0CcePH8fGjRvx6aefYsiQIejUqRMEQYCtrS1ee+01fPzxx/jyyy9x+PBhREVFobS0tNYreREREVi5cqWB3ymixlNYWIjk5OQn1r1796r8macJgGFhYRAEAXl5eU3RPrVADIBEElNZWYnt27ejY8eOSEtLe6aQp1KpcO3aNZw5cwbbtm3D3LlzMWrUKLz00kswMTFB69at0adPH4wfPx6ff/459u3bB6VSiZycHFRUVPCWLRlMfUaqZGVlwc3NDVZWVrC3t8fChQtx//59A3f6uKcJgLt370br1q1RXl7exF1RS8EASCRBFRUVmDp1KlxdXWsctaLb+SItLQ0+Pj747rvvsHjxYri5uaFXr16wsLCAqakpXnnlFfz973/HggUL8O2338LPzw83b97EgwcPGPKoWahrpMrDhw/Rt29fvPPOO4iNjUVAQAA6deqEBQsWiNDtn7KyshAXF4d169bBxsYGcXFxiIuLg0ajAQB4eXlh7969SEhIQFpaGvbt24e2bdti0aJFovVMxocBkEiiysrK8Ne//hVTpkxBcHAw9u3bhxUrVmDSpEno378/2rRpA0EQ8OKLL2L48OGYPXs2tmzZgjNnziA5ORnl5eUMeWQ0aruaplAoYGJiguzsbP2xo0ePwsLCAiqVypAt6s2YMaPGZwSDg4MBAD4+PhgwYABsbGxgZWWFvn374ptvvsGDBw9E6ZeMEwMgURO6cOEC3Nzc4OjoWOPcrsrKSshkMjg6OqJ169YYNmwYrl27ZrD+bt68CRMTE9ja2uLNN9/E9OnTsWHDBhw7dgyxsbHQaDQMedQi1BYA16xZg1dffbXKsZKSEgiCgKCgIEO1R2RwDIBETUihUGD16tU4efJkjQFw06ZNaNOmDU6ePImEhAT9FnNqtdpgPWZlZTHkUYtXWwCcNWsWRo8e/dhxc3Nz/Pbbb4ZojUgUDIBEBlI9AFZWVqJjx47YtGmT/lh5eTlsbW3xww8/iNEikVFozJEqs2bNwt/+9rfHjpuZmeHo0aNN9hqIxMYASGQg1QNgeno6BEFAbGxsla8bN24cpk+fbuj2iIxGY45U4S1gkioGQCIDqR4Aw8PDIQhClYfPgdqvSBA1N05OTo9deVuxYoXYbdWorkUgOTk5+mPHjh0TdREIkSEwABIZSG0B8NETDwD885//xJgxYwzdHtFTc3Jywvr165Gbm6sv3aiS5qKukSq6MTAjR45EbGwszp8/jxdffFHUMTBEhsAASGQgvAVMLY2TkxN27twpdhtPVNdIFeDPkDh27FhYWlrCzs4OCxYs4EBlavEYAIkMpLZFIJs3b9Yfu3//PheBkNFwcnJCx44dYWdnh/79+8PDw6NZ7KBBRHVjACRqQhqNRn/LSRAE7NixA3FxccjKygLw5xgYW1tbnDp1CgkJCZg8ebLBx8AQPasdO3ZAqVTi6tWr2LdvHzp06ICZM2eK3RYR1QMDIFETCg4OrvH204wZMwD8/yDojh07wsLCAkOHDkVCQoK4TZOkPcuIFZ0TJ05AEAQUFRUZuGsieloMgEREpPcsI1Z07ty5A0EQEBUVZeCuiehpMQASERmYp6cnnJ2dYWFhARcXF4SEhIjdUqOQy+UQBEH/iAMRNV8MgEREBnTs2DGYmZlh3759SEpKwuLFi2FtbW10oSkiIkL/TGtGRgaOHz+OTp06Ydy4cWK3RkT1wABIRGRArq6umDt3bpVjPXv2xMqVK0Xq6NlcvnwZr7/+OmxtbdG6dWv06NEDMpkMZWVlYrdGRPXAAEhEZCD379+HqakpTp06VeX4okWLMHToUJG6IiIpYgAkIjKQ7OxsCIKA8PDwKsc3btyI7t27i9QVEUkRAyARkYHoAmBERESV4x4eHujRo4dIXRGRFDEAEhEZCG8BE1FzwQBIREalpkHFDg4OYrdVb66urpg3b16VY7169TK6RSBEZNwYAInIqMhkMvTp0we5ubn6KigoELutetONgTlw4ACSkpKwZMkSWFtbIzMzU+zWiEhCGACJyKjIZDL0799f7DYaxNPTE05OTjA3N4eLiwsuXLggdktEJDEMgERkVGQyGaysrODo6AhnZ2e4u7sjPT1d7LaIiIwKAyARGRWFQoETJ04gPj4eAQEBGDZsGBwcHFBUVCR2a0RERoMBkIiMmlarhYODA7Zv3y52K0RERoMBkIiM3qhRox7bXo2IiGrHAEhERq28vBydO3fGunXrxG6FiMhoMAASkVFZtmwZlEolMjIyEBUVBTc3N7Rp04ZjVIiIngIDIBEZFXd3dzg6OsLMzAydOnXCxIkTkZiYKHZbRERGhQGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGL+D46LC9CUHHwlAAAAAElFTkSuQmCC\" width=\"640\">"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib notebook\n",
    "\n",
    "X,y = make_blobs(n_samples=1000, n_features=3, centers=3, cluster_std=1.0, random_state = seed)\n",
    "X = X.T\n",
    "\n",
    "from mpl_toolkits.mplot3d import Axes3D\n",
    "plt.figure()\n",
    "plt.axes(projection='3d')\n",
    "plt.plot(*X, '.')\n",
    "#plt.axis('equal');\n",
    "\n",
    "m, n = X.shape\n",
    "\n",
    "Xbar = X.mean(axis=1).reshape(-1,1)\n",
    "Y = X - Xbar\n",
    "U, s, VT = np.linalg.svd(Y)\n",
    "\n",
    "for col,u in enumerate(U):\n",
    "    plt.plot(*(np.c_[-u, u]*15 + Xbar));\n",
    "\n",
    "plt.gca().view_init(20,70)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Die Hauptkomponenten sind"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-24T09:28:15.287735Z",
     "iopub.status.busy": "2021-09-24T09:28:15.287292Z",
     "iopub.status.idle": "2021-09-24T09:28:15.289416Z",
     "shell.execute_reply": "2021-09-24T09:28:15.289023Z"
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-0.6248, -0.6964,  0.3531],\n",
       "       [ 0.2762, -0.6201, -0.7342],\n",
       "       [ 0.7303, -0.3612,  0.5798]])"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "U"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Für die Varianzen erhalten wir"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-24T09:28:15.292375Z",
     "iopub.status.busy": "2021-09-24T09:28:15.291933Z",
     "iopub.status.idle": "2021-09-24T09:28:15.293909Z",
     "shell.execute_reply": "2021-09-24T09:28:15.293517Z"
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([52.785 ,  4.4244,  1.0172])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vk = s**2 / (n-1)\n",
    "vk"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In Scikit-Learn stehen entsprechende Methoden zur Verfügung, mit denen man diese Berechnungen sehr einfach durchführen lassen kann. Wir erhalten"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-24T09:28:15.297348Z",
     "iopub.status.busy": "2021-09-24T09:28:15.296946Z",
     "iopub.status.idle": "2021-09-24T09:28:15.398237Z",
     "shell.execute_reply": "2021-09-24T09:28:15.397801Z"
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "/* Put everything inside the global mpl namespace */\n",
       "/* global mpl */\n",
       "window.mpl = {};\n",
       "\n",
       "mpl.get_websocket_type = function () {\n",
       "    if (typeof WebSocket !== 'undefined') {\n",
       "        return WebSocket;\n",
       "    } else if (typeof MozWebSocket !== 'undefined') {\n",
       "        return MozWebSocket;\n",
       "    } else {\n",
       "        alert(\n",
       "            'Your browser does not have WebSocket support. ' +\n",
       "                'Please try Chrome, Safari or Firefox ≥ 6. ' +\n",
       "                'Firefox 4 and 5 are also supported but you ' +\n",
       "                'have to enable WebSockets in about:config.'\n",
       "        );\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure = function (figure_id, websocket, ondownload, parent_element) {\n",
       "    this.id = figure_id;\n",
       "\n",
       "    this.ws = websocket;\n",
       "\n",
       "    this.supports_binary = this.ws.binaryType !== undefined;\n",
       "\n",
       "    if (!this.supports_binary) {\n",
       "        var warnings = document.getElementById('mpl-warnings');\n",
       "        if (warnings) {\n",
       "            warnings.style.display = 'block';\n",
       "            warnings.textContent =\n",
       "                'This browser does not support binary websocket messages. ' +\n",
       "                'Performance may be slow.';\n",
       "        }\n",
       "    }\n",
       "\n",
       "    this.imageObj = new Image();\n",
       "\n",
       "    this.context = undefined;\n",
       "    this.message = undefined;\n",
       "    this.canvas = undefined;\n",
       "    this.rubberband_canvas = undefined;\n",
       "    this.rubberband_context = undefined;\n",
       "    this.format_dropdown = undefined;\n",
       "\n",
       "    this.image_mode = 'full';\n",
       "\n",
       "    this.root = document.createElement('div');\n",
       "    this.root.setAttribute('style', 'display: inline-block');\n",
       "    this._root_extra_style(this.root);\n",
       "\n",
       "    parent_element.appendChild(this.root);\n",
       "\n",
       "    this._init_header(this);\n",
       "    this._init_canvas(this);\n",
       "    this._init_toolbar(this);\n",
       "\n",
       "    var fig = this;\n",
       "\n",
       "    this.waiting = false;\n",
       "\n",
       "    this.ws.onopen = function () {\n",
       "        fig.send_message('supports_binary', { value: fig.supports_binary });\n",
       "        fig.send_message('send_image_mode', {});\n",
       "        if (fig.ratio !== 1) {\n",
       "            fig.send_message('set_dpi_ratio', { dpi_ratio: fig.ratio });\n",
       "        }\n",
       "        fig.send_message('refresh', {});\n",
       "    };\n",
       "\n",
       "    this.imageObj.onload = function () {\n",
       "        if (fig.image_mode === 'full') {\n",
       "            // Full images could contain transparency (where diff images\n",
       "            // almost always do), so we need to clear the canvas so that\n",
       "            // there is no ghosting.\n",
       "            fig.context.clearRect(0, 0, fig.canvas.width, fig.canvas.height);\n",
       "        }\n",
       "        fig.context.drawImage(fig.imageObj, 0, 0);\n",
       "    };\n",
       "\n",
       "    this.imageObj.onunload = function () {\n",
       "        fig.ws.close();\n",
       "    };\n",
       "\n",
       "    this.ws.onmessage = this._make_on_message_function(this);\n",
       "\n",
       "    this.ondownload = ondownload;\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._init_header = function () {\n",
       "    var titlebar = document.createElement('div');\n",
       "    titlebar.classList =\n",
       "        'ui-dialog-titlebar ui-widget-header ui-corner-all ui-helper-clearfix';\n",
       "    var titletext = document.createElement('div');\n",
       "    titletext.classList = 'ui-dialog-title';\n",
       "    titletext.setAttribute(\n",
       "        'style',\n",
       "        'width: 100%; text-align: center; padding: 3px;'\n",
       "    );\n",
       "    titlebar.appendChild(titletext);\n",
       "    this.root.appendChild(titlebar);\n",
       "    this.header = titletext;\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function (_canvas_div) {};\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function (_canvas_div) {};\n",
       "\n",
       "mpl.figure.prototype._init_canvas = function () {\n",
       "    var fig = this;\n",
       "\n",
       "    var canvas_div = (this.canvas_div = document.createElement('div'));\n",
       "    canvas_div.setAttribute(\n",
       "        'style',\n",
       "        'border: 1px solid #ddd;' +\n",
       "            'box-sizing: content-box;' +\n",
       "            'clear: both;' +\n",
       "            'min-height: 1px;' +\n",
       "            'min-width: 1px;' +\n",
       "            'outline: 0;' +\n",
       "            'overflow: hidden;' +\n",
       "            'position: relative;' +\n",
       "            'resize: both;'\n",
       "    );\n",
       "\n",
       "    function on_keyboard_event_closure(name) {\n",
       "        return function (event) {\n",
       "            return fig.key_event(event, name);\n",
       "        };\n",
       "    }\n",
       "\n",
       "    canvas_div.addEventListener(\n",
       "        'keydown',\n",
       "        on_keyboard_event_closure('key_press')\n",
       "    );\n",
       "    canvas_div.addEventListener(\n",
       "        'keyup',\n",
       "        on_keyboard_event_closure('key_release')\n",
       "    );\n",
       "\n",
       "    this._canvas_extra_style(canvas_div);\n",
       "    this.root.appendChild(canvas_div);\n",
       "\n",
       "    var canvas = (this.canvas = document.createElement('canvas'));\n",
       "    canvas.classList.add('mpl-canvas');\n",
       "    canvas.setAttribute('style', 'box-sizing: content-box;');\n",
       "\n",
       "    this.context = canvas.getContext('2d');\n",
       "\n",
       "    var backingStore =\n",
       "        this.context.backingStorePixelRatio ||\n",
       "        this.context.webkitBackingStorePixelRatio ||\n",
       "        this.context.mozBackingStorePixelRatio ||\n",
       "        this.context.msBackingStorePixelRatio ||\n",
       "        this.context.oBackingStorePixelRatio ||\n",
       "        this.context.backingStorePixelRatio ||\n",
       "        1;\n",
       "\n",
       "    this.ratio = (window.devicePixelRatio || 1) / backingStore;\n",
       "\n",
       "    var rubberband_canvas = (this.rubberband_canvas = document.createElement(\n",
       "        'canvas'\n",
       "    ));\n",
       "    rubberband_canvas.setAttribute(\n",
       "        'style',\n",
       "        'box-sizing: content-box; position: absolute; left: 0; top: 0; z-index: 1;'\n",
       "    );\n",
       "\n",
       "    // Apply a ponyfill if ResizeObserver is not implemented by browser.\n",
       "    if (this.ResizeObserver === undefined) {\n",
       "        if (window.ResizeObserver !== undefined) {\n",
       "            this.ResizeObserver = window.ResizeObserver;\n",
       "        } else {\n",
       "            var obs = _JSXTOOLS_RESIZE_OBSERVER({});\n",
       "            this.ResizeObserver = obs.ResizeObserver;\n",
       "        }\n",
       "    }\n",
       "\n",
       "    this.resizeObserverInstance = new this.ResizeObserver(function (entries) {\n",
       "        var nentries = entries.length;\n",
       "        for (var i = 0; i < nentries; i++) {\n",
       "            var entry = entries[i];\n",
       "            var width, height;\n",
       "            if (entry.contentBoxSize) {\n",
       "                if (entry.contentBoxSize instanceof Array) {\n",
       "                    // Chrome 84 implements new version of spec.\n",
       "                    width = entry.contentBoxSize[0].inlineSize;\n",
       "                    height = entry.contentBoxSize[0].blockSize;\n",
       "                } else {\n",
       "                    // Firefox implements old version of spec.\n",
       "                    width = entry.contentBoxSize.inlineSize;\n",
       "                    height = entry.contentBoxSize.blockSize;\n",
       "                }\n",
       "            } else {\n",
       "                // Chrome <84 implements even older version of spec.\n",
       "                width = entry.contentRect.width;\n",
       "                height = entry.contentRect.height;\n",
       "            }\n",
       "\n",
       "            // Keep the size of the canvas and rubber band canvas in sync with\n",
       "            // the canvas container.\n",
       "            if (entry.devicePixelContentBoxSize) {\n",
       "                // Chrome 84 implements new version of spec.\n",
       "                canvas.setAttribute(\n",
       "                    'width',\n",
       "                    entry.devicePixelContentBoxSize[0].inlineSize\n",
       "                );\n",
       "                canvas.setAttribute(\n",
       "                    'height',\n",
       "                    entry.devicePixelContentBoxSize[0].blockSize\n",
       "                );\n",
       "            } else {\n",
       "                canvas.setAttribute('width', width * fig.ratio);\n",
       "                canvas.setAttribute('height', height * fig.ratio);\n",
       "            }\n",
       "            canvas.setAttribute(\n",
       "                'style',\n",
       "                'width: ' + width + 'px; height: ' + height + 'px;'\n",
       "            );\n",
       "\n",
       "            rubberband_canvas.setAttribute('width', width);\n",
       "            rubberband_canvas.setAttribute('height', height);\n",
       "\n",
       "            // And update the size in Python. We ignore the initial 0/0 size\n",
       "            // that occurs as the element is placed into the DOM, which should\n",
       "            // otherwise not happen due to the minimum size styling.\n",
       "            if (fig.ws.readyState == 1 && width != 0 && height != 0) {\n",
       "                fig.request_resize(width, height);\n",
       "            }\n",
       "        }\n",
       "    });\n",
       "    this.resizeObserverInstance.observe(canvas_div);\n",
       "\n",
       "    function on_mouse_event_closure(name) {\n",
       "        return function (event) {\n",
       "            return fig.mouse_event(event, name);\n",
       "        };\n",
       "    }\n",
       "\n",
       "    rubberband_canvas.addEventListener(\n",
       "        'mousedown',\n",
       "        on_mouse_event_closure('button_press')\n",
       "    );\n",
       "    rubberband_canvas.addEventListener(\n",
       "        'mouseup',\n",
       "        on_mouse_event_closure('button_release')\n",
       "    );\n",
       "    rubberband_canvas.addEventListener(\n",
       "        'dblclick',\n",
       "        on_mouse_event_closure('dblclick')\n",
       "    );\n",
       "    // Throttle sequential mouse events to 1 every 20ms.\n",
       "    rubberband_canvas.addEventListener(\n",
       "        'mousemove',\n",
       "        on_mouse_event_closure('motion_notify')\n",
       "    );\n",
       "\n",
       "    rubberband_canvas.addEventListener(\n",
       "        'mouseenter',\n",
       "        on_mouse_event_closure('figure_enter')\n",
       "    );\n",
       "    rubberband_canvas.addEventListener(\n",
       "        'mouseleave',\n",
       "        on_mouse_event_closure('figure_leave')\n",
       "    );\n",
       "\n",
       "    canvas_div.addEventListener('wheel', function (event) {\n",
       "        if (event.deltaY < 0) {\n",
       "            event.step = 1;\n",
       "        } else {\n",
       "            event.step = -1;\n",
       "        }\n",
       "        on_mouse_event_closure('scroll')(event);\n",
       "    });\n",
       "\n",
       "    canvas_div.appendChild(canvas);\n",
       "    canvas_div.appendChild(rubberband_canvas);\n",
       "\n",
       "    this.rubberband_context = rubberband_canvas.getContext('2d');\n",
       "    this.rubberband_context.strokeStyle = '#000000';\n",
       "\n",
       "    this._resize_canvas = function (width, height, forward) {\n",
       "        if (forward) {\n",
       "            canvas_div.style.width = width + 'px';\n",
       "            canvas_div.style.height = height + 'px';\n",
       "        }\n",
       "    };\n",
       "\n",
       "    // Disable right mouse context menu.\n",
       "    this.rubberband_canvas.addEventListener('contextmenu', function (_e) {\n",
       "        event.preventDefault();\n",
       "        return false;\n",
       "    });\n",
       "\n",
       "    function set_focus() {\n",
       "        canvas.focus();\n",
       "        canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    window.setTimeout(set_focus, 100);\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function () {\n",
       "    var fig = this;\n",
       "\n",
       "    var toolbar = document.createElement('div');\n",
       "    toolbar.classList = 'mpl-toolbar';\n",
       "    this.root.appendChild(toolbar);\n",
       "\n",
       "    function on_click_closure(name) {\n",
       "        return function (_event) {\n",
       "            return fig.toolbar_button_onclick(name);\n",
       "        };\n",
       "    }\n",
       "\n",
       "    function on_mouseover_closure(tooltip) {\n",
       "        return function (event) {\n",
       "            if (!event.currentTarget.disabled) {\n",
       "                return fig.toolbar_button_onmouseover(tooltip);\n",
       "            }\n",
       "        };\n",
       "    }\n",
       "\n",
       "    fig.buttons = {};\n",
       "    var buttonGroup = document.createElement('div');\n",
       "    buttonGroup.classList = 'mpl-button-group';\n",
       "    for (var toolbar_ind in mpl.toolbar_items) {\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) {\n",
       "            /* Instead of a spacer, we start a new button group. */\n",
       "            if (buttonGroup.hasChildNodes()) {\n",
       "                toolbar.appendChild(buttonGroup);\n",
       "            }\n",
       "            buttonGroup = document.createElement('div');\n",
       "            buttonGroup.classList = 'mpl-button-group';\n",
       "            continue;\n",
       "        }\n",
       "\n",
       "        var button = (fig.buttons[name] = document.createElement('button'));\n",
       "        button.classList = 'mpl-widget';\n",
       "        button.setAttribute('role', 'button');\n",
       "        button.setAttribute('aria-disabled', 'false');\n",
       "        button.addEventListener('click', on_click_closure(method_name));\n",
       "        button.addEventListener('mouseover', on_mouseover_closure(tooltip));\n",
       "\n",
       "        var icon_img = document.createElement('img');\n",
       "        icon_img.src = '_images/' + image + '.png';\n",
       "        icon_img.srcset = '_images/' + image + '_large.png 2x';\n",
       "        icon_img.alt = tooltip;\n",
       "        button.appendChild(icon_img);\n",
       "\n",
       "        buttonGroup.appendChild(button);\n",
       "    }\n",
       "\n",
       "    if (buttonGroup.hasChildNodes()) {\n",
       "        toolbar.appendChild(buttonGroup);\n",
       "    }\n",
       "\n",
       "    var fmt_picker = document.createElement('select');\n",
       "    fmt_picker.classList = 'mpl-widget';\n",
       "    toolbar.appendChild(fmt_picker);\n",
       "    this.format_dropdown = fmt_picker;\n",
       "\n",
       "    for (var ind in mpl.extensions) {\n",
       "        var fmt = mpl.extensions[ind];\n",
       "        var option = document.createElement('option');\n",
       "        option.selected = fmt === mpl.default_extension;\n",
       "        option.innerHTML = fmt;\n",
       "        fmt_picker.appendChild(option);\n",
       "    }\n",
       "\n",
       "    var status_bar = document.createElement('span');\n",
       "    status_bar.classList = 'mpl-message';\n",
       "    toolbar.appendChild(status_bar);\n",
       "    this.message = status_bar;\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.request_resize = function (x_pixels, y_pixels) {\n",
       "    // Request matplotlib to resize the figure. Matplotlib will then trigger a resize in the client,\n",
       "    // which will in turn request a refresh of the image.\n",
       "    this.send_message('resize', { width: x_pixels, height: y_pixels });\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.send_message = function (type, properties) {\n",
       "    properties['type'] = type;\n",
       "    properties['figure_id'] = this.id;\n",
       "    this.ws.send(JSON.stringify(properties));\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.send_draw_message = function () {\n",
       "    if (!this.waiting) {\n",
       "        this.waiting = true;\n",
       "        this.ws.send(JSON.stringify({ type: 'draw', figure_id: this.id }));\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_save = function (fig, _msg) {\n",
       "    var format_dropdown = fig.format_dropdown;\n",
       "    var format = format_dropdown.options[format_dropdown.selectedIndex].value;\n",
       "    fig.ondownload(fig, format);\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_resize = function (fig, msg) {\n",
       "    var size = msg['size'];\n",
       "    if (size[0] !== fig.canvas.width || size[1] !== fig.canvas.height) {\n",
       "        fig._resize_canvas(size[0], size[1], msg['forward']);\n",
       "        fig.send_message('refresh', {});\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_rubberband = function (fig, msg) {\n",
       "    var x0 = msg['x0'] / fig.ratio;\n",
       "    var y0 = (fig.canvas.height - msg['y0']) / fig.ratio;\n",
       "    var x1 = msg['x1'] / fig.ratio;\n",
       "    var y1 = (fig.canvas.height - msg['y1']) / fig.ratio;\n",
       "    x0 = Math.floor(x0) + 0.5;\n",
       "    y0 = Math.floor(y0) + 0.5;\n",
       "    x1 = Math.floor(x1) + 0.5;\n",
       "    y1 = Math.floor(y1) + 0.5;\n",
       "    var min_x = Math.min(x0, x1);\n",
       "    var min_y = Math.min(y0, y1);\n",
       "    var width = Math.abs(x1 - x0);\n",
       "    var height = Math.abs(y1 - y0);\n",
       "\n",
       "    fig.rubberband_context.clearRect(\n",
       "        0,\n",
       "        0,\n",
       "        fig.canvas.width / fig.ratio,\n",
       "        fig.canvas.height / fig.ratio\n",
       "    );\n",
       "\n",
       "    fig.rubberband_context.strokeRect(min_x, min_y, width, height);\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_figure_label = function (fig, msg) {\n",
       "    // Updates the figure title.\n",
       "    fig.header.textContent = msg['label'];\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_cursor = function (fig, msg) {\n",
       "    var cursor = msg['cursor'];\n",
       "    switch (cursor) {\n",
       "        case 0:\n",
       "            cursor = 'pointer';\n",
       "            break;\n",
       "        case 1:\n",
       "            cursor = 'default';\n",
       "            break;\n",
       "        case 2:\n",
       "            cursor = 'crosshair';\n",
       "            break;\n",
       "        case 3:\n",
       "            cursor = 'move';\n",
       "            break;\n",
       "    }\n",
       "    fig.rubberband_canvas.style.cursor = cursor;\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_message = function (fig, msg) {\n",
       "    fig.message.textContent = msg['message'];\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_draw = function (fig, _msg) {\n",
       "    // Request the server to send over a new figure.\n",
       "    fig.send_draw_message();\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_image_mode = function (fig, msg) {\n",
       "    fig.image_mode = msg['mode'];\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_history_buttons = function (fig, msg) {\n",
       "    for (var key in msg) {\n",
       "        if (!(key in fig.buttons)) {\n",
       "            continue;\n",
       "        }\n",
       "        fig.buttons[key].disabled = !msg[key];\n",
       "        fig.buttons[key].setAttribute('aria-disabled', !msg[key]);\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_navigate_mode = function (fig, msg) {\n",
       "    if (msg['mode'] === 'PAN') {\n",
       "        fig.buttons['Pan'].classList.add('active');\n",
       "        fig.buttons['Zoom'].classList.remove('active');\n",
       "    } else if (msg['mode'] === 'ZOOM') {\n",
       "        fig.buttons['Pan'].classList.remove('active');\n",
       "        fig.buttons['Zoom'].classList.add('active');\n",
       "    } else {\n",
       "        fig.buttons['Pan'].classList.remove('active');\n",
       "        fig.buttons['Zoom'].classList.remove('active');\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function () {\n",
       "    // Called whenever the canvas gets updated.\n",
       "    this.send_message('ack', {});\n",
       "};\n",
       "\n",
       "// A function to construct a web socket function for onmessage handling.\n",
       "// Called in the figure constructor.\n",
       "mpl.figure.prototype._make_on_message_function = function (fig) {\n",
       "    return function socket_on_message(evt) {\n",
       "        if (evt.data instanceof Blob) {\n",
       "            var img = evt.data;\n",
       "            if (img.type !== 'image/png') {\n",
       "                /* FIXME: We get \"Resource interpreted as Image but\n",
       "                 * transferred with MIME type text/plain:\" errors on\n",
       "                 * Chrome.  But how to set the MIME type?  It doesn't seem\n",
       "                 * to be part of the websocket stream */\n",
       "                img.type = 'image/png';\n",
       "            }\n",
       "\n",
       "            /* Free the memory for the previous frames */\n",
       "            if (fig.imageObj.src) {\n",
       "                (window.URL || window.webkitURL).revokeObjectURL(\n",
       "                    fig.imageObj.src\n",
       "                );\n",
       "            }\n",
       "\n",
       "            fig.imageObj.src = (window.URL || window.webkitURL).createObjectURL(\n",
       "                img\n",
       "            );\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        } else if (\n",
       "            typeof evt.data === 'string' &&\n",
       "            evt.data.slice(0, 21) === 'data:image/png;base64'\n",
       "        ) {\n",
       "            fig.imageObj.src = evt.data;\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        var msg = JSON.parse(evt.data);\n",
       "        var msg_type = msg['type'];\n",
       "\n",
       "        // Call the  \"handle_{type}\" callback, which takes\n",
       "        // the figure and JSON message as its only arguments.\n",
       "        try {\n",
       "            var callback = fig['handle_' + msg_type];\n",
       "        } catch (e) {\n",
       "            console.log(\n",
       "                \"No handler for the '\" + msg_type + \"' message type: \",\n",
       "                msg\n",
       "            );\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        if (callback) {\n",
       "            try {\n",
       "                // console.log(\"Handling '\" + msg_type + \"' message: \", msg);\n",
       "                callback(fig, msg);\n",
       "            } catch (e) {\n",
       "                console.log(\n",
       "                    \"Exception inside the 'handler_\" + msg_type + \"' callback:\",\n",
       "                    e,\n",
       "                    e.stack,\n",
       "                    msg\n",
       "                );\n",
       "            }\n",
       "        }\n",
       "    };\n",
       "};\n",
       "\n",
       "// from http://stackoverflow.com/questions/1114465/getting-mouse-location-in-canvas\n",
       "mpl.findpos = function (e) {\n",
       "    //this section is from http://www.quirksmode.org/js/events_properties.html\n",
       "    var targ;\n",
       "    if (!e) {\n",
       "        e = window.event;\n",
       "    }\n",
       "    if (e.target) {\n",
       "        targ = e.target;\n",
       "    } else if (e.srcElement) {\n",
       "        targ = e.srcElement;\n",
       "    }\n",
       "    if (targ.nodeType === 3) {\n",
       "        // defeat Safari bug\n",
       "        targ = targ.parentNode;\n",
       "    }\n",
       "\n",
       "    // pageX,Y are the mouse positions relative to the document\n",
       "    var boundingRect = targ.getBoundingClientRect();\n",
       "    var x = e.pageX - (boundingRect.left + document.body.scrollLeft);\n",
       "    var y = e.pageY - (boundingRect.top + document.body.scrollTop);\n",
       "\n",
       "    return { x: x, y: y };\n",
       "};\n",
       "\n",
       "/*\n",
       " * return a copy of an object with only non-object keys\n",
       " * we need this to avoid circular references\n",
       " * http://stackoverflow.com/a/24161582/3208463\n",
       " */\n",
       "function simpleKeys(original) {\n",
       "    return Object.keys(original).reduce(function (obj, key) {\n",
       "        if (typeof original[key] !== 'object') {\n",
       "            obj[key] = original[key];\n",
       "        }\n",
       "        return obj;\n",
       "    }, {});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.mouse_event = function (event, name) {\n",
       "    var canvas_pos = mpl.findpos(event);\n",
       "\n",
       "    if (name === 'button_press') {\n",
       "        this.canvas.focus();\n",
       "        this.canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    var x = canvas_pos.x * this.ratio;\n",
       "    var y = canvas_pos.y * this.ratio;\n",
       "\n",
       "    this.send_message(name, {\n",
       "        x: x,\n",
       "        y: y,\n",
       "        button: event.button,\n",
       "        step: event.step,\n",
       "        guiEvent: simpleKeys(event),\n",
       "    });\n",
       "\n",
       "    /* This prevents the web browser from automatically changing to\n",
       "     * the text insertion cursor when the button is pressed.  We want\n",
       "     * to control all of the cursor setting manually through the\n",
       "     * 'cursor' event from matplotlib */\n",
       "    event.preventDefault();\n",
       "    return false;\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function (_event, _name) {\n",
       "    // Handle any extra behaviour associated with a key event\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.key_event = function (event, name) {\n",
       "    // Prevent repeat events\n",
       "    if (name === 'key_press') {\n",
       "        if (event.key === this._key) {\n",
       "            return;\n",
       "        } else {\n",
       "            this._key = event.key;\n",
       "        }\n",
       "    }\n",
       "    if (name === 'key_release') {\n",
       "        this._key = null;\n",
       "    }\n",
       "\n",
       "    var value = '';\n",
       "    if (event.ctrlKey && event.key !== 'Control') {\n",
       "        value += 'ctrl+';\n",
       "    }\n",
       "    else if (event.altKey && event.key !== 'Alt') {\n",
       "        value += 'alt+';\n",
       "    }\n",
       "    else if (event.shiftKey && event.key !== 'Shift') {\n",
       "        value += 'shift+';\n",
       "    }\n",
       "\n",
       "    value += 'k' + event.key;\n",
       "\n",
       "    this._key_event_extra(event, name);\n",
       "\n",
       "    this.send_message(name, { key: value, guiEvent: simpleKeys(event) });\n",
       "    return false;\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onclick = function (name) {\n",
       "    if (name === 'download') {\n",
       "        this.handle_save(this, null);\n",
       "    } else {\n",
       "        this.send_message('toolbar_button', { name: name });\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onmouseover = function (tooltip) {\n",
       "    this.message.textContent = tooltip;\n",
       "};\n",
       "\n",
       "///////////////// REMAINING CONTENT GENERATED BY embed_js.py /////////////////\n",
       "// prettier-ignore\n",
       "var _JSXTOOLS_RESIZE_OBSERVER=function(A){var t,i=new WeakMap,n=new WeakMap,a=new WeakMap,r=new WeakMap,o=new Set;function s(e){if(!(this instanceof s))throw new TypeError(\"Constructor requires 'new' operator\");i.set(this,e)}function h(){throw new TypeError(\"Function is not a constructor\")}function c(e,t,i,n){e=0 in arguments?Number(arguments[0]):0,t=1 in arguments?Number(arguments[1]):0,i=2 in arguments?Number(arguments[2]):0,n=3 in arguments?Number(arguments[3]):0,this.right=(this.x=this.left=e)+(this.width=i),this.bottom=(this.y=this.top=t)+(this.height=n),Object.freeze(this)}function d(){t=requestAnimationFrame(d);var s=new WeakMap,p=new Set;o.forEach((function(t){r.get(t).forEach((function(i){var r=t instanceof window.SVGElement,o=a.get(t),d=r?0:parseFloat(o.paddingTop),f=r?0:parseFloat(o.paddingRight),l=r?0:parseFloat(o.paddingBottom),u=r?0:parseFloat(o.paddingLeft),g=r?0:parseFloat(o.borderTopWidth),m=r?0:parseFloat(o.borderRightWidth),w=r?0:parseFloat(o.borderBottomWidth),b=u+f,F=d+l,v=(r?0:parseFloat(o.borderLeftWidth))+m,W=g+w,y=r?0:t.offsetHeight-W-t.clientHeight,E=r?0:t.offsetWidth-v-t.clientWidth,R=b+v,z=F+W,M=r?t.width:parseFloat(o.width)-R-E,O=r?t.height:parseFloat(o.height)-z-y;if(n.has(t)){var k=n.get(t);if(k[0]===M&&k[1]===O)return}n.set(t,[M,O]);var S=Object.create(h.prototype);S.target=t,S.contentRect=new c(u,d,M,O),s.has(i)||(s.set(i,[]),p.add(i)),s.get(i).push(S)}))})),p.forEach((function(e){i.get(e).call(e,s.get(e),e)}))}return s.prototype.observe=function(i){if(i instanceof window.Element){r.has(i)||(r.set(i,new Set),o.add(i),a.set(i,window.getComputedStyle(i)));var n=r.get(i);n.has(this)||n.add(this),cancelAnimationFrame(t),t=requestAnimationFrame(d)}},s.prototype.unobserve=function(i){if(i instanceof window.Element&&r.has(i)){var n=r.get(i);n.has(this)&&(n.delete(this),n.size||(r.delete(i),o.delete(i))),n.size||r.delete(i),o.size||cancelAnimationFrame(t)}},A.DOMRectReadOnly=c,A.ResizeObserver=s,A.ResizeObserverEntry=h,A}; // eslint-disable-line\n",
       "mpl.toolbar_items = [[\"Home\", \"Reset original view\", \"fa fa-home icon-home\", \"home\"], [\"Back\", \"Back to previous view\", \"fa fa-arrow-left icon-arrow-left\", \"back\"], [\"Forward\", \"Forward to next view\", \"fa fa-arrow-right icon-arrow-right\", \"forward\"], [\"\", \"\", \"\", \"\"], [\"Pan\", \"Left button pans, Right button zooms\\nx/y fixes axis, CTRL fixes aspect\", \"fa fa-arrows icon-move\", \"pan\"], [\"Zoom\", \"Zoom to rectangle\\nx/y fixes axis, CTRL fixes aspect\", \"fa fa-square-o icon-check-empty\", \"zoom\"], [\"\", \"\", \"\", \"\"], [\"Download\", \"Download plot\", \"fa fa-floppy-o icon-save\", \"download\"]];\n",
       "\n",
       "mpl.extensions = [\"eps\", \"jpeg\", \"pgf\", \"pdf\", \"png\", \"ps\", \"raw\", \"svg\", \"tif\"];\n",
       "\n",
       "mpl.default_extension = \"png\";/* global mpl */\n",
       "\n",
       "var comm_websocket_adapter = function (comm) {\n",
       "    // Create a \"websocket\"-like object which calls the given IPython comm\n",
       "    // object with the appropriate methods. Currently this is a non binary\n",
       "    // socket, so there is still some room for performance tuning.\n",
       "    var ws = {};\n",
       "\n",
       "    ws.binaryType = comm.kernel.ws.binaryType;\n",
       "    ws.readyState = comm.kernel.ws.readyState;\n",
       "    function updateReadyState(_event) {\n",
       "        if (comm.kernel.ws) {\n",
       "            ws.readyState = comm.kernel.ws.readyState;\n",
       "        } else {\n",
       "            ws.readyState = 3; // Closed state.\n",
       "        }\n",
       "    }\n",
       "    comm.kernel.ws.addEventListener('open', updateReadyState);\n",
       "    comm.kernel.ws.addEventListener('close', updateReadyState);\n",
       "    comm.kernel.ws.addEventListener('error', updateReadyState);\n",
       "\n",
       "    ws.close = function () {\n",
       "        comm.close();\n",
       "    };\n",
       "    ws.send = function (m) {\n",
       "        //console.log('sending', m);\n",
       "        comm.send(m);\n",
       "    };\n",
       "    // Register the callback with on_msg.\n",
       "    comm.on_msg(function (msg) {\n",
       "        //console.log('receiving', msg['content']['data'], msg);\n",
       "        var data = msg['content']['data'];\n",
       "        if (data['blob'] !== undefined) {\n",
       "            data = {\n",
       "                data: new Blob(msg['buffers'], { type: data['blob'] }),\n",
       "            };\n",
       "        }\n",
       "        // Pass the mpl event to the overridden (by mpl) onmessage function.\n",
       "        ws.onmessage(data);\n",
       "    });\n",
       "    return ws;\n",
       "};\n",
       "\n",
       "mpl.mpl_figure_comm = function (comm, msg) {\n",
       "    // This is the function which gets called when the mpl process\n",
       "    // starts-up an IPython Comm through the \"matplotlib\" channel.\n",
       "\n",
       "    var id = msg.content.data.id;\n",
       "    // Get hold of the div created by the display call when the Comm\n",
       "    // socket was opened in Python.\n",
       "    var element = document.getElementById(id);\n",
       "    var ws_proxy = comm_websocket_adapter(comm);\n",
       "\n",
       "    function ondownload(figure, _format) {\n",
       "        window.open(figure.canvas.toDataURL());\n",
       "    }\n",
       "\n",
       "    var fig = new mpl.figure(id, ws_proxy, ondownload, element);\n",
       "\n",
       "    // Call onopen now - mpl needs it, as it is assuming we've passed it a real\n",
       "    // web socket which is closed, not our websocket->open comm proxy.\n",
       "    ws_proxy.onopen();\n",
       "\n",
       "    fig.parent_element = element;\n",
       "    fig.cell_info = mpl.find_output_cell(\"<div id='\" + id + \"'></div>\");\n",
       "    if (!fig.cell_info) {\n",
       "        console.error('Failed to find cell for figure', id, fig);\n",
       "        return;\n",
       "    }\n",
       "    fig.cell_info[0].output_area.element.on(\n",
       "        'cleared',\n",
       "        { fig: fig },\n",
       "        fig._remove_fig_handler\n",
       "    );\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_close = function (fig, msg) {\n",
       "    var width = fig.canvas.width / fig.ratio;\n",
       "    fig.cell_info[0].output_area.element.off(\n",
       "        'cleared',\n",
       "        fig._remove_fig_handler\n",
       "    );\n",
       "    fig.resizeObserverInstance.unobserve(fig.canvas_div);\n",
       "\n",
       "    // Update the output cell to use the data from the current canvas.\n",
       "    fig.push_to_output();\n",
       "    var dataURL = fig.canvas.toDataURL();\n",
       "    // Re-enable the keyboard manager in IPython - without this line, in FF,\n",
       "    // the notebook keyboard shortcuts fail.\n",
       "    IPython.keyboard_manager.enable();\n",
       "    fig.parent_element.innerHTML =\n",
       "        '<img src=\"' + dataURL + '\" width=\"' + width + '\">';\n",
       "    fig.close_ws(fig, msg);\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.close_ws = function (fig, msg) {\n",
       "    fig.send_message('closing', msg);\n",
       "    // fig.ws.close()\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.push_to_output = function (_remove_interactive) {\n",
       "    // Turn the data on the canvas into data in the output cell.\n",
       "    var width = this.canvas.width / this.ratio;\n",
       "    var dataURL = this.canvas.toDataURL();\n",
       "    this.cell_info[1]['text/html'] =\n",
       "        '<img src=\"' + dataURL + '\" width=\"' + width + '\">';\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function () {\n",
       "    // Tell IPython that the notebook contents must change.\n",
       "    IPython.notebook.set_dirty(true);\n",
       "    this.send_message('ack', {});\n",
       "    var fig = this;\n",
       "    // Wait a second, then push the new image to the DOM so\n",
       "    // that it is saved nicely (might be nice to debounce this).\n",
       "    setTimeout(function () {\n",
       "        fig.push_to_output();\n",
       "    }, 1000);\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function () {\n",
       "    var fig = this;\n",
       "\n",
       "    var toolbar = document.createElement('div');\n",
       "    toolbar.classList = 'btn-toolbar';\n",
       "    this.root.appendChild(toolbar);\n",
       "\n",
       "    function on_click_closure(name) {\n",
       "        return function (_event) {\n",
       "            return fig.toolbar_button_onclick(name);\n",
       "        };\n",
       "    }\n",
       "\n",
       "    function on_mouseover_closure(tooltip) {\n",
       "        return function (event) {\n",
       "            if (!event.currentTarget.disabled) {\n",
       "                return fig.toolbar_button_onmouseover(tooltip);\n",
       "            }\n",
       "        };\n",
       "    }\n",
       "\n",
       "    fig.buttons = {};\n",
       "    var buttonGroup = document.createElement('div');\n",
       "    buttonGroup.classList = 'btn-group';\n",
       "    var button;\n",
       "    for (var toolbar_ind in mpl.toolbar_items) {\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) {\n",
       "            /* Instead of a spacer, we start a new button group. */\n",
       "            if (buttonGroup.hasChildNodes()) {\n",
       "                toolbar.appendChild(buttonGroup);\n",
       "            }\n",
       "            buttonGroup = document.createElement('div');\n",
       "            buttonGroup.classList = 'btn-group';\n",
       "            continue;\n",
       "        }\n",
       "\n",
       "        button = fig.buttons[name] = document.createElement('button');\n",
       "        button.classList = 'btn btn-default';\n",
       "        button.href = '#';\n",
       "        button.title = name;\n",
       "        button.innerHTML = '<i class=\"fa ' + image + ' fa-lg\"></i>';\n",
       "        button.addEventListener('click', on_click_closure(method_name));\n",
       "        button.addEventListener('mouseover', on_mouseover_closure(tooltip));\n",
       "        buttonGroup.appendChild(button);\n",
       "    }\n",
       "\n",
       "    if (buttonGroup.hasChildNodes()) {\n",
       "        toolbar.appendChild(buttonGroup);\n",
       "    }\n",
       "\n",
       "    // Add the status bar.\n",
       "    var status_bar = document.createElement('span');\n",
       "    status_bar.classList = 'mpl-message pull-right';\n",
       "    toolbar.appendChild(status_bar);\n",
       "    this.message = status_bar;\n",
       "\n",
       "    // Add the close button to the window.\n",
       "    var buttongrp = document.createElement('div');\n",
       "    buttongrp.classList = 'btn-group inline pull-right';\n",
       "    button = document.createElement('button');\n",
       "    button.classList = 'btn btn-mini btn-primary';\n",
       "    button.href = '#';\n",
       "    button.title = 'Stop Interaction';\n",
       "    button.innerHTML = '<i class=\"fa fa-power-off icon-remove icon-large\"></i>';\n",
       "    button.addEventListener('click', function (_evt) {\n",
       "        fig.handle_close(fig, {});\n",
       "    });\n",
       "    button.addEventListener(\n",
       "        'mouseover',\n",
       "        on_mouseover_closure('Stop Interaction')\n",
       "    );\n",
       "    buttongrp.appendChild(button);\n",
       "    var titlebar = this.root.querySelector('.ui-dialog-titlebar');\n",
       "    titlebar.insertBefore(buttongrp, titlebar.firstChild);\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._remove_fig_handler = function (event) {\n",
       "    var fig = event.data.fig;\n",
       "    if (event.target !== this) {\n",
       "        // Ignore bubbled events from children.\n",
       "        return;\n",
       "    }\n",
       "    fig.close_ws(fig, {});\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function (el) {\n",
       "    el.style.boxSizing = 'content-box'; // override notebook setting of border-box.\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function (el) {\n",
       "    // this is important to make the div 'focusable\n",
       "    el.setAttribute('tabindex', 0);\n",
       "    // reach out to IPython and tell the keyboard manager to turn it's self\n",
       "    // off when our div gets focus\n",
       "\n",
       "    // location in version 3\n",
       "    if (IPython.notebook.keyboard_manager) {\n",
       "        IPython.notebook.keyboard_manager.register_events(el);\n",
       "    } else {\n",
       "        // location in version 2\n",
       "        IPython.keyboard_manager.register_events(el);\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function (event, _name) {\n",
       "    var manager = IPython.notebook.keyboard_manager;\n",
       "    if (!manager) {\n",
       "        manager = IPython.keyboard_manager;\n",
       "    }\n",
       "\n",
       "    // Check for shift+enter\n",
       "    if (event.shiftKey && event.which === 13) {\n",
       "        this.canvas_div.blur();\n",
       "        // select the cell after this one\n",
       "        var index = IPython.notebook.find_cell_index(this.cell_info[0]);\n",
       "        IPython.notebook.select(index + 1);\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_save = function (fig, _msg) {\n",
       "    fig.ondownload(fig, null);\n",
       "};\n",
       "\n",
       "mpl.find_output_cell = function (html_output) {\n",
       "    // Return the cell and output element which can be found *uniquely* in the notebook.\n",
       "    // Note - this is a bit hacky, but it is done because the \"notebook_saving.Notebook\"\n",
       "    // IPython event is triggered only after the cells have been serialised, which for\n",
       "    // our purposes (turning an active figure into a static one), is too late.\n",
       "    var cells = IPython.notebook.get_cells();\n",
       "    var ncells = cells.length;\n",
       "    for (var i = 0; i < ncells; i++) {\n",
       "        var cell = cells[i];\n",
       "        if (cell.cell_type === 'code') {\n",
       "            for (var j = 0; j < cell.output_area.outputs.length; j++) {\n",
       "                var data = cell.output_area.outputs[j];\n",
       "                if (data.data) {\n",
       "                    // IPython >= 3 moved mimebundle to data attribute of output\n",
       "                    data = data.data;\n",
       "                }\n",
       "                if (data['text/html'] === html_output) {\n",
       "                    return [cell, data, j];\n",
       "                }\n",
       "            }\n",
       "        }\n",
       "    }\n",
       "};\n",
       "\n",
       "// Register the function which deals with the matplotlib target/channel.\n",
       "// The kernel may be null if the page has been refreshed.\n",
       "if (IPython.notebook.kernel !== null) {\n",
       "    IPython.notebook.kernel.comm_manager.register_target(\n",
       "        'matplotlib',\n",
       "        mpl.mpl_figure_comm\n",
       "    );\n",
       "}\n"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<img src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAoAAAAHgCAYAAAA10dzkAAAgAElEQVR4nOzdeXSj9X0vfvf2Ns1pWifN8qOQ085AWpbBBAIpNCENSRq4DdlIwyXJ7SHcm+2SewpJmpCZhBCHrQmEJWwhLIGBpAEGCo+sxdoXy7JlS5Zt2dZmSZZlWbYsW5bkdYYZv39/GAnZlmxZeuxHtt6vcz7nMJb0+DtmmOfN9/t8P98GEBEREVFdaZB6AERERES0uxgAiYiIiOoMAyARERFRnWEAJCIiIqozDIBEREREdYYBkIiIiKjOMAASERER1RkGQCIiIqI6wwBIREREVGcYAImIiIjqDAMgERERUZ1hACQiIiKqMwyARERERHWGAZCIiIiozjAAEhEREdUZBkAiIiKiOsMASERERFRnGACJiIiI6gwDIBEREVGdYQAkIiIiqjMMgERERER1hgGQiIiIqM4wABIRERHVGQZAIiIiojrDAEhERERUZxgAiYiIiOoMAyARERFRnWEAJCIiIqozDIBEREREdYYBkIiIiKjOMAASERER1RkGQCIiIqI6wwBIREREVGcYAImIiIjqDAMgERERUZ1hACQiIiKqMwyARERERHWGAZCIiIiozjAAEhEREdUZBkAiIiKiOsMASERERFRnGACJiIiI6gwDIBEREVGdYQAkIiIiqjMMgERERER1hgGQiIiIqM4wABIRERHVGQZAIiIiojrDAEhERERUZxgAiYiIiOoMAyARERFRnWEAJCIiIqozDIBEREREdYYBkIiIiKjOMAASERER1RkGQCIiIqI6wwBIREREVGcYAImIiIjqDAMgERERUZ1hACQiIiKqMwyARERERHWGAZCIiIiozjAAEhEREdUZBkAiIiKiOsMASERERFRnGACJiIiI6gwDIBEREVGdYQAkIiIiqjMMgERERER1hgGQiIiIqM4wABIRERHVGQZAIiIiojrDAEhERERUZxgAiYiIiOoMAyARERFRnWEAJCIiIqozDIBEREREdYYBkIiIiKjOMAASERER1RkGQCIiIqI6wwBIREREVGcYAImIiIjqDAMgERERUZ1hACQiIiKqMwyARERERHWGAZCIiIiozjAAEhEREdUZBkAiIiKiOsMASERERFRnGACJiIiI6gwDIBEREVGdYQAkIiIiqjMMgERERER1hgGQiIiIqM4wABIRERHVGQZAIiIiojrDAEhERERUZxgAiYiIiOoMAyARERFRnWEAJCIiIqozDIBEREREdYYBkIiIiKjOMAASERER1RkGQCIiIqI6wwBIREREVGcYAImIiIjqDAMgERERUZ1hACQiojVWVlawsrKCkydP4vXXX8eJEyewvLyMpaUlLCwsYG5uDtlsFktLSzh16pTUwyWiCjAAEhHVkZWVFZw6dSof7I4fP47l5WUsLi5ifn4e2WwW6XQas7OzCIVCCAQCmJmZQSqVwszMTP6fU6kU5ubmsLi4iNdffx0rKytS/9aIaBsYAImI9oFcsMvN2uWCXeGsXSaTwezsLGZnZ4sGulzNzs4inU7D5XKhu7sbmUwG2Wx2Qy0uLubr+PHjnA0k2kMYAImIalix5djjx49vWI7NzdoVBrlcwMuFvFz4S6fTyGQy+SoW7rLZLFwuFxwOR8nXl5aW8iFzcXERS0tLOHnyJGcDifYABkAiIokUztoVW46dm5vLB7vCWbtis3e5YFcY7koFt3Krp6cHTqez5OuLi4tYXl5eEwJzs4EMgUS1jQGQiEhkWy3HTk9PI5lMFl2OXb8kW2zWrtpgt50A2NPTU1YALAyCCwsLWF5e5pIwUQ1jACQi2gYxlmO7u7vR09OzreVYKcrpdG4aAHNLwKVCIDeIENUuBkAiImy+HJsLd6WWY9f/81bLsQ6HAy6XS/KAt1VtNc5SAbDYkjBnA4lqCwMgEe1r65dji/W0W787ttQOWbGWY/dSAOzt7a0oAK4PgtwgQlRbGACJaM8q1tMuF+zW97Qrd3fsbjxrt1WwqpXq7u6uOgBygwhRbWIAJKKaU2o5drNZu9HRUdhstm0vx9ZisKqV6urqQl9fX9UBcP2zgTxBhEh6DIBEtGvKOWIsk8lsmLUrZzk2EolArVZLHpr2UwC02+3o7+8XLQAW/rseHx/H/Pw8ZwOJJMIASESiKLUcW+yIsfUnT4ixHDs6OsoAuAcCYC4EarVajI2NsV0MkUQYAIloU5UsxxYGut1ajo1GowyAIldnZyfcbrfoAXB5eRlarRaxWCy/JMwNIkS7iwGQqE6Vsxw7OTmJeDxedDl2fbCTqllxYQBsbW2VPDSVU1s9W1cr1dHRgYGBgR0JgBqNBvF4nBtEiCTCAEi0DxVbji08Yiyb3bgcW+yIMYfDAYfDUXObKBgAdy8ADg4Olny90vC3vLwMtVqNiYmJohtETp48KfV/QkT7HgMg0R6y1RFjYi/HulyuTc+CraUaGxtjABS5bDbbjgXA1tZWTE5OFt0gsri4iBMnTnA2kGgHMQAS1YBylmOz2eKzdpv1tKt2xq63txcOh0PyIFJOMQCKX+3t7RgaGkI6nUYikUA0GsXw8DAGBwfR09OD4eHhipeBVSoVEonEpj0DuUGEaOcwABLtsO0sx64/YqzYs3a7uRzb29uL7u5uyYNIOTU2NgaVSiX5OMqpWgmAmUwGMzMziMfjGBkZgc/nQ39/P7q7u2G1WiGXyyGXyyEIAmQyGVpbW2E0GmGz2eB0OqFWq9HR0YFsdvuzgUqlEslkcst2MUtLSzxPmGgHMAASVaiS5Viv1wu3270nmhVns1n09fWhq6tL8nGUU3spAG7VXkWMymQySCaTiMViCIVC8Hg86O3tRWdnJ8xmM7RaLVpaWiAIAuRyObRaLSwWC+x2O3p7e+HxeGAwGOByuZBMJov+mcxkMujo6IBarUY8Ht9WAJTL5Zient6yXQw3iBDtDAZAonWKLccWHjFWajl2qyPGMpkM+vr6YLfbJQ8g5VZ/f/+eGW8sFqubADg7O4vJyUmMjo4iEAhgYGAATqcTNpsNRqMRra2tkMlkEAQBSqUSer0eVqsV3d3d6O/vh8/nw8jICCYmJpBKpUr+D0dbWxu8Xm/JcSwtLWFpaQl+vx9yuRyDg4NlLwm3tLRsGQC5QYRo5zAAUl0p1dMutxw7Nze3o8uxbrcbnZ2dkgeQcmsvjTcWi0GpVEo+jnKqVADMLcmOj48jHA7D6/XmZ2Hb2tqg0+mgUCjyS7JqtRomkwkdHR3o6enB4OAghoeHEY1GkUgkkE6nqxqnxWKBz+cr+Xph2JuamoJOp4PFYkE6nd4y1MlkMqRSqbJnDAtnA7lBhKh6DIC0L42Pj0MQhLJ2x251xJiYy7GDg4P5Z6b2QjEAilfpdBpTU1MYGxuD2WxGW1sbXC4XOjs7YTKZoFar80uyCoUCOp0ObW1t+ecFvV4vwuEwxsfHMT09vSuPCVgsFvj9/pKvr5/tm5+fh8PhgFKpRDQa3TTQCYKA2dnZbS0bc4MIkXgYAGlf0mg0OPPMM7fcHbvbz9oNDQ2hvb1d8jBSbg0MDOyZwDo+Pi5ZAEylUpiYmEAkEoHf74fb7YbD4UB7ezsMBgNUKhUEQYAgCFCpVFAqldBqtXA4HHC73fD7/YhEIvklWal/lrkym83bCoC5CofDkMvl6O3tzYe19UFOEISyZgq5QYRoZzAA0r5kNptxxhln1Mxmilx5PB5YrVbJx1FuDQ4OwmazST6OcmonAmAmk8H09DTGx8fXbKSw2+2wWCzQarX5XbItLS3QaDQwm83o7OyEy+XC0NAQgsEgxsbGMDU1lV+S3eqItVopk8mEQCBQ8vXNQloqlYLRaITBYNjwrN/i4iIEQUAmk6koABbOBuaeyWUIJNoeBkDalzo6OvCe97yn5gKgz+eDxWKRfBzl1l6asRwfH4dCoSj7/aV623V0dOSXZMvZSBGPxzEzM7OtP2t7JQAajUYMDw+XfH2rkLa4uIi+vj7I5XKEQqH81xcWFiAIQlnX2KpGRkZgNBq5QYRomxgAaV9yOp1obGysuQDo9/thNpslH0e55fF49lwA3Kq3nV6vh1KpLNnbbmBgAIFAAKOjo5icnMTs7KzoY91LATAYDJZ8vdyQlmvR093djfn5eczPz0MQBMzNzYkSAM1mMzeIEG0TAyDtS4ODg3jrW99acwEwEAjAZDJJPo5yy+v11tSSdbHedrmNFAaDIb8Uu1lvu1AohFgsVrK33W7UXgmABoNBlAC4vLyMdDoNq9UKrVab36Q1Pz9fdQAMh8OwWCz5GceFhQVuECEqAwMg7UvDw8P4oz/6o6rbYIhdwWAQRqNR8nGUW16vF21tbbvyvbbb285gMKC9vT2/kaK/vx8tLS01t5GiWHV2dmJgYEDycWxVer0eoVCo5OuVPLc3NDSUD+liBMBgMIi2trY134MbRIi2xgBI+1I0GkVDQwOmp6clv4kWVigUgl6vl3wc5ZYYzywWbqSoprfd2NjYpr3t4vE45HK55D+zcqqjo2PPBMBwOFzy9UpDWzQahSAI+ccLqgmAw8PDsFqtG4ImTxAh2hwDIO1LU1NTaGhoQCwWk/wmWljhcBg6nU7ycZRbWz2zWNjbLhgMYmhoCC6Xa81Git3qbTcxMcEAKHLpdLodCYDpdBqCIKCzsxOtra0YHx+v+FqBQAA2m63kjGPhCSIMgkRvYgCkfSmTyaChoWHTm5cUNTIyAq1WK/k4yqlUKoWBgQHodLqyetsZDIb8Rgq3273jGynWFwOg+KXVajEyMlL0tWo2cMzOzuYbtQcCAcjlcgwMDBTtGbhV+Xw+dHR0bLrszA0iRBsxANK+tLy8jIaGhk2PsZKiRkdHoVarJR1Dbkm2cCNFqd52MpkMLS0tJXvbJZPJmnnOcmJiAi0tLZKPo5zq6OjA4OCg5OPYqrRaLSKRSNHXqgmAqVQKMpks/+tkMgm9Xg+z2bzt00G8Xi86Ozu3fB83iBCtxQBI+9LJkyfxR3/0R0XPW5WyotEoWltbd+z6s7OzJXvbFdtIsVVvu0AgsGc2reylAGiz2fZEANRoNDsSAKenpyGXy9d8bWFhAU6nEwqFApFIpOxreTwedHV1lfVebhAhehMDIO1Lp06dwlvf+lY4HA7Jb6KFFYvFoFKptv25anrb5TZSBAIBRKNRJBKJspdkg8EgDAaD5D+3cmpycpIBUORSq9UYHR0t+lo1ATCZTG4IgLmKRCJQKBTo6enJz9htVkNDQ+ju7i77e3ODCNEqBkDal1ZWVtDY2FhzTYyLnVZR2NuucCNFZ2cnzGYzNBqNZL3t9tKuZQZA8au1tRXRaLToa9UGQKVSuekSsclkgl6v33CM3PoaGBiAw+HY9hhyQZAbRKheMQDSvrSysoL3vOc9ks9ere9t53Q6IZPJ1vS2W7+RorC3nd/vRyQSkay3XTgc3lMBUCaTST6OcmovBcCxsbGir1UTABOJBFQq1abvWVxcRH9/P+RyOYLBIJaWloq+z+12o6enp6JxcDaQ6hkDIO1LKysreO9731vRcms5VWlvO6vVCkEQ1mykmJqaqpmNFOtrZGRkz7StSSQSeyoADg0NST6OrUqlUu1IAJycnERra2tZ7809NmG324t+z/7+frhcrorHkguC3CBC9YYBkPallZUVnHXWWRUFgnQ6jUQike9tV7iRotrednsppGSze6ttzV762ba3t++JAKhUKkv20qwmAE5MTECtVpf9/kwmA6vVCo1Gg8nJyTWv9fb2ore3t6oAWBgCw+EwlpaWOBtI+x4DIO265uZmNDQ0rKnTTjtN9O9z7rnn4tixY2tuWqlUChMTE4hEIlv2tivcSOF0OjEwMFB1b7upqSkIglBzZxSXqkgkAo1GI/k4yqlEIgFBECQfRzm1lwLg+Ph40deqCYDxeBwajWbbAc3r9aKlpQVerze/JOxyudDf3191AMx9D5lMhpmZGRw/fpyzgbSvMQDSrmtubsb555+PiYmJfE1NTVV93UwmA6fTCUEQ8Nhjj+HAgQO44oorcMkll+DYsWP53nYtLS3QaDQbetsVbqTYqSXZ6elpCIJQs0u+66sW+haWWwyA4pdCodiRADg+Pg6tVlvRZycnJ6HRaNDe3o5MJoOenh643W7RAqAgCJidneUGEdr3GABp1zU3N+PCCy8U/bpHjx7Fu9/9blx44YW4+uqrccYZZ+Daa6/F3XffDbfbne9tJ+Xs28zMDARBkGRDRyW1030Lxazc7KrU4yin9koAlMvliMfjRV+rJgDGYjHodLqKPz83Nwe73Q6VSgWr1YqBgQFRAuDCwkL+zxA3iNB+xwBIu665uRl/9md/htNPPx0HDx7El770JYRCoaqvu/4v6I985CP49a9/LflNtLBSqRQEQcDMzIzkYymnxsbGGAB3oNrb2+HxeCQfx1Yll8sxMTFR9LX5+fmKg9bY2Bj0en3Vs3XBYBAymQwWi6WiY+TW1/z8PARBWBNuuUGE9isGQNp1KpUKr7zyCtxuN3Q6Ha644gqcdtppmJ6eFvX7fOITn8CDDz4o+U20sHJnoE5PT0s+lnJqbGxsx3ZSi1176flKq9W6JwJgS0vLjgTAaDQKg8Egyqxde3s7VCoVTCYTUqlUVdfKZrMQBGFDA+pcCFxcXOQJIrRvMACS5Obn53Haaafh/vvvF/W6n/rUp3DPPfdIfhMtrEwmA0EQkEwmJR9LORWLxaBUKiUfRzmVTCYZAEUumUyGycnJoq9VEwBHR0dhNBpFCYCdnZ355ukKhQIjIyMVXyv332ex2cT1S8KcDaS9jgGQasInP/lJ3HjjjaJe8/Of/zzuvPNOyW+i60sQBCQSCcnHUU4VO7mkVisXAPfCBpu9FABL/VmtJgBGIhGYTCZRAmBHRwd8Pl8+WCoUCjidzrKOkVtfuRn6Uk2nC4MgN4jQXscASJJbXl7Ge9/7Xtx+++2iXve6667DbbfdJvlNtNhNtdSsSq1VPB6HXC6XfBzlFAOg+LXZ/6xUEwBHRkZgNptFCYA2mw2BQGBNiDObzdDr9Ugmk9u6ViqVgkwm2/J93CBC+wEDIO2673//+zCbzQiHw7Db7fjMZz6Dv/iLv0AkEhH1+1x//fX44Q9/KPlNdH1t9mB9rdXExMSeCYB7qcWO1WqF1+uVfBybVW45dGpqqujr1QTAcDgMi8UiSgC0Wq0YHh5e87XFxUUMDAxALpcjEAhsOqNXWNPT05DL5WV/79yzgUtLS1wSpj2HAZB23Ze+9CWcfvrp+JM/+ROcccYZ+Jd/+RcMDQ2J/n2+8Y1v4Lvf/a7kN9L1tVlvtVqriYkJtLS0SD6OcmovBcC2traaC4CBWBJ6dxSB2Orzqel0etPnVasJgMFgEG1tbaIEwLa2NgSDwaKvjY+Po7W1FZ2dnWW1rUkmk1Aqldv6/oUbRE6cOMHZQNozGABp3/p//+//4dvf/rbkN9b1tdnxWrVWk5OTe+Z4tVwArOSElt2uXQ2AmQzm4oFN33O0zY8zjyhw4LACZx5R4GibPx8AS+1YrzYAWq1WUQKgxWJBOBwu+Xo2u3r2slqtxsTExKbXSiQSUKlU2x5D4ZLw8jLbxdDewABI+9a///u/42tf+5rkN/v11drairGxMcnHUU7tpdM1GACL12LPS1i54z1Y1t5V9PVALJkPf7k684gCvtHJHQuAgUAANptNlABoNpu33Pm7tLQEn88HuVyOoaGhkkvC2z2juNRsIDeI0F7AAEj71pEjR3D99ddLfrNfX2q1GqOjo5KPo5zaS731cqesMAAWVGoKpx5oApobsaz6SdH36N3RNeEvV9q+kU2bllcTAP1+v2gB0Gg0IhKJlPXeRCIBrVYLq9WKdDq94fVKzijebDaQG0SoljEA0r7V3NyM6667TvKb/frSaDSIRCKSj6Oc2ku99fbSMXsWiwU+n2/Hv8+y9i6guREn7j0bRpcv/3xfYZWaAfRGJvIBcGhkHC12H+z9Png8HrhcLnR3d2NqaqqikOTz+dDZ2SlKADQYDIhGo2W/f35+Ht3d3VCpVBgbG1vzWrVH1G02G0hUaxgAad+6++67cc0110h+s19fOp0OIyMjko+jnNpLrVUYANfW3LgPK3eeBjQ34js//tGa5/ty78lkMpiZmcFvtO41zwD++HdG/PoVPY6+JODIky04eFiOA4cVOHhYgTv/0wCHwwGHwwG5XF5yA8Zm5fV6YbfbRQlaer1+Q5Arp0KhEORyOfr6+vLP7olxRN36ENjW1oZwOMwNIlRzGABp37rvvvtw9dVXS36zX196vR6hUEjycZRTe+m5utw5ywyAq3Xiha8CzY1w3PZBHHgjwOUC3mtqE7RaLVpaWiAIAhQKBV6Sa/HEa0bc+jsTDr7x3oOH5fl/Lvx8IJbEwsJC/qSY7TZe9ng86OrqEiVkabVaxGKxij47PT0Ng8EAo9GImZkZUU8oyZXRaMTIyAg3iFDNYQCkfeuRRx7BP/3TP0l+s19fBoMBwWBQ8nGUU3tpVq3eAuDs7CwmJiYQiUTg8/nQ39+Prq4utLW1ofu/HgGaG7HS/HZcfeSRDc/3vWRyYWRkBBMTE2t+XsWWg4uVYSCaD3yzs7MwmUwwGo1ln8U7NDSE7u5uUQKWRqPB+Ph4xZ9fXFxEb28vFAoFent7RTuhJFe5JerCJWGeJ0y1gAGQ9q0nn3wSH/nIRyS/2a8vk8mE4eFhycdRTuVCVamNALVUe2msFosFfr+/5Ouzs7OYnJzE6Ogo/H4/3G43HA4HrFYr9Ho9FAoFBEFAS0sLNBoNzGYz7HY7ent74fUMYfmhy4DmRqT+8+tFn+8r9ixgNlt6Q0ipGcDCEOVyuaBUKstajh0cHITD4RAlYJXT3qWcikajkMvlUKlUVW1wWV86nS4/Q8kNIlRLGABp33ruuedw6aWXSn6zX19ms3nTm38tVe5s1L0QqvZKAEyn0zAYDOjt7UUgEMDAwAAcDgfa29thMBigVCohCAJkMhnUajVMJhM6Ozvhcrng8XgQCoUwPj6O6enpoptzlmxPrM7+3f1ezE2Gi/b4KzW2YjOAB9+o9Z8vtuQbDochl8sxMDCw6ekbAwMDcDqdogSs1tZWTE5OinItr9cLhUIBnU5X8QaX9aXRaBCPx9d8jRtEqBYwANK+9dJLL+EDH/iApDf7Yjfo3doBKkblAmCpXnC1VLUQVtPpNKampjA2Nobh4WEMDg7C6XTCZrPBYDBApVJBEIT8c3cmkwkdHR1wuVwYGhpCMBhELBZDMpmsbOd1MoZT95wFNDdiyXhf/uuBWBKGgWjJmb/CKgyMBw/LcbTNX/TzxQJgJJFGa08QL7Zo0N7ejmw2WzQUud1u9PT0iBKwlEolEomEKNcaHh6G1WrF4OAg5HI5/H5/2cfIbRZQi81QFs4GcoMISYEBkPatV199FYcOHZIsDMz09mL4c59H0mRe8/W9cA5sYaDZ7DiwWqqdDoCZTAbJZBKxWAzBYBBDQ0Po6elBR0cHjEYjWltbIZPJIAgCVCoVjEYjbDYbenp6MDg4iGAwiLGxMUxNTcFkMu3ILHDqle8BzY04/sBFyM5WHtoDsSSUjgCOHit9Csz6APg7W2jNTOPdL5igVquLhrP+/n64XC5RQptCoUAymRTlWn6/Hx0dHVheXu0J2Nraio6ODmSzxYNstQGVJ4iQlBgAad9SKpV43/veJ1kgGbnx2/Cccy4855yLkZtuRuqN5/7a29sxNDQkeWAqN/QIgoCpqSnJx7JVVTNbmclkMD09jfHxcYRCoXyvu87OTphMq0EmF+6USiUMBgPa29vhdDoxMDCAQCCAaDSKRCJRVsucnXgM4NVWDU789C+B5kbc8KO7N13qXV/rzwHOZlePAcydA13s9cIAGEmkiz5r2OYcgFwuRyAQWDOT1tfXh97eXlFCm1wux/T0tCjXWt+fMJvNoqOjA2q1esMyrpgBlRtESAoMgLRvGQwG/PVf/7VkTYzTY2MYvfUn8Jx3CJ5zzoX3og8g9sgjsJnNGBwclDwwlVuCICCRSEg+ji1/3iXOrs31uhsfH0c4HIbX60Vvby/sdjvMZjM0Gs2adih6vR5WqxUOhwNutxuBQACjo6OYnJwUrR2O2WxGILD5+bzbqcDYFEy3/SPQ3AjdT67YcrNHYZV6RnBiYgItLS0lXy8MgGZvvOiGEYs3jng8DpVKhe7u7vxnent70dfXJ0pok8lkmJmZEeVaxdrTLC0twe/3Qy6XY3BwcNtLwi0tLWWNjxtEaLcxANK+1d7ejtNOO03yUyymHU4Er/tSfjZw4IorMPD885IHpnJLJpPVdABMpVKIx+MIhUIQBAE9PT3o6uqCxWLZ0OtOp9PBarWiu7sb/f398Pv9iEQiooa7cspkMokaAHs1z68u/f70L3HFkafWtGspNnuXza7O6h2zB0v2+YvH43j+FXnJXcQLCwvoDiZwj3IQv9J4ir4vklg9bi2dTsNisUCv12NmZgYulwv9/f2ihLZc6x8xrrVZe5qpqSnodDpYLJaix8iVCnW5PprljoEbRGi3MADSvtXd3Y13vOMdkgfAbHZ1FmrixZfg+/Dl+SAY/ta3kNoDzwLKZDJMTk5K8r0363Wn0+kgl8shCALkcjm0Wi0EQUBHRwf6+vrg8/mK9rqrhRI1AKaSOH7/BUBzIx6/9V/XBLCHNENFZ+8KZ/VK9fmLx+N44PelX//OHxwbvn6w4Hv9zhZaE2wWFxfR19cHhUIBq9UKt9tddWDLBaxyA9lWtdXu5Pn5eTgcDiiVyrKOn1tcXMwfpbjd3xc3iNBOYwCkfcvtduNtb3tbTQTAXKXjcfR997sYOnT+ahC84P0Y++V9SNfwM3YtLS2YmJgQ/bpV9brzehEOhxGPxzEzM4NMJrOnNqyIGQCX9b8AmhuxcPeZaDryckH4Gyw6K2fzxTYNf2ceUeCYPQhdTwC3P6vYMEN44LACR17uKfn5Z9uG8aojkrU/mAYAACAASURBVJ/9W1+RSAQymQxmszm/+aHSqjRglar+/v6ydifn2t309vZu+nuYn5+HIAir/54q/P3lls25QYTExgBI+5bf78d//+//vaYCYDabhcPhQJ9CgdD1X83PBvo/9nEkZLKaG2s2m4VcLkc8Ht/WZ9LpNBKJBKLRaFm97sxm87Z63RWr3IaVegqAc/FhrNx1OtDciMXO365p11KqqfOjOk/J8FYs7JX7tfWvFZsFzFVHRwdUKhXa2tqqCm/VBqz1tZ3NKTMzMzAajflj5Iq9J5tdfYa2msbS3CBCO4UBkPatkZERNDQ01Nw5tk6nEy6XC5lMBpOCAP8VH8sHwdBXb8CM2y35GAtLoVBgfHw8/+tye93JZDK0trZu6HUXCoWq63VXovZSADQajaKcBnPi2NeB5kacfPyjyGbW7j6+q6W/6OyezRcrGtyue9y65Skg263C5wALq6urCwMDA+jq6irZJ6+cmpubqzpgFZbL5drW5pTcsrZcLkcotDHs5v5MVjvTuX6DCJEYGABp35qYmEBDQ8OOLF9WUz09PXA6nW8GqqkpjN37S3gueP9qEDx0PqJ33YW0BOMu1uuupaUFFotlW73uxA535Y59r7SsESMAzvstQHMj0NyIef/aXpMPaQaLBrJvPNuJ56x+0YPeZmXxvtk+JZJIw+yNQ2mywev1rtlh6/P5tr3DNptdnWEr1pS6kurp6ano2cSxsTEolUp0d3evCaO51kTVNpMuDJzLy8tS/9VK+wQDIO1bqVQKDQ0NiEQikt/wC8vlcsHhcGz4esrjRfib38rPBvou/wgmXzomWpCqtNddS0sLnE7ntnvdSVF1EwAzaZz8zRVAcyNOHPv6mtcCseSmS7S7WYUzgIWNog8eVuDOlzth9sYRSaQxMTGB1tZW2O32bc3m5Z77rHaGLVcOhwMDAwMVfTadTqOtrQ1arTbf+HlmZgYtLS2ijI0BkMTGAEj71sLCAhoaGkRttyFG9fX1oaurq+TridZWBD55ZT4IBr/8ZUwXzBgWq+30ulMqlSV73RULdyqVCmNjY5L/3MqpvdKzsNoAuNj5zOp5v3edjrn42uuUevZvd6oFB350LP/rx/S+/MxfqY0nuWcFs9nVU3J0Ol3ZjZ3FnmHr7u7G0NBQxZ9fWlrKz5r7fD5MTU1BoVAwAFJNYgCkfevkyZNoaGjAwMCA5Df8wurv74fdbt/0PZnpacQeehjeCy9aDYLnHULw8GGE3W54vd58iNyNXnetra2IRqOS/9zKqb0UAIPBYGWfn47j1L1/CzQ3Yln/8w2vSzYD+KNXcO4j1+K8x/8ZB468hkffCH/Ly6UbRa+fKVxaWoLb7YZcLsfIyMiWgSiVSkEmk4kWsOx2O7xeb9XXmZiYyG9uEjsA8hlAEgsDIO1bp06dwlve8ha4XC7Jb/iF5Xa70dnZmf/1Zr3ujMeOoetLBU2kL7kEXXfdja7Ozl3rdadWqzE6Oir5z62cqvWm1bkyGAwVB8BlxZHVps/3XwBDb7DoaR+lngHcsaXe257BoSc+hqajTTj/mYvwvtsfze8AjiTSeNUR2TKUFj4rGI1GoVAotmyzMj09LeoSa2dnJ3w+nyjXmpubQ1tbGwRBwPj4OAMg1RwGQNq3Tp06hT//8z9fE7Z2u4r1ujMajVCpVEV73VkslqK97ia0WgQ+dXU+CA5/4V8wvUu/LwZA8avSADg32o+VO94NNDfi6z++PT97Vuzc34c0Q7sS/s6685c4/7eXoOloEw49dTnObP5Nflw/+a++fHPog4c3bx/zqN6X3yQSSaSRSqVgMBhgNptLNnpOJpOizrDZbDYEAgHRrjc+Pg6lUgm5XI6BgQFR+h4yAJJYGABp31pZWcE73/lOmM3mqm/YxarSXncWiwU6nW7bve4yqRTGn3gS3osvyQfByC23YHaHN7loNJqa20hTqqQ8tWQ7VWkAfP25LwLNjTDf9hEcOCxfs4S6fibwIc1QWUvBl92tqzD8yfF39/47zn/2AjQdbcK5v/4MDv74hS2Xel9zRnDTk9oNrx184/Xc+35nC2FhYQEOhwMqlaroLNrU1BSUSqVogc1qtWJ4eFi068ViMeh0OiSTyfwxcts5Fm59LSwsMACSaBgASTKPPfYYDh48iD/90z/FxRdfjLa2NlGvv7KygtNPPx0ajWbbN1qxet0VC3cejwdWq7Xi8DA7MoLID27Jh0DvJR/E+JNPIbNDS8BarRYjIyOSh6Zyaj8HwAV3C9DciFM/eyc+ceSJDQGqsAH0XbK+nZ35O/JfOOfhr6DpaBOajjbh7Ae/gQOHhbI+a/HG8firhq2XlQt2EAeDQcjlcng8njUbPiYnJ6FSqUQLbG1tbUX7+VVa0WgUBoMhH96cTieUSiVGR0crDoAnTpwQ9e9Jql8MgCSJF198EX/yJ3+Cp556Ch6PB9/5znfwtre9DaOjo6J9j5WVFRw4cAByuXzNjbRYr7uenh50dHTsSq87n88Hi8VSdYhI2mwYvuaaN5eFP/0ZJE3iz3bqdDqEw2HJQ1M51dLSsmcCYCgUKv8zszM4+dDFQHMjUi9/d8OO2oOHFThyrPTxbGLWwZ88h/N+88nV5/2evRB/+/Oflv3ZXKh7tdW46XF0hWExF34SiQQ0Gg06OjowNzeH5eU3N1uIFdjMZjPC4bBo14tEIjCZTGu+NjIyAoVCAZfLte3+hQyAJCYGQJLEpZdeihtvvHHN184991wcOXKkquuurKwgkUjA6XTitddew8GDB3HNNdfgyiuvhFqtXtPrTqVS5XvdOZ1ODAwMYHh4eMd73fn9ftGWpTOzs4gffQ6+Sy/LB8GRm25GSoRTJnKl1+u3F1YkrJ06t1js2u7PdMl0/+rs3y/ORDAUQvOrvZLs9D3r9gdx/tOXrj7v9/Q/4KyfPbqtzz/2xnN+v35Fj5+/5ljTF3D976fYKSLZbBY2mw0ajQZTU1OIx+PQaDSiBTaTyYRIJCLa9cLhMMxm84avp1IpmEwmGAyGslveMACS2BgAadcdP34cf/zHf4xXX311zddvvvlmfPSjH63q2k8++SQaGhrwrne9CxdeeCHe9a534aqrrsLNN9+M3t7ekr3udrMCgQBMJpOo10yPjWH01p/Ac96h1WXhiz6A2COPIDM9XfW1tz1bJWHtxwA4NxnGyt3vBZobYXvhnk1D086VHH97z+H8837nPf7POHjrf27rGo/qfWuaQZ95ZPVrFm8c3cEEbpe5yzpHONdrTy6Xo7+/H1qtVrTAZjAYEI1GRbteKBRCW1tb0dcWFxfR398PuVyOYDBYVi/DxcVFvP7661X9HUmUwwBIu258fBwNDQ2w2Wxrvn733Xfj7LPPruraCwsLWFhYyP/6kksuwdGjRyW/4RdWMBiE0WjckWtPO5wIXvdm25jAlVdhSr39ZyALq5qWJbtdcrl83wXA46/cCDQ3YumRD+F9R1p2fdbvwJHXcM6vrs8/73fOr76KA0de29Y1uoOJos2gzzyyOitYGArvkLk3zPwV7g7OfS0Wi0Eul0OhUIh2FJxer8fY2JhoAXB4eBjt7e2bvicWi0GlUsFut+eXthkAaTcwANKuywXAjo6ONV+/6667cM4554j6vT70oQ/hiSeekPyGX1ihUAh6vX7Hrp/JZDDx4kvwffjyfBAMf+tbSHm9FV1PjHNrd6v2WwCcH7ZhpfntQHMjHMbyNlmIWQdv/U+c9/g/v/G83/vxt784gsLdx+XWa85IyWbQB4uEwsKgt37WsHBmMBgMoqWlBUajEalUqurAptVqRevZt7y8DL/fj46Oji3fl8lkYLVaodFoMDk5yQBIu4IBkHbdTi4Br/exj30MDz/8sOQ3/MIKh8PQ6XQ7/n3S8Tiid9wJz6HzV4PgBe/H2C/vQ3qbZ+WaTKaaO06vVMnlcsTjccnHsVXp9fqtN9ZkMnj9iX9aPe/3hesRiCXL2jghVp31s0dx6Ol/WA1/T1+Ks27/VcXXes0ZKToDWGoJO7f5o9SsYS4gjo6OwmAwwOVyQalUVj17p9FoEI/Hq7pGYXm9Xtjt9rLeu7S0BI/Hg5aWFni93qJLwgsLCwyAJBoGQJLEpZdeim9/+9trvnbeeedVvQlkvauuugr33Xef5Df8wopEItBqtbv2/Wb6+xG6/qv52UD/xz6OhExW9g5ms9kMv39jo+FarL0SAMvZWb3Y9fzqeb93noa5cR+y2SyOtvk3fQbw4BEFfvZab9Xh729/fhvOf/bC1ef9fvNJHPzJ81VdLxfYij0DuFnAKzVrmAuIIyMj+V224XA433C50rOBW1tbN52B224NDQ2hq6trW5+ZnJyERqNBe3s7MpkMAyDtGAZAkkSuDcxvf/tbeDwefPe738Xb3vY2RCIRUb/PZz/7Wdx9992S3/ALa3R0FGq1ele/ZyaTwaQgwH/Fx/JBMPTVGzDjdm/52b0UABUKxf4IgDOTOPXLs1fP+9Xeuea1QCwJw0AUx+zBouHIMBDFN57trDCsCTj7wa+/+bzfw1/BgSOvVhX+Cs8Ezs3qPfSCCn3+SNFQWLjEu9UMYDgchsViyb8/mUxCq9Xm+2xuN7CpVCokEgnRAmCuOfx2Pzc3Nwe73Q6VSoVYLJb/OpeASUwMgCSZxx57DAcOHMBb3vIWXHzxxbBYLKJ/j2uvvRbNzc2S3/ALKxqNorW1VZLvnZ6awti9v4TngvevBsFD5yN6111Ib/LcnMVigc/nk/znVk7tlwC43HrbatuX+89Hdqb40XbFloTPPKKAzReraKn44I9fwLm//swbz/tdgL+79/uo5Hm/wtnIx9aFv8KZtomJiTVBz7Juk0euNguIxXbZzs3NobOzE2q1etthTqFQIJlMihYA3W43enp6Kvrs0tJSvgF2f38/FhcXsbi4iJMnT4r+9yTVJwZA2tf+9V//FT/60Y8kv+EXVm7Xn5RjSHm8CH/zW/nZQN/lH8HkS8eKLgu3tbXBW+EGkt0uhUKB8fFxycexVel0upKnq8yNDWLljvcAzY1Y7Hlx0+sULgnnzgTWu6PbDmtnNv8Gh5768Gr4++0lOOvO+6peRn7NWbqfXrGZtmI7fbcKiMPDw7BarUXDk9frhVwuRyAQKHtJWC6Xb6sv31bV19eH3t7eqq4xPT0NvV4Pk8mEmZkZBkASDQMg7Wtf+9rX8P3vf1/yG35hjY+PQ6FQSD6ObDaLRGsrAp+8Mh8Eg1/+MqadzjXvsVqt8Hg8ko+1nFIqlXs+AJ74/ZeA5ka8/ttPIVvGc5q5JeHcWcA2X2xb/QHfd/cdOP+Zi1af93vi4zh427NVh78Dh0vP/kUSaTzwnwoMhN5c2txslm+zCgQCsNlsJV+Px+NQqVTo7u4uq1WMTCYTZTdxrlwuF/r7+6u+zsLCAnp6emA2mxkASTQMgLSv3Xjjjfi3f/s3yW/4hRWPxzccTydlZaanEXvoYXgvvGg1CJ53CKM/uQ3psTFks1m0t7djaGhI8nGWU3slAJY6X3lhsHV148fP/hLz4e5tX7dwRnDrkuHs+2/MP+937iPX4sCPXhEl/OWC3DNtw3jVESm5CeR3ttCWz/ltVuW0WUmn07BYLNDr9ZiZmSn5vqWlJQiCgHR66+9bbuVOGBLjWktLS8hkMgyAJBoGQNrXbr75Znzzm9+U/IZfWBMTE2hpaZF8HOsrFRjGyL/d9Oay8GX/gPhzz8Nmte6pABiLxSQfx1ZVNACmUzj5yN8DzY04/trN275msWcCS84E/vglnPvoNfnw93f33YQDh3euyfTBw2sbPhcGvVcdkaKfKTwHuFR5vV50dnZu+b7FxUX09fVBoVBgdHS05HsEQdiw87aa6u7uxuDgoGgBcGFhgQGQRMMASPvaLbfcghtuuEHyG35hJRIJyGQyycdRqqaMRgQ+dXU+CLr/xz9j8NVXJR9XObWXA+CS5aHVjR8/P4BsYnTb1yz32b8zf/o0Dj350dXn/Z65GO+76+c7FvzWhMASM5OvOSMVzwB6PJ6y++wtLy8jEolAoVDkN1UUvjY/Pw9BELY8jWM7Zbfb4fF4RAuAi4uLOHXqlNR/rdI+wQBI+9ptt92Gr3zlK5Lf8AtramoqP9Mg9VhKVSaVwvgTT8J78SX5IBj5wS2YjUQkH9tmlWubIfU4tiqtVotI4c8yMYqVn//16pFvbY9WdM1yGkW/766f4/xnLkbT0SYcevKjOPOnT+9K+Cs1I5kLepU+Azg0NITu7u5tBancpoq2trY1s31zc3MQBAHz8/OiBcDOzk74fD4GQKpJDIC0r91555344he/KPkNv7Cmp6fzzxpJPZatanZkBH3f+EY+BHov+SDGn3wKmVRK8rEVq70aAI+/ehPQ3IiTj1yKbLr0z3Z2dhaTk5OIRCLw+Xzo6+uD3W6HxWKBRqPB/3mo1DJuC/7uvpvefN7vsWtw4Mcv7Wr4W9/4uVjPv1KtYEpVpX325ufn0dXVtaYdTSaTgSAIG2YGqymbzYZAIMAASDWJAZD2tXvvvRef+cxnJL/hF9bMzAwEQcDs7KzkYymn7HY73C+/jOFrrskHwcDVn8aUyST52NaXSqXC2BubV2q5NBpNPgDOh+xY+dk7gOZGpPtW29iEQiEMDQ2hp6cHNpsNBoMBKpUKgiBAJpNBo9HAYrGgq6sLfX198Pl8iEQi6B+Obnzu70ev4NxHrs2Hv7PvvxEHDst2NfwdOPxmQ+iRyVn88jkZ/NHqGy673W44nc6KA5Xf74dcLofP58Ps7CwEQaj4FJFiZbVaEQwGGQCpJjEA0r728MMP46qrrpL8hl9YqVQKgiBgZmZG8rGUU3a7Hf39/cjMziJ+9Dn4Lr0sHwRHbroZqeFhyceYq9bW1poNgOl0GolEAqOjo1AqlbDb7eju6sLsg5cDzY0Yf/CTEAQBSqUSer0e7e3tcDqdGBwcRDAYRCwWQzKZ3PLRgTXHxd32LM574uNvPO93Ed73H7fv+LJuqcpt6hBzs0V/fz9cLldV15icnIRarUZ7ezsEQRAt/C0vL8NisSAcDjMAUk1iAKR97fHHH8cVV1wh+c2/sHIzDdPT05KPpZzKzTLlg8zYGEZv/Qk85x1aXRa+6AOIPfwIMjXw+5EyAKZSKcTjcYTDYXg8HrhcLnR0dMBoNKK1tTU/e6dWq9HS0rIaDhQPrm78uOM9SAScos0K23wx3CL/PS46+verz/s9dTnObP7Njizr3i5zl/Xe7uDqjN/CwgIEQUA2u/2j2tZXb29v1Y2Wl5eXkc1mYTKZ8v9dihUATSYTRkZGRA2AKysrUv+1SvsEAyDta8888ww+9KEPSR5MCiv3rFEymZR8LOVUd3c3ent7N3x92uFE8LovvbksfOWVmFJrJB1ra2srotHojvw7m5qaQjQaRSAQgNvthsPhgNVqhU6ng1wuhyAIkMvl0Ol0sFqtcDgcGBgYQCAQwNjYGKampvKzdxqNBtGQH6fuP7R63m/rT0Ub67MWH86+9/s4/9kLVp/3+/WncfDHL2wazr76lG3b4e8e1SAiiXTRPn6bzQCKudvW5XKhr69PlIA1NTUFmUwGuVwuWmgzGo0l284wAJLUGABpX/vDH/6Aiy++WPIQtb4EQUAiUfyM11orh8MBl8tVMhhNvPgSfB++PB8Ew9/8FlIeaY6OqzQAzs7OYmJiAiMjI/B6vejt7UVnZyfMZjPUajVkMhkEQUBraytMJhM6Ojrgcrng8XgQDocRj8eR2sbGGLVajVnhyOrs333nljzvd7vlHo3i3Ie/8ubzfg9+HQePCFuGs7ta+rd1esiBwwr88NibS6/f+UPPljOFuc0dYu627enpEeWkjeXlZSQSCahUKkSjUSgUCvT29la9IUSv12NsbIwBkGoSAyDta6+88gqampokD1HrSyaTYXJyUvJxlFNOpxM9PT2bvicdjyN6x53wHDp/NQhe8H6M/fI+pKemdnWsarV6QwDMZDJIJpOIxWIIBoMYHByE0+lEe3s79Ho9FAoFBEFAS0sLtFot2tra0N3dDbfbDb/fj9HRUSQSCVF3bZtlv8epO/6/1fN+u38vyjV9Ez5c9dJnV5/3e/ZC/O3Pb8sHsJftIbxsD2075G32jN/Bw4qSM4AHj7z5uYOHFfjO4/L8TFg2u/o/QOUczbZViXnSxuTkJFpbW7G8vIxUKgWDwQCz2VzVySBarRbj4+OijG9xcZEBkETFAEj7mlwux9lnny15iFpfcrkcExMTko+jnOrp6dkyAOZqpr8foeu/mp8N9F/xMSQE2Y72PCxsjaJQKGC329e0RmlpaYEgCFCpVDAajbDZbOjp6cHQ0BBCoRDGx8cxMzOzq30Z4w//8+p5v09dWdZ5v1uVMWjE5X+4fDX8Pf0POOtnj66ZfcudE7y9o+LKW9Y1e+NFXzt8zJVv/nzmEQV+9JQcQ0ND+WdgxWi3IuZJG/F4HBqNJv/rhYUFOBwOqFSqikOcWq3Ot5lhAKRawwBI+5pOp8PBgwdrrumyQqHYE2fWZrOrAdDpdJb9/kwmg0lBgP+Kj+WDYOirN2DG7d72985kMpieni67NUpLSwva2trWtEaZnJysqZY7Cx7tG+f9vgPzwc6qrpXJZPCU6ylc+NyFaDrahGuFa/Gw0bam197RNv+azwRiSTyq84iyAaTUDOCZR4o3fX5RpobNZhOt3UpXVxeGhoZECVjj4+PQarUbvh4MBiGXy+HxeLY95tbWVkxOTooWAJeWlhgASTQMgLSvWSwWnH766TUXAPfKkWXZbBYulwsOh2Pbn0tPTWHs3l/Cc8H7V4PgofMRvfMupAtmPgtbo/j9fvT396O7uxttbW3QarX52btyW6Oo1WqMjm7/GLVdq/QsTj76D0BzI+Ze+HpV10rOJvFD4w/zz/v9wPgDJGdXZ/oCsSQMA9H8zN/6KufUkNzybbEl4IOH1zZxXn+SR6mdwYaBaH637ezsbNWhyG63w+v1ihKwxsbGoNfri76WSCSg0WjQ0dGxrc0rSqUSU1NTogZAIrEwANK+Zrfb8a53vavmAmAt96tbX729veju7q7os6lUCmNOJ3w3/O/8bODgZZeh4xe/QGvB7J1arYbZbIbdbkdvby+8Xi9GRkYwMTGxrdm7wgbLtVhL7b8Gmhtx4o7TEfP3VXydUCKE/yn7n2g62oT3H30/nux5ctt/xguXg888osBDmkE0v9abD3xnHlHg5mdMRYPcy/bQhtmwwpM8Ss0KRhJpJJPJfKiPx+NVhSIxj1qLRqMwGAwlX89ms7DZbNBoNGWHOrlcLlpbGQZAEhsDIO1rvb29+PM///OaC4A1P1NVUH19fejq6trw9e22RrE/9msMfvSKN58P/OK1iNtsov67qekAOBXFqV8cBJob4Xnmporb1dhGbPjoCx9F09EmfPgPH4YhaKh4TMVmCnNfU1vs0LY7iga5QCy55XJoqfN9p6enIZfLMTw8DLlcXtVRaTabDX6/X5SAFYlEYDKZNn3P0tIShoaGIJfLyzrhQyaTIZVKMQBSTWIApH3N4/HgLW95S80FwJoOKm9UrjVKZ2cnTCaTKK1RMtPTiD30MLwXXrQaBM87hNGf3Ia0SLOh68/YraU6Lvve6nm/D38QaqW8ogD4u/7f4aLnLkLT0SZ8/tXPwzuxc+12cv0f188U5p4pLOd5uGLn+yaTSSgUCiwvr268UKlU6OnpqWhTSHt7O4aHh0UJWOFwGBaLpaz3xmIxKJVKOJ3OkruZl5aW8md+ixUAl5eXpf4rlfYRBkDa10KhEBoaGkRt4SFG6XQ6jIyMSPb9t9MaRalUQqVSidoaJRUYxsi/3ZSfDfRd9g+IP/c8MlVu1qjVADg/4sTKz/4SaG7Eglux7UcAZtIzuM1yW/55v5t0N2EytbNthApPgCk2U1jpJo6pqSkolcr8r2dnZ2E0GmGxWJDNbu90EDHP2g2FQmhrayv7/bOzszCZTDAajUVn+cQ88YQBkHYCAyDta+Pj42hoaKi5Uzf0ej1CodCOXb+wNYrP50NfX1/FrVHcbjc6O6vbrVqqpoxGBD51dT4IDl/zBSQ7Oiq+nlarlTRYF61MBq8/85nVZ/+e/5/IZrNQqVRlB8DIVAT/Kv9XNB1twgVHL8DD3Q/vyoz2+iMA11elATDXcHl9WLLb7VCr1dvaNGGxWBAKhSoax/oaHh5Ge3v7tkOZy+WCUqnc0PBZzBNPct/r+PHjUv+VSvsIAyDta9PT02hoaKi5DRcGgwHBYLCiz263NYrFYsnfzCtpjTIwMICOKkLZlr+fVArjTzwJ78WX5INg5Ae3YLaCmbxaDICLrpdX277c8W7MRd3IZssPgN2j3fjES59A09EmXPr7S6Hyq3Zt3Ha7Hf39/SVfrzQAFjZcLqzC5+sikUhZ1zKbzaId2xYIBGCz2Sr6bDgchlwux8DAQP7nks2K1/CaAZB2AgMg7WvZbBYNDQ07OttWSZlMJgwPDxd9TezWKNXW4OAgbDbbjv9MZkdGEPnBLfkQ6L3kgxh/8ilktnHEmtRL6xsqNYVTD1ywet6v8sf5r5cTAF8efBkXP38xmo424epXroZ73L2rY+/s7IR7k96NlQbAiYkJqNXqkq/njmIrDFOlymQylR0Wtyqfz4fOzs6KP59MJqHVamG1WvP/HYvV75ABkHYCAyDtaydOnEBDQwM8Ho/0YeCNyh0z5XQ64fF44HK50NHRAaPRiNbW1rJao2zn3Nlqa2hoaFcCYK6SNhuGr7kmHwQDV38aUyZTWZ+ttQC4rPuP1fN+f/l3yE6/2f9wswCYSqdwt+3u/PN+32z9Jsand79p+GYBMJPJVBxk1p+4Uaymp6eh1WrR0dGx6ZnBRqMxf8RcteX1emG326u6xvz8PDo7O6FWqxGJRCCTyUQZGwMg7QQGQNrXTp06hT/+4z/e9FkmMavc1ii53bNWqxUOhwMDj4+hvgAAIABJREFUAwMIBAIYGxvD1NRUTe1aHhoaQnt7+65+z8zsLOJHn4Pv0svyQXDkppuRKjFrmiudTodwOCz5zyybzWJu3I+Vu/5q9bxf+7NrXlOpVEUbgcemY/g/qv+TD3/3dt6L2Yw0p5h0dHRgYGCg5OuVBplSJ26sr2w2C6vVCr1eX7KVil6v3/DsXaU1NDSE7u7uqq+ztLQEr9eLlpYWyGQy0WYAFxYWGABJVAyAtK+dOnUKf/Znf1a0j10llWuNMjIyAq/XW3FrFKvVCq9351p4iFkejye/rLXblR4bw+itP4HnvEOry8IXfQCxhx9BZnq66Pv1en3NBMATL/7v1bYvv/n4hvN+i50E0zvWi6tevgpNR5vwwd99EIJHkHT8NpsNg4ODJV+vNMjEYjHodLqyZ716e3vzP6/1r+t0uqJfr6QGBgbgcDhEudby8uqmEkEQ0N3dLcpzgAsLCzhx4oTUf6XSPsIASDXlwIEDaGhoWFOHDx+u+HorKyt4xzveUVaA2U5rFK1Wi7a2topbo7S3t2NoaEjykFJOeb1etLW1STqGaYcTweu+9Oay8JVXYkqt2fC+WgmA8z4T0NwINDdiPrDxz976ANjibcHf/+7v0XS0CVceuxI90R7Jfw82m23TP6OVBpnNjlwrVaFQCHK5HD6fb82Mmlarxfj4uCiBze12o6enR7QAmEgkoFQqYbFYoNfrMTMzwwBINYUBkGrKgQMHcMcdd2BiYiJfc3NzFV9vZWUFp512GvR6PSYnJ0VrjSLGzXWz2ZVaKp/Pl+/RJmVlMhlMvPgSfB++PB8Ew9/8FlKeN2dSd7q9TlmVSePkr/9xte3LsW8iEEtC717bQy8XANOZNO63359f8r1BeQOiycpOCBG7tvqflEqDzFZHrpWq3O5hh8OR74mnVqsxMTEhSmDr6+uDy+USLQDmNrssLi6ir68PCoWiqucVGQBJbAyAVFMOHDiABx98sKpreDwe/OpXv8L3vvc9fPGLX0RjYyPe/va347/9t/+GV155RZTWKNXWVs9X1VL5/f6aCIC5SsfjiN5xJzyHzl8Nghe8H2O/vA/pqamaCICLHU+ttn25+wy8qLMVPUVDqVTCH/HjRvWN+fB3h/UOpNK7t7lnq7JarZtunqo0yIyOjsJoNFb02XQ6DZPJBJPJhHQ6jdbWVkxOTooS2Hp7e9HX1ydaAFz/rGMkEoFCoUB/f39Fp54sLi7i9ddfF+lvWiIGQKoxBw4cwF/91V/hne98Jy688ELcdddd237wWSaT4fOf/zxuuukm3Hffffibv/kb3HPPPRgcHKyZE0G26rFWS+X3+2E2myUfx/qa6e9H6Pqvvnm28BUfg+2++yvur1hOFZvNW1PJcZy6531AcyOmVP9R8hzdo7KjuPrlq9F0tAkXP38xXhx4UfKf5/ra6jnVSoNROWfubjUT1t3djdbWVigUCiQSCVECW09PD/r7+0ULgMWWuqenp6HX62GxWLa9k5oBkMTGAEg15YEHHoDZbEZ/fz+eeuopvPvd78bXv/71qq556NAhvPhibd1gtzploZYqEAjAVGYblt2uTCaDSUGA/4qPvdk/8Mtfwcwm/euK/h63CnbZbMkzcQvruPyW1bYvv7oIhr7QmvCXq0fa/wsffO6DaDrahI+/+HHYI3bJf47Fqq2tDT6fr+hr1ZxuEQ6HYTabqwpXuZ22giBgaGhIlMDmdDoxMDAgWgAsNdM5Pz+Prq4utLa2bmv5emFhgQGQRMUASDuuubl5w8aO9eVwOIp+9pVXXkFDQwOmp6cr/v4XXXQRnn/+eclvqIXlcDjQ29sr+TjKqeHhYRiNRsnHsVmlp6Ywdu8vMXR+02oQPHQ+onfehfTExJafLSfYBWLJkrN5uffMRfqwcvu7Vs/77XutyGfkOPveH+CCoxeg6WgTrnvtOoxM1U7PwvVlsVh2JACGQiFYLBZRQlZLSwvkcjn6+vqqbrfS3d0tWphcXl7GyMhIyZnOpaUl+P3+ohtbGABptzAA0o5LJpPwer2b1tLSUtHPxmIxNDQ0wG63V/z9L7vsMjz99NOS31ALy+l0wuVyST6OcioYDMJgMEg+jrJCy0vH4P3qDfnZQN/lH4Hv6H9C3z8Kmy+2YZavnGCXzWZxzB4sOptnGHhzw8brR68Bmhvx+tFr8l/Lh8sjr+Lch/9X/nm/r73wNYyM1W74y2azMJvN8Ps3huFstroAGAwGYbVaRQuAY2Nj0Ol0aG9vr2pcXV1d8Hg8ogXAcDi8ZdCdnJyEWq2G3W7ftOE1AyDtBAZAqmlyuRwNDQ0YHR2t+Bof/ehH8dhjj0l+Qy2snp4eOJ1OycdRToVCoT0TAI1GI4LBIBKtrQh88sp8EDz24U/h4//3iQ2zfHp3dMtgd7TNj4NF3nPgsAIPaVZ3yS70vba68eP2d2EusnZpv314AP/jpc+h6WgTLnzuQjzX9xwUCgXi8bjkP6/Nymw2IxAIFH2tmqA1PDyM9vZ2UUKWIAhIpVKYm5uDzWaDVqvF9PR0Rdfq7OyEz+cTLQCWG3Sz2Wy+UfxmY19cXMTJkydF/NuV6h0DINWMjo4OPPDAA+jt7UU4HMZLL72EM844A5/73Oequu4nP/lJPPDAA5LfUAvL5XLB4XBIPo5yKhQKQa/XSz6OrSoQS+Kxl3Xo6FvdueoPj+OHX/geXIcugOecczFwznn4xT9/Ded/71h+lq/UDGButtDmi214fcNs4eg4Tv3qIqC5Ecflt6wZkylkwuV/uBxNR5vwjy/8I9rCq/0U90IANJlMOxIAA4EAbDZb1QFraWkJgiAgnU7nf+12u6FQKCo6HcRms8Hv94sWALfz+8yNXS6XIxwOMwDSrmAApJrR09ODyy67DG9/+9vx1re+Feeccw6am5uxsLBQ1XU//elP4xe/+IXkN9TC6uvrE+10kp2ucDgMnU4n+Tg2q2LP8eWWbS+56fd4/BNfzs8Gdl5wCf7vl34CQ9/qEuxNv+9eE+o+97Alf61SM3+F1f7srasbP+55H4LhMPTuKPxjU3ja9TQufO5CNB1twheFL2J48s1j7ORyec0HQKPRiOESR+9VEwD9fj86OjqqDliLi4sQBGHDbtqRkRHI5XJ4PJ5tPRfY3t6O4eFh0QKgz+dDZ2fntj4TjUahUCjgcrnWtIpZWlrCwsICAyCJigGQ9r0vfOELuP322yW/oRZWf38/7Pba3P25vkZGRqDVaiUfR6kqNYv3pMm75muf/dqvoPrgFfkg6PncNfDrzJvO8G1VHzz8e2R/ehrQ3IjfP3bHG8/7vYZzfnVD/nm/fzf8O5Kza58p3CsBsFRLne22MKk2GBWr+fl5CMLqcXnrX0skElCr1ejq6ir7GLa2tjYEg0HRAqDH44Hdbt/251KpFAwGA8xm85rZTQZAEhsDIO17X/7yl3HrrbdKfkMtLLfbjc7OTsnHUU5FIpGaDoClnuN72R7aMIN31i0yfOfaw3BfdHE+CP7qyuvx/u++UFEAfOnWzwLNjei77SIcPNyCg7f+Aec9/ik0HW3C+c9egL+754d41rJxJ22tB8B0Og2dToe+vr41Z16bTCaoVCoIgrCtcFVYXq+3omC0vubm5iAIQsnNE5lMBmazGUajMR+kNiuLxVJy+bWSGhoaQnd3d0WfXVhYgMPhgEqlwvj4OAMg7QgGQNr3brjhBvzgBz+Q/KZaWIODg+jo6JB8HOXU6OgoNJqN5+7WSm22k7dwA8fBwwo0v9aHQCyJ2ZERRH5wSz4Edp9/EW669jDOvEVWdvj77JGH8uf9fuHIAzjzZ4/h0NMfWg1/v/17nHXHgyV3FcvlckyU0aJmp2p2dhaTk5MYGRlZE/CMRmM+4OWORbRYLPkzrwOBAMbGxjAxMQGTyZRvEL7dmbGurq6qA1Y2m4UgCJuG0MXFRTidTqhUqi177plMJkQiEdEC4MDAAJxOZ1XXCAaDkMvlGBoawsLCAk6dOiX1X6e0jzAA0r73rW99CzfffLPkQaWwhoaG0N7eLvk4yqnR0VGo1WrJx7FZbdbLz+aL4VGdBzZfbMPnkrb/n70zD2+rvPK/2+fpDFAwDAwzHbokYc1ikhYolAIFylZKf7SlndKZFuhGaSnQwgBOSIgTCC1bA5QChbCEpRASApKudsmSJcuyrNWyLdmSV1myZFuLJVmyDEn8/f0h7o0ka9eVJSfv93m+Txvr6t5XstD9+Jz3nNOBtiu+w4Cg4OtX4I57/1EEAFIwPnw+0NKI/Vu+izP/3IJ1ryf3+635x1VYueXNnFXF0Wj1AXBmZgY+ny8N8LRabRrg8Xg8SKVSqNXqRYDn9/shlUoxMjKS9fyzs7OIx+PQ6XSQSCQlVd5WEhlLdSQSAYfDKThWLbXnXr4Ur0KhqGhWb6atVisrs4WnpqYgkUhgMBgIABKxKgKAREe87rrrLvzud7+rOaSk2m63o729vebrKMbj4+MQiUQ1X0chOz1+vLRfDo25j/lZPjB0evzYqxvCGc08/O7mLeg893Ba+MVv/xTn3/NOTgD846aNQEsjwltPxYadtzD7/c75281YsXH/ouN3cK3YqxvCXt0QnB4/eDxeRQCYCXhms7kowBscHGQALxKJ5L2GTCbD6Gj2XoV0EUgikYDVaoVAIMDExERRQNPX1weDwVAxGM3MzIDD4RRd6DExMQGBQLCowIK2XC4vq3o4l9mcLRyJRDA5OUkAkIhVEQAkOuJ1//3345e//GXNASXVAwMDUKvVNV9HMXa73RAKhTVfRzFObV5cKDWc+di6e/fiie/8Cr3nrIH9nNUwrV2P+394H868n5N23JrmD+DbugKB7Sfh2y9ezMDfWU/eixXNvILRw5XNfPzxJR74emfO0XOpgGe32wsCnsFgQG9vb0mAV8hSqbQgANKmU5XDw8MFYYaN1Oj8fLJYgsvllvwcuVzO/LeX+phMJoPH42ENANmcLZxIJDA3N0cAkIhVEQAkOuK1adMm/OxnP6s5nKTa4XAw+6fq3R6PBwKBoObrKMapAJi7OGQob+XvlXe8jH0XH04LS8+7FDf94mnm8b9v/jlsj52CK3et+3S/3/k4/dEnSywgoRggfYarSwM8gUCQBnjt7e1pgOfxeBAIBCoGvEKWSCQYGxvL+li2wouJiQnw+Xz09vbmjcr19PTAZDJVDEXBYBA8Hq/k58ViMXR2dkIikcDv9zM/l0gk8Hq9rAEg/TsjAEhUryIASHTEa/v27fjxj39cczhJtdPphFKprPk6ijGdOqv1OoqxSqXKGwFc2czHnylrYUB7kMIv/nc7NOu/zoDga5f/CD+8+6/gPPFFnP/6WjTtbsLal6/AqodfL6uCODUqKVbplhzwClkikcDlcmV9LFflbSAQgEQigU6ny1mcwdbeOL/fD4qiygaqvr4+UBTF7PsTi8UFC0VKMZuzhQkAElVDBACJjnj95S9/wY033lhzOEn10NAQFApFzddRjOnITq3XUYxVKhUGBg63XXlO0pdWBVxMY+e0dO99H+DRG+5Az+q1sJ+zGj1rV2P7r9bgvF3rsPrvN2HFpn0Fo3zFeJ9uuObvXabFYnHJADg/n9yvlq9C2GKxwGKxVAxF09PTEAgEFZ1jbGwMFEWhr68PAoEAU1NTrAGgTqdDf38/qwC4sLBQ669ToiNIBACJjnjt3LkT119/fc1vqKleLuPVotEovF4vKIqq+TqKcSoAZtvnV64v/cML2Hf1BiYaqD5/Pf7nlsew4sHiIa/QvsDUApV6sEgkwvj4eNbH8gHg/Px83gphs9nMSnHE1NQUhEJhxefx+/2QSCTgcrmYnJxkDQC1Wi1ro+UIABJVQwQAiY54vfDCC/j2t79d8xtqqpfDeDXaPp8PPB6v5usoxjQAZkv/lp2iffg1rH3lW2h6Yx1+37wG+vPWMSD4zqU34pI7K0sBp0JgtlY1tbJQKITb7c76WCEApKGlp6dnUYWw2WxmpTjC5/NBJBKxAljRaLKnoFQqRSgUYuWcbI6WIwBIVA0RACQ64vXqq6/i0ksvrfkNNdX1Pl0j1ZOTk3UDgE6PH/Ke8ZzVs2q1Gv39/djT4WQFys549Amse+08NO1uwrW7VkO34ys4///ewpYb70L3mqZkWnj1Wjxywx1Yfd8HR1QkUCgUwuPJDqTFACDtzApho9GInp6eiqHI6/VCLBazAljz8/PgcDjQ6/UltbTJZzZHy83NzREAJGJdBACJjni9/fbb+PrXv17zG2qql0NzZdpTU1Pgcrk1X0dmT79/SHswPDwMm80Gk8kEjUYDiqKwaRevpP132c3DWU/9kWnxcutLZyO0/SRseuhe5piL7noTr13+IyYa2L7hQvzifx+pOC2cbXJILSwQCFgBwPn5wxXCPT090Ov1rFTHTkxMQCqVshZh43A4mJmZweDgICiKgtPpLLrHYDa3tbWxNlqOACBRNUQAkOiI1969e7Fhw4aa31BTvVyaK0ejSQDkcDhLdr1IJIJgMIiJiQkG8CTqrkUFHKua+djLl0Gj0cBoNMJms2GfQF5yoccib/oA5zz/Ewb+bnvmUnzS0oi+h8/Fqix9/n5821OQnXcpA4J7v3k9rrzj5QLXobBtvzHnWjMnh9TCfD4fExMTWR8rFQDn5w9XCIvFYlYA0O12QyaTsQZYHA4HkUgE8/PJ9LJQKITRaCw4aSSX2RwtNzc3h0QiUeuvUqIjTAQAiY54cTgcrF69uuY31FR7PJ5l01x5enqauTmycb5MwOvr64PRaIRGo4FcLgdFUeBwOODz+ZDL5dBoNNgt0hYFSi9/pCwa9O55R784BbtlN9a8/O1kf7/XN+Cyv9yFQ1tPBFoa8eONT+c815n3c/DAD++Fee25sJ+zGr3nrMHj3/kV1t27N+vxNz7Bw9TUFDoGPIvBtk4igHw+H16vN+tj5QDg/HyyQlggEDDR70qgaHx8HHK5nBXAisfjzB859M9mZmagUCigUqkYMCzFra2tGB8fJwBIVLciAEh0xEskEuH000+veV+1VC+n3np+vx8cDgfhcLgkwPN4PBgaGkoDPJlMtgjwOjo6YDKZYLPZMDw8DK/Xi1AolHbOfFM9Uo/jSFVZgaoYIDz9kb9i3WsXJPv77fomVrW8AN3DFwItjeBtubaoc5x/9zt46ds/ZaKBneeejztu3oJVD6ZHDlc2U+gZShZY5BtXV0tTFMU6AM7Pz6OzsxOtra0lzxDOtMvlgkKhYAWwZmdnweFwFr0uuppZLBZjenq6pHOyOVmEACBRNUQAkOiIl1KpxJe+9KW6AsDl1FolEAgw+6NowAsEAosAr729fRHgtba2pgHeyMhIVsArxsWA0qP/bE0HrY187OBZC6SFKZz1xANY98a5aNrdhDUvfhcrN7+LP2zaDLQ0Ym7rv+MbzW+WlEb+f796FsILLmdAkHPR1bj29hfSjnlGYGUA1unxo7U3d3FLLZxvXnGlANjf34+enh4mzVzOecbGxqBUKlkBrGg0WQWcLd2bSCRgt9tBURRGR0eLPqdEImGlmIQGwPn5+Vp/lRIdYSIASHTES6vV4j/+4z/qCgDrubVKJuBZrVZwOByo1eo0wBMIBGmAZ7fbKwK8YpwPlJwef1bQyxYBvG6nIvn/N36Ic577GbPf7+xnfokVGzk4p3k/PFtXAS2N+OtDvyprL+HpD3Bxz4+bYVj3VQYEn73mFqz/03ugi1Tqqeo30zweD5OTk1kfqwQAU/vj0RXC5VTLjoyMoK2tjRXACofD4HA4eYs+xsfHmUKWYopDRCIRa5NFCAASVUMEAImOeBmNRpx44ol1BYC1rKzNBLze3l4YDAYmgsfj8dIAT6PRgMPhoK+vr+qAV4lzzf7N5ZWb38aaf1yT3O/3xnqc+fhmBsye3fwLoKUR7q2n4+zmD8sCQNob/vgenr3mFgYC9eu+irt/3IxVD3CZY1L7/xVqdbNUphsjZ3ss15i3YtzR0QGn08n8O7VCuJSq2+HhYajValYAKxQKgcvlFjwuEAhAKpWio6OjIASzOVmEACBRNUQAkOiIV19fH4499ti6AkC2CytyAd7g4GAa4Eml0kWAp9VqYTabc0bwQqEQOBwOgsFgzd+3fM4VAcwandv+HNa+elFyv9+rF+H0bX9jHvtm8xtIbD0FaGnE7zY9XBH8pUUdb38BnIuuZkBQ8PUr8L1fPZsWCayn/YAcDgdTU1NZH6sEALM1SC5mhnCmh4aG0N7ezgpgBYNB8Hi8oo6dnZ1l/lgKBoM5j+Pz+fD7/awB4Mcff1zrr1KiI0wEAImOeDmdTnz2s5+tKwCk99UVW1iRDfDcbnca4KnV6kWAp1Ao0gBvdHQUPp+P2c9XjGdmZsDhcBAIBGr+vuWz0+PHL1+UFgAxCmc+/hDWvbE+ud/vpeuwcvM7acfwt1wNtDRCu+UbqLyfYEY6+kEefnfzFnSeex4Dgi9++6c4/553sGrj4lnFtaoIjkQi4HA4mJ6eZh0A29vbs6Z8o9Eo2tracs4QzrTT6YRGo2EFsPx+P/h8fklA1t3dzfRKzHYMj8fLC4gEAIlqLQKAREe8XC4XGhoa6iqCFQwG0worMm++fr+fATy6eW4m4AmFwjTA6+/vLwvwCpneH1XPAFjU3N+NHJz97C+Y/X7nPPczrNiYnt796aYngJZGHNx6Iq7b+CKr8JfqdffuxRPf+RV6z1kD+zmrYVq7Hvf/8D6ceT9n0bG16AlIA6Dfnx0+KwFAtVrNTAXJdL4Zwpl2OBzQarWsANb09DQEAkHJzxseHgZFURgYGEhLX6c2liYASFSvIgBIdMRramoKDQ0NOVta1OLm6vP5wOFwmGrIQoBnsViqBniVwkCtXWju78qNfKzc/C7WvPjdT/f7nYszn3hwUXTv9GYu7A+vA1oa8ebmH1UN/lJ95R0vY9/F32GigbLzLsWPfvFUzSOANPRXAwBVKlXeCRn0DOFCFcIDAwPo7OxkBbAmJyfLnitMP1ev1zPvC91YOhwOEwAkqlsRACQ64hUOh9HQ0ICxsbElAyY6gud0OhnAU6lUkEgkDOBxOBy0trais7OzpoBXzOvJlw6stfMVf6zayMfNb/wTa3ddnIS/1y7A6Y/szHrs5of+CLQ0Ymbrf2FD83tLAoArmvlY8SCFX/zvdpgu+AYDgq9d/iNcfPebNdsDWCjtXwkAKpXKotqpFKoQ7u/vh06nYwWwfD5fRXOFw+EwlEollEolwuEwYrHYosbSlTgejxMAJGJdBACJjnglEgk0NDTA4WDnZkoD3vj4OAN4XV1dDOBxuVwmgqdUKhcB3uTkJFNYUa9RtUznKwiotbNFAFc287FPN4wn1a9h3esbkvv9/nEVVm7ZjRXNfOzgdqcdv755D0JbTwNaGvHwQ3cvHfyl2OEYw/gjj8K+dl0SBJvWw/3kUwjXALxpAMy1baJSACx2RFq+CmGbzQa9Xs8KYLExVzgej0Ov10MkEsHj8WRtLF3JuT/55JNaf5USHWEiAEhUc+3YsQMXX3wxjj32WJx44olZj3G5XPje976H4447Dqeccgruvvvuov8iPnjwID7zmc+gp6enaMCbnp5mAM9qtWYFPJFItAjwxsbGMDk5WVQEr56hKtP5WoLUgzOrZ19T2bBNve3wfr+/3YwVm/YzsHXnW11p8PXG5h8DLY3of3gdTm/mLhn00W75yMK8lqDViuFbbmWigY7Lr8AUh7ukRUz0Hyi52v1UAoAKhQIul6vo43NVCPf19cFgMLACWG63m5WxcolEAgMDA0yUv9w5wgQAiZZCBACJaq6tW7di586duO+++7IC4MGDB9HU1IQrr7wSZrMZMpkMp512Gu66666izn/o0CEcc8wxMBqNiwDP4XAUDXgDAwMM4JVTvZvpfI126835pkIU42J621Xa/87p8eNVngo8jQq38G853Nz5qT9hRTMvJ3xds/ElHNh6EtDSiJ9uemLJ4S/bPr9IJIJJDgeOy69gQHD4llsRtFqX5PdNFylVAwDlcnnJM3Kj0WSFsFKpZOby9vb2wmg0sgJY4+PjaG1tZeVc8/PJ4hAOh4Pu7m5WIJAAIFE1RACQqG70xhtvZAVAoVCIz372s5iYmGB+9t577+Ff//VfEYlEsp7L7/dDpVLhzTffxPbt23Hqqadiw4YNOOuss7ICXnd3N+uAV8gURVUEVUvpStZaTG87tvrfva94H1e8ewWadjfhwncuBH+Aj5YPu/MAGAXNlouBlkYIt1xVE/jL91rD09NwP/kU7OeuT4Lg2nUYf3QHwlUuaMpXpR6NVgaAMpkMbre7LAhKrRC2Wq0wmUysABubY+Xm55N9BblcLmQyGdrb2zE7O1vR+ebm5nDgwIGqffcSHZ0iAEhUN8oFgA8//DDWr1+f9rNQKISGhgYoFIqs5/rb3/6GL3/5y7jssstwyy234NRTT8W9996Ld999d8kAr5DpKsdar6MYUxRVVhV1tv15mRGvYo4pxvtt+3Hem+ehaXcTrnn/OrzTpYHT48/aIJr+9283bQVaGjG/9RRcsvH1JYQ/Cu9pBop+jSF7P0Zu/y0TDRz45iXw7Xm/amnhQn0qKwFAqVSas3deIScSCfT29oLP5zOReTaAbXR0lLWxcvPzyb6CFEUhFouho6MDUqm0YFsbAoBESy0CgER1o1wAePvtt+Oaa65Z9PN/+Zd/wbvvvlvwvAsLCzjttNMgFotrDlKpppvI1nodxTgVVktJ1eaq0E3tbVfMMfk8E5nBX7R/YVK+1719M1Y9tDctwnb3O/q0c9/9jh6PfqDD+MNnAC2NeG7zrUsa+fvFs9yy+ipOiURwXn0NA4JDN9+MgMHI+u/b7/dXDQAlEkne9i7FeGhoCFwul7VJIGyOlZufn8fU1BTTVzC1rU2pqW/a8XicACAR6yIASFQVtbS0oKGhIa8NBkPac/IB4LXXXrvo55+6s6YXAAAgAElEQVT73Ofw3nvvFVzLwsICVq1aBR6PV3OQSrVIJILb7a75OooxDaulpmqrHQH0BDz4jeg3DPz94cP/w8qM/X65Jmy4PtgCtDRiYutKrG7+YGnTvu+XP1ovEgjA89zf0L/hq0kQXLMWrs1bEGbxs0QDYK4IYyUAKBaL4fP5KoYsrVYLHo9X8gzhXEDJFkzOz2fvKzg6OgqKomC320teLwFAomqIACBRVeT3+9Hf35/XiUQi7TlspoAztXr1auzbt6/mIJVqsVgMl8tV83UUY6FQCKnJUdaosmzQmBlFLGcPoNVjxXf2fQdNu5tw/tvnY79tP16l1EVB2EXNb+LgI/8JtDTinoceWtLo34pmPu5+gVvxZJqQcxCjd919OC184UXwvvkWIiz0kKRnVed6vJLCBpFIhMnJyYohy2g0wmQylTxDOJudTic6OjpYA0Cv1wuJRLLo59PT0xCLxejq6ippvQQAiaohAoBEdaNCRSBer5f52Z49e/IWgWRq/fr1+Oc//1lzkEq1RCJZsubUldjp8ePnz+SGmdRULQ12HQOeNMBzevxo7U3+OxfspR5TaE38AT4ufOdCNO1uwlXvXwXDuAHRaBQCZUdWSM38GXfLd4CWRhx45Wp09LsXPV6sf/matuAx2c9N4RlRDyu/n2mFAs7rv8uA4OAPfgB/R0dF55yamgKXy60KAAqFQkxNTVUMWQaDAX19fYhGF1cIl+qBgQHWxsrNz8/D4/FAJpNlfSwSiUClUkGhUBQ9Km5ubg4HDx5k7buWiAggAEhUB3K5XLBYLNi+fTuOP/54WCwWWCwWzM7OAjjcBuaqq66C2WyGXC7Hl770paLbwADABRdcgNdff73mMJVqmUyG0dHRmq8jn3erHQUBp2PAwxybDb5So3mVFnyEI2E80/UMk/K9lX8rXP7DUVSdToenPuxYBJip0Pnfm54GWhqx0HIiXN3KvJNECnmfbignPK7amHx8r24oJxiyNeYtEgph4uVX0H/e+QwIjt3/AGbK/HxNTk5WDQAFAgGmp6crhiy9Xg+bzcZEyLq6uoqaIZzNbE4VmZ9PtpXJ11dwbm4ORqMRQqGwqHQ4AUCiaogAIFHNddttt2XdI6hUKpljXC4XbrjhBhx77LE4+eSTcdddd2F+fr7oa1xyySV46aWXag5UqZbL5RgeHq75OnI5W/VsrghgvmNTAS8XbO3TDRVcjy/ow52SOxn429a+DaFwep86nU4Hq9WaNZro9PjR2jOKqScvAFoa8e7m72PVRj5+84auLPhbtTEJvy0fWrI+lhrZzPXeFFvoUqxnRkcxdv8DDAT2n3c+Jl5+BZEc/fzyAWDmntlwOIypqSmMjY3B5XKVve+Oz+fD7/dXDFk6nQ79/f3Mv1MrhEutMrbZbOjq6mINAMfGxqBQKPIek0gk4HQ6QVEUBgcH8x4Xj8cJABKxLgKAREeFrrzySjz33HM1h6pUt7a2YmioMPgsNfTt/TRqlStylemNe434w9tdeY+hQadjwJMVhvLt+3N6/HhPr8P1+76Hpt1N+NpbX8O7Pe9mPbarqwvd3d25wUb+d6ClEZGtX8B5zf8sO/K3sjlZSZwaaWz5qBsdA56saeznJLacYFxpA+xs9nd0YPAHP2BA0PndGzCtVOZ9TiQSQSAQgNvtRm9vL7hcLrq6utDW1gaxWAwulwsulwuJRAKBQACj0VhWJJCiqIpaotDu7OzEwMDAop8PDw/nnSGczWxOFZmfn8fIyEjRbWUmJiYgEAhgNpuzvp8EAImqJQKAREeFvvOd7+Cpp56qOWClWqlUYnBwsObroJ2Zwi13X1yuVGdqGrZQpDB1TWc+shPrXrsATbubcNHbl0E7ps35GvIC4LQbicdWAC2NeOShOwuuOd86eaaRklPZz0n6mPOubKbwmrKftQbYWYFuZgbe3W9i4MKLGBAc+cMf4LFaMTw8DJvNBqPRiPb2dshkMmZ8mVAohFwuB5fLZRqku1wuTE9PM1XBoVAIcrkcWq225OILHo+HYDBYMWRptVo4HI68UGW1WouKVPb09LA2VYSG0FLayoRCIbS2tkKtViMajRIAJFoSEQAkOir0/e9/Hzt27Kg5ZKW6ra0NDgd7N/xinCvaVGy6t1xnA6ZsTp2J63BP46wnH8S6N85F0+4mrHnxuzh9y7t5ISsfAPa9+jugpRGDD6/GGc2cvIC3g2tNgbXDQEhDWrm9C50ePySWEex+n4P+MR8rDbBTHQ6HMTk5ibGxMfT398NisaBTKoXh9t/CtnoN7OesRl/TudA9+CA0SiVMJhPsdjtGRkbg8/mYyR9erxd8Pj/nNWKxGGZnZ6FSqdDW1rYIWvKZy+UiFApVDFkajQZOpzPn44FAAFKptKgKYavVCrPZzBoADg4OQqPRlPScWCyGzs5OSCSStBR5IpHA3NwcDh06VOuvUaIjTAQAiY4K/eQnP8HWrVtrDn2pVqlUGBgYWLLr5Ys2VVIIwabpaOF0aBq3ce86PM/32V9gxUZOQcjS6/VZAXC0txOfbP03oKURt2x6LO16manc5yS2rGC2TzeUVtVcLrzRY9YkltGSITI1Tet0Opk51nSalsPhMGlalUoFvV6P3t5eDA4OwtXWhsH//snhtPA112BKtLg5ejgchtvtBp/PRygUQjAYRDAYRCAQYBwOhzE3N4d4PI7Ozk7IZLKiKloTiQQzYq5SyGpvby+Y5o1Gi6sQpgvP2AJAh8NRVlVxIpFAX18fKIrC2NgYAUCiqooAINFRoVtuuQXNzc01h75Ut7e3o7+/f0muVQhYqh0BLMX7TEbc9NFNaNrdhHVvrMeZj2/GimaKATa66jgXAFoslvSfRyIIvHg90NII6ZYr0q+lG2ZeP713r9joXrnpWxoAc0UA+8d88Hq9GB4eRl9fX840beYc68w0bS6A9L73Hga+eQkDgsO/+Q2menrSAM/tdoPL5WJoaAjBYBChUAgzMzOYmZlBOBxGJBLB7Ows5ubmMDc3B7PZDJFIVLC4gwbAcDhcMWSp1WoMDw8XPK6YCmGz2Qyr1coaAPb396Ozs7Ps57tcLlAUhb6+PuY9JgBIxLYIABIdFfr1r3+Ne++9t+bQl2qNRgO73b4k1yoGarK1cVlqn7H9b/jmPy9F0+4mXPrupXhU8mFWSMoFW9kAMG7Z/+m835PxrY2vFozYlRLdK6V3IW16ysbIyAie4XYdhshmPh56lQ8OhwOKoiCXy9HR0ZEzTZvP4XAY4XAYMzMzTBQvFfCmXS6MbtsG+9p1SRA8dz1cjz+BkM/HjH9zOp3g8XgYHh5GPB5PM50Cnp2dRTwex9zcHGw2GzMyMBfYzM3NMRNGKoWstrY2jIyMFB1Zy1chbDQa0dvbyxoAslFV7Pf7IZFIoNUm97wSACRiWwQAiY4K/f73v8edd95Zc+hLdUdHB9PIttrOBjUrmpPpzrQ1DXhw666OGsAfhbMe34z1uzegaXcTbvroJjgmHcyaip1AYjAY0gEw5MehZ84FWhrR8/pdRUfsKinOiEQi8Pv9edO0HA4HEokEarUaorZO/FNmQGd3PzweD4LBYN4oXi7Ay4Q82nQEj47i0RG8aDSKUE8PRm69jYkGOi6/An4ehVgshng8zkSi+vv7F0FgPB5nIDAWiyGRSGBoaCgtfZnpWCzGTBipFLKUSmXO6+QyXSGc2XaFbirNFgD29vayUlUcjUahVqthMpkIABKxLgKAREeF7r33XvzmN7+pOfSlWqvVore3d8mu95ykbxF4pYJUtseXwis3cvD99w7v9/uT/E+YDk0z6y6l4CITAOflfwFaGnHoybMQDXhLitjlOzYUCpWcpnU4HHC5XPB4PMw+uEKQR6dds0XxCkFeNBpNi9LlciwWwzSPB8flVxyuFr71NoR7+xCPx9MqamkwzAeB4+PjoCgqa4UuDYCzs7MVw5FCoYDL5Sr5eV6vd1GFcFdXF+x2O2sA2NPTA5PJxMq54vE4olESASRiXwQAiY4KNTc349Zbb6059KWablq8VNfL14Q5W5+6JYG/ze9izYvf/XS/37m4k/fEouhXKSlZg8EAs9mchB/vIBZ2/BfQ0oi5ztKmwGRW05rNZmi1WrS2tkIgEFSUpg0EAuBwOAgGg3nTtJmAl7oXLzWKVwjwivVsIICJp5+G/dz1SRBctw6eHY9hdmoKU1NTEIlEMBgMWSEwFoshGo0iHo8jkUhgcnIyaxuWaDQKDoeDWCxWMRjJ5XKMj4+X9Vy6QrizsxPxeHxRU+lK3d3dzVpRCV0EsrCwUOuvUaIjTAQAiY4KtbS04Oabb6459KW6UNNitp0rDVyrfX+rWl7E2l3fTMLfaxfg9Ef+ihXN2cGu2JSs0WhkAPCT938FtDTi4EvfQjQSTjsuX5pWJBKlVdOq1WqmmnZoaKjsNC0NeT6fDxwOBx6PJy/glRLFY9NhhwOjv73jcFr4kkswtXcfAoEAZDIZsyctFwTSkcBAIACJRAK9Xs80OI5EIuBwOBWNkqMtk8ngdrvLfn40mqzEVyqV0Gg0OXsKlmM2i0oIABJVSwQAiY4KPfbYY/jhD39Yc+hL9aL9akvgeij0WNHMxxl/3oZ1ryf3+615+dtYuWU389j/vNyRE2ALpW9pAIw52oCWRqClER4dJy1NK5VKC6ZpC1XTpkJeKWla2jqdDmKxGFNTU0sOeMU6IJHAefU1h6uFb/4ppru6oFQqoVKpEA6HC0JgOBxGa2srNBoNYrEYwuEwOBxO2WPkUi2VSvMWnBTjubk5dHV1gcfjoaenhzUANBqNrJ2PACBRtUQAkOio0NNPP40bbrih5tCXDVaW8ppOjx9/pqw1hD8uzt75W2a/3znP/wQrNu1fdFy+Vi+p8JWZppVIJBDwKcw8+TWgpRHjf7thUZp2dHS0omrafMUWxaZpY7EYjEYjxGIx/H5/zWEvl2MzM/A+/zz6N3w1CYJr1sK1eQs0IjHkcjlCoVBBCIzFYlCr1VAoFJiammINAMViMbxeLyuAJZFIwOPxSp4hnMt6vZ61ohK6DQwBQCK2RQCQ6KjQ888/j6uvvrrm0Jdqk8kEo9G4ZNcrNIqt2l750PtY/cKNDPyd9dSfsKKZl/XYF2R2Jk1r6h/G3rZutHaa8qZpDQYDlEolHO+3AC2NWNjxX4hOOEsCvHxRvEzAK7XYIhsoWSwWCIVCTE9P1xz28jkyPIyxu+9hooEDF14E8+NPQPJp7798EEgDDB315HA4rICRSCSCz+dj5VwqlQomkylrhXA5ZrOohAAgUbVEAJDoqNDLL7+Mb33rWzWHvlSbzWYYDIaqX8fp8WOvbqimqd9VW1/B2lcu/XS/33k449En8h7/j31i8Hg8NL/MTRvJ1vKeOm+a1tLVjk/+vDLZ90/+OAN5paZpy6mmLSvCFovBarVCIBBgcnKy5qBXyMG2Ngx+9wYGBPuuvx7yF16Ez+fL+tro942GGIPBAA6Hg6mpqYrBSCgUsnKe+flkT8HR0dGsFcLluLOzEwMDA6wBYCKRqPVXKNERKAKAREeFdu/ejW984xs1h75Ud3d3Q6/XV/UatY76rWjm44wdj2Hd619D0+4mrH3lcqx6+LW8x3/vrzK8JutGq2UoR+/C9N6JqVE871u3Jws/dq5HYGqiJtW05bivrw8CgQBer7fmkFcQWsNh+HbtQv/55zMg2PXzWzBmtRaEQL/fDy6XC4qiKk63CgQCTE9PswJZqS1lgsFgWoVwOefr6OjIO6eYACBRPYgAINFRoT179uC8886rOfSl2mq1QqfTVTXyV1v44+Gspw/391v99x9ixaa9FZ93ZTMf9jHvIrgLDRlwaNvJQEsjAtp3WEnTLqXtdjszSaPWaynG0fFxuB54kIHA3g1fhXPnTsQikUXH0u/95OQk+Hw+RkZGQFFU0ZM8spmiqIKj54p1a2trWkuZaPRwhXA5U0uKmVNMAJCo1iIASHRU6MMPP8S6detqDn2p7unpQWdnZ9XOn6vv35J40z6s/vuPGPg7+693Itd+v3IssYwsStMeePMmoKURkReuhlgkQiAQqDkkleqBgQFQFAW3213ztRTrUKcOQz/4IQOC9muvRVClSoM/v98Pl8vFpLsTiQQ8Hg8zZaQcMOLxeAgGg6xAlkwmWxSRpCuE6UKdUs6nUqmKmlNcLADOz8/X+iuU6AgUAUCio0ICgQBnnnlmzaEv1X19fUxPtaWKAK5srn7fv1UPv461L1+R3O/3+ldxxmM72D3/Rj6GfcE0CEn0UsnCj+0nI+6ywGQyQSKRIBgMLikMseHBwUFQFAWXy1XztRTrmWAQY6+8AtsFX2dA0PLzn0O+531wudy0Yp2enh6mQnhqagpCoRAWi6XkPXdcLhehUIgVyMrVUiaRSKCvry/nDOFcViqVGB0dJQBIVNciAEh0VKi1tRVf+cpXiurttlS22WzQaDRVvUbqHsClgL/TH30S6147P7nfb9elWNXycsWw92eqh1n3qo18vKUZTAeQ6AwOPZds+/IJ9QDi8eTeM4PBsGwhcHh4GDweD6OjozVfSzyejOIFAgGMj4/D4XCgu7sbnZ2daG1tBZ/PB4fDgUAggJKiYP7d72BbvSYJghs2wPXMM5hNaReT2SYmFApBKpVCp9MV3SA6kUgw4/TYgCyxWJy3ophOWRdbIVzumDoCgERLKQKAREeF2tvb8Z//+Z91BYB2ux3t7e2snCsUCmFiYoKZTWswGJimx2/u4+GpN7l4dLegivDHw1lP3Xt4v98L/w8rH9pT8XkfeN+EYV8Qw74glDbPoshfPB7Hx21/TUb/nliFeMiXBhp6vR4ymSxrv7p69+joKCiKwvDw8JJcLxwOw+fzYWRkBDabDQaDAWq1GhKJZFEUz2g0wmazMaPvMptC+7SdsFz/XSYa6LzmWgSk0pwQGIlEoFQqoVarixoTNzc3Bw6HU9b+vGwWiUQFK4pLqRCWy+UVTSnJfK0ff/xxrb9CiY5AEQAkOirU1dWFk08+ua4AcGBgAGq1uqhj6abHo6OjTNPjjo6OtAgMn89nmh6bzeZFTY+rVhSycT/O+dvNh/f77bwdK5q5rJ1/ZXOWqB/t6TEsPPbFZNsX7SuLHo/FYujq6oJMJsPMzEzNoa5Uu1wuJvJU6blisRgCgQDcbndaFE+hUDDzjQUCARQKRdpkFLfbjUAgUHIBTSgUQufjj6P36xcyIDj62zsQdjiyQmAsFoNGo0FrayvC4XBeKIrH4+BwOIhGo6xAVrEVxcVWCLMxpYQAIFG1RQCQ6KiQ1WrF8ccfX1cA6HA40NbWhmj08GxaOsVGVwhnNj2WSqVM02N6Nu3ExASCwWDB62WOgVvZzMd97+org7Mtb2LNP676dL/fBpz555aqRBiz7fuLx+M48MHvgJZGHHrxEsRnF8+njceT6cvOzk7I5fKs48vq3W63GxRFwfEpOOVzJBLB5OQkE8Wjx99li+IZDIa8UTw2HI1GoZFKob/zD7CvXZcEwXPXY+LppzEbCGRtGK3X6yGRSBAIBHJCUSwWA4fDwezsLCuQRVFU3uulOhotXCHM1pQSAoBE1RQBQKKjQgMDA/jc5z5XUwCk07RDQ0Po6+uDSqUCn89Pm00rEomgVCqh0+mYCMz4+HhJs2mzjS6zjU5kLQhR9Y1hZZlRwdMfeQbrXvt6cr/fqxdj1bYXqgJ/tJU2TxpczA1rsdByItDSiIRDmRdEZmdnodVqmehSraGuVE9MTIDP58NutzNRPKfTCavVuiiKx+fz0dramhbFGx8fLyuKx4ZnZ2eh0+nQ+uabGPzZz5looOPyK+DnUUyLGLpXYCKRQE9PD9McOxeEcTicotLFxbjUimIaVHNVCItEopxrL9XxeJwAIFFVRACQ6KjQ6OgoGhoaipr/Wq6LTdO2traio6MDGo0GYrG4otm0xY4uk3aP5oSqbR+VOhuYwplPPIh1b5yLpt1NWPPS9Vi5+Z9Vhb9FEcBYDAdfuQpoacSBPbcVDSIajYaJ3NQa6nKZjuKNjo6mRfHoSDCHw0mL4vX19VU1iseGY7EYzGYzhAIBXHv2wHH5FQwIjtx6G2Z6exdBoMPhAEVRaf35aEciEXA4nKKLRgq5nIKSfBXCbDapjsfj+OSTT2r9FUp0BIoAINFRIZ/Ph4aGBkxOTpYNeOWkafv6+nKmaYeHhyGXy7NCHtujy4Z9wUURQBqqdE5v8TC28SOc89wtzH6/c569DSs2flRV+FvZzMfzMnsaUMwb3v503u9/Ym6y+P1xs7OzaG9vR1tbW80gMBaLIRgMpkXxdDodlErloiieVqtNi+KNjY2Bz+czrVRqDXalvm4amMYHBzHx9NOwn7s+CYLr1sGz4zFEfD7mc5tIJDA2NgaKohY1VQ6Hw+BwOBWNa0sFOQ6HU3DfYS7TFcKpkz9KSSkTACSqlQgAEtWFduzYgYsvvhjHHnssTjzxxKzHNDQ0LPJLL71U1PmDwSAaGhrgcrnyQl4wGExL09KVkLnStFarlbk5+/3+ktK0Q0NDkEqlDMQVC3jlji57SzPIQGBqOxVFn6c4ENv8Dta8dF1yv98b63Hm45uwopmqKvylwipTCDLjx6GnzgFaGvGx/LGSQSQajUKtVkOlUiEazb5vsFKnRvHsdjsTxZNKpeByueBwOBCLxWl/KAwPD8Pr9SIcDueFu9TeecsNAuPxOJxOJ3g8HkZGRhB2ODD62zsOp4UvuQST77/P7AtMJBJM+ttmszHAFwqFwOFwWAOsSgtK6Arh7u5uJBIJVnsUEgAkqpYIABLVhbZu3YqdO3fivvvuywuAb7zxBnw+H+O5ubmizh+LxdDQ0IC+vj6YzWbmxlwoTZtaTTs5OclE2NhI046NjYHH48HlcjGAV+3RZdnaqWSLDmb69G3PY+2r3/h0v9+FOH37c0sCftkilp+IH04WfuxsQjxSXo8/eiM/XYVd6vPpKJ7H41kUxRMKheBwOKAoioniWSwWDAwMMH8oVPp79fv9EIvFMJlMyxIC6cjewMAA4vE4AhIJnFdfw4Dg8M0/RcBgZCDQ7/dDJBLBZDIhkUggGAyCx+OxAlh0QUml+wnpCmGtVltRRDHTc3NzOHDgAJtft0REAAgAEtWZ3njjjbwA+NFHHxV9Lp1Oh23btuHWW2/FpZdeimOOOQaf+cxncNpppxWdpi2l2KKcNG1fXx9EIlHNmxVnRgf/9+WONCg847FHkvv9/nENVm55qyRw+/ZTirR//3a3ruyG1J1dnVh45NRk2xfLvopecyQSQVtbG9rb27MCWWYUz2QyZY3iqVQq6PX6tCjezMxM1cEsEAhAIpHAYDAsSwj0eDxp6ezYzAy8zz+P/g1fTYLgmrVwbd6CyMQEEokEZmZmIJPJ0NnZienpaVAUxQpg0QUl8Xjuti6lnKutrQ0cDoe1PYAEAImqJQKARHWlQgD4xS9+EaeccgouuOACvPTSSzh06FDOc+3Zswe33XYbtm3bht27d+OEE06AQCAoq9iiUJqWjvSVmqalmxW3trbWvDBh2BfEdo6VgbOVzXxs51ghsIxh414zznjsUazY+GHJ0KZzeqFzevFiaz90Ti8DnKVC4KqNfMR2/zfQ0oiDr9+AeIXQE4vFMD09DZlMBrlcju7u7qxRPLlcnhbFc7lcmJ6erklFbabpqJNOp6uL9ZTqqakpiEQiGI1GBmIjw8MYu/seJho4cOFFmHzrbcx92jKmra0NCoWCNQBkcz9hKlCWM0OYACDRUooAIFFdKR8APvroo8yN+Omnn8Zxxx2HRx99tKjzHjp0CMcffzx0Ol3ZxRaZkMdWmnZ2dhZqtRoajaamkZx8hSLFpIlzeb9+JOf1lDYPnpfZs173j+8a0qKSUu4/k4Uf2/4Nc2Omol5TNBrF1NRUWhRPo9FAJpOl7emkKApisRg9PT0YGhpasigeGw6FQgykLkcIDAQCTHPl1PUH29ow+N0bGBAc+sEPMdPVhXg8DpVKBS6Xy8oouJmZGdb2E6YCYE9PD/h8fsUTQeLxOAFAoqqIACBR1dTS0pK1cCPVBoMh7Tn5ADBTTz/9NBobG4s69tChQ/j85z8PoVBY9WKLchwOhyGTyWA2m2t2I97Gyd4Ohu6/l5omXtnMx627tEVF8fIB4Af6EXzQNQKd0wulzcP8L71HkdmzODGFQ89/PTnvl/sn5hype/EGBwfR09ODrq6urFG8jo6OnFG8cDgMuVy+CEKWi2dmZtDa2srMlq71espZv0KhgFqtTouEx8Jh+HbtQv955zMgOP7gg5iw2cDj8ViJsrG5nzAzopitQpgAIFG9iAAgUdXk9/vR39+f14lEIu05pQCgRqNhWrsU0scff4xTTz0V1157LUKhEJPirVaxRTn2+/0QCATMxvil9LAvmBXmMvvvPS+zM42jV21MtmfZrx/Ba22OrPC3sjn7BI/MFHC+cW/RaBRR2VPJnn+PfRndncpFUbzUvXj0hJRSo3gzMzOQy+Xo6upaFpG/TIfDYWae7nKEwEgkgvb2drS2ti6a3RwdH4frgQcYCLSfdz40Gzeiz2qFQCCAz+crG7ACgQBr6eT5+WSFMpfLZf7t8/nSKoRLPR9JARNVSwQAiepKpQDg888/j2OOOQbz8/NFHe/1evHFL34Rjz32WN3e4CcmJkBRFFwu15JeN1crmG0cK3NMvhRxMc/Pdx4aArXddiaK19bWBqFQCOEHb+PjR74AtDRiaM8mmM3mtCgem7ATCoUgk8mg1+vr9jOSz5FIBCqVqqZ9DisxPTVEIpHA7/cvejyg1cJ54/cZEBy84Xtw7P+Q+W+mHGCbnp6GQCBgDQCzAWWxM4RzAeDBgwcr+VolIsoqAoBEdSGXywWLxYLt27fj+OOPh8VigcViwezsLACAx+PhlVdeYaI7u3btQmNjI+65556SrqPVanHccceBy+XW/GaXy08KgpUAACAASURBVENDQ+Dz+ZicnFyya2aDsszoXS7I2/9pGjff8+m9eGNjY9ijMOdMF7+0X54WxZuYmMD8/juTbV/+/o2c837ZdDAYXNbVtdFoFO3t7VAqlXU7GaTQ+nU6Hfh8Prq7u2E2m9Mjvh9+iPatW2E7/4LD/QN/eweEb79dVqqVLkRhCwCnp6fB5/MX/ZxuPaRQKIpuEZNIJAgAElVNBACJ6kK33XZb1j2CSqUSACASifDVr34Vxx9/PI477jg0NTXh2WefLSs1smvXLvzbv/0bent7a36zy+Xe3l6IxeJFqbBqOlejaNqFCkFWfmr6+c0vcyGTyZgpKTweD3K5HDx5e9Z0c7Z08dyoHgvbTkrO++2XLdl7QUPgcu2zR4+9a21txczMTM3Xk+lwOAyfz4eRkRGm4bpKpYJYLGY+KwKBAFwuF1qtFv39/Yv3bXo8GN+yBfY1a5MguOGr0Nx3H3rN5pJSrT6fD2KxmDUApBt154rm5ZshnA0A4/E4AUCiqogAINFRp4WFBfz+979HU1MTpqaman4zzOZYLIauri4oFIol3c+VrVE07Wg0iqf53XkLPlY28/HkuzIIlVp0dnaCy+ViYGAAoVAoDaTe0gwyewlz7gGMxXDw1euSe//e/dmS/w4CgQDEYjHMZvOyhcDOzk7IZLIl/UMiHl9cnEM3ylYoFGnj7hQKBTo7O2G1WuF0OuF2uxEMBpn32+FwgMfjYXR0NOd1AgYjhn9yMxMNtF56Gcyvvlr0nOCJiQlIpVLWALAQUKbOEC5UIUwAkKiaIgBIdFTq448/xmWXXYabbrqpLgpAct3A29raoNVqlwRAYrEYQqEQMwovdS8eHcV7+i1ewapfumo4Ho9jYGAgZzp72BfEfv0I9utHsgLnvGlPsu3Lo/+BOe/SF8bE43FmAkV3d/eyhcCuri5m5CDb5/b7/XC5XBgYGGCm6mS22MnWKLuU1PTo6CgoioLD4cj5uY1Go5jauxeOSy5lQLD75p8iPDBQENg8Hg9kMhlrAFgsUBZTIUwAkKiaIgBIdNTK5/PhS1/6Eh555JG6vbnPzMxAKpWiu7ublfNFo1FMT09jbGwM/f39zE1bLpczN22hUIi2tjZ0dXWhp6cHg4ODmJiYQCgUwpA3kDcNnFk1HI8n09l0+52i1xr249Bf1yTbvki21/R3MD09DaFQiJ6enpp/HspxLBaDwWCARCIp7XcQPzwNZWRkBDabjZmNLZFIwOVyweVyIZVKodFoYDKZ0N/fj7GxMUxNTbEauaanhvT29mb9b5WGwOjkJDyPPQb72nWwn7MatnVNcD/5FOby9AscHx9Ha2srawDodrshl8uLjhbmqxCm9wDma3hPRFSuCAASHdXq7OzEcccdx4yCqkfTVYq5IiCZN8KZmRkmitfb2wu9Xp8WxePxeJDJZGXftNP6AW5M3/eXrZVLLBaDyWSCVCotej/ax9JHkoUfT69GPLy4GnSpPTU1BYFAUNf7Rgt9LkwmE0QiEaanp9N+Tkd9c/VRpGdja7VadHd3w+FwwO12IxAILOkfTpOTkxAKhTn3ZdIQGIvFELHZMHLrbYeniVx+OYJ8QVbIcrlcUCgUrAGgy+UqCShTZwhnziMmAEhUTREAJDrq9dprr+Gkk06C1bq4ZUm92OPxgMfjYXx8HLOzs5ienl6UepPL5aAoioniKZXKtCiex+NJ219ViVP3CubbN5h6c9bpdEWNvJvzObDw6H8k5/2a3qv5e58KIAKBADabreZrKcWpqVq1Wg0ejwe1Wr3o80JHfcvto7gUzjU1JPVzRkNgPB5HgKJgS0kLj952G6J2expkjY6Ooq2tjTUAHBsbg1KpLOk5uSqECQASVVMEAImOei0sLOAPf/gD1q1bt6StV3KZjuJ5vd60KJ5UKgWHw8kaxbPb7RgdHWU99cY2iLS3txdsVHzg3Z8n5/3uurbieb9se3JyEnw+H/39/TVfS6ozR94ZjUa0t7enpWolEgna29uhUCjA4/Fgt9vr+vOSy/TUk8ypIbRjsRhmZ2cRi8UwNzeHeCgE+5aH0beuKQmC65ow8ec/Yy4QYPbiqVQq1gCw3PNlqxAmAEhUTREAJCJCsijkW9/6Fr7//e8vSVFIZhTPYrFAq9VmjeLpdDomitfV1bXk7WHYdCQSgVKpzFnYkuiXfTrv9yTMjXTVfL3Z7PV6wefzl3RiS7Y/ClKbZaeOvKPnZTscDoyPjyMQCCz6TNtsNvD5fHi93pq/n+V+jtRqNRQKRdZtBTQEzs7OYm5uDolEAoPtGhhuuulw78BLLoX/g/0YHBxEe3s7awA4NDRU9vkSiQTzu3G73QQAiaoqAoBERJ9qcnISX/7yl7Ft27aK016pN+zh4WH09fVBr9en9TrL3EBfTBQvFouhs7MTSqVy2UVuaM/MzEAmky1utDwbxaEXLk4Wfnz4h5qvM58nJibA5/OL2pdZymcmEAjA7XbD4XCgu7sbWq0Wra2t4PP5OVP7ExMTZaVq+/v7wefz4fF4WHsNS2m6zY1UKs1Z3JI65jGRSMDtdqP1ySdhv+JKBgT7//QnaDQa1gDQ6XSio6Oj4igiXfk8NzeHhYWFWn89Eh2BIgBIRJSirq4ufP7zn8f+/fuLugGltsGgo3jZbtg6nQ5Wq5WVvXjRaBRKpRKdnZ11tT+rFAeDQYhEorR9lx9rXkxG//78ZcT94zVfYyF7PB5QFIXBwewzjHP97uiJKHa7HSaTCe3t7ZBKpWmpWrVaDYPBAJvNhpGREUxOTlZltJvD4QBFURgfr//3O5vp4hahUJhz+wYNgbFYDIlEIrmXk8tF77Zt6P/aeRjg8qDValkDwIGBAXR2dlZ8HrpC2G63EwAkqooIABIRZWj37t046aSTYDabMTIyAovFUjCK197enhbFq9YNm/bMzAwkEkldF64UMl1Z29/fj3hwAguPrwBaGvGx+rmar61Yu91uUBSFoaEh5mfhcHhR5DezClsul6OjoyNtrrHf769JT8qhoSFQFIWxsbGav5/lOBaLobe3N280M7U4JJFIMJNe9Eol7HY7K8BG2263Q6fTsXKuYDCIQCBAAJCoKiIASET0qd5++208+OCD+PGPf4xTTz0VxxxzDBoaGnD99denRfGyTSyohaempsDn80uKQNWbvV4vKIrCzHt3JNu+/O0CxKP1Pb+WnnLhdrvhdDqZiScSiYSJ/AoEgqyR38yJKPXikZER8Hg8jIyM1Hwt5ZqOZuabGkJvm0gkEohEIlAoFExVMVsAaLPZoNfrWTnX3NwcSQETVU0EAImIPtU999yD3//+93jqqafw/vvv47LLLsONN95Yt5NC4vE4xsfHwePx4Ha7a76Wcu21KnCo5dN5vzZRzdcTjx8u0kltmK3RaNKmXIjFYqhUKhgMBnR1dYHH48Fms5U05aKePDY2tiiaudw8OjoKHo8Hp9OZ9fHMSGAsFoNEIoFAIEAkEmEF2np7e2E0GgkAEtW9CAASEeXQ1NQUvvKVr2Dr1q11GbWh7XA4IBAI0hr8LhvHYjj4+veAlkb4nvn2klalhsNh+Hw+jIyMoK+vDwaDIS29n9pqx2w2Mw2zp6ens/5RQI8sW66p1Hg8+QcFPZ6s1msp1263G3w+H319fXkbRs/OziKRSKCnpwcSiQQSiQTBYLBiaLNarTCbzawBYCKRIABIVBURACQiyiODwYDPf/7z2LdvX81vbPnc3d0NiURS9KSNevF89wfJwo9HTsWQQQ6BQICpqSlWzk1PufB4PBgcHITVaoVOp4NCoYBAIGCmXCgUCnR2drKS3qerN10uV83f23JN72tcyjY3bLuUqSFWqxUmkwnd3d0QCoWYmpqqCNosFgssFgurAEhEVA0RACQiKqC33noLJ510EiwWS81vbLkci8Wg1WrR1tZW1ynrNEeCOLSzKdn2RbQF8XgcVqsVIpGo6Jm1mZXY2WYbi0QiqFQq6PV69PX1YXh4GF6vt2qp2uHhYVAUtazT8nSbm+U29STVfr8fEokEOp1u0X8Ts7OzTONstVrNROz6+/uZ31250GYymWC1WgkAEtW9CAASERXQwsIC/vjHP2L16tXw+Xw1v7HlcjQahUKhQFdXV12nrGl/LP9zsvDjqbMRn0mmr2OxGIxGI2QyGRPNjEQimJycxMjICGw2GwwGA9RqddqUi8xK7FJmG1fDg4ODoChq2fbYi8fjTBuS5Tr/OBqNwu12QywWQyqVQq/XQ61WZ03xu1wuBrboKO7w8HBZ0GYwGNDb20sAkKjuRQCQiKgIffLJJ7jyyivxve99r64bMIdCIYjF4rq/ac9NDmFhxxeShR/6txAKhTAxMcGkakUiESiKSkvVtra2QqvVoru7Gw6HA263G4FAoG5h1+l0gqIoTExM1Hwt5ZpOpVqt1rp8n1P/OEht00S33KGnowgEAgiFQtjt9qwp/syG0XQE1G63I5FIlARter0eNpuNNQCcn5+v9dcf0REqAoBEREVqenoaK1euxJYtW+ryZph60+bz+XVVzTk7O4tAIIDx8XE4HA4Ed/0IaGnEzNNfB/VpqlYoFKKtrY2ZciGTyaBQKGrebqcSDwwMLOuRa/F4HNPT0xCJRDCbzTX5PdB9FXONwKP/OKD3cWZruVPK1BC6Qnh6ehpCoRAWi6UkCNTpdOjv7ycASFT3IgBIRFSCjEYjjj/+eLz//vs1vzHns8vlWvLoEz3lYnR0FHa7HUajEe3t7WmpWolEgm7q5WThR8uJmDDwc6ZqI5EIc2NfrgAYjx8euVbP2wcK2e/3QywWw2g0sv67oMcm0hHgnp6eRcU6dF9F+o+DoaEheL3ekkbg0dsLRCJRzkKjzDYxoVAIUqkUOp0O8Xi8KGjTarVwOBysAeDHH39c6689oiNUBACJiErUO++8gxNPPBEmk6nmN+Z8HhgYgEAggN/vZ/VGnS8aQ6fctFotLBYLBgYGMD4+fnjKRWwWB1+8FGhpxIEP7ih4zZmZGUil0pzVnMvFNpsNAoEg57iy5eBAIJCcnqHXl/y7oJtnezweOJ1OdHd3o7OzE62traAoKi0CnFqs4/P5WC3WicVi6OnpKWpqSDyeTAfToxfVajVmZ2cLQptGo4HT6SQASFT3IgBIRFSiFhYWcN999+Hss8+u+9Se2WyGTCYr+iYai8UQCATgdrvhcDjQ3d2dc74xHY0ZHBzExMREUVMu5jtfTUb/Hvsi4tPF9csLBAIQiUR1v6+xkPv6+lhtc1MLh0IhyGQydHZ2LqqspT87dJrfYrEsqsgWi8VLNuc4nwcGBvL2bMzWMLqjowOtra0Ih8N5oU2tVmNoaIgAIFHdiwAgEVEZOnDgAK666irccMMNdV0UEovFoNFomOhFPH44VUtPuTCZTNBoNJBKpeByueBwOJBIJOzfqEM+LDxxenLer/KvJT2XHnu3nHvTxeNx9PT0QCgULs+m3fHkPjmv1wuxWAy5XM58dmQyWVqav729HUajsS4qsnOZHn9X7NSQubk5GAwGiMViBAKBnNDW1taGkZERAoBEdS8CgEREZcrv92PVqlV46KGH6i49SW+cHx4ehtVqZaog6epIugVGR0cHzGYzBgYG4HK5ck65YMOf8B9Mtn157muIR0tvWD0xMQGKopb1vFq68bBIJGItNc+2U8fg2e12mEymrHs5+Xw+xGIx7HY7XC7X4TR/HbyGYl3s1BAaAunJIQKBAD6fLyu0KZVKjI2NsQKA8XicACBR1UQAkIioAlksFhx//PHYs2fPkoNEMBiE2+2G0+mE1WpFZ2cnFAoFk6oVCARQKBTQ6XQwmUzg8/kwGo01qaqdG7diYfvJybYvvbyyzzM2NgYej4fx8fGaw0MlvzuLxcJEkmqxhmg0isnJSYyOjqb1VkztkSeXy9HR0QGLxQKHw4Hx8fG0tjuRSARtbW1QqVRLnsJl0z6fD0KhMGeVcywWY6qD6apcusXP+Pj4ImhrbW3N+vNyAfCTTz6p9dcc0REqAoBERBXq3XffRWNjI4xGI6s3ptRITH9/P8xmM5NuS91TpVKpYDAY0NfXh5GRkZwb530+X80iaAff/AHQ0oiDb95U8bnoJsv1vv8yn2OxGEwmE8RiMYLBYFWuEYlE4PP5MDw8zPTIa2trS+uRl9pbsZwxeNFoFGq1Gm1tbVWbrLIUzjc1hP590W1i6ObMY2NjoCgKg4ODadAmk8kqmiRCAJBoqUQAkIioQi0sLOD+++/HWWedVXLbFfomTTeyNRgMUKlUWacVmM1m9Pf3Y2xsrOxU7ejo6JLDU6KHmyz82H4K5tzsFHLY7XYIBIJlu5eOhgqj0QiJRFI2BKb2yOvp6UFXVxeUSuWiWcc6nY7pkTcxMVFS+5RCjkajaG9vh0KhWNYQGAqF0Nraivb29qz7FVMhMB5PVgh7vV4mhUz3CpRIJJiYmCAASFT3IgBIRMSCDhw4gGuuuQbXX3992s0jFoshFArB4/EwUy50Ot2imzTd767cSEwpttvtEAqFS5N+jM3i0HPnJef9Cjayeu7u7u6qRtCWwrFYDHq9HlKpFKFQKOvjmVNSMlP9qVXZvb29TI+8pYSx2dlZpuKXHuG3HB0Oh6FSqaBQKHK+jsyG0X6/HyKRCCaTCYlEAiKRKOf+wHKKQA4cOFDrrzeiI1QEAImOWK1YsQINDQ1pbm5uZv06n3zyCQYHB7F3716ccsopuPLKK/Gtb30Lt99+O5OqFYlETI+z3t5eDA8PL/lNOhUqjEYj5HL5kuzdmhvV48A//wfxELs98Gh4KqXNTT16dnYWWq2WKahIbb1D98gTiURQqVRMjzw61V9Pe+/oaRsymWxZQ3k0GoVWq807NSSzOGRmZobpf0m3+iEASFTvIgBIdMRqxYoVeOSRR+Dz+RjPzs6yeo2FhQU0Njbic5/7HM4++2xccsklWLVqFW677TZ89NFHddn+gr5Zt7e3o729fdlVbma+jo6ODiiVyrp8n1OBge6RNzAwwPTIS93PyePxQFEUurq6YLPZMDo6Wrefn3yvU6/XQyKR1KzAha3XUezUkHj8cMNolUoFDocDr9fLWgqYACBRtUQAkOiI1YoVK/DMM89U/ToTExM4ePAg8+89e/agsbERBoOh5jeyfI5EIpDL5VUZ77WUpm+8tYZZumjH5XJl7a+Y2iPPZDIt2s9JRwLpZsO1fl/LNQ1PYrG4blvdFPs66Kkhufb2ZkYC4/E400dzZmaGACBRXYsAINERqxUrVuALX/gCTj75ZGzYsAE7duxYkp5aCwsLePDBB3HGGWfkHDdVLw4EAhAKhbDb7TVfSyUOh8NobW1FV1dXVWE227xjtVqd1iMvtWiH7q9YbI88OqK53AsqYrEYzGYzRCLRsi7UiceLmxqS2iaGw+Ggq6uL6fVIAJCoXkUAkOiI1c6dO9HW1gar1Ypdu3bh3//93/HrX/96Sa594MABXHfddbjuuuvqPoXn9XpBURRGR0drvpZKHAqFIJFIcvZzK9aRSASTk5MYGRlheuRlVmanzjt2OBxwu91pPfIq8ezsLDQaDZRKZV3t8SvVdNNroVC4rGcgx+OHp4YMDg7mfK3RaBTRaBQcDgczMzOw2Wzg8/kVpYPn5ubSsgtERGyKACDRslJLS8uiwo5MGwyGrM/94IMP0NDQgEAgsCRrDQaDOPPMM/HAAw/UfYp1ZGQEFEXB5/PVfC2V2O/3QygUoq+vL+9x4XCY6ZHX29vL9MgTCoWLKrOtViucTic8Hk9R847ZcGp/vXr/A6KQe3t7mckZtV5LJaanhthsNsTj6X8o9PX1MS14OBwOIpEI5ufnmZ6V5U4GIQBIVE0RACRaVvL7/ejv78/rRCKR9bkejwcNDQ3Q6XRLtt6enh6ccMIJeOedd2p+Aytkm80GkUi0rCs44/E4JicnQVEUbDYbJiYmmB55me13BAIBlEoldDodenp6qtIjrxLTexvVavWyh0C73Z53L129mp6YQkNeR0cHuFwuU52d2meR/gx5vd60htHj4+OgKAoOh6Mk+KP3FBIAJKqWCAASHTWiKAoNDQ1wuVxLet29e/fihBNOgF6vr/kNLZ/pCs7W1tZlkXpM7bGYOg4vtX0Kn89HW1sb0yOvlu13ygWQtra2nM2Jl5PpvXRut7vma8l8j1MhT6/XQ6VSMRNTMqPBNpsNQqEQnZ2dRTeMnpychEAgQE9PD9MwmgAgUa1FAJDoiJRWq8XOnTthsVgwMjKC999/H6eddhpuvPHGJV/LwsICNm3ahNNPP73ubn6Znp2dhVqthkajqYsoGN0+xe12w+FwMD3y5HJ5Wo/FzHF4k5OTGBoaqkvgKNWRSARKpRIajWZZt+yJx+PMDF2Xy7Wk100t3knd15k5Fo+GvELR4FAoBLlcDo1GkxPMMxtGBwIBiMViGAwGZqYwAUCiWooAINERKZPJhIsuuggnnngijjnmGJxzzjloaWlBPB6vyXoOHjyI66+/Htdcc03dR9fC4TBkMhnMZvOSXG92dhZ+v5/pkZdt5rFEIoFarYbRaITdbi+6R57T6QSfz1/2+8/C4TAUCgU6OjqWPQQODQ2Bx+OxXnSUDfLUajVTvENRFFO8Q0/cqWRfJz01RKlU5pwaktkmhq5Wp//AKgSAc3NzOHToUE2+s4iOfBEAJCJaIoVCIZx11ln4v//7v7qIruWz3++HQCDAwMAAK+eje+SNjY0xPfLa29vTeuRJpVJoNJqsPfIquTadslvOPelo4KCjVMsdAkdHR8Hj8TAyMlL25yi1DU+2Cm02IK+Q6akh+aafZDaMpqPsdPNyAoBEtRIBQCKiJVRvby8aGxvx9ttv1/wmXMgTExMlpesyIzDZbs4ymQwdHR1Mj7zx8XEEAoGqAk0sFoPFYoFEIsk6b3c5mR45ptPp6v6PiEIeGxsDRVGLWqvkgjyJRJK3DU+1ZmcX8/kyGAxFTQ2hI4Fzc3PQ6XTMDGgCgES1EAFAIqIl1gcffIATTjgBXV1dNb8JF/Lw8DD4fD7Txy2z9UXmhvnMNFutb86pN+Curi7I5fJlUwCSyzMzM5DJZFVvel1N05DX29sLLpfLTHKhG2qn/rFQD5BXzOfLarVCIBDknRqSuicwkUjAYrFAKBRmnR1MAJCo2iIASES0xFpYWMDmzZuxatWquixQCIfD8Hq9GBoaQm9vL+RyObhcLtM+hW59kbphfil75FUCHRqN5ojorRcKhSCVSmEwGOr2Paf3dqaOxkuFPHpqilKpBPf/t3fnYVHV+x/AjxAgi6JgIZpCt9w1vaS0mEsu11uSuTxFLmldr9t1vZrpo+mkYtfdzMhyLa3UriuDwyYwsiMKCgKCgKCyb85C4lV4//7omfkJgqDAHIbzfj3P558TymeGifP2nPP9fM+eRWRkpP6KcHN9TXVVcnLyE6+aV78SeO/ePf2fuXPnDgMgGRQDIJEIHj58iLFjx2LkyJEGXxSi1WpRWlqK7Oxs3LhxA1evXkVUVBSCgoIem5EXHR2N+Ph4KJVK+Pv7N9srMPUt3ViVlrCitri4GH5+fqLu5axbpV19AU9Nz3ZWv+3/aM85OTnw9vauc4C3MVR6enq9dg3RbR1XXl6O9PR0yOVyZGRkMACSwTAAEomktLQU3bp1w7///e9GP4FrtVoUFxfrZ+RduXLlsRl5CoUCSqUSFy9exLVr15Ceno7c3Nwab5FqNBoolUpEREQYdQAsK/vzCuf58+dx8eJFo38tutEiDd3+rq7P0tOGvKfZ/1hXj87KM/afi274s27XkJreU92YGN3A6Nu3b0MulyM5OblKAKysrBT7VxW1UAyARCJKTEyEra0tfvrppwadmFNSUhAXF4fw8PAqM/J8fX0REhKCmJgYJCYm6mfkPcst0NLSUvj7++PKlSuin2AbWsXFxfD19W0Rr6WwsBA+Pj64cuXKMwen6iFP91kKCAjQhzw/Pz+EhobqV2k/S8irq/Lz86FQKBr0WppL5eTk4Ny5c4iLi6vxtdQ0MPrR1//HH38wAFKTYgAkEtmpU6dgY2ODyMjIx04S1Z+jenRGXvUTs25GXmZmZr1m5D1LFRQU4Ny5c0hJSRH9BNtYr6W2qzTGVAUFBVAoFLh69WqtX6O7Kqwbqv1oyHt03uKjIa+xRvE87Wvx9fXF5cuXjT4E6l5LdHR0re9h9YHRxcXF8Pf316/0ZgCkpsIASJLn6ekJZ2dnWFhYwMXFBSEhIQb9/vfu3cO8efPQtWtXLF26FB999BFOnDhR5WH5R2+xNdXVl/rWnTt34OXlhVu3bol+gm1o5ebmwtvbu9bntYypdIE2Li7uiTun6K4KP/oPBkOHvLqqqKgIfn5+zXqRS32ruLj4qXcNUalUCAoKQmJiIgMgNRkGQJK0Y8eOwczMDPv27UNSUhIWL14Ma2trg+wXfPjwYTg5OaFVq1awtLSEra0t+vbti08++QRKpbJZr4i8ceMGvL29a517Zkyle/YqMzNT9F7qU9X3QK4p5Hl7e9e4c0pzCnl1le5K2JOunhlL3b17F0qlEsHBwbWOIao+MFq3UIQBkJoKAyBJmqurK+bOnVvlWM+ePbFy5com/95paWlQKpW4c+cOKioqUFpaip49e2Lx4sXNMvRVr6tXr8LPz6/WbbCMqTIyMvSjOMTuRRcGqoe86ot4Ht0DOTExUR/ydM+eJSUlif46Glq6PXcjIiKMPgSq1Wr9Lfe6dg3RrRC+d+9ek/8eIuliACTJun//PkxNTXHq1KkqxxctWoShQ4eK0lNSUhJsbW1x8OBB0U9YdZVWq0VkZKR+Syux+2lopaSkVBl6bYj3r/o4nvqGvLreb92t7cbayk/MKi0t1e+fa+yfM41Gg4sXL8LX1xcFBQW1fi7UajXUajUKCwtF+T1E0sAASJKVnZ0NQRAQHh5e5fjGjRvRvXt3kboCzpw5AxsbG0RERIh+LmcPqAAAGRBJREFUwqqr1Go1goODERkZaRRXLeuqhIQEKBQKFBUVNcrf96SQ5+3tXeM4noas1H60dLP1WsKCnbt37yI4OBghISFGHwJ1u4a4u7tDLpcjNzcXoaGhOHjwIFavXg13d3e89tpraNeuHVxcXET7PUQtHwMgSZYuAEZERFQ57uHhgR49eojU1Z87hchkMnTt2rXe+/CKWaWlpfDz83viClRjKa1Wi8uXLz/1vsGP7p4SHx+vH6xdV8hr6iHguv2cU1NTRX9vG1oqlQpKpRJKpdLgw9Mb+pkqLi5GTEwMfvvtN2zYsAHTp0/HkCFDYG5uDkEQ0L59ewwaNAhTpkyBTCbDL7/8gujoaKhUKtF+D1HLxwBIktUcbwHrPHz4EB988AGGDx9uFHvX5ufnt5jVtFqtFlFRUQgMDKwSNGoLebrdUxQKhX73lISEBP1gbbHDim6RS1pamujvbUNLrVYjJCTkiYspxPrMqFQqxMfH49SpU9iyZQtmz56NESNGwMnJCSYmJrCyskK/fv0wceJErFixAvv378eaNWtgaWmJI0eOiPr7hqSJAZAkzdXVFfPmzatyrFevXgZZBFKXu3fvolevXliwYIFR3F69desWvLy8muX+xvUtXci7ceMG/Pz8oFAoat0i79GQ15zCSG0/G7lcjvT0dNF7aWip1WqEhYUhMDDQoAuQdIObU1NToVAosGvXLixcuBDvvfceunfvDnNzc5iZmaFbt2547733sHjxYnh6eiIgIABZWVmoqKiocUWvn58f1q5dK8JvGJI6BkCSNN0YmAMHDiApKQlLliyBtbU1MjMzxW4NAJCcnIx27dph//79op9461MpKSk4d+5crQ+4N4dSqVTIzc1Feno6EhISEB0djeDg4MdCXmRkJHx8fBAUFITs7OxmH/LqqqysLMjlcty8eVP0XhpaGo1GP/rmaW7V16e0Wi1u3bqFwMBA/PDDD1i+fDkmTJiAfv36wdraGq1atYKTkxNGjBiBOXPmYPv27fDy8kJKSgru37/PsS1kNBgASfI8PT3h5OQEc3NzuLi44MKFC2K3VIWXlxesra0RHh4u+om3PnXlyhXRx8OoVCrk5eUhIyMD165dw8WLF6FUKqFQKPRz8oKCghAVFYX4+HikpaUhJyfnsZBXWlqKgICAFjGQuKysDJmZmUY18/BJpdFoEB0dDX9//1rHqtRWWq0WeXl5CA8Px08//YQvv/wSkydPxqBBg9C+fXsIggAHBwcMHjwYn376KTZu3Ijff/9d/5wrQx61BAyARM1cZWUl1q9fj65duxrFiVur1SIiIgJKpbJJZ7ep1ep6hbzIyEhcvXoVN27cQHZ2NkpLS58qzOn2DW4Ji1zKyspw8+ZNyOVyo1hgVFdptVrExMTA19cXycnJj/23kpISXLp0CceOHYOHhwc+/fRTDBkyBI6OjhAEAba2thg4cCA+/vhjrFmzBkeOHEFUVBRKS0sZ8qjFYwAkMgIVFRWYMGEChg4dahS3ItVqNYKCghAdHd2gK2ePhrzExETExMTgwoUL8PHxwZkzZyCXyxEYGNjgkFdX6bZZqx4yjLXS09Mhl8uNfjs/rVaLu3fv4quvvoK9vT1WrVqFOXPmYNSoUXB2doaJiQksLS3Rt29fjB8/HsuXL8f+/fuhVCqRm5uLiooKsf/XJhINAyCRkVCpVOjduzfmz59vFLcjS0pK4Ovri4SEhCd+nVqtRn5+Pm7evFnvkHfnzh2UlJQY9H3IyclpMatpy8rKkJaWBrlc3uwX7egWX9y4cQM+Pj7YvXs3Fi1aBDc3N/Ts2RPm5uYwNTVFz549YWFhgalTp2L37t3w9/fHzZs38eDBA17NI6oBAyCREUlJSUH79u2xd+9e0U/M9am8vDx4e3sjNTX1sZAXEhICX19fnDlzBl5eXvotv65cuYLU1FRRQl5dpVvp3BJun5aVlSE1NRVyuRzZ2dmi96LVanH79m0EBwdj7969+OKLLzBp0iT0798fNjY2EAQBXbp0wTvvvIPZs2dj69atOHv2LJKTk1FeXo6KigrIZDJ06NCBO2gQ1QMDIJGR8fb2hrW1NUJDQ0U/aT9aGo0GBQUFyMzMRFJSEi5duoSQkBD9M3nVQ15KSgpu376N4uLiZhXy6ird7dPmEJoao3Rb4OXk5DT599JqtSgoKEBkZCQOHz6MtWvXYsqUKXj99ddhb28PQRDw/PPP46233sL06dPh4eGB48ePIzY2Flqttl5X8qoPdieimjEAEhmZyspKeHh4oEuXLgYf6VFTyAsNDYWfn58+5AUEBCA8PBxxcXH6kJeQkABvb28UFhaKHngao65fv45z584hPz9f9F4ao5KTk+Ht7Y3c3NwG/126xReXL1/G8ePH8fXXX+Ozzz7DsGHD0LlzZwiCgLZt28LFxQUfffQRVq9ejZ9//hmRkZEoKSnh7VoiA2EAJDJCFRUVmDRpEoYMGdLo41Y0Gg0KCwuRlZWF5ORkXL58WR/yzp49i7Nnz9YY8oqKip54JS82NhYBAQFGsYilPhUfHw8fH59G2zdY7EpKSsK5c+eQl5dX59dqtVqo1WokJibCy8sL27dvx7x58zB69Gj85S9/gampKSwsLNC7d2988MEHWLZsGfbu3Yvg4GBkZ2fXOhSZiAyHAZDISKnVavTp0wfz5s176luo9Q15YWFhiI2NxfXr13Hr1q06Q15doSEsLAwhISFNOh7GUKXVanHp0iX4+/uLOvOwMevatWvYvHkzlEqlfvFFeno6fH194enpiSVLlsDNzQ29e/dG69atYWpqipdffhljxozB/PnzsWvXLvj6+iIjI4OLL4iaOQZAIiOWmpqK9u3b48cff6wxoBQVFeHWrVu4fv06YmNjERYWBn9/f33I8/f3b9SQV1epVCoEBgbi4sWLRvXc35NCYGRkJIKCgkTf87chryE7OxtKpRL79+/H+++/j7Zt26JXr15o27YtBEFA586dMWzYMMyaNQubN2/G6dOnkZSUhHv37jHkERkpBkCiZyCTySAIQpVycHAweB8VFRU4dOgQWrdujUWLFmHatGnYunUrAgIC9CHPz88PoaGhuHz5MpKTk5GVlYXCwkLRrsIVFxfDx8cHiYmJooefxiiNRoOQkJBmfWVTq9WisLAQUVFROHLkCL766itMmzYNb7zxBp5//nkIggB7e3u88cYbmDZtGoYPH4527drh9OnT0Gg0DHlELRADINEzkMlk6NOnD3Jzc/VVUFBgsO8fFhaG3r17w8LCAqamprCzs8PLL7+MiRMn4vDhw6KHvLoqNzcXcrkcGRkZovfSGKVSqfS7joh1ZVOr1aK0tBRxcXH473//i//85z+YOXMmhg8fji5dukAQBNjY2GDAgAH48MMPsWrVKhw6dAjh4eEoKiqqEvIqKyuxfPlyTJ061WCfaSIyLAZAomcgk8nQv39/0b6/br6ebgP6iooKfPjhhxg8eLDRPI+m25fWEONHDFGlpaXw9/fHpUuXmiwE6hZfJCcnQy6XY+fOnZg/fz7GjBmDV155Bc899xzMzc3Rs2dPvP/++1i6dCn27NmDoKAg3Llz56kWX1RWVqK8vLyJP8lEJBYGQKJnIJPJYGVlBUdHRzg7O8Pd3R3p6emi9qRWq9GvXz/MmTPHaJ6vS0pKgkKhaDEraYuKiuDj44P4+PgGhTyNRoOMjAz4+/vj+++/x9KlSzFu3Dj06dMHlpaWMDExwUsvvYTRo0fjX//6F3bu3AmFQoG0tDT873//4y1bIqoTAyDRM1AoFDhx4gTi4+MREBCAYcOGwcHBAUVFRaL2dePGDdjZ2WHPnj2ih6H6hp1Lly7h/PnzLWY8TH5+Pry9vXH9+vU6X3tOTg5CQkJw4MABrFq1Ch999BFcXFzQrl07CIIAR0dHDB06FDNnzsSmTZtw8uRJXLt2jYsviKjBGACJGoFWq4WDgwO2b98udivw8fGBlZUVlEql6GGoPqXRaBAaGorQ0NBm+8zi01Z2djbmzp2LXbt2oaioCBcvXsSvv/6K9evX45NPPsFbb72FF154AYIgwM7ODq6urpg6dSrWrVuHX3/9FTExMVCpVAx5RNRkGACJGsmoUaMwd+5csdtAZWUlNm3ahE6dOiE9PV30MFSfUqlUOH/+fJM+P9dUpdVqcffuXVy9ehUnT57Epk2bMGvWLIwaNQpmZmYQBAHW1tZ49dVXMWnSJKxcuRIHDx5EaGgo8vPzGfKISBQMgESNoLy8HJ07d8a6devEbgXAn+Nh3N3d8dZbb6GkpET0kFSfKioqgkKhQFJSkui91BTyNBoNUlJScO7cOXzzzTdYsGAB3n33XXTr1g1mZmYwMzND9+7dMXbsWCxZsgTff/89VqxYARsbG0RFRTHoEVGzwgBI9AyWLVsGpVKJjIwMREVFwc3NDW3atEFmZqbYrelpNBq8+uqrmDVrltFcVcvJyYFcLjf4HsePBr2srCycP38ee/bsweeff47x48ejX79+sLKyQqtWreDs7IyRI0di7ty52LFjB+RyOVJTU3H//v0aQ97OnTuxZcsWET4BRES1YwAkegbu7u5wdHSEmZkZOnXqhIkTJyIxMVHsth6TlpYGOzs7eHp6ih7u6lsZGRmQy+XIzc1tspCXl5eHsLAwHDp0CKtXr8bHH3+MgQMH6hdfdOzYEW+//TY+++wzfP311/oFP2VlZbySR03Ow8MDb775JiwtLWFra1vj11QfRC8IAvbs2WPgTsmYMQAStXB+fn6wsrJCUFCQ6OGuvpWYmAgfHx8UFxc/c8grLi5GTEwMjh49ig0bNmDGjBl4++230bFjRwiCgHbt2mHQoEGYPHky1q5di19++QXR0dG4e/cuQx6Jau3atdixYweWLl36xAB46NChKsPo//jjDwN3SsaMAZCohausrMTWrVvh6OhoNItCtFotYmJiEBgYWOseu1qtFiqVCgkJCTh9+jS2bNmC2bNnY+TIkXB2doaJiQksLS3Rr18/TJgwAV988QX279+PCxcuIC8vDxUVFWL/aIie6NChQ08MgKdPnzZwR9SSMAASSUBFRQUmT56MN954w2gWhWg0GuzcuRMjRoxAUlISfHx88O2332LhwoUYO3YsevToAXNzczz33HPo1q0b3n33XSxatAjfffcdAgICkJmZiYcPH/JqHhmtugJg586dYW9vj4EDB2LPnj38Rw09FQZAIonQarUYMGAA/vGPfzS7RSFarRa3bt1CUFAQfvzxRyxfvhwTJkyAi4sL7O3tYW5ujq5du2LEiBGYM2cOtm3bBi8vL1y/fr3WxRdExu5JAXDDhg2IiIhAXFwctm3bBisrK2zYsMHAHZIxYwAkkpCMjAzY29tj9+7dooS8/Px8hIeH4+eff8aaNWswefJkuLq6ws7ODoIg4IUXXsDgwYMxY8YMbNy4Eb///ju8vb3RoUMH7N69W+y3j+iZyWSyGhduPFoxMTFV/syTAmB127ZtQ9u2bZuidWqhGACJJOb8+fOwsrJCYGBgk4S8kpISXLp0CcePH8fGjRvx6aefYsiQIejUqRMEQYCtrS1ee+01fPzxx/jyyy9x+PBhREVFobS0tNYreREREVi5cqWB3ymixlNYWIjk5OQn1r1796r8macJgGFhYRAEAXl5eU3RPrVADIBEElNZWYnt27ejY8eOSEtLe6aQp1KpcO3aNZw5cwbbtm3D3LlzMWrUKLz00kswMTFB69at0adPH4wfPx6ff/459u3bB6VSiZycHFRUVPCWLRlMfUaqZGVlwc3NDVZWVrC3t8fChQtx//59A3f6uKcJgLt370br1q1RXl7exF1RS8EASCRBFRUVmDp1KlxdXWsctaLb+SItLQ0+Pj747rvvsHjxYri5uaFXr16wsLCAqakpXnnlFfz973/HggUL8O2338LPzw83b97EgwcPGPKoWahrpMrDhw/Rt29fvPPOO4iNjUVAQAA6deqEBQsWiNDtn7KyshAXF4d169bBxsYGcXFxiIuLg0ajAQB4eXlh7969SEhIQFpaGvbt24e2bdti0aJFovVMxocBkEiiysrK8Ne//hVTpkxBcHAw9u3bhxUrVmDSpEno378/2rRpA0EQ8OKLL2L48OGYPXs2tmzZgjNnziA5ORnl5eUMeWQ0aruaplAoYGJiguzsbP2xo0ePwsLCAiqVypAt6s2YMaPGZwSDg4MBAD4+PhgwYABsbGxgZWWFvn374ptvvsGDBw9E6ZeMEwMgURO6cOEC3Nzc4OjoWOPcrsrKSshkMjg6OqJ169YYNmwYrl27ZrD+bt68CRMTE9ja2uLNN9/E9OnTsWHDBhw7dgyxsbHQaDQMedQi1BYA16xZg1dffbXKsZKSEgiCgKCgIEO1R2RwDIBETUihUGD16tU4efJkjQFw06ZNaNOmDU6ePImEhAT9FnNqtdpgPWZlZTHkUYtXWwCcNWsWRo8e/dhxc3Nz/Pbbb4ZojUgUDIBEBlI9AFZWVqJjx47YtGmT/lh5eTlsbW3xww8/iNEikVFozJEqs2bNwt/+9rfHjpuZmeHo0aNN9hqIxMYASGQg1QNgeno6BEFAbGxsla8bN24cpk+fbuj2iIxGY45U4S1gkioGQCIDqR4Aw8PDIQhClYfPgdqvSBA1N05OTo9deVuxYoXYbdWorkUgOTk5+mPHjh0TdREIkSEwABIZSG0B8NETDwD885//xJgxYwzdHtFTc3Jywvr165Gbm6sv3aiS5qKukSq6MTAjR45EbGwszp8/jxdffFHUMTBEhsAASGQgvAVMLY2TkxN27twpdhtPVNdIFeDPkDh27FhYWlrCzs4OCxYs4EBlavEYAIkMpLZFIJs3b9Yfu3//PheBkNFwcnJCx44dYWdnh/79+8PDw6NZ7KBBRHVjACRqQhqNRn/LSRAE7NixA3FxccjKygLw5xgYW1tbnDp1CgkJCZg8ebLBx8AQPasdO3ZAqVTi6tWr2LdvHzp06ICZM2eK3RYR1QMDIFETCg4OrvH204wZMwD8/yDojh07wsLCAkOHDkVCQoK4TZOkPcuIFZ0TJ05AEAQUFRUZuGsieloMgEREpPcsI1Z07ty5A0EQEBUVZeCuiehpMQASERmYp6cnnJ2dYWFhARcXF4SEhIjdUqOQy+UQBEH/iAMRNV8MgEREBnTs2DGYmZlh3759SEpKwuLFi2FtbW10oSkiIkL/TGtGRgaOHz+OTp06Ydy4cWK3RkT1wABIRGRArq6umDt3bpVjPXv2xMqVK0Xq6NlcvnwZr7/+OmxtbdG6dWv06NEDMpkMZWVlYrdGRPXAAEhEZCD379+HqakpTp06VeX4okWLMHToUJG6IiIpYgAkIjKQ7OxsCIKA8PDwKsc3btyI7t27i9QVEUkRAyARkYHoAmBERESV4x4eHujRo4dIXRGRFDEAEhEZCG8BE1FzwQBIREalpkHFDg4OYrdVb66urpg3b16VY7169TK6RSBEZNwYAInIqMhkMvTp0we5ubn6KigoELutetONgTlw4ACSkpKwZMkSWFtbIzMzU+zWiEhCGACJyKjIZDL0799f7DYaxNPTE05OTjA3N4eLiwsuXLggdktEJDEMgERkVGQyGaysrODo6AhnZ2e4u7sjPT1d7LaIiIwKAyARGRWFQoETJ04gPj4eAQEBGDZsGBwcHFBUVCR2a0RERoMBkIiMmlarhYODA7Zv3y52K0RERoMBkIiM3qhRox7bXo2IiGrHAEhERq28vBydO3fGunXrxG6FiMhoMAASkVFZtmwZlEolMjIyEBUVBTc3N7Rp04ZjVIiIngIDIBEZFXd3dzg6OsLMzAydOnXCxIkTkZiYKHZbRERGhQGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGIYAImIiIgkhgGQiIiISGL+D46LC9CUHHwlAAAAAElFTkSuQmCC\" width=\"640\">"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib notebook\n",
    "\n",
    "from sklearn.decomposition import PCA\n",
    "\n",
    "plt.figure()\n",
    "plt.axes(projection='3d')\n",
    "plt.plot(*X, '.')\n",
    "#plt.axis('equal');\n",
    "\n",
    "pca = PCA(3)\n",
    "pca.fit(X.T)\n",
    "\n",
    "for col,u in enumerate(pca.components_.T):\n",
    "    plt.plot(*(np.c_[-u, u]*15 + Xbar));\n",
    "\n",
    "plt.gca().view_init(20,70)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "mit Hauptkomponenten"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-24T09:28:15.400497Z",
     "iopub.status.busy": "2021-09-24T09:28:15.399489Z",
     "iopub.status.idle": "2021-09-24T09:28:15.402242Z",
     "shell.execute_reply": "2021-09-24T09:28:15.402572Z"
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-0.6248, -0.6964,  0.3531],\n",
       "       [ 0.2762, -0.6201, -0.7342],\n",
       "       [ 0.7303, -0.3612,  0.5798]])"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "pca.components_.T"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "und Varianzen"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-24T09:28:15.405231Z",
     "iopub.status.busy": "2021-09-24T09:28:15.404840Z",
     "iopub.status.idle": "2021-09-24T09:28:15.406558Z",
     "shell.execute_reply": "2021-09-24T09:28:15.406884Z"
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([52.785 ,  4.4244,  1.0172])"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pca.explained_variance_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### PCA bei Klassifikation von Ziffern"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wir wenden die oben erklärten Methoden auf die Klassifikation\n",
    "von handschriftlichen Ziffern an.\n",
    "\n",
    "In Scikit-Learn gibt es einen entsprechenden Datensatz mit\n",
    "8x8-Pixel Graustufenbildern."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "code_folding": [
     10
    ],
    "execution": {
     "iopub.execute_input": "2021-09-24T09:28:15.413445Z",
     "iopub.status.busy": "2021-09-24T09:28:15.408209Z",
     "iopub.status.idle": "2021-09-24T09:28:16.591121Z",
     "shell.execute_reply": "2021-09-24T09:28:16.590828Z"
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjwAAADVCAYAAABT2yr5AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAVU0lEQVR4nO3df8heZ33H8c+3S9nWdk3SCZ0Wl8TSWqeYxP0x2GBJsFpb/0jUwcpKTfOPo10lDdb5lzSpghMGSVjrpFCePLNj0EGXaCfKsEmZHThlaQShlrIkVjDYjiT2x1q1u/bH/aSEcD5Pct25zjl3v8/7Bc8/V27u+/s95zrn/uY81/e5opQiAACAzC4ZOwAAAIC+UfAAAID0KHgAAEB6FDwAACA9Ch4AAJAeBQ8AAEiPggcAAKQ3asETEddFxGsR8ciYcfQlc34R8fI5P29ExN+NHVcrEfGbEfFwRByPiJci4nBE3Dx2XK1FxOqI+GZEnIyIExHxQEQsGzuuVrKfx+z5nRERhxbupWfuNz8eO6aWst9PJSki3hMRT0TE6Yh4LiI+NnQMYz/heVDS90eOoU9p8yulXHHmR9LVkv5X0j+PHFZLyyQ9L2mDpOWSPi/p0YhYPWZQPfiKpJ9LerukdZrke9eYATWW/Txmz+9sd59133n32MG0lP1+uvCfqAOSHpd0laRPSXokIq4fMo7RCp6IuFXSKUnfGSuGPmXP7xx/psmX5r+PHUgrpZRXSik7SynHSin/V0p5XNJRSX84dmyNrZH0aCnltVLKCUnfkvTekWNqJvt5zJ7fEpXufirpBknvkLS7lPJGKeUJSU9Jun3IIEYpeCLiSkn3S/rMGJ/ft+z5ddgq6R9K4n1KIuJqSddL+tHYsTS2V9KtEXFZRFwj6WZNip6UEp9HSenz+1JEvBgRT0XExrGD6VHG+2mYsfcNGcRYT3i+IOnhUsrzI31+37Ln96aI+H1NHqfPjx1LXyLiUkn/KGm+lPLM2PE09qQmT3R+Iemnkn4gaf+YAfUl+XnMnt/nJL1L0jWSHpL0jYi4dtyQ2kt8P31Gk6dWn42ISyPiw5rkedmQQQxe8ETEOkk3Sto99GcPIXt+HT4p6bullKNjB9KHiLhE0tck/VLS3SOH09RCbt+W9JikyyW9TdJKSV8eM64+ZD6PUv78SinfK6W8VEp5vZQyr8mvQ24ZO64epLyfllJ+JWmLpI9KOqHJbz8e1eQ/WYMZoxtjo6TVkn4SEZJ0haTfiIg/KKV8YIR4Wtuo3Pmd65OS/mbsIPoQkxP4sCaLCG9ZuGgzuUrSOyU9UEp5XdLrETEn6YuS/nrUyBrKfh6z52cUdf+a5K0u7f20lPJDTZ7qSJIi4j808JOsGPrXhBFxmaQrzxq6V5MC4c5SyguDBtOD7PmdLSL+WNK/Sfq9UspLY8fTWkR8VZPOpRtLKS+PHE4vIuK/NfkVwd9qUpzPSXq1lHLbqIE1lP08LoH8Vkj6I01+/fprSX+uyZz9QCklTXv6Erifvl/Ss5r8ZukuSX8l6YaF/2wNYvBfaZVSXi2lnDjzI+llSa9lKQay53eOrZIeS3pxrpL0l5p8kZw46+9jpCkEFnxc0kckvSDpOU2+UHaMGlFD2c9j9vwWXKrJU8cXJL0o6dOStmQqdhakvZ8uuF3SzzRZy/NBSR8astiRRnjCAwAAMLSx//AgAABA7yh4AABAehQ8AAAgPQoeAACQHgUPAABIj4IHAACkR8EDAADSo+ABAADpUfAAAID0KHgAAEB6FDwAACA9Ch4AAJAeBQ8AAEiPggcAAKRHwQMAANKj4AEAAOlR8AAAgPQoeAAAQHoUPAAAID0KHgAAkB4FDwAASI+CBwAApLfsPP9eWnzIsWPHOsc3btxY9fopxAW8pkmOp06d6hx3OW7ZsqVzfOfOnbUf3TzHp59+unN8z549nePz8/M1b6+DBw92jrtjpfPn2Jmfy8Mde3cOXd533HHHecK6YM3PoYvNnatVq1Z1ju/du7dzfPPmzTXhSBeRozsv99xzT+e4O+/79+/vHF+9evV5wrpgzc/jihUrOsddzIcOHap6nylMdS06tee2Nr8pjsdg89Rdi8uXL+8cd/N6ivnbfJ664+zuQwPcaztz5AkPAABIj4IHAACkR8EDAADSo+ABAADpnW/RchNuEbJbuPRW5BYbu9zdwtlZsG7dus7xffv2dY67hWbufdziu0UWLU/FLaRz884tZHSLDhsuWp6am1+1C8ldLlMsTm6udtGn43J082QWuDnp5vBQ11Yr7t7h5rW7z7qFuw0Xa0/N5bJ27drO8SNHjlS9T8NF9+d14MCBzvFW32fbtm3rHG91r+UJDwAASI+CBwAApEfBAwAA0qPgAQAA6VHwAACA9Abp0nIr8d3KbrcS/OjRo53jroujD26lvPsT/Nu3b+8cr+2+mIVuA8fF7MZd91ZrtZ0D7vWuw8V1xAyVn+Q7NNy8c/N0165dnePr16/vHB+ye8vN/dpry+U4y9wcc+Oui2lWO9Fc56cbn9VuM6l+ewsnonvnB/c+s3xMWml1r+UJDwAASI+CBwAApEfBAwAA0qPgAQAA6VHwAACA9Jp2abkuF7e63K28du8z5N5FrhvLrYh3+6K4romVK1d2js/NzXWOD5l7bVeBy9E5ffp01eun5TqYXFdf7d5us7wXXKv9dVatWtXkfS6G6/J0XIePMwt7FDmu88fFPKtc5+3x48c7x909pbaLdRa609z8rY1t//79Va+vvS9fCNed6Tqo3bXocnH7iLl5QpcWAADAOSh4AABAehQ8AAAgPQoeAACQHgUPAABIb6ouLdcx5Fbi33fffZ3jbgW3W6k95H5SrpOntqugtsNkyH2YXMeD65Kr7a5yex0tX7686n2G4uajO06z0MXjuHnnuq7cteXmdW3HSB/ceXGxuXk3y+fRcTnOai6ui7V2TzbXJeuu3SH3n3LXSqs93Nz34pB7STpu3tV2irkcT548WRlRN57wAACA9Ch4AABAehQ8AAAgPQoeAACQHgUPAABIL0opi/175z+6lde1e4O4Lgu3T4wbX6R7Ky4ghs4c3V5XrbiuBbe/2CLOm6PMeXTccXbdAO71U+TinC/HqvxcN+HWrVs7x938cp0hrtPuYuapTI4uF9dp57odXNdPq33H1MM8dfebTZs2dY67a65hx1PzHN1xdvcn1xHrcnHvs0j3VNNrsZWI7rAOHjzYOb5Il9bU59DdB9335fz8fOe4626t7fhdRPN5Wqv22nXzepEusM4cecIDAADSo+ABAADpUfAAAID0KHgAAEB6FDwAACC9qfbSqt0fw3F7ck3R5TI19567d+/uHHcdHa6bwu0NMuSeWbXcMXEdQbOwr1IX1wngurHcXmFu3HUUzM3NdY67+X4xjh492jnu9szqu8NyyP3uao+nuxbXrFnTOd6wk3JqtXs+1e7b5OZJ7R5XQ3GdkbOwT5r7LNfp6Lq0XIflLKvd166246zVeeQJDwAASI+CBwAApEfBAwAA0qPgAQAA6VHwAACA9Kbq0qrlVnC7bge3En9IbmW9G3dc18osd2m5PWE2bNjQOT6rHR21XFeOO4dufMguC9ep5Lqu9u7d2znuulxcN8WQ3ViOO84ud9fpUXt+h1R7n3Cvd91etV1gQ3FxuXtTw/3QmnPX0Cx0lrXivstd7u48un3EWnW48oQHAACkR8EDAADSo+ABAADpUfAAAID0KHgAAEB6UUoZOwYAAIBe8YQHAACkR8EDAADSo+ABAADpUfAAAID0KHgAAEB6FDwAACA9Ch4AAJAeBQ8AAEiPggcAAKRHwQMAANKj4AEAAOlR8AAAgPQoeAAAQHoUPAAAID0KHgAAkB4FDwAASI+CBwAApEfBAwAA0qPgAQAA6VHwAACA9Ch4AABAehQ8AAAgPQoeAACQHgUPAABIj4IHAACkR8EDAADSo+ABAADpUfAAAID0KHgAAEB6FDwAACA9Ch4AAJAeBQ8AAEiPggcAAKRHwQMAANKj4AEAAOlR8AAAgPQoeAAAQHoUPAAAID0KHgAAkN4oBU9EHIqI1yLi5YWfH48RR98i4rqFPB8ZO5bWImJ1RHwzIk5GxImIeCAilo0dV0tLIcczks/V90TEExFxOiKei4iPjR1TSxFxVUT8S0S8EhHHI+Ivxo6ptaVwLWafp9L453HMJzx3l1KuWPh594hx9OlBSd8fO4iefEXSzyW9XdI6SRsk3TVmQD1YCjmekXKuLtxMD0h6XNJVkj4l6ZGIuH7UwNp6UNIvJV0t6TZJfx8R7x03pOZSX4tLZJ5KI59HfqXVk4i4VdIpSd8ZOZS+rJH0aCnltVLKCUnfkpTtJrsUcsw+V2+Q9A5Ju0spb5RSnpD0lKTbxw2rjYi4XNInJH2+lPJyKeW7kr6uJPmdJfu1mHqenmXU8zhmwfOliHgxIp6KiI0jxtFcRFwp6X5Jnxk7lh7tlXRrRFwWEddIulmTyZtJ+hyXwFwNM/a+oQPpyfWS3iilPHvW2BHlKgak/Ndi9nl6xqjncayC53OS3iXpGkkPSfpGRFw7Uix9+IKkh0spz48dSI+e1OSm+gtJP5X0A0n7xwyoB0shx+xz9RlNHqF/NiIujYgPa/IY/bJxw2rmCkmnzxk7Lel3RoilT9mvxezz9IxRz+MoBU8p5XullJdKKa+XUuY1eXR3yxixtBYR6yTdKGn3yKH0JiIukfRtSY9JulzS2yStlPTlMeNqaYnkuE7J52op5VeStkj6qKQTmjzJelSTm20GL0u68pyxKyW9NEIsvVgK1+ISmKczcR5nZQ1PUfcjvbeijZJWS/pJRJyQdK+kT0TEf40ZVGNXSXqnpAcWitb/kTSnJEXrgqWQ40bln6sqpfywlLKhlPK7pZSbNHm6/J9jx9XIs5KWRcR1Z42tlfSjkeLpw1K4FrPPU2kGzuPgBU9ErIiImyLityJiWUTcJulPNan8MnhI0rWarEBfJ+mrkv5V0k3jhdRWKeVFSUcl3blwDldI2qrJ2oEUlkKOWgJzVZIi4v0L95vLIuJeTTpE9o0cVhOllFc0+R/z/RFxeUT8iaTNkr42bmTtLJFrMfU8lWbjPI7xhOdSSV+U9IKkFyV9WtKWUkqKv8VTSnm1lHLizI8mj5xfK6W8MHZsjX1c0kc0OY/PSfq1pB2jRtRe6hyX0Fy9XdLPNFkj8UFJHyqlvD5uSE3dJem3NcnvnyTdWUrJ9IRHSn4tLsg+T6WRz2OUUob6LAAAgFHMyhoeAACA3lDwAACA9Ch4AABAehQ8AAAgPQoeAACQHgUPAABIj4IHAACkR8EDAADSo+ABAADpUfAAAID0KHgAAEB6FDwAACA9Ch4AAJAeBQ8AAEiPggcAAKRHwQMAANKj4AEAAOlR8AAAgPQoeAAAQHoUPAAAID0KHgAAkB4FDwAASG/Zef691LzZHXfc0Tk+Pz/fOb58+fLO8WPHjnWOr1ixoiYcSYoLeE1Vjs6pU6c6x7ds2dI57o6VG19E8xx37tzZOb5r167O8cOHD3eOr1u3rnPcHatFzu/5cqzK78CBA53ju3fv7hzfuHFj57g7Tj3kJzWap08//XTnuMtx//79Va9fRPMcDx061Dm+b9++znGX+z333NM5PgvXostl27ZtneNr167tHF+9enXV+w91LbprxR17d85n+TvDxebuH+6+6ebpFJrnWHu+3Otdjq3OI094AABAehQ8AAAgPQoeAACQHgUPAABIL0pZdG1S1cKliO61UBs2bOgc37FjR+f45s2baz520ZAu4DVNFoO6xclu0ZZbQOkWFy6ieY7uPK5atapzfO/evZ3j27dv7xx3ObpjpcYLJd25couZ3cLd++67r3N8ikWwgy3o3bRpU+e4u0YXOSe1ps7RLWxduXJl57ibp27ePfnkk53jtYvxNeC16Bo+XGwux5MnT3aOD7Vo2cV75MiRznF3zblry+UxZAOBm3fHjx+veZuWTT5T5+g+a82aNZ3j7lp0C7bdvZlFywAAABeIggcAAKRHwQMAANKj4AEAAOlR8AAAgPTOt7VEJ7dS23Er8d2q81nmVpe7Dp+5ubnOcbfq3B3bKbq3puY6IdzWEm5lvevS2rNnzzRhNeM+33UkufxcZ5N7/ZDcPHXXXO0xcfN3kQ6mqbmORqf2T/avX7++c9x1h/WhVY6uQ9Dl7joQp9hWYyquG6u2I8l1Cbn8GnYCv8ldK64bq3YrGzdP3fYgDbeieJP7HnLny11Dtd9zra5FnvAAAID0KHgAAEB6FDwAACA9Ch4AAJAeBQ8AAEhvkC4tt9eSG3f7b7hV8H10MLnPcp1KriPJdey4mF13xJCdTS5ml3vtfiljc8fe5T0/P985Xrt/Tx/cvHB7J7muFccdk9puoIvhulZcl9y2bduafO7p06ebvM+FqO1Cqe3Aceel9l7eWm1HqLsWXcdTH91YTu18qe3Mc/q45mq581L7XbJ169ZWIXXiCQ8AAEiPggcAAKRHwQMAANKj4AEAAOlR8AAAgPSm6tJyq8LdCmvXzeLeZ+XKlZ3jQ+4z5bog3J4hLhf3Pm5F/yysuK/tAHF7xbjOkyG7mLq4uGo7mIbca8lx++g4rivGXVtung6119JiXCelG3c2bdrUOX748OHO8SH3Yer7/cfupGx1Dc3CfdPNC3dfcZ29fc+FPrguudr9MvvuGuQJDwAASI+CBwAApEfBAwAA0qPgAQAA6VHwAACA9Kbq0nJdNrUdI7Urst3eI26vnYvhcnRdK63273F7jwzJ7cPkuhAOHDjQOT5kV10Nt/9U7V44rpuitsvtYtR2vB05cqTq9WvXru0cH/scLqb2fuA6SdavX98gmgtT+1luDrtrznVSuo7YodTup+jycPfNWeikdPfN2m6/iOgcH/JadPc8950xNzfXOe5idh2Trrao7RblCQ8AAEiPggcAAKRHwQMAANKj4AEAAOlR8AAAgPSm6tJyXOeAWylf29XVRzeW4/Y/cTG7vVzcKnLXXTPkPlOu681x3VhO3/uiTKs2b2cWOpVq95NynWiuy2UWugYdNx+PHj1a9T7umNTuA3QxNmzYUBXDjh07ev3cobj83HeGe31th+WQ3LXY6j40pNp9vtzra++drb5LeMIDAADSo+ABAADpUfAAAID0KHgAAEB6FDwAACC9pl1atXtsua4r19nkOqH64HJptU/SkB1njjueW7du7Ryfn5+vev2sdvi02nendh+XIbn5VZvjLMxTx+3ztX379s5xl7t7/ZC5u/uN63x1e/e5Pah27txZ9blDcZ1KLu9Z2L+ulusgc53Abl8q11E3ZLdo7XGu7SJ1ObY6vzzhAQAA6VHwAACA9Ch4AABAehQ8AAAgPQoeAACQXpRSxo4BAACgVzzhAQAA6VHwAACA9Ch4AABAehQ8AAAgPQoeAACQHgUPAABI7/8BCgPUAKPZw9UAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x288 with 20 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "from sklearn import datasets\n",
    "from sklearn import svm\n",
    "from sklearn import metrics\n",
    "\n",
    "ds = datasets.load_digits()\n",
    "X = ds.data\n",
    "y = ds.target\n",
    "\n",
    "def digiplot(imgs, target = [], n = 10, mi = 8, pi = 8):\n",
    "    # Anzahl der Subplots\n",
    "    if imgs.ndim < 2:\n",
    "        images = imgs.copy()\n",
    "        images.shape = np.r_[1,imgs.shape]\n",
    "        ni = 1\n",
    "        m = 1\n",
    "        n = 1\n",
    "        fig, ax = plt.subplots(m, n, figsize = [n, 2.0*m])\n",
    "        ax.axis('off')\n",
    "        ax = [ax]\n",
    "    else:\n",
    "        images = imgs\n",
    "        ni = images.shape[0]\n",
    "        m = (ni -1) // n + 1\n",
    "        \n",
    "        if m==1:\n",
    "            n = ni\n",
    "        \n",
    "        fig, ax = plt.subplots(m, n, figsize = [n, 2.0*m])\n",
    "        ax = ax.flatten()\n",
    "        for axk in ax:\n",
    "            axk.axis('off')        \n",
    "\n",
    "    for k, axk in enumerate(ax[:ni]):\n",
    "        axk.matshow(images[k].reshape(mi,pi), cmap = plt.cm.gray_r)\n",
    "        axk.set_title(str(target[k]))\n",
    "\n",
    "ndigi = 20\n",
    "digiplot(X[-ndigi:], y[-ndigi:])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Die 1797 Bilder sind in Form einer Matrix mit Spaltenvektoren der Dimension 64 abgespeichert:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-24T09:28:16.593703Z",
     "iopub.status.busy": "2021-09-24T09:28:16.593246Z",
     "iopub.status.idle": "2021-09-24T09:28:16.595598Z",
     "shell.execute_reply": "2021-09-24T09:28:16.595237Z"
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1797, 64)"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Klassifikation mit k-Means-Clustering ohne PCA"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wir lassen den Datensatz jetzt direkt mit k-Means-Clustering in 10 Klassen einteilen. Die einzelnen Daten $x_j$ haben Dimension 64. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-24T09:28:16.598766Z",
     "iopub.status.busy": "2021-09-24T09:28:16.598362Z",
     "iopub.status.idle": "2021-09-24T09:28:22.258009Z",
     "shell.execute_reply": "2021-09-24T09:28:22.257703Z"
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjwAAABeCAYAAAAnrM7iAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAJiUlEQVR4nO3dP2xdZx3G8eetOhS1TSxUtkh2hFRR/jmeGeyoFQMSOBJiQ7IHYHUywILAyQISDLYlFtTB9oBUWHAkBFIHiBfE5iRQtR2QHQUmhthAG4nlMPhavU3f5/q+1++5vufn70eKovve23PO4/eck59uz89vappGAAAAkT133gcAAADQNgoeAAAQHgUPAAAIj4IHAACER8EDAADCo+ABAADhVSl4Uko/TSndrLGtU/bzjZTSW23vx+w7dMbo+Xr7JmO9/ZCxvf2GztfbNxnr7YfzdFhN05zpj6TPSPqnpE/1jb0u6T1JH0r6k6Tpwm2uSNqX9IGkdyW92vfe3yR9+azHTcaLk4+MZOxKxuj5yBgjY1fz1Qj+fUlv9r1+RdKRpG9JekHSzyX9pWB735H0UNLnJSVJn5X06b73fyjpF+Oa2IuQMXo+MpKxKxmj5yNjjIxdzVcj+B8lfbvv9fck/bnv9YuSnkr63BDbek7SY0mvD/jMVyTtj2tiL0LG6PnISMauZIyej4wxMnY1X41neL4k6f2+11+Q9ODkRdM0H0j6e2/8NFd6f76YUnqcUtpPKd1JKfUf57uSZlJKl85+6EOLnjF6PomMZPykScwYPZ9ExggZO5nv+bP8xz1Tkv7T9/olSf965jNHkl4eYltXen9/Vcc/0ClJb0v6h6Q3e++d7GtK0r9LD3ZEU4qdcUqx853si4xk7DeJGacUO9/JvsjY7YxT6mC+Gt/wPNHHQ/1X0rNV2CV9/IfjPO39/bOmaQ6bpjmQ9EtJX+v7zMm+DouPdHTRM0bPJ5HxBBk/MokZo+eTyHiiyxk7ma9GwfNQ0qt9r9+RNHvyIqX0oo4fQHpniG29L+l/kpoBn3lN0kHTNOOq1KX4GaPnk8hIxk+axIzR80lkjJCxk/lqFDy/lzTf9/q3Ov5/cd9MKb0g6ceSHjZN854kpZSWU0oHuQ01TfOhpF9L+kFK6eWU0hVJ35X0u76PzUv6Q4XjLhE9Y/R8EhnJ+IwJzRg9n0TGCBm7ma/C09qv6Pj/tfX347+h4378p5LuSZrpe+9Hkn41YHuXJL2l46/CHvd+cKnv/b9Kmj3rcZPx4uQjIxm7kjF6PjLGyNjVfLXC/0TSzSE/+7ak10bcz9cl/WZck3qRMkbPR0YydiVj9HxkjJGxi/lSb4MAAABhsXgoAAAIj4IHAACER8EDAADCo+ABAADhUfAAAIDwKHgAAEB4FDwAACA8Ch4AABAeBQ8AAAiPggcAAIRHwQMAAMKj4AEAAOFR8AAAgPAoeAAAQHgUPAAAIDwKHgAAEB4FDwAACI+CBwAAhEfBAwAAwqPgAQAA4VHwAACA8J4/5f2mxk4WFhay44eHh9nx+/fv19itJKUhPlMl4+3bt7Pj6+vr2fGDg4Ps+NTUVOmuq2d081I6jzs7O9nxa9eulRyOdHrGKnN49+7d7PjKykp2/N69e9nxmZmZ0l2P7TxdXl7OjrtrzmWchPPUuXnzZnbcnadbW1s1ditNwDyWXosjqHotll5z7r5Z0bnPoZsrdy22cD+Vgp6nfMMDAADCo+ABAADhUfAAAIDwKHgAAEB4pz20XMQ9gLa7u5sdX11drbn7c1X6cOcID32OjXvQ+sGDB9nx2dnZ7PikZnQPzLnz0T0UOMLDydW5LKUPC7rPT+ocSv483djYyI6vra21eDTtcA9gb29vZ8c3NzdbPJr63JzcuHFjvAfSIneeujlcXFzMjo/wcPLYuKYHl3FpaanFo/H4hgcAAIRHwQMAAMKj4AEAAOFR8AAAgPAoeAAAQHipaQb+BumiXy/tniJ33T17e3tF2xlB9V+h7Z5Gn5uby467LgTXfTGCkTO6X9N+9erVogOYgHms8uvsXWfI/v5+drxil9bY5nB+fj477roMK6p+LZaeXxWXyXCqZ3TH5s69isvyOFWvRZev4jIKpcZ2nrprt+KyQ071jG5ppTt37uQ3PrjuqIGlJQAAwMVEwQMAAMKj4AEAAOFR8AAAgPAoeAAAQHhV19Jy6/S4tZYmeW0Qp7QLYpLXhHHz5bgOn0lYU6rEkydPsuOXL1/Ojrsuw0nI7To6nEleG6uUy+46RrqY3R1z1+6d7l5zdHSUHe9avkHcebqwsJAd7+J56rrqpqenx3sgp+AbHgAAEB4FDwAACI+CBwAAhEfBAwAAwqPgAQAA4Y2lS8t1s6yvr2fHXWfTJHTFlHY2uTWNXOeaW3tkcXGxaL/DePToUdHnXbfB7u5udryNYy7h5sqtY+Y6RpaWlrLjrstia2srOz4J3Reum8J1NrmMbrwN7pjdfLksOzs72XF3v6m43t2p3Lnqxks71JaXl7Pjk3BPzXH53By6rq5xnqeOO09L59Cdj5NwX3FZ3L8x7phdTeDO31J8wwMAAMKj4AEAAOFR8AAAgPAoeAAAQHgUPAAAILzUNM2g9we++Sz3pLxbi8h1KrnP7+3tFe1XUnJv9CnK6J4ud0/ir6yslGzediEMWDNp5IxuXbC5ubkhNjm6zc3N7PiAJ/FPy5jN535mrivHdcS4n5PbjjsfXQeCzjCHroPp+vXrQ2zyI6XXYgtzKBXOo+uAdGu+ue4XNy/uWnddeGrhfuO6jFxnZCnXgTjgZzLStXhe9xp3/23jWnRK/81w3Fp/I3Q2nft56q5Rd57Uysg3PAAAIDwKHgAAEB4FDwAACI+CBwAAhEfBAwAAwqu6lpZ7YvrWrVvZcbeOS611VNrgumJcx47j1kXZ2NjIjrufyVnWwnH/7fT0dHa8dO0tx3W51Fov5UTpGm6us8kdb+m6am1w3RFra2vZcXctup+9y166ZtNZuHlcXV3Njrtjdl0xbvvufjNO7r5S2v3ibG9vZ8fdPI66NpW7R7vOI8fNiZtzdz8d0KVVnZtD97Mvvf+WroPXxvpppeepOwZ3T611v+EbHgAAEB4FDwAACI+CBwAAhEfBAwAAwqPgAQAA4Y2lS8t1GLkn693T5aWdUG1w3Qau68o9Xe66BxYXF7PjbTxZ79Z4cZ0QLqNb/8Rt/7zn0Z1fLrc7Xtdh0kanUik3V447Tx3XYTJO7pjdteI610rXXBsnN4+l91R333Ln/KjdWKXc/t015DopXXeam/NxqrhWW5ab2zb+zXBKz1PX7eyOuda1yDc8AAAgPAoeAAAQHgUPAAAIj4IHAACER8EDAADCS03TnPcxAAAAtIpveAAAQHgUPAAAIDwKHgAAEB4FDwAACI+CBwAAhEfBAwAAwvs/+UxoCLOTx8UAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x144 with 10 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjwAAABeCAYAAAAnrM7iAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAKDUlEQVR4nO3dO2hd2RXG8W8HezJTZCRDUkwR5AeEcZHIjz6ywSYMSUZOYVcDvpVV2qqsJthVMqokl65sQUKCnMKCMAORQTZMExI/NAmENLYMCQRS2MoUCXntKXw9Ucz+jnyOzn0t/X+gQusen3OW9j73Lu7sNTvlnAUAABDZlwZ9AwAAAL1GwQMAAMKj4AEAAOFR8AAAgPAoeAAAQHgUPAAAILxWCp6U0o9TSpfaONc213k/pfTzXl/HXDt0jtHz616bHNu7Djn27rqh8+temxzbuw7z9HXlnHf0I+lrkv4s6a3u729I+oWkDUlZ0oma51uT9FdJf5O0Lmn6ldd/L+lbO71vctw9+ZEjOY5KjtHzI8cYOY5qfm18w9OR9FHO+e9bYp9I+kDSXxqc76Kkd3LOb0u6IOknKaV3trz+s268nzqKnWNHsfOTyJEcy4Ytx45i5yeRY4QcOxrB/NooeN6TdO/lLznnf+acF3POn0j6T92T5Zw/zTn/++WvkvZK+vqWQ+5K+m7z220keo7R85PIkRwLhjDH6PlJ5Bghx5HMr42C55uS/tjCeb6QUvplSukfkn6tF4n+dsvLf5C0P6X0dpvX3Eb0HKPnJ5FjI+TIs9gD5NjAkOU4kvnt2ck/7hqX9FkL5/lCzvl7KaW9kk5Jejfn/N8tL7+81rhe/Pe+fhhX7BzHFTu/l9cix5rIkWexB8ZFjrUNWY7jGsH82viG55mkr7Rwnv+Tc/5XzvljSd9JKb2/5aWX13re9jUrRM8xen4SOTZGjpJ4FttEjg0NUY4jmV8bBc+nkr7RwnmcPZIObfn9sKSNnHO/KnUpfo7R85PIsQ3k2HvR85PIsQ2DznEk82uj4PlI0tTWQErpyymlN7u/vpFSejOllLqvdVJKG6UTpZTeTSm9l1J6K6W0N6X0gaRva8viqO61Pm7hvuuInmP0/CRylMhRr/y7Ycwxen4SOUqjn+No5rdd3/p2P5K+KulP6vbjd2MberHSeuvP/u5rP5T0U3Ouw3qxYOkzvfjq6jeSfvDKMb+TNLnT+ybH3ZMfOZLjqOQYPT9yjJHjqObXVvI/knTpNY/9laTDDa/zfUnL/RrU3ZRj9PzIkRxHJcfo+ZFjjBxHMb/UPSEAAEBYbB4KAADCo+ABAADhUfAAAIDwKHgAAEB4FDwAACA8Ch4AABAeBQ8AAAiPggcAAIRHwQMAAMKj4AEAAOFR8AAAgPAoeAAAQHgUPAAAIDwKHgAAEB4FDwAACI+CBwAAhEfBAwAAwqPgAQAA4VHwAACA8Ch4AABAeBQ8AAAgvD3bvJ7rnGx+fr4Yn5ubK8YPHDhQjN+/f78Y37dvX53bkaT0GscUc9zY2CgevLi4WIzfvHmzGB8fHy/Gz5w5U4x3Op1i/MiRI8W4dpDjs2fPige7cVxdXS3GHzx4UIy73G/dulWMnzp1qhjX9jnWmqeXLl0qxq9du1aMT05O1jqPG8MKjcfw+fPnxYPdvd2+fbsYP3HiRDFed15XaJyj4/7OLkd3zy5H9zep0HqO+/fvL8afPn1ajE9MTBTjV69eLcZ7MFeL+T169Kh48NGjR2td3OVX91msmL+tf2a49/q6x7tchmGeus+n9fX1YnxsbKwYr/v+VKGYI9/wAACA8Ch4AABAeBQ8AAAgPAoeAAAQXsq5cm1S8UW3CHl5ebkYd8fPzMwU425xbMWiVqfx4qy7d+8WD3aL49xCM7eo1C2QvXLlSjHuFh1qBzm6xeFuvI4fP/4al/ofN46Oux+1vGjZLVJ1i+Ld2NqbqX6mShqPoVsM6hb/Oe48bv6656NC6wsl3T3UbThwCyufPHlSjLuFxOpBju6e3XgtLS3VOb0ePnxYjO+gSaLW4vqK97Ui9+xubm4W42tra8V4xSLY1sdwZWWlGHcLd91njJvXbi70c566e3aLw937k3sW25qnfMMDAADCo+ABAADhUfAAAIDwKHgAAEB4FDwAACC8Rl1ajx8/Lh7sulzqdve48zfQ+mp0x3UPuC6Eut0vvdhaoi1uvFxXnevOu3z5srtEq11ajhsr1ynjOmKmp6frXrpvY+g6Ri5evFiMu3nqOkZ68b/sr6vuNgYudzfuFQY+jnU7Ct32MjsYx1byc3/72dnZYnxqaqoYr7vNiIbg/dSNoRvzYehEq6vuVhQLCwvFuOsOE11aAABgt6LgAQAA4VHwAACA8Ch4AABAeBQ8AAAgvD1N/tHBgweLcdet4/akcV08rnPAdYENg7b2LqrY/2Tg3PgeOnSoGD927FgxfuHChdbuqU2ui8d1dExMTPTwbnrjxo0bxbjrxnLztKLLZeDqPkMVHZBDa3Jystbxbo++YR1H1wXouC7ZYc2viuvGcp1oFd1YA+feV1w3lsuxohurFr7hAQAA4VHwAACA8Ch4AABAeBQ8AAAgPAoeAAAQXqO9tOpyXVeuS8u5c+dOMV7RvdW3PUNcV4HrAHEr6+t2e2kI9kVxXXvz8/PF+NmzZ+teoi/79ziuQ8B1hjTowOvbGLq92tz+PS12Ew58nnY6nWLcPbvub1Vh4Dm695W6e/cNei8tN+9cfm4fvAbdPQMfQ/eZ4Z65UfzMcPPLvQ+599oK7KUFAAB2JwoeAAAQHgUPAAAIj4IHAACER8EDAADC60uXluO6t2ZmZopx1w304YcfuksMfDV63e4tt+K+Yr+Ugefoxmt1dbUYd3tyVRhol5bjOgrqdsRoCMbQzS/XGdJW18QrepqjGxfX5bm2tlaMD/Oz6ObYyZMni/GFhYVivKK7aaQ6Jt2YVxj4GLqOM/fZ4DraKgxtjouLi8V4W+PINzwAACA8Ch4AABAeBQ8AAAiPggcAAIRHwQMAAMLb0+bJ5ubminG3Z5br0nLdPefOnWt2Yw24VeH37t0rxl0ubtX55uZmMe66uvrJ7YHlclxeXi7GG6ys7wvXyeLirgvCHe868PrJ3bMbE9eNtbS0VIy7eV2xB1Nj7p5dp4frSHLz16m7n1MvrKysFOOug6zBvkp94caw7l5Xbj46bgx78Yy6HN2YuPd613HWYP+6vnF/Zxev+7nojq87f/iGBwAAhEfBAwAAwqPgAQAA4VHwAACA8Ch4AABAeK3upeW6e65fv17nNDp9+nQr59EO9gxxK+g7nU6tG6jbFdOgy6L1fVHc3kIuF9eF58bL7YlWodX9e9weWK6jwI2V69Zx3UMVWh9D14Wyvr5ejI+NjRXj7m81DHtpuXFx8bpdV64zpKJbZuDj6Jw/f74Yb9Bt1+hZdO+nbn65/Kamporx2dnZYnx6eroYr9C3z4y6+yy6Z65BZ2Tr89Tdg+u6ctx4NcidvbQAAMDuRMEDAADCo+ABAADhUfAAAIDwKHgAAEB423VpAQAAjDy+4QEAAOFR8AAAgPAoeAAAQHgUPAAAIDwKHgAAEB4FDwAACO9zAzDMoxb2vwIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x144 with 10 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjwAAABeCAYAAAAnrM7iAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAKcUlEQVR4nO3dT4hcWRXH8d8dEiUQpzujDogwScCFgjNJOwtFjUnc6EJNXIoy6UYQd8noyoV0guCACkkvBHXTyULHfwOJgyPRwaQRQXSTdhxEETqRURDRTpzBWQjzXFQFKjX3V9X39a3XVSffD/SiT1XfV6fue69Pqu/JTU3TCAAAILIHdvoFAAAATBoFDwAACI+CBwAAhEfBAwAAwqPgAQAA4VHwAACA8KoUPCmlp1JKZ2qMNeY4n0gpfX/SxzHHDp1j9Pz6xybHeschx8kdN3R+/WOTY73jcJ5uVdM02/qS9FZJf5O0p//9+yT9QtK/Jf1T0o8kvW2LYz0i6ZWhr0bSFwee8wdJj233dZPj/ZMfOZKjGethSU9L+rukO5J+Lem9Q8/hWpziHPs//35Jv5X0sqTfS/rgTs7h/TCPs5pfjU94FiU91zTNq/3v90n6jqQDkvb3T8LVrQzUNM1fm6bZe/dL0qOSXpP0zMDTnpb0uQqvu8SiYue4qNj5SeRIjq+3V9LvJD0u6SFJlyT9NKW0d+A5XIv1LapSjimlhyT9RNLXJc1L+pqkZ1NK+waeNtM5Tuk8LmoW86tQ6f1S0mdGPP4eSS+3HHtZ0rWh2AckbdSqVMkxfn7kSI4F4/9H0uM7lSNzWJajpI9JenEo9mdJn42S4zTO46zmV+MTnkcl/WnE4x+S9GLLsZ9Q719dg/4o6UBK6cGWY7YRPcfo+UnkKJHjSCmlw5LeIOkvA2Guxfpq5pj6X8Oxdw98P+s5DpuGeZzN/CpUev+T9E7z2GPq/U3vSItxj6j3t7y9Q/Hd6v1975EOq9nQOUbPjxzJcQvjPijpBUlf2skcmcOyHCW9WdJtSZ/q53JKvT+HfDtKjtM4j7OaX41PeDYlvWk4mFJ6h6SfSTrdNM2vWox7StIzTdO8MhS/e6zbLcZsK3qO0fOTyJEcjZTSHknPSvpN0zRPDT3MtVhftRybpvmXpBOSviDpH5I+Kul5SS8NPG2mcxwyLfM4m/lVqPSel/Tpodh+STclfb7lmHvU65r4cOaxnfh7bOgco+dHjuQ4Yrw3Sroq6XuSHtjpHJnDdjkOjLNL0i1JH4mW4zTN46zmV+MTnuckHb37TUrp7eotaPpm0zTfGn5ySmkxpXRzzJifVK+Su5Z57Kh6FWSXoucYPT+JHMnx9Y/tlvRjSa9KeqJpmtcyT+NarK9qjimlhZTS7v76jm9IeqlpmqsDT5n5HPumaR5nM78Kld5b1Pv48G4//rJ6f2u7p69+4PlflvTdMWNelfQV89gLkg51VcneDzlGz48cydGMdbT/s/8d+vkjA8/hWpzyHNVrWb7T//qBpIej5Tht8zir+dVK/quSzmzxuT+X9K6Wx/m4pB92Nan3U47R8yNHcpyVHKPnR44xcpzF/FJ/QAAAgLDYPBQAAIRHwQMAAMKj4AEAAOFR8AAAgPAoeAAAQHgUPAAAIDwKHgAAEB4FDwAACI+CBwAAhEfBAwAAwqPgAQAA4VHwAACA8Ch4AABAeBQ8AAAgPAoeAAAQHgUPAAAIj4IHAACER8EDAADCo+ABAADhUfAAAIDwKHgAAEB4u8Y83pQMduHChWz89u3b2fjly5ez8fX19Wx8bm4uG79582Y2Pj8/n7IP3KsoR5eLy93FT548mY1fvHix5OVIUvUcnTNnzmTj7jUfPnw4G79+/XrpocflWJSfy2NlZaVkGMud1ydOnHA/Un0Ob9y4kY273NfW1kqG1+rqaja+uLjofqSz89Rdo+58dNw8jhineo7u2nL3FXfvbDFfTtVr0eXn4m5u3Xk9gfykStei+x1w69atbHz//v3ZuPv9N0L1HN1rcNfQ2bNns/EDBw5k4+53xvz8vHtJ2Rz5hAcAAIRHwQMAAMKj4AEAAOFR8AAAgPBS04xcm1Rl0bLjFv+VLn4esQi29eIst9DMLYIbsXB6Cy9h/DgjVF+A5t5P9544bh4nkGM2v9LFq8eOHcvG3UK6c+fOZePLy8vZuFuopwnMoTtP3Xvicn/yySezcbcA2y1SVIeLlktzd+ejW1Q6iXl019bCwkI27hawuhzv3LmTjW9ubmbjpYtBB2Tzc4uQl5aWsvHTp09n4+59covuNzY2snF3TWsC52npYnnHLUifhhzdsVzcvSfu/uGu6dJrkU94AABAeBQ8AAAgPAoeAAAQHgUPAAAIj4IHAACEN25riSLuv/d23Apr1zXRYkuC1tx/7+1Wl9fq6poGrmPHxR3XmeHeq1rdDHe5TpPS9760+9B193SpdIsS95643Eu7D7vkziN3f6p93rVR2rXiuuTcPdV1FLqurtrz645Tyt07Dh06lI2P6FTqTOm2CO48ddfoNOdY+tpKO4FL8QkPAAAIj4IHAACER8EDAADCo+ABAADhUfAAAIDwWnVpuRXZpV1Upd0vpftsbIfrgnB72Fy5ciUbd6/ZdfJ0uRLfzZd7zbW6nrrqKnAr/l1+7vml57XrlBmxz1Rr7rxbXV3Nxl03kOuicd2KpR2ZXXKvrXRvt2nI0d2HaqnVPTWOey/dteW6DN2eYKW/S7rk7gfufuPiLnf3+6+0U3M7at3T3flYa3w+4QEAAOFR8AAAgPAoeAAAQHgUPAAAIDwKHgAAEF6rLi23YrpWl4vrZindy2kSFhYWqoxz6dKlbHwa9hFbWVnJxufm5rJx1z3gdNVt547j9hVyXN6l+xlNwrVr17Jx173l4qVcB8gkumVKu0JdvLRrr8v9wtxrc/e80mvO6XIec0rv9e53zzT8bnDnnbuflnL3my67Cd39Y9L7B+7bt6/KOHzCAwAAwqPgAQAA4VHwAACA8Ch4AABAeBQ8AAAgvNQ0zajHRz645YOklI27FfoV94/JH/he2Rzdnh6ue8Gt0HddV64jya12d3sgaRs51uI6J9z8jsjFGZdjUX5uTg4ePJiNnz9/Phuv2B1R/Twt7UhynSRuT64WHXWtc3SdREtLS0UvoGIuTuscXffL8vJyNl6691lpR+E27jdF16I7T103rOtKrNilVf1+WrpnoYu7cVrsM9U6R3fvXF9fz8Y3NjaycXdNu/O6Re7ZHPmEBwAAhEfBAwAAwqPgAQAA4VHwAACA8Ch4AABAeFW7tFzXiluR7VZ8V9zDprMOJtfp4boQXLyFznJ081trD7URqnaGuM4Ut8fW5uZmNj6L56nrAnRz6K7RFqrnWNoB6bjzwXWAlHaGDKkyj64rqXQeW5zDVa9F15Xj9k5yXT8tOpWczs5T1wnnzt+K+5t1dp66+XXz5faYbNG5TZcWAAC4P1HwAACA8Ch4AABAeBQ8AAAgPAoeAAAQ3q6ag7muHLfyumKXy45zq9Er7vGy41y3gev8mVZur5pTp05l47N4nrpr0e3Z5N6TaVa6h5u7Fo8fP140TsWOoLHcPK6trWXjbt+3aT2HS/dT7PK9r8XdN10urmtwFrmOMxevuI9mFp/wAACA8Ch4AABAeBQ8AAAgPAoeAAAQHgUPAAAIb9xeWgAAADOPT3gAAEB4FDwAACA8Ch4AABAeBQ8AAAiPggcAAIRHwQMAAML7P03U+cHqNt5uAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x144 with 10 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjwAAABeCAYAAAAnrM7iAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAJiElEQVR4nO3dT2idWR3G8eeMdTHqTG5EV6OkCxE6MKYV92mRAUXGtOBGEJq4cNsU12MbEN3ZVl0ILtrOZtRN21mMMAhpFy5Eoa0Io6umC0UUTOo41M1wZpEbqPE8Nz1vzn3z3l+/H+gi517e933u+yc/bs8vJ+WcBQAAENlzh30AAAAA00bBAwAAwqPgAQAA4VHwAACA8Ch4AABAeBQ8AAAgvCYFT0rphymltRbb2mc/X08p/WLa+zH7Dp0xer7xvsnYbj9knN5+Q+cb75uM7fbDdfq0cs4H+ifp05L+Kun58c8vS/qDpK3xv99IernDdpckZUnf3zP+J0lfOOhxk/HZyUdGMs5Kxuj5yBgj46zma/ENz4qkt3POj8c//03SNyR9UtKnJL0lqao6Syl9VNIVSb8rvPympO90PdiOVhQ744pi55PISEZjYBlXFDufRMYIGVc0g/laFDxflXRn94ec83bOeTPvlGVJ0geSPle5ze9KekfSnwuv3Zb0tW6H2ln0jNHzSWQkozekjNHzSWSMkHEm8x056AYkvSLpL3sHU0rbkj6hnaLqe0+7sZTSgqRvS/qipJ8W3vKupKMppRdzzv/ucsAdRM8YPZ9ERjIWDDBj9HwSGSNknMl8LQqekaT39g7mnEcppY9LOivpYcX2fizp9Zzzf1JKpdd39zWS1NfFO1LsjCPFzre7LzKSca+hZRwpdr7dfZFxtjOONIP5WhQ8W5JeKL2Qc34/pfQzSf9MKR3LOf9j0oZSSq9JeiHn/MsJb9vd13aXg+0oesbo+SQyknGPgWaMnk8iY4SMM5mvRcHzR0mfl/R78/pzkj4m6SVJE4NL+rKkL6WU/j7+eU7SBymlV3LOy+OxY5I2e/xqUoqfMXo+iYwSGfcaYsbo+SQySrOfcSbztZi0/LZ2WskkSSmlV1NKJ1JKH0kpvSjpR9qpBt8dv76SUto023pdOx/i8fG/tyT9XNLqE+9ZkvTrBsddI3rG6PkkMpLx/w0xY/R8EhkjZJzJfC2+4XlD0r2U0vPjFrWRpJ9I+oykx9qpAL+Sc/7v+P2flfTb0oZyzu/pif8XTCk9lvR+zvlfT7ztm5K+1eC4a0TPGD2fREYy7jHQjNHzSWSMkHE28+U2f4ToB5LWnvK970g61nE/r0n6VYtjJuOzlY+MZJyVjNHzkTFGxlnMl8YbBAAACKvFHB4AAIBBo+ABAADhUfAAAIDwKHgAAEB4FDwAACA8Ch4AABAeBQ8AAAiPggcAAIRHwQMAAMKj4AEAAOFR8AAAgPAoeAAAQHgUPAAAIDwKHgAAEB4FDwAACI+CBwAAhEfBAwAAwqPgAQAA4VHwAACA8Ch4AABAeBQ8AAAgvCP7vJ5b7GR7e7s4vrKyUhy/efNmi91KUnqK9zTJePHixeL45cuXi+Obm5vF8dFoVLvr3jI6J0+eLI5fu3atOH706NHaXeyXsUm+tbW14rjLMYvn0GVZXV0tjm9sbBTH3TmfoLeM7jw67h7t4NAzXrlypTje43lsku/evXvF8dOnT1e9f8j3ont+1GbsoLeMtb/jXa3QQTEj3/AAAIDwKHgAAEB4FDwAACA8Ch4AABDefpOWm3ATJY8fP97H7ptyk6paTdB12+8w+a65Hibv9uL27dvFcTfhc2lpqTg+1HySv47cBN25ubni+JDvUTeJ053Hc+fOTfFoDsadL9cM4TIuLi4Wxzs0ChwqN9l11nJMUvs8HTI30do9a934tPENDwAACI+CBwAAhEfBAwAAwqPgAQAA4VHwAACA8Jp2adV2MLk/j147S73PmfvumF129ye03TG7P/XecLmNfbkZ9G75gUuXLhXHXUeQ6zzpi+soWFhYKI6769dtx+WbRsdT7bIt9+/fL46762sInWi1ywycPXu2OF67zEufzxV3bTx8+LA47q5V95kMtbvJnZPa69SdwyF0Gd66das4vr6+XhyvzdjnuXXny/3OcFkO67zwDQ8AAAiPggcAAIRHwQMAAMKj4AEAAOFR8AAAgPCadmnVrg3iOklcJ5TrGJlG14+bjX79+vXiuOtUcjPoHz16VBwfQleB+/zdWkTu/Sml4rj7TNz10JXrHHCfvdu/u35d94W7Tt39cRDuOnXH5jqYlpeXi+PuM3HdhK3PoeQzug6mM2fOVG3HPT+msUac66pz426NM7eWluvSql1brTXXaec+e3c9OidOnCiOP3jwoDjeZ2eTu+fcGn1ufH5+vji+sbFRHHf36EG4riu3L/f7zF0Pbvut1lbjGx4AABAeBQ8AAAiPggcAAIRHwQMAAMKj4AEAAOF16tJyHSDnz58vjrtZ6o7rQLh69WrVdg6idj0v14FT2wXhug2mwR2zW8fGzbh3nSHONDp5SmrXH3Pn3HWhOX2uFeY60RzX1eAyuq7E2s7Lg3Cfp+s8cs8b1523uLjY5bA6cR1e7nN291btPXfYa6K5jjp3Tpza3wHu/pjGdVrbFeqev7XPD7ffaXRpuWNz47VdVO6zatVlyDc8AAAgPAoeAAAQHgUPAAAIj4IHAACER8EDAADCSznnSa8XX3Szwl3nQG13hOsS2traKo5P6EAoL+b0v4oZ3axw183iOoJc9oWFheJ4bXeYDpDRcV14N27cKI67dVHcedznmivZL2PVBt05dN2BzhDOoduX66ir7Ypx3NpxEzraml+njuvccF2kd+/eLY53WNeueUZ3ft2ztuGz02l6L7pOJTd+586d4rj7XeK2M+HcNj+HriPM/R51HW1Oh7W0ersXXffW+vp6cfzChQvF8Q5raRUz8g0PAAAIj4IHAACER8EDAADCo+ABAADhUfAAAIDwOnVp1XJdPG4WuetAcDPuJ+htNrrr6pqfny+Ou/V+hpzRcce8urpaHD/sLi2ntqPArevTYZ2e3s6h6ww5depUcXx5ebk4XrtOmXrM6K5H10Hm7t0OesvonpGuq8s9gzvo5V5058R15bh7t3YdPA3geVq7ZmGHtft6y+iehe46rV0bcAK6tAAAwLOJggcAAIRHwQMAAMKj4AEAAOFR8AAAgPCO9LETt16LW9enQ5fLoXPdAHNzc8XxWczouK4Yty7KUNV260xYq2awart1OnSAHDrXyeOeN27tONehhulz92KrteBm0YR1owardq2/aeMbHgAAEB4FDwAACI+CBwAAhEfBAwAAwqPgAQAA4e23lhYAAMDM4xseAAAQHgUPAAAIj4IHAACER8EDAADCo+ABAADhUfAAAIDwPgQdAR+mTEyfdQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x144 with 10 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjwAAABeCAYAAAAnrM7iAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAKCElEQVR4nO3dT2hdaRnH8d8rs1DoTBPGwoBI0gjCIJqm6c5FG7QuBE0LpishWYkr27gJKJK6GVsQ0m5cdJMWESVx0WxGcGFb3LhosPUP6iJtioqza6PioJvjIslwyby/m7wn557c++T7gS7y5M4553fPOTcPd87Dm6qqEgAAQGQfOeoDAAAA6DUaHgAAEB4NDwAACI+GBwAAhEfDAwAAwqPhAQAA4TXS8KSUfpBSutbEtvbZz1dTSj/r9X7MvkNnjJ5vZ99kbG4/ZOzdfkPn29k3GZvbD9fpQVVVdah/kk5J+rukj2V+tyipkvTFgu2NSnog6T+S/rz3v5X0B0mfO+xxk/H45CMjGQclY/R8ZIyRcVDzNfENz5ykd6uqer+zmFL6lKSvSfpH4fZ+Kum3kt6U9F1JP08pndrz+2/UPtp65hQ745xi55PISMa8fss4p9j5JDJGyDinQczXQKf3K0lfz9R/IenLkjZ1wE5P0qcl/VfS6x21X0v6ZsfPn5f0vK1O9jhkjJ6PjGQclIzR85ExRsZBzdfENzyflfSXzkJKaUbS/6qqerdwW5+R9Kyqqn911J7u1Hf9SdJoSumNOgdbU/SM0fNJZCTjh/Vjxuj5JDJGyDiQ+V47zH+8Y0jSBweaUjoh6R1JX6qxrROStvbUtiR9ouPn3X0NSfpnjX3UMaTYGYcUO9/uvsh4cGTcxr3YvCGRsUQ/ZhzSAOZr4huel5Je7/j5+5J+XFXV8xrb+rekvR3cG+p4Yzv29arG9uuKnjF6PomMpci4jXuxeWQs048ZBzJfEw3P77T9/+B2fUHSt1JK76WU3pP0SUkrKaWFA2zrj5LGUkqdb+T4Tn3X25I2q6pqq1OX4meMnk8iIxk/rB8zRs8nkTFCxsHM18DDS9+WdKfj5zclvdXx76+SZiSd2Pn93M6Bu+39RtIPJX1U0mVtd3SnOn7/HUk/auvhrOOQMXo+MpJxUDJGz0fGGBkHNV8TwT8u6W/KzOPv/H5THU9rS/qepJ902d6opIeS3tf2Q1F75/F/L2m85Ys3dMbo+chIxkHJGD0fGWNkHNR8TYV/R9K1A772l5Lerrmfr0haaeukHqeM0fORkYyDkjF6PjLGyDiI+dLOBgEAAMJi8VAAABAeDQ8AAAiPhgcAAIRHwwMAAMKj4QEAAOHR8AAAgPBoeAAAQHg0PAAAIDwaHgAAEB4NDwAACI+GBwAAhEfDAwAAwqPhAQAA4dHwAACA8Gh4AABAeDQ8AAAgPBoeAAAQHg0PAAAIj4YHAACER8MDAADCo+EBAADhvbbP76uSja2urmbrCwsL2frFixez9Rs3bmTrw8PDJYcjSekArynK+OrVq2z9+vXr2frdu3ez9QsXLmTr9+/fLzkcqQcZnzx5kq3Pzc1l66Ojo9m6y3jt2rWSw5H2z1iUz+V4+PBhtn7mzJls3Z1z9/ouGj+H7rpzx+zOlXu9O+ddNJ5xc3MzW3fvvztm9171w3l0nzcui6u7a3toaKjkcKSG70V3Dt316N6Pe/fuZevT09MlhyO1eA5dRnfdueu0hsYzus9Ud35L78Uashn5hgcAAIRHwwMAAMKj4QEAAOHR8AAAgPBSVXV9NqnowaWxsbFs/fnz59n6zMxMtu4efl5ZWSnajnrwcNalS5ey9bW1tWx9cXExWy99qNQ9FKY+yFjKXQ9dHoSt9aCke1BzamoqWx8fH8/W3UPW7ly5h767PCDa+DksfRjVvffuocPSB7zV4oOS7sF/95Coy+IeNu2i8Yy3bt3K1ufn57P1kZGRbN29V+4a7qKVAQL3ELJz8uTJbN1dv724F0vPVamlpaVsvQdDIJLJ6D7bJiYmSo8h68GDB9m6u3e74KFlAABwPNHwAACA8Gh4AABAeDQ8AAAgPBoeAAAQ3n5LS2Str69n6276ZmNjI1t3U11uyQm33y5TWrW5p/vdpNLs7Gy27qYg3ASIewq+F9x0ist49erVbL3BJRaOlJvucRNM7vVuAq/GNMW+3HW6tbWVrbvrtHR5Bfd6N6lyGO6ecJM8y8vL2bqbCOoyAdkadw+5e9Tdi+4ac5OXpcvF1FV6Dt116rjtuP3WmPrZV+lUn5sKLV3KphefK467Ht2UnHu9++x0WZr6u8g3PAAAIDwaHgAAEB4NDwAACI+GBwAAhEfDAwAAwqs1pfXy5cts/ezZs9m6m8ZyJicni4+paaVrEZVOepRuvx+UTuC8ePGiNwdyQG5CwCmdTHHn8PTp00XbOYzS6+jy5cuNbN+tR9YLpdMvbnLNKV0rrcb6U/ty73PpxI7bjsvo7pGmJ9dKPwvcFKDjcpReC21yk0qlr3cTTL2YknX3ottXad2dd7ff0s8/vuEBAADh0fAAAIDwaHgAAEB4NDwAACA8Gh4AABBeo1Nabg2sprY/PDzcyPYPos01rY5K6XoypU/Knz9/Plt3T+L3Yvqll9qacOmmdEphZGSk6PXunLt7tBfc++yyuHWjnNIpsF5wk31uyqj0vJdOxRw1d1yla3+1eW4nJiaydbfOlOOyuHPo7o9BW8tQ8tOEbkK49G8G3/AAAIDwaHgAAEB4NDwAACA8Gh4AABAeDQ8AAAiv1pSWm5ZaX18v2o6b9Hj8+HG2fuXKlaLtH0bpE+5bW1vZupsScFNg/TCp5KYK3LG5J+hd9tI1q46ay+HWB2pzOsIdmzuH7phLpxJLJ6EOw+3LXXdNrYnW5np309PT2frs7Gy27qZZXPbSdZia5iY23XVaem6bWmvpMFxGx01XuenZR48eZeulaxwehptEK10XzCmdRCvFNzwAACA8Gh4AABAeDQ8AAAiPhgcAAIRHwwMAAMJLVVV1+332l8+ePcu+eHJyMlu/c+dOtr66upqtb2xsZOulU2CS0gFe0/UN2Kt0/anSiZEaa9s0ntFNgLgn8d0UTelkSJeJiv0yZvO5J/unpqaydTc16NbvcduvsX5P4+cwpfwm3TSQOyfueu+H67R0zSF3nbrpw9u3b2fr7j1Ui/eie//de/L06dNsfXl5OVvvsh5crXvRcedkbW2tZDN22qsf7kWX0X1+lJ7Dff6G57R2L7rryN1zpWuldZlqzmbkGx4AABAeDQ8AAAiPhgcAAIRHwwMAAMKj4QEAAOHVWktrbGwsW79582a2vrCwkK2fO3cuW68xjdUaN3nkpinc9EuNKZfWlGZx0wYuY1vr27gJIzdl49aIc2vkNLW+Sy8sLS1l6/Pz89m6e0/aXKenlLu+3OSGu65dxi7TWK1xx1Y6Ibi4uJitd5nGakXp9bW5udnIdtpUOlFXeg77Qelkr5ukdH8bmrpO+YYHAACER8MDAADCo+EBAADh0fAAAIDwaHgAAEB4+62lBQAAMPD4hgcAAIRHwwMAAMKj4QEAAOHR8AAAgPBoeAAAQHg0PAAAILz/A3BdbWlZQ/gMAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x144 with 10 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjwAAABeCAYAAAAnrM7iAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAALDklEQVR4nO3dXYhcZx3H8d9To2jBZrdWKSJuEsFSqm6WpIhV2Ea8UWlTxK3Ul256k4I3TS9KC1LbG7UXQhPxpimStlhaNzcNiNJU2V18QexLmipaBbu7EPGl1c1WizeWx4uZyLA8v0mek2fOzDz7/UAp+c/kzPnNOXPyZ/b89wkxRgEAANTskmHvAAAAwKDR8AAAgOrR8AAAgOrR8AAAgOrR8AAAgOrR8AAAgOoVaXhCCN8KIRwqsa3zvM6NIYQnB/065rWrzlh7vu5rk7Hc65BxcK9bdb7ua5Ox3Otwnl6oGONF/Sfp3ZL+LOkd3T/vkBQl/bvnv3sztrcq6T89f/fkpsd/K+kjF7vfQ864W9LPJG1IOiPp68PMOIB810n6taR/SXpJ0icqPIZkJCP5yLglM45rvhLB75L0cM+fzwXf1nB7q5I+1efxr0n6blsHdkAZfyfpG5LeIukDkv4i6cZhZSyZT9Llkl6TNNfN92VJ65ImazmGZCQj+ci4lTOOa74SP9L6tKTlAtu5UEuSPtvi60nlM+6Q9HiM8c0Y458k/VzSNT2PL6ndjCXzXSfpbzHG491835f0qqTP9TxnSeN9DMnYsSQyllZ7PomMuUYx41jmK9HwfFjSHxL1tRDCmRDCsRDCFZnbfDyE8GoI4WQIYXrTY7+XtCOEcFmjvW2mdMbDkm4NIbw1hHCVpI9J+knP421nLJkvdP/bXPtQz5/H/RiSsYOM5dWeTyJjDRnHMl+JhmdCnZ+7nfOapGslTUnaI+mdkh7P2N6X1PkGZErSoqSnQwgTPY+fe60JtWdCZTP+UNLn1blX6WVJ34sxPtvzeNsZJ1Qu3y8lvTeEcEu3oZtX58d2l/Y8Z9yPIRk7yFjehOrOd+61yDjeGSc0jvkK/Czv75Ku7fP4ler8bO+yhtt/WdINm37e13h7w87Y3f/XJd0qaZuk90n6laSvDitj6WMoaVbSs5L+KekJSSfVcwPbuB9DMpKRfGTcyhnHNV+Jb3hekvTBPo/H7v83f2V1oeKmv3u1pNUY4+sNt9dEyYy7JL0ZY3wsxvjfGOMZSU9K+kzPc9rOWPQYxhiXY4zXxhgvl/QVSVepcwf+OeN+DMnYQcbyas8nkVEa/4xjma9Ew/MjdbozSVII4aMhhKtCCJeEEN4l6TuSlmKMG93HD4QQVlMbCiG8P4Tw8RDC20IIbw8h3CXpCkm/6HnarKQfF9jvHMUySvpj5ynhi92/f6WkL0g63fOctjOWzKcQwkz3q8nLJH1b0pkY49M9Txn3Y0jGDjKWV3s+iYw1ZBzPfAW+2rpCnd8lc24e/xZJK5LeUGfc+jFJV/Y8/151JpRS27pGnc7xDUn/kPRTSXs3Pec3kqYvdr+HlbH7+CfV+fpuQ9JfJT0s6dJhZRxAvie62TYk/UDSeyo8hmQkI/nIuCUzjmu+UuG/KenQBT73pKSrG77ODZIW2jqoWylj7fnISMZxyVh7PjLWkXEc84XuBgEAAKrF4qEAAKB6NDwAAKB6NDwAAKB6NDwAAKB6NDwAAKB6NDwAAKB6NDwAAKB6NDwAAKB6NDwAAKB6NDwAAKB6NDwAAKB6NDwAAKB6NDwAAKB6NDwAAKB6NDwAAKB6NDwAAKB6NDwAAKB6NDwAAKB6NDwAAKB6NDwAAKB6NDwAAKB6287zeMzZ2PXXX5+s79ixI1l/5JFHcjbfRLiA52RlPHToULL+1FNPJesHDhzI2s7ExETO7kgDyHjTTTcl62fPnk3Wl5aWcjbfxPkyZuVz593999+f9Xx3vjfQ2nnqjlXuedpA8Yyrq6vJujt/Xd0d9waKZ9y9e3dW/fDhw8l6g+uK0+iz6K4d7r0/cuRIsj49PZ2sFzx/Gx9DlzH3euqO4Shfb26//fZk/ejRo8n6wsJCsj43N5fzsv0kM/INDwAAqB4NDwAAqB4NDwAAqB4NDwAAqF6Ise+9SVk3Lrmbk9fW1nI2o6mpqWTd3aTYR/Gbs9yNY7n75t6rBjcAN87o9nnnzp25+5DkbjB88cUXczdV9KZldxPhiRMnkvX5+flkveBN962dp8vLyzmb0crKSrLuzt8+imd07/9tt92WsxkdO3YsWXc3wvbROKM799y56q6R7ibdFm8+T+ZzN+K6uju2ru6uZW1eT91r7du3L3cfkk6dOpWsuxvY+yj+Wdy1a1eyvr6+nqxPTk4m6+7m57vvvjtndyRuWgYAAFsVDQ8AAKgeDQ8AAKgeDQ8AAKgeDQ8AAKje+ZaWyOJ+fbmb0tq+fXuy7iZM3K/iLvhr08/L3RGfu3yG22d3p3/BXyv+f+79dGZnZ5P1ghNnrXDH0E2PPfroo8m6+7X4DSaYWuMm506fPp2s5y5F0SY36eGuK+645C4LMwhunx23b27qqeCUViPueueuQe6YuM/i/v37G+zVeMmdaGvTPffck6y76aq9e/cm6w899FCyfvDgwWTdXQMcvuEBAADVo+EBAADVo+EBAADVo+EBAADVo+EBAADVK7qWVu4aRc6DDz6YrDeYNCi+Zoib5JmZmUnW3WRT7tovfSZ/Gmd0ExLuzne3Loo77u69yp0OU+G1tNzruxyOmzxxEyZ9jOw6U6O8rp3jplnuvPPOZN1NSA3gPJUy17Vz05m56xO2uF5Y1jF01/Tc98Md81E4T3M/i+4z5475ef4NT2nts+jWxnruueeS9RdeeCFZX1hYSNbn5ubcS7OWFgAA2JpoeAAAQPVoeAAAQPVoeAAAQPVoeAAAQPWKTmk5bvLITfG4aYoG01vF70Z3Wfbt25ezGa2srCTrDdZhKp7RrTXl1lu64447knU3neCO+0VMohU5T3O5qS63xpZ7X9Xi1ITjpll27tyZrJ86dSpZH+WMTu4aW21mdJNi7jPk9tldt1y9z/qEI/lZdPs7gGuNNKSMIaR3bXFxMVnvs/7iyGa8+eabs57vprfElBYAANiqaHgAAED1aHgAAED1aHgAAED1aHgAAED1trXxIn3uFs/SYF2Uxtzd/W4a67777kvW3T67CR83ZdFgeqsxl93V+0ytJLmpugZrUA2VWwvHTWmNcj53frkJPLd2kZvMG2XufByF4+imj9w11dXdZ9RlabDG1lC53G4KbdzySdL09HSy7q7Lpf7dbdOePXuS9QceeKDI9vmGBwAAVI+GBwAAVI+GBwAAVI+GBwAAVI+GBwAAVK/olNaJEyeS9e3btyfrbgrCcZNNg+CmVlwWN+nhprRmZmaSdTflkvteDYKb9HDZXZZhTyvlrk/kuPXQ3OfAnQuDmMBzWdzUijsmbp/X1taSdXeejsKUoTvu7rx2a/q1eRzdueTWMnPXFTex445X6Skmd0zc67hrvcvntu8mKQfBnV+5dfcZLXXdGoTnn38+WT9+/Hiyvr6+nqwfPXq02D6l8A0PAACoHg0PAACoHg0PAACoHg0PAACoHg0PAACoXtEprcXFxWT9yJEjWduZn59P1ttcGyR3DZvJyclk3U117d+/P1l3E09tcvuQO/3ipg1y194qzU2Puakcx61t446tO6cGIXcay3GTR+4YtjmN5aal3GfU7Zs7f93xbZNbo89xa5xtbGwk621ldOeLO1a5E55uO7Ozs/13rCB3HrlJtOXl5WR9amoqWXfn7yj8m/HKK68k688880zWdg4ePJiss5YWAADABaLhAQAA1aPhAQAA1aPhAQAA1aPhAQAA1QsxxmHvAwAAwEDxDQ8AAKgeDQ8AAKgeDQ8AAKgeDQ8AAKgeDQ8AAKgeDQ8AAKje/wCphI0rIhUl6gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x144 with 10 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjwAAABeCAYAAAAnrM7iAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAKUUlEQVR4nO3dS4gdWR3H8d8ZM5oZxunOoCKiJLhwiK8kzGKQQTpxoSA4iSgIIpPORlGETjbCLDRpFWeZ7o0LRTrtawZ1kUZnfCDmIaIwi6SN+EKwlVFcDCQdHwEdLRd9I2U8v06fm1N1q/9+PxBCn1tdVf/7P1X3z039c1LTNAIAAIjsrkmfAAAAQNcoeAAAQHgUPAAAIDwKHgAAEB4FDwAACI+CBwAAhFel4EkpPZFSOl5jX7c5zqMppae6Po45dugYo8c3OjYx1jsOMXZ33NDxjY5NjPWOwzzdqqZp7uiPpJdL+oOke1pj90r6rKTnJa1Luliwv09JuiLpBUmnMq//TNKb7/S8JxWjpFdIelLSH0e/9yNJD08yxg5yuF/SD0e/95ykT0TKITES43aIb4j3GmJknk4yhzW+4ZmV9EzTNDdaY5+T9ICkvaO/TxTs7zeSPibpafP6k5I+WH6ad2RW9WK8T9Kzkh4a/d6ypKdTSve1tuk7xlnVzeFXJV0c/d6MpA+nlB5tvb7dcygRo0SMXZhV7HuNRIzM0//WXw4rVHo/kPSB1s8PSrou6f473O+Xlf+G5xFJv+2rku0yxtb+rkt6aFIx1o5P0t8kvb7189clPR4ph8RIjNshvsz+J3qvIUbm6SRzWOMbnjdJ+lXr54cl/U7SfErp+ZTSlZTSeyoc56ZfSNqTUrq/4j5vp7MYU0r7Jb1YG99s3dR3jLXjW5D0WErp7pTSg5LeIun7rdcj5HBBxEiM9UW/10jEyDzdRJc5rFHwTEv6c+vnV0t6ozb+Le5Vkj4qaTmltLfCsdQ61nSl/W3FtDqIcZS8L0mab5pmvfVS3zFOq25835L0Xkk3JP1S0heapnm29XqEHBIjMXZhWrHvNTePRYzM0//RdQ5rFDxXJb209fMNSf+Q9Ommaf7eNM0FSeckvb3CsdQ61rVK+9uK6jGmlO6R9E1JP2ma5olbXu47xmrxpZQekPQdSZ+UtFPSayS9I6X0kdZm2zqHxPgfxFhf9HuNRIzM04w+clij4PmppNfd8nOX9kpaa5rmesfHaasaY0rpJZLOauMp9w9lNuk7xprxvVbSP5um+WLTNC80TfOcpKckvbO1zXbPITFuIMb6ot9rJGIsxTytFF+NgucZbTw1ftNFSb+X9HhKaUdK6RFJByV9V5JSSrMppTW3s9G/Ue4cnduOlNLOlNKLWpvMSPp2hfMuUS3GlNLdkr6hjYr4saZp/pXZrO8Ya+bw1xubpPenlO5KKb1S0vskrba22dY5FDHeRIz1Rb/XSMTIPG3pNYcVnqZ+mTb+X4B2P/4bJP1Y0l8l/VzSu1uvfVzSVzbZ3xlJzS1/ZluvX5G0r8aT4JOIcZS4RhtP3f+l9eetk4qxgxy+TRtthuuS/iTp85LujZJDYiTG7RDfEO81xMg8nWQOawX/GUnHt7jt9yTtHfM475L0tb6S+v8UY/T4iJEYt0uM0eMjxhgxbsf40miHAAAAYbF4KAAACI+CBwAAhEfBAwAAwqPgAQAA4VHwAACA8Ch4AABAeBQ8AAAgPAoeAAAQHgUPAAAIj4IHAACER8EDAADCo+ABAADhUfAAAIDwKHgAAEB4FDwAACA8Ch4AABAeBQ8AAAiPggcAAIRHwQMAAMKj4AEAAOFR8AAAgPB23Ob1psZBDh48mB2/du1adnx+fj47fvjw4dJDpy1sUyXG8+fPZ8ePHDmSHd+/f3/RfjZRPcbZ2dnsuDvnM2fOZMdd3hcWFkpOR7p9jFXic/Px+PHj2XEX3xiq57D0PXY5XF1dzY6fO3cuO77Je1I9RndtTU9PZ8fX1tay46dOncqOj5Hf3u43Kysr2fG5ubns+J49e7LjLu9ue1W+Fp3Lly9nx11O3Fxw14GbI+ogh+6e7sbd59/Zs2ez40P+XHT3VHfNuWvUxb6JbIx8wwMAAMKj4AEAAOFR8AAAgPAoeAAAQHi3e2i5CveA2IULF7Lj7oHIMR7Oqs49THfo0KHs+NTUVHbcPZw1BC5fLna3vXsg0j0EvMmDkr1wDxGW5srtZ5MHJatz773jcuL2U/pQaRfcA5EuX+7c3LV79erV7HifeXQP3bqHPkvz6N6rSV+LLj73cPLy8nJ23DUo9DlP3QO3i4uL2fGTJ09mx5eWlrLjQ/hcdNz77JohusY3PAAAIDwKHgAAEB4FDwAACI+CBwAAhEfBAwAAwqvapeU6N0qXS3BLGAyBe+J+37592XHXVeD++/AhKP1v2l1Hh+tmmXQHiDu+6yhwOXf7GUIHk7uGXNeK6+JxOXT76ZObp6UdSU6f3Vil5+DmmOtcc3N4qPdaF4fLrYtvCN2wrguwdNkFl6uhdtpJfv6ePn06Oz7GskNF+IYHAACER8EDAADCo+ABAADhUfAAAIDwKHgAAEB4Y3Vpla7vsr6+XrT/PrtZSpWuA+W2H/L6J64bYGVlJTt+9OjR7Hjpek59Ke2CKO02G8L8defmOkZcDl33yxC4TrETJ05kx11XjFu7bwhcjC6W0g7LIXSi5bjzdd1pzhA6ldy93q2ZVWtNvyFwsZR+XtbCNzwAACA8Ch4AABAeBQ8AAAiPggcAAIRHwQMAAMIbq0vLPUntOgp27dpVtH/XgdAndw6ue6C0m2WoHUyS7/Rw64W5vE+6A8Tl0J3v6upq0f5LO0a64M6hdH4NeT46pR0dbj4OoavOKZ3DLpYhx5jj5rXLubsvDzlud26uk9J1dQ2hE8118Lo1s+jSAgAA6AgFDwAACI+CBwAAhEfBAwAAwqPgAQAA4Y3VpdU194S+W+uoC269pcXFxaL9uO6tSXcwjcOti+K6utz2k+bO69KlS9lx18HkOmX67Hhy88udm7u2XJfFkNd8c3l0nSHuvXIdPl13jLS5+838/Hx23HVMDnXts9J7h1t/cW5uLjvu5vsQuNy6+8Tu3buz4wcOHKh0RuNz14pbv85ZWlqqcDbl+IYHAACER8EDAADCo+ABAADhUfAAAIDwKHgAAEB4g+zSGgL31L/rNnDrMLkuBNf9cuzYsaLt74Rbp8etc+K2X1tbq3I+tblOODdeui6Vy22f3Ho87pxdbl2H2hC6tEq7/Vw3p+tgcu9Vn11argPHdeyU3m9cd01f6zC5eeqO7+Ir7YB09/E+O35dp52bj64TbWpqqtYpjc29b+6c3bXrPufce+KuxdK10viGBwAAhEfBAwAAwqPgAQAA4VHwAACA8Ch4AABAeFW7tFz3i+v0cOv3uCe7+1wvxT2N7jp53LhbR8XF7roWuuiWcfly3VjuifjS9cUmzc0jlys3PoT1e9w8dTlcXl7Ojk9qbZutcDG6cZcXd225DqY+uevbjbtupdKOQrd9X11M7r7pPgNcF48bdznvs0vLXVsuJzMzM9nxPs/ZcZ8Bpd1SruvK5b3W2pN8wwMAAMKj4AEAAOFR8AAAgPAoeAAAQHgUPAAAILzUNM2kzwEAAKBTfMMDAADCo+ABAADhUfAAAIDwKHgAAEB4FDwAACA8Ch4AABDevwE/RsNZs4Ug5wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x144 with 10 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjwAAABeCAYAAAAnrM7iAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAJWUlEQVR4nO3dz2tdaR3H8c8jg4wbcyvjxoVtN8IgmBZ1NYs0OJtBxqbrgUn6FzRZzEokzUaXScGdizTCwCAjJJsRC2KKuE7juBBEkoIuVKSNP1dyXDTFNDyfe+9z8txz7/nm/YJZzHNPzzmfnh/3y53nO09qmkYAAACRfWbaJwAAADBpFDwAACA8Ch4AABAeBQ8AAAiPggcAAIRHwQMAAMKrUvCklH6QUlqtsa8Rx/lOSumjSR/HHDt0xuj5To9NxnrHIePkjhs63+mxyVjvONyn42qa5kL/SPqipD9J+tzpv78n6Z9n/vm3pEbS18fc37Gk/5z584/Off5bSV+76HmT8fLkIyMZ+5Ixej4yxsjY13w1gn8g6UdDPl+R9AdJqSD420M+/66kH3Z1YS9Dxuj5yEjGvmSMno+MMTL2NV+N/6T1jqTHQz5flvTj5vSsK9iX9O1K+xpX9IzR80lklMjYxr54FmsjY/8z9jNfhUrvr5K+aT67Kum/kq4X7O9Y0p9P9/tI0vy5z7+gFz+Vfb7DajZ0xuj5yEjGvmSMno+MMTL2NV+NX3gGkv5hPntf0q+apjkq2N97kq7pxV/aLyX9PKU0OPP5y2MN1J2BYmccKHa+l8ciIxnPm7WMA8XO9/JYZOx3xoH6mK9CpfcX+Urv95LuXnD/v5P07rQq2cuQMXo+MpKxLxmj5yNjjIx9zVfjF57fSPrK+cGU0luSviTp4wvuv5GUzvz7m5KOm6b5+wX3WyJ6xuj5JDKScTzTzhg9n0TGCBl7ma9GwfOJpIXM+LKknzZN88rPXimllZTScW5HKaUvp5TeSil9NqX0ekrpA0lvSPr1mc0WJP2swnmXiJ4xej6JjGQ8Z0YzRs8nkTFCxn7mq/DT1huS/qjTfvzTsdclPZf0rcz235P0odnXV/WicvyXpL9J+oWkb5zb5lOdm9DUwc93oTNGz0dGMvYlY/R8ZIyRsa/5aoX/vqTVMbd9JOnNlsd5V9JPurqolylj9HxkJGNfMkbPR8YYGfuYL53uEAAAICwWDwUAAOFR8AAAgPAoeAAAQHgUPAAAIDwKHgAAEB4FDwAACI+CBwAAhEfBAwAAwqPgAQAA4VHwAACA8Ch4AABAeBQ8AAAgPAoeAAAQHgUPAAAIj4IHAACER8EDAADCo+ABAADhUfAAAIDwKHgAAEB4FDwAACA8Ch4AABDeayM+b3KDe3t72Y03Nzez48+fP8+OHx4ejjj8q46OjrLj165dc38kjbHbbEbn1q1bRePOYDDIjq+urhbtRxfIeHx8nN14aWkpO156vZzbt29nx3d3d90fGZUxm+/hw4fZjbe2trLjGxsb2fGDg4MRh3+Vu4bumusC1/DJkyfZjd39WHp/uWdrZWWlaD+6QEb3/nDn5u6j0me0heoZ3Tm7Z/fGjRtjnML/ra2tZcfdM6qWz6JTer7u3eTuxyHfDU5n3xmPHz8u2Y3W19ez4/fv3y/ajyaQ0b3bTk5OSnajhYWF7Lh7l5d+9/MLDwAACI+CBwAAhEfBAwAAwqPgAQAA4Y2atJy1vb2dHXeTsObm5rLjbhKWm+TVYgJaa24yqMvoxt3kvw4mUI7kJiG78eXl5ez4nTt3suPuupdOVGzLTex0+dyEyFIVJ/qO5Ca7usmCbmK2Mz8/nx3v8hktzbi4uFi0/6tXr2bH9/f3s+OTyOjeN+5ede9Ox53zkMnJVblGl9JGCLe9u1ZufBJKvzPcs+XeQzdv3mxzWlW5d6p7Fmt9B7h3QCl+4QEAAOFR8AAAgPAoeAAAQHgUPAAAIDwKHgAAEF6rLi03w9rNUnfbt/hf8HfGnYPr6HDZZyGL8+zZs6Lt3XV0fydddWM5pd009+7dy467HG7/XXbgld5fLqN7FrvsjHRKO21c59G0OkPG4d4frpOnxXICU+WWDCjt4umy66pU6bPolkuY9ntzGNel5bj3yrTuX37hAQAA4VHwAACA8Ch4AABAeBQ8AAAgPAoeAAAQXqsuLefp06dF4242eulM8EkoXeOldC2tWeA6Q5y1tbWi7d2aa5NYU6qGBw8eZMddJ8nOzs4kT2cspZ0hLqPrGHFdMV12kly5cqXKftxaRLPwjLqOMPcuLH13uq4Y10VTm7tPXQ7397G1tZUdd++ULrtkSzsa3Tm7Lk+XvUulXXKla5xNusOVX3gAAEB4FDwAACA8Ch4AABAeBQ8AAAiPggcAAISXmqYZ9nn2QzeDfnd3t+jgd+/ezR90+DmVSGNskz1YaXfEyclJdtytXVRxxn3rjK5Ly13H0u4BNxO/xRpFozIW3TDuvNw1XF5ezo5XXO+n9TV03DVxXSul6/q0uH+rZ1xaWsqOl3YfVlwHr3XG0neq61w7ODjIjpd2ywxR9Vl03WMbGxslu7Frdc3Cs+i6sUq7PDc3N7PjLTrtqmd03w2ue9kedMLf/fzCAwAAwqPgAQAA4VHwAACA8Ch4AABAeBQ8AAAgvFZdWqX29vay467LwnUatFi/p/psdMd1dV2/fj07fnR0lB0vXY9FHWZ0nSSl3TLT7tJy3Pm6NZgqduBN/Rq6Z8t1X7iuriE6y1j6vpnl7pdS7nq5jrPSzlp19Cy67ip3vm6NuBbfJVN/Ft19595PpV2J6jBj6Tt1fX09O+66+YagSwsAAFxOFDwAACA8Ch4AABAeBQ8AAAiPggcAAIT3Wps/5GaXu3Uz3FpE8/Pz2fEW3VidqbUmjevqatGlVZ3L4roHDg8Ps+Pb29uVzqgdd5+6LirXATI3N5cdd2vkzALXqeQ6y9zfVYvuiM64TjG3Rp8zy+8b955wHWfuOrbo5OmEO1+3rl3pMzoL71N3Dd371D277nu0Sy6L+86YtfuOX3gAAEB4FDwAACA8Ch4AABAeBQ8AAAiPggcAAITXai0tNyPbdQ64GfcV15Nyqq8ZUrpulNu+xXpLTuuMpet/uU4I121QscOn1fo97j5dXFzMjrt8bj8Vu3taX0PX5eKeIbemkrt/3fYtVH8WS98TrqvLrT/VQuuMpe8Pd87uvdLhdSy6hq7TcWdnp2Q3tiO0RSdlZ98ZrhtrYWEhO17aCTxEZ+8b993fwbuWtbQAAMDlRMEDAADCo+ABAADhUfAAAIDwKHgAAEB4o7q0AAAAeo9feAAAQHgUPAAAIDwKHgAAEB4FDwAACI+CBwAAhEfBAwAAwvsfDpos/BCJznIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x144 with 10 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjwAAABeCAYAAAAnrM7iAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAJSklEQVR4nO3dT2hlZx3G8ee1LVKUzoy0BSmYiQuxGyeDliJlSAbclGInK1diZhB0I0xQXIgIk40uMwPddZHJQpTiItlY6KKTCF1IhUys6EY0A/6BVplJlSpVelzcFK7hfZK8N+85OeeX7wcGmjeHc89zz5/74/b95U1N0wgAACCyj5z0AQAAALSNggcAAIRHwQMAAMKj4AEAAOFR8AAAgPAoeAAAQHhVCp6U0o9SSos19nXI67yYUvpp269jXjt0xuj59l6bjPVeh4ztvW7ofHuvTcZ6r8N1elRN0xzrn6QnJP1Z0qNjY1+R9DtJ/5D0W0nzR9zXk5J+IukvknYlvSHp2X3b/EbS54573GQ8PfnISMahZIyej4wxMg41X43g35X08tjPT0l6X9LzkpKkFyS9J+nJI+zr05K+LemTkh6S9A1Jf5P08bFtvi/ppY4v3tAZo+cjIxmHkjF6PjLGyDjUfDWCvy7pq2M/Pyvp7X3bvCPpixPu/11Jnx/7+TlJf+zqxJ6GjNHzkZGMQ8kYPR8ZY2Qcar4awd+R9MzYzw9J2pT04t5/z0v6k6SPTbDvGUn/lnRmbOwTkhpJj3V4ckNnjJ6PjGQcSsbo+cgYI+NQ89UI/h9Jn9039nVJ/5T0X42+1nphgv0+JuktSd/bN/7IXvBPdXFiT0PG6PnISMahZIyej4wxMg41X43gb+v/K70vSfq7pC9o1AX2jKS/Spop2OejGlWLL2d+dxLVeuiM0fORkYxDyRg9HxljZBxqvhpt6b+W9Jmxn2ck/aJpml81TfNB0zRvSvrl3htyqJTSRyWtaTQD/JuZTZ6WtNM0zbvHOehC0TNGzyeRkYwZPcwYPZ9ExggZB5mvRsHzc0mzYz+/KelSSmlGklJKFyVd0ugNUkrpakppJ7ejlNIjkn4m6V+SvtY0zQeZzWYlvVrhuEtEzxg9n0RGMu7T04zR80lkjJBxmPkqfLX1uEaTk8b78b8l6fca9eP/QdJ3xn73A0k/Nvua1ehrq/c0+n+BH/67NLbNW5Iu1Px67rRnjJ6PjGQcSsbo+cgYI+NQ89UK/0NJi0fc9jVJT0/4Ol+W9EpXJ/U0ZYyej4xkHErG6PnIGCPjEPOlvR0CAACExeKhAAAgPAoeAAAQHgUPAAAIj4IHAACER8EDAADCo+ABAADhUfAAAIDwKHgAAEB4FDwAACA8Ch4AABAeBQ8AAAiPggcAAIRHwQMAAMKj4AEAAOFR8AAAgPAoeAAAQHgUPAAAIDwKHgAAEB4FDwAACI+CBwAAhEfBAwAAwnv4kN83JTtbX1/Pji8vL2fH19bWsuNnz54tedmDpCNsU5Txxo0b2fGlpaXsuMt45cqVkpc9yMQZHzx4kN14bm4uO76zs5Mdv3v3bnb8/PnzhxzWkR2WsegcuhzT09PZ8QsXLmTHNzY2suMTXL8Tn0OXZWZmJju+u7ubHV9ZWcmOX7169dADO6Lq96LL6K5rl2VxcTE73uV5dG7fvp0dv3nzZnbcvScuo9v+AFXvRZfDPWcd95x1z7IDVH+ezs/PZ8fdveuO2V0LE6h+nbqM7j1xz86Kshn5hgcAAIRHwQMAAMKj4AEAAOFR8AAAgPAOm7RcZGFhITvuJv+5SVhugl2X3GSr0knIbjJX0xTNCWuFy+Im0zlu4qEbP2luwtyZM2ey49vb29nxPly/7tjc5GSX0U0SrThpeWLuXnTZHddY4CbXd5ndNXy4a8lNbF1dXc2Olz7Paitt9hiizc3NonHHnUN3PU4wMXti7pnurl/HbV+xmSeLb3gAAEB4FDwAACA8Ch4AABAeBQ8AAAiPggcAAIRXtUvLdTu4rhjXwdSHLi3XWeb+FLvr2HHbu+UYJvhT7xNzx+Def9e91cGfCa/KvceuA+HatWvZcdf50gduOQyXvc/n0N2LW1tb2fF79+5lx0v/xH+X3Ply16Tr2Ekpv2pAX69V1zXonkF97upyHUauI9fdc5cvX651SNWVLr/jOindvUuXFgAAwDFR8AAAgPAoeAAAQHgUPAAAIDwKHgAAEN5EXVquq8F1gLguiz50R5Ry3QOl6/24jrYuuW6Aqamp7LjL7s5vX9XqhOvDOXRdDW7crbvThyylSjsmnT6sF9aH9bza5NbSKl3DzXV1dbmeVC2uS9Zl7LKD13HPldnZ2ez4uXPn2jycYnzDAwAAwqPgAQAA4VHwAACA8Ch4AABAeBQ8AAAgvIm6tFxHgZtx77g1b1zHUx+6gdxMedfB5Gav9yGLm3Hvugc2Nzez4ysrK7UOqRPu+nKdIW6dmD530LiM7hwuLy+3eDTtcBlv3bqVHb9+/Xp2fIgdam6NLaev16o7h2tra9nxvuY4iOtGLu1Q68NnhlPaGXnx4sV2DuQQfMMDAADCo+ABAADhUfAAAIDwKHgAAEB4FDwAACC8ibq0HNft4GZwuzVD+jwb3XUwuYyrq6vtHcwxra+vZ8cXFhay4+58zc/P1zqkqtw5cd0Rrmvwzp07lY6oO65rcGjnUPKdPG6dHtdVV9rZ1AfuHnXXsLt3+9rd5Lqxdnd3s+Puuu6z0utuiBldJ5p73rju5bbxDQ8AAAiPggcAAIRHwQMAAMKj4AEAAOFR8AAAgPCqdmm52eVubRs3g9vtx3VvuQ6ENtbIKe1m2draKhp3+3dreB3H/fv3s+OuQ6JWh487j65jY1Kuo851YznueixdV62Nc+gyuu5A18XjOtrcPeTOeRsdlm79L8cd88bGRtF+5ubmirZvg1unzt2jpdlPOqPr7nHcderuA/d+dNmx5z7/pqamsuOl61J1+ZnhlH7Guy7D0udN6Wc83/AAAIDwKHgAAEB4FDwAACA8Ch4AABAeBQ8AAAivapeW65ZyM/HdLHLXreM6QFynQRtdWq4Dxx1zaWdI6X6O0xXjZr67Y3DnsbSLpqt1VFwnhsvttnfv/fb2dtHxlHZfHIU7/647otbabm59qzbWAXLXi1szy3XsuM4Qp/TebYN7RrqMpc8P95501b3lriNnaWkpO+6ukT6sIeY6I925cuPunJS+h21w77/7zCj9PJuens6O06UFAACwDwUPAAAIj4IHAACER8EDAADCo+ABAADhpaZpTvoYAAAAWsU3PAAAIDwKHgAAEB4FDwAACI+CBwAAhEfBAwAAwqPgAQAA4f0PkOXo5uc8dcQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x144 with 10 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjwAAABeCAYAAAAnrM7iAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAJuUlEQVR4nO3dz4tdZx3H8c9jU20kdi6iFbpoMohCoe1k6MKFlEn+AM240JU0dyG6TAbBnU5CodkUkll04yYJKhUxMEEaf4A2sxAENxkriFDIRFTcTVJru5E+Lu4NXNPzuZnnznPOnPvN+wWzuM+9c879zHPOmW/unG+elHMWAABAZB876DcAAADQNgoeAAAQHgUPAAAIj4IHAACER8EDAADCo+ABAADhVSl4UkoXUkpna2zrIfv5akrpp23vx+w7dMbo+cb7JmO9/ZCxvf2GzjfeNxnr7YfjdK9yzvv6kvRZSf+QdHhi7FuS3pH0nqRfSXq6YHuvSHpb0n8lnWt4/s+SXtjv+yZjO/kkPTP+nsmvLOm7zCEZyfho56udUdJTkt6Q9E9J9yT9XtKXyDg/+bo8Tmt8wjOUdCPn/IEkpZRWJL0q6ZSkT0u6PZ6svXpH0vckvWmef0PSt2d9szMaKnbGoSrlyzn/Led85P6XpOclfSjp2sTLmMN2DEXGec84VOx8Ut2MRyT9UdKL4++9KunNlNKRideQsb6h5vE4rVDp/U7SNycevybp9YnHT2v0L/zPF273x2qu9L4s6XZXleyjkLGtfOPvXZf0FnNIRjKSr82ME9//rqQXyTh/+do+Tmt8wvO8pL9OPE7jr8nHkvRchX1J0l8kHUspPVlpe3sRPWOb+V7W6F8kk5jDdpBx/jNGzye1mDGldFzSxzX6xOA+MtY3l8dpjYJnIOnfE49vSPpGSumFlNJhST/QqNL7ZIV9aWJfg0rb24uBYmccqIV8KaWXJH1O0s8feIo5bMdAZJz3jAPFznd/X21cb56U9CNJ53PO9yaeImN9A83hcVqj4NmV9Kn7D3LOv9XozxjXJN2RtKPRm/17hX1pYl93K21vL6JnbCvfaUnXcs7vPTDOHLaDjPOfMXo+qYWM41+yv5D0h5zzhQeeJmN9c3mc1ih4/iTpi5MDOefXc85fyDk/pdEP4JBGd1nX8KyknZzzu5W2txfRM1bPNz45v66P/jlLYg7bQsb5zxg9n1Q5Y0rpE5I2Neoa+k7DS8hY31wepzUKnhuSVu4/SCk9kVJ6Lo08I+mHkjZyzrvj54cppR23sZTS4ymlJ8bv7dB4e49NvGRF0i8rvO8S0TNWzTf2NY2q8bcanmMO20HG+c8YPZ9UMWNK6XGN/mT+gaSXc84fNryMjPXN53Fa4W7tz2j0sdXh8eOBRtXffyT9S9IFSY9NvP77kn4yZXtXNPrb3+TXcOL5tyUt7fd9k7G9fOPX/FrSK+Y55pCMZHwE89XOqNEvwizpff3///31EhnnI1+Xx2mt8K9KOrvH1/5G0rMz7ucrkn7W1aQ+Shmj5yMjGeclY/R8ZIyRcR7zpfEGAQAAwmLxUAAAEB4FDwAACI+CBwAAhEfBAwAAwqPgAQAA4VHwAACA8Ch4AABAeBQ8AAAgPAoeAAAQHgUPAAAIj4IHAACER8EDAADCo+ABAADhUfAAAIDwKHgAAEB4FDwAACA8Ch4AABAeBQ8AAAiPggcAAIRHwQMAAMKj4AEAAOEdesjzuc2dHzt2rHF8MBg0jt+8ebPo9ZLSHt5GlYw7OzuN44uLi0XbuX37duO4+1mpw4znzp1rHL9y5UrRdtw87iNjq8fpiRMnGsfX1tYax0+dOlW6i87m8O7du43jw+GwcdzNyaVLl0p33VlGx82j+5n0+XrjuPk6e/Zs0fgUB3ouuvfr5urWrVuluzjwOVxdXW0cdxk3Nzcbx93xrh5cb44fP9443vbvfj7hAQAA4VHwAACA8Ch4AABAeBQ8AAAgvIfdtFzF9evXG8fv3LlTNO5ugJpy41Jn3E3LkbgbIt3Nce5mOnfjoXt9V9wcbm1tFW1nhpuWO+NuMHfn6NLSUovvph3u5no3jwsLC43jfb7euJs+3bVz3q5P7v1ubGwUbafPc+huqHbnouOOd3ejbx+U/u53x4M7Dxw+4QEAAOFR8AAAgPAoeAAAQHgUPAAAIDwKHgAAEF4nXVpnzpwpev3Kykrj+JSlB6pzd/e7O+vdnfJOHzKWcssPlC5LcNAdEqXLKzgHnWMa16FRuiTElP+e/sC5+bp69WrRdlynRx/ORZdxe3u7aDt9yFLC5XO/S1z3YZ/PUdeVevTo0cZx18HUJddBdvny5cbxWp1ipd1YDp/wAACA8Ch4AABAeBQ8AAAgPAoeAAAQHgUPAAAIb6YuLdfl4tZI6sPd5aXcXf9ra2vdvpEWufVJXHbXoeaOBzfvXXWMuHyu86V0zaw+dICUdiq5db7cXPUho+v0KO3Gctzx2yXXPecyrq+vN46fP3++cXx1dXWWt3VgStejO+h1+GaxvLzcOO6uW+5YKO0u3Q+37pw7R+/du1dl+7XwCQ8AAAiPggcAAIRHwQMAAMKj4AEAAOFR8AAAgPBm6tJyd5G78dK1QWqtm7EfruPMrS1U2mXhflZdct1YrtOjlq46f1zngNu/6/Q4ffp043gf1plyc1jaaefWyOnzGkwXL15sHHfXj5MnTzaOu3O9S67TZnFxsXF8d3e3aPvuXOiyw6eEO07duVi6lmEbXLefOxfdnLhzcWlpqXG8yzl01zyX3c2L+x3T9jzyCQ8AAAiPggcAAIRHwQMAAMKj4AEAAOFR8AAAgPBm6tJyXRCld5279V3cXe2uE6pLLntpN0sful/cHfGld8q7eTzozhC3HzfuOg3cejCukySSPnQTus6Q0o4Rpw8ZXeegW1PKnVt95a7dpWsTup9HHzrt3HVzY2OjaDuuq3ne5lwqP7fa7nzlEx4AABAeBQ8AAAiPggcAAIRHwQMAAMKj4AEAAOHN1KVVamFhoej1Xa21VFNp19XW1lbjuLurvQ9dXa4TwnXhubWO+jq/89jdU8qtX+fMY8ZHQR+uByXctcN1HpWu+ea6wFzXTxvrNbruT3e9c93I7hx1XWB96F52So9TN++15otPeAAAQHgUPAAAIDwKHgAAEB4FDwAACI+CBwAAhJdyztOen/rkXrnuF3cH/fb2duP47u5u4/iUrp809Y2NVMnoulncnfvubvTNzc3G8SlrjHSWsXTtIteBMUOX1sMyVsnn5mR5eblxfH19vXG8dC0y9eA4dXPrssywHlpnGZ3SdXpmWLuotxndOeeuN1N0ci46rtvLdTy543RKZ1Nnc+ium+49uw61lZWVxvEpx++BX2/c+ovu9e7aPKULrDEjn/AAAIDwKHgAAEB4FDwAACA8Ch4AABAeBQ8AAAjvYV1aAAAAc49PeAAAQHgUPAAAIDwKHgAAEB4FDwAACI+CBwAAhEfBAwAAwvsfYLVPngMDLnwAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x144 with 10 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.cluster import KMeans\n",
    "\n",
    "km = KMeans(n_clusters = 10, random_state = seed)\n",
    "yd = km.fit_predict(X)\n",
    "\n",
    "for k in range(10):\n",
    "    ii = np.argwhere(yd == k).ravel()[:10]\n",
    "    digiplot(X[ii], list(zip(yd[ii], y[ii])))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Zur besseren Übersicht wandeln wir die Klassennummern in die (vermeintlich) erkannten Ziffern um."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "code_folding": [
     0
    ],
    "execution": {
     "iopub.execute_input": "2021-09-24T09:28:22.262175Z",
     "iopub.status.busy": "2021-09-24T09:28:22.261884Z",
     "iopub.status.idle": "2021-09-24T09:28:23.327628Z",
     "shell.execute_reply": "2021-09-24T09:28:23.327244Z"
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjwAAADVCAYAAABT2yr5AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAUmklEQVR4nO3dX8jeZ33H8c+3pmxruyYphU5lS8Kw1FXME3e2HSTB6uw8SHQ7KJP6JCeOdoMkqGMnkqQ6mDBIwlrnhPLk0Y5BB10yOlHEJmXqiUKSgay4sidZZQY7SGL/LFW7awf3ky6E6/Mk153fn9vv835BT765ue/re/+u33V/+8v1zRWlFAEAAGR2y9gDAAAA6BsFDwAASI+CBwAApEfBAwAA0qPgAQAA6VHwAACA9Ch4AABAeqMWPBHxroi4HBFPjTmOvmTOLyJevea/NyPir8ceV1ci4pci4smIOBcRr0TEqYh4cOxxdS0iNkbEVyPiQkScj4jHI2LN2OPqSvbrmD2/KyLi3RHxXERciogXI+IjY4+pS9nXU2k2ruHYT3iekPTdkcfQp7T5lVLuuPKfpHsk/Y+kfxh5WF1aI+klSVslrZX0GUlPR8TGMQfVgy9I+rGkt0ua0yTfR8ccUMeyX8fs+Wm5AD8u6VlJd0n6hKSnIuLeUQfWoezr6axcw9EKnoh4SNJFSd8cawx9yp7fNf5Qkx/Nfxl7IF0ppbxWSjlQSjlbSvnfUsqzkpYk/fbYY+vYJklPl1Iul1LOS/qapPtHHlNnsl/H7Pktu0/SOyQdKqW8WUp5TtK3JT087rB6k2491Yxcw1EKnoi4U9Jjkj45xuf3LXt+FfOSvlwSn1MSEfdIulfS98ceS8eOSHooIm6LiHdKelCToielxNdRUtr8wsTeM/RABpJxPZ2JazjWE57PSnqylPLSSJ/ft+z5vSUifkOTx+mLY4+lLxFxq6S/k7RYSnlh7PF07HlNnuj8RNIPJX1P0rExB9SX5Ncxc34vaPLE49MRcWtEfFCTNee2cYfVvcTr6Uxcw8ELnoiYk/SApENDf/YQsudX8XFJ3yqlLI09kD5ExC2SviLpp5L+dOThdGo5t69LekbS7ZLulrRe0ufHHFcfMl9HKXd+pZSfSdop6cOSzmvy5PxpTQr0bFKup7NyDcfoxtgmaaOk/4wISbpD0tsi4rdKKe8bYTxd26bc+V3r45L+cuxB9CEmF/BJTTYR/v7yTZvJXZJ+XdLjpZQ3JL0REQuSPifpz0YdWYeyX8fs+UlSKeVfNXkiIEmKiO8o31MQKfF6OgvXMIb+a8KIuE3SnVeFPqVJgfBIKeXlQQfTg+z5XS0ifkfSNyT9WinllbHH07WI+KImnUsPlFJeHXk4vYiI/5D0JUl/pUlxviDp9VLKx0YdWIeyX8fs+UlSRLxX0g80+VuJRyX9iaT7lgv1FFbBejr6NRz8r7RKKa+XUs5f+U/Sq5IuZykGsud3jXlJzyS9OTdI+mNNfkjOX/XvY6QpBJZ9VNKHJL0s6UVJP5e0b9QRdSj7dcye31UelvQjTfaBvF/SBzIVO8vSrqfLRr+Ggz/hAQAAGNrY//AgAABA7yh4AABAehQ8AAAgPQoeAACQHgUPAABIj4IHAACkR8EDAADSo+ABAADpUfAAAID0KHgAAEB6FDwAACA9Ch4AAJAeBQ8AAEiPggcAAKRHwQMAANKj4AEAAOlR8AAAgPQoeAAAQHoUPAAAID0KHgAAkB4FDwAASI+CBwAApLfmOn9euviQs2fPVuPbtm1rev0U4gZe00mOFy9erMZdjjt37qzGDxw40PrRned4+vTpavzw4cPV+OLiYsvb68SJE9W4+650/Ryr+bk83HfvrqHLe9euXdcZ1g3r/Bq6sblrtWHDhmr8yJEj1fiOHTtahiPdRI7uuuzdu7cad9f92LFj1fjGjRuvM6wb1vl1XLduXTXuxnzy5Mmm95nCVPei03ptW/Ob4vsYbJ66e3Ht2rXVuJvXU8zfzuep+57dOjTAWlvNkSc8AAAgPQoeAACQHgUPAABIj4IHAACkd71Ny51wm5DdxqVfRG6zscvdbZydBXNzc9X40aNHq3G30cy9j9t8t8Km5am4jXRu3rmNjG7TYYeblqfm5lfrRnKXyxSbkzvXuunTcTm6eTIL3Jx0c3ioe6srbu1w89qts27jboebtafmctm8eXM1fubMmab36XDT/XUdP368Gu/q92z37t3VeFdrLU94AABAehQ8AAAgPQoeAACQHgUPAABIj4IHAACkN0iXltuJ73Z2u53gS0tL1bjr4uiD2ynv/gn+PXv2VOOt3Rez0G3guDG7uOve6lpr54B7vetwcR0xQ+Un+Q4NN+/cPD148GA1vmXLlmp8yO4tN/db7y2X4yxzc8zFXRfTrHaiuc5PF5/VbjOp/XgLJ6J+8oN7n1n+TrrS1VrLEx4AAJAeBQ8AAEiPggcAAKRHwQMAANKj4AEAAOl12qXlulzc7nK389q9z5BnF7luLLcj3p2L4rom1q9fX40vLCxU40Pm3tpV4HJ0Ll261PT6abkOJtfV13q22yyfBdfV+TobNmzo5H1uhuvydFyHjzMLZxQ5rvPHjXlWuc7bc+fOVeNuTWntYp2F7jQ3f1vHduzYsabXt67LN8J1Z7oOancvulzcOWJuntClBQAAcA0KHgAAkB4FDwAASI+CBwAApEfBAwAA0puqS8t1DLmd+Pv376/G3Q5ut1N7yPOkXCdPa1dBa4fJkOcwuY4H1yXX2l3lzjpau3Zt0/sMxc1H9z3NQheP4+ad67py95ab160dI31w18WNzc27Wb6OjstxVnNxXaytZ7K5Lll37w55/pS7V7o6w839Lg55lqTj5l1rp5jL8cKFC40jquMJDwAASI+CBwAApEfBAwAA0qPgAQAA6VHwAACA9KKUstKfV//Q7bxuPRvEdVm4c2JcfIXurbiBMVRzdGdddcV1LbjzxVZw3RxlrqPjvmfXDeBeP0UuzvVybMrPdRPOz89X425+uc4Q12l3M/NUJkeXi+u0c90Oruunq3PH1MM8devN9u3bq3F3z3XY8dR5ju57duuT64h1ubj3WaF7qtN7sSsR9WGdOHGiGl+hS2vqa+jWQfd7ubi4WI277tbWjt8VdD5PW7Xeu25er9AFVs2RJzwAACA9Ch4AAJAeBQ8AAEiPggcAAKRHwQMAANKb6iyt1vMxHHcm1xRdLlNz73no0KFq3HV0uG4KdzbIkGdmtXLfiesImoVzlWpcJ4DrxnJnhbm46yhYWFioxt18vxlLS0vVuDszq+8OyyHPu2v9Pt29uGnTpmq8w07KqbWe+dR6bpObJ61nXA3FdUbOwjlp7rNcp6Pr0nIdlrOs9Vy71o6zrq4jT3gAAEB6FDwAACA9Ch4AAJAeBQ8AAEiPggcAAKQ3VZdWK7eD23U7uJ34Q3I7613ccV0rs9yl5c6E2bp1azU+qx0drVxXjruGLj5kl4XrVHJdV0eOHKnGXZeL66YYshvLcd+zy911erRe3yG1rhPu9a7bq7ULbChuXG5t6vA8tM65e2gWOsu64n7LXe7uOrpzxLrqcOUJDwAASI+CBwAApEfBAwAA0qPgAQAA6VHwAACA9KKUMvYYAAAAesUTHgAAkB4FDwAASI+CBwAApEfBAwAA0qPgAQAA6VHwAACA9Ch4AABAehQ8AAAgPQoeAACQHgUPAABIj4IHAACkR8EDAADSo+ABAADpUfAAAID0KHgAAEB6FDwAACA9Ch4AAJAeBQ8AAEiPggcAAKRHwQMAANKj4AEAAOlR8AAAgPQoeAAAQHoUPAAAID0KHgAAkB4FDwAASI+CBwAApEfBAwAA0qPgAQAA6VHwAACA9Ch4AABAehQ8AAAgPQoeAACQHgUPAABIj4IHAACkR8EDAADSo+ABAADpUfAAAID0KHgAAEB6oxQ8EfHuiHguIi5FxIsR8ZExxtG3iHhXRFyOiKfGHkvXImJjRHw1Ii5ExPmIeDwi1ow9ri6thhyvSD5XU683EXFXRPxjRLwWEeci4o/GHlPXVsO9uArm6ej5DV7wLE/S45KelXSXpE9Ieioi7h16LAN4QtJ3xx5ET74g6ceS3i5pTtJWSY+OOaAerIYcr0g5V1fJevOEpJ9KukfSxyT9TUTcP+6QOpf6Xsw+T2clvzGe8Nwn6R2SDpVS3iylPCfp25IeHmEsvYmIhyRdlPTNkYfSl02Sni6lXC6lnJf0NUnZFtnVkGP2uZp6vYmI2yX9gaTPlFJeLaV8S9I/KUl+V8l+L6aep5qR/MYoeMLE3jP0QPoSEXdKekzSJ8ceS4+OSHooIm6LiHdKelCTRSiT9Dmugrmafb25V9KbpZQfXBU7o1zFgJT/Xsw+T2civzEKnhc0eTT56Yi4NSI+qMnjydtGGEtfPivpyVLKS2MPpEfPa7Ko/kTSDyV9T9KxMQfUg9WQY/a5mn29uUPSpWtilyT96ghj6VP2ezH7PJ2J/AYveEopP5O0U9KHJZ3X5P8sn9ZkEv/Ci4g5SQ9IOjTyUHoTEbdI+rqkZyTdLuluSeslfX7McXVpleQ4p+RzNft6I+lVSXdeE7tT0isjjKUXq+FezD5PZyW/KKUM+Xn1QUR8R9JiKeVvxx7LzYqIvZL+Qv+/4Nwh6W2S/q2U8r6xxtWliLhb0suS1pVSLi3Hdkr6XCklxSPYVZLjXiWfqzXJ1pvbJV2QdH8p5d+XY1+W9F+llD8fdXAdWQ33Yk2meVozRn5jtaW/NyJ+efnvYz+lyc77o2OMpQdfkvSbmnQSzEn6oqR/lvR74w2pW6WU/5a0JOmRiFgTEeskzWuydyCF1ZCjVsFclXKvN6WU1zR58vFYRNweEb8raYekr4w7su6sknsx9TyVZiO/sf7hwYcl/UiTv9N7v6QPlFLeGGksnSqlvF5KOX/lP00eOV8upbw89tg69lFJH9Lk/7xelPRzSftGHVH3Uue4iuZq2vVm2aOSfkWT/P5e0iOllO+PO6TOpb4Xl2Wfp6PnNxN/pQUAANAnjpYAAADpUfAAAID0KHgAAEB6FDwAACA9Ch4AAJAeBQ8AAEiPggcAAKRHwQMAANKj4AEAAOlR8AAAgPQoeAAAQHoUPAAAID0KHgAAkB4FDwAASI+CBwAApEfBAwAA0qPgAQAA6VHwAACA9Ch4AABAehQ8AAAgPQoeAACQHgUPAABIb811/ry0vNmuXbuq8cXFxWp87dq11fjZs2er8XXr1rUMR5LiBl7TlKNz8eLFanznzp3VuPuuXHwFned44MCBavzgwYPV+KlTp6rxubm5atx9Vytc3+vl2JTf8ePHq/FDhw5V49u2bavG3ffUQ35SR/P09OnT1bjL8dixY02vX0HnOZ48ebIaP3r0aDXuct+7d281Pgv3ostl9+7d1fjmzZur8Y0bNza9/1D3ortX3Hfvrvks/2a4sbn1w62bbp5OofMcW6+Xe73LsavryBMeAACQHgUPAABIj4IHAACkR8EDAADSi1JW3JvUtHEpor4XauvWrdX4vn37qvEdO3a0fOyKQ7qB13SyGdRtTnabttwGSre5cAWd5+iu44YNG6rxI0eOVON79uypxl2O7rtSxxsl3bVym5ndxt39+/dX41Nsgh1sQ+/27durcXePrnBNWk2do9vYun79+mrczVM3755//vlqvHUzvga8F13Dhxuby/HChQvV+FCblt14z5w5U427e87dWy6PIRsI3Lw7d+5cy9t02eQzdY7uszZt2lSNu3vRbdh2azOblgEAAG4QBQ8AAEiPggcAAKRHwQMAANKj4AEAAOld72iJKrdT23E78d2u81nmdpe7Dp+FhYVq3O06d9/tFN1bU3OdEO5oCbez3nVpHT58eJphdcZ9vutIcvm5zib3+iG5eeruudbvxM3fFTqYpuY6Gp3Wf7J/y5Yt1bjrDutDVzm6DkGXu+tAnOJYjam4bqzWjiTXJeTy67AT+C3uXnHdWK1H2bh56o4H6fAoire43yF3vdw91Po719W9yBMeAACQHgUPAABIj4IHAACkR8EDAADSo+ABAADpDdKl5c5acnF3/obbBd9HB5P7LNep5DqSXMeOG7Prjhiys8mN2eXeel7K2Nx37/JeXFysxlvP7+mDmxfu7CTXteK476S1G+hmuK4V1yW3e/fuTj730qVLnbzPjWjtQmntwHHXpXUt71prR6i7F13HUx/dWE7rfGntzHP6uOdauevS+lsyPz/f1ZCqeMIDAADSo+ABAADpUfAAAID0KHgAAEB6FDwAACC9qbq03K5wt8PadbO491m/fn01PuQ5U64Lwp0Z4nJx7+N29M/CjvvWDhB3VozrPBmyi6nGjau1g2nIs5Ycd46O47pi3L3l5ulQZy2txHVSurizffv2avzUqVPV+JDnMPX9/mN3UnZ1D83CuunmhVtXXGdv33OhD65LrvW8zL67BnnCAwAA0qPgAQAA6VHwAACA9Ch4AABAehQ8AAAgvam6tFyXTWvHSOuObHf2iDtr52a4HF3XSlfn97izR4bkzmFyXQjHjx+vxofsqmvhzp9qPQvHdVO0drndjNaOtzNnzjS9fvPmzdX42NdwJa3rgesk2bJlSwejuTGtn+XmsLvnXCel64gdSut5ii4Pt27OQielWzdbu/0iohof8l50a577zVhYWKjG3Zhdx6SrLVq7RXnCAwAA0qPgAQAA6VHwAACA9Ch4AABAehQ8AAAgvam6tBzXOeB2yrd2dfXRjeW480/cmN1ZLm4XueuuGfKcKdf15rhuLKfvc1Gm1Zq3MwudSq3nSblONNflMgtdg46bj0tLS03v476T1nOAbsbWrVubxrBv375eP3coLj/3m+Fe39phOSR3L3a1Dg2p9Zwv9/rWtbOr3xKe8AAAgPQoeAAAQHoUPAAAID0KHgAAkB4FDwAASK/TLq3WM7Zc15XrbHKdUH1wuXR1TtKQHWeO+z7n5+er8cXFxabXz2qHT1fn7rSe4zIkN79ac5yFeeq4c7727NlTjbvc3euHzN2tN67z1Z3d586gOnDgQNPnDsV1Krm8Z+H8ulaug8x1ArtzqVxH3ZDdoq3fc2sXqcuxq+vLEx4AAJAeBQ8AAEiPggcAAKRHwQMAANKj4AEAAOlFKWXsMQAAAPSKJzwAACA9Ch4AAJAeBQ8AAEiPggcAAKRHwQMAANKj4AEAAOn9H9SKvraxKm5JAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x288 with 20 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def ytrans(yd, y):\n",
    "    ydcl = np.sort(np.unique(yd))\n",
    "    \n",
    "    tt = np.zeros(ydcl.shape, dtype = np.int32)\n",
    "    for k,ydk in enumerate(ydcl):\n",
    "        ii = np.argwhere(yd == ydk).ravel()\n",
    "        u, uc = np.unique(y[ii], return_counts = True)\n",
    "        tt[k] = u[uc.argmax()]\n",
    "    \n",
    "    ttt = np.sort(np.unique(tt))\n",
    "    if ttt.shape[0] != ydcl.shape[0]:\n",
    "        print(\"**** Fehler ****\")\n",
    "        tt = np.array([], dtype = np.int32)\n",
    "    return(tt)\n",
    "\n",
    "tt = ytrans(yd, y)\n",
    "ydt = tt[yd]\n",
    "digiplot(X[-ndigi:], ydt[-ndigi:])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Die Beurteilung des Ergebnisses erfolgt wieder mit den Standardmethoden von Scikit-Learn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-24T09:28:23.330035Z",
     "iopub.status.busy": "2021-09-24T09:28:23.329673Z",
     "iopub.status.idle": "2021-09-24T09:28:23.336259Z",
     "shell.execute_reply": "2021-09-24T09:28:23.335965Z"
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.99      0.99      0.99       178\n",
      "           1       0.60      0.30      0.40       182\n",
      "           2       0.84      0.83      0.84       177\n",
      "           3       0.88      0.85      0.86       183\n",
      "           4       0.98      0.90      0.94       181\n",
      "           5       0.93      0.75      0.83       182\n",
      "           6       0.97      0.98      0.98       181\n",
      "           7       0.83      0.98      0.90       179\n",
      "           8       0.45      0.57      0.51       174\n",
      "           9       0.56      0.78      0.66       180\n",
      "\n",
      "    accuracy                           0.79      1797\n",
      "   macro avg       0.80      0.79      0.79      1797\n",
      "weighted avg       0.81      0.79      0.79      1797\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(metrics.classification_report(y, ydt))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Klassifikation mit PCA und k-Means-Clustering "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wir führen zunächst eine (partielle) PCA durch und schauen uns die Varianzen der\n",
    "Hauptrichtungen an"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-24T09:28:23.338879Z",
     "iopub.status.busy": "2021-09-24T09:28:23.338453Z",
     "iopub.status.idle": "2021-09-24T09:28:23.347364Z",
     "shell.execute_reply": "2021-09-24T09:28:23.347044Z"
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([179.0069, 163.7177, 141.7884, 101.1004,  69.5132,  59.1085,\n",
       "        51.8845,  44.0151,  40.311 ,  37.0118,  28.519 ,  27.3212,\n",
       "        21.9015,  21.3244,  17.6367,  16.9469,  15.8514,  15.0045,\n",
       "        12.2345,  10.8869,  10.6936,   9.5826,   9.2264,   8.6903,\n",
       "         8.3656,   7.1658,   6.9197,   6.1929,   5.8849,   5.1559,\n",
       "         4.4902,   4.2463])"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.decomposition import PCA\n",
    "\n",
    "nu = 32\n",
    "pca = PCA(nu)\n",
    "pca.fit(X)\n",
    "pca.explained_variance_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "bzw. die relativen Varianzen bezogen auf die Varianz von $X$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-24T09:28:23.350730Z",
     "iopub.status.busy": "2021-09-24T09:28:23.350314Z",
     "iopub.status.idle": "2021-09-24T09:28:23.352698Z",
     "shell.execute_reply": "2021-09-24T09:28:23.352373Z"
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.1489, 0.1362, 0.1179, 0.0841, 0.0578, 0.0492, 0.0432, 0.0366,\n",
       "       0.0335, 0.0308, 0.0237, 0.0227, 0.0182, 0.0177, 0.0147, 0.0141,\n",
       "       0.0132, 0.0125, 0.0102, 0.0091, 0.0089, 0.008 , 0.0077, 0.0072,\n",
       "       0.007 , 0.006 , 0.0058, 0.0052, 0.0049, 0.0043, 0.0037, 0.0035])"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pca.explained_variance_ratio_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Die Varianzen fallen stark ab. Kumuliert erhalten wir"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-24T09:28:23.355957Z",
     "iopub.status.busy": "2021-09-24T09:28:23.355505Z",
     "iopub.status.idle": "2021-09-24T09:28:23.357594Z",
     "shell.execute_reply": "2021-09-24T09:28:23.357335Z"
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.1489, 0.2851, 0.403 , 0.4871, 0.545 , 0.5941, 0.6373, 0.6739,\n",
       "       0.7074, 0.7382, 0.762 , 0.7847, 0.8029, 0.8206, 0.8353, 0.8494,\n",
       "       0.8626, 0.8751, 0.8852, 0.8943, 0.9032, 0.9112, 0.9188, 0.9261,\n",
       "       0.933 , 0.939 , 0.9447, 0.9499, 0.9548, 0.9591, 0.9628, 0.9664])"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "varcum = pca.explained_variance_ratio_.cumsum()\n",
    "varcum"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Mehr als 90 Prozent der Varianz von $X$ erreichen wir schon mit sehr wenigen\n",
    "Hauptkomponenten"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-24T09:28:23.360553Z",
     "iopub.status.busy": "2021-09-24T09:28:23.360150Z",
     "iopub.status.idle": "2021-09-24T09:28:23.362387Z",
     "shell.execute_reply": "2021-09-24T09:28:23.362115Z"
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "20"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "n90 = np.argwhere(varcum > 0.9).min()\n",
    "n90"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wir klassifizieren den Datensatz erneut indem wir\n",
    "    \n",
    "- zunächst mit PCA die Dimension der Daten reduzieren\n",
    "\n",
    "- dann wieder k-Means anwenden"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-24T09:28:23.366720Z",
     "iopub.status.busy": "2021-09-24T09:28:23.366397Z",
     "iopub.status.idle": "2021-09-24T09:28:24.804294Z",
     "shell.execute_reply": "2021-09-24T09:28:24.803984Z"
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjwAAADVCAYAAABT2yr5AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAUmklEQVR4nO3dX8jeZ33H8c+3pmxruyYphU5lS8Kw1FXME3e2HSTB6uw8SHQ7KJP6JCeOdoMkqGMnkqQ6mDBIwlrnhPLk0Y5BB10yOlHEJmXqiUKSgay4sidZZQY7SGL/LFW7awf3ky6E6/Mk153fn9vv835BT765ue/re/+u33V/+8v1zRWlFAEAAGR2y9gDAAAA6BsFDwAASI+CBwAApEfBAwAA0qPgAQAA6VHwAACA9Ch4AABAeqMWPBHxroi4HBFPjTmOvmTOLyJevea/NyPir8ceV1ci4pci4smIOBcRr0TEqYh4cOxxdS0iNkbEVyPiQkScj4jHI2LN2OPqSvbrmD2/KyLi3RHxXERciogXI+IjY4+pS9nXU2k2ruHYT3iekPTdkcfQp7T5lVLuuPKfpHsk/Y+kfxh5WF1aI+klSVslrZX0GUlPR8TGMQfVgy9I+rGkt0ua0yTfR8ccUMeyX8fs+Wm5AD8u6VlJd0n6hKSnIuLeUQfWoezr6axcw9EKnoh4SNJFSd8cawx9yp7fNf5Qkx/Nfxl7IF0ppbxWSjlQSjlbSvnfUsqzkpYk/fbYY+vYJklPl1Iul1LOS/qapPtHHlNnsl/H7Pktu0/SOyQdKqW8WUp5TtK3JT087rB6k2491Yxcw1EKnoi4U9Jjkj45xuf3LXt+FfOSvlwSn1MSEfdIulfS98ceS8eOSHooIm6LiHdKelCToielxNdRUtr8wsTeM/RABpJxPZ2JazjWE57PSnqylPLSSJ/ft+z5vSUifkOTx+mLY4+lLxFxq6S/k7RYSnlh7PF07HlNnuj8RNIPJX1P0rExB9SX5Ncxc34vaPLE49MRcWtEfFCTNee2cYfVvcTr6Uxcw8ELnoiYk/SApENDf/YQsudX8XFJ3yqlLI09kD5ExC2SviLpp5L+dOThdGo5t69LekbS7ZLulrRe0ufHHFcfMl9HKXd+pZSfSdop6cOSzmvy5PxpTQr0bFKup7NyDcfoxtgmaaOk/4wISbpD0tsi4rdKKe8bYTxd26bc+V3r45L+cuxB9CEmF/BJTTYR/v7yTZvJXZJ+XdLjpZQ3JL0REQuSPifpz0YdWYeyX8fs+UlSKeVfNXkiIEmKiO8o31MQKfF6OgvXMIb+a8KIuE3SnVeFPqVJgfBIKeXlQQfTg+z5XS0ifkfSNyT9WinllbHH07WI+KImnUsPlFJeHXk4vYiI/5D0JUl/pUlxviDp9VLKx0YdWIeyX8fs+UlSRLxX0g80+VuJRyX9iaT7lgv1FFbBejr6NRz8r7RKKa+XUs5f+U/Sq5IuZykGsud3jXlJzyS9OTdI+mNNfkjOX/XvY6QpBJZ9VNKHJL0s6UVJP5e0b9QRdSj7dcye31UelvQjTfaBvF/SBzIVO8vSrqfLRr+Ggz/hAQAAGNrY//AgAABA7yh4AABAehQ8AAAgPQoeAACQHgUPAABIj4IHAACkR8EDAADSo+ABAADpUfAAAID0KHgAAEB6FDwAACA9Ch4AAJAeBQ8AAEiPggcAAKRHwQMAANKj4AEAAOlR8AAAgPQoeAAAQHoUPAAAID0KHgAAkB4FDwAASI+CBwAApLfmOn9euviQs2fPVuPbtm1rev0U4gZe00mOFy9erMZdjjt37qzGDxw40PrRned4+vTpavzw4cPV+OLiYsvb68SJE9W4+650/Ryr+bk83HfvrqHLe9euXdcZ1g3r/Bq6sblrtWHDhmr8yJEj1fiOHTtahiPdRI7uuuzdu7cad9f92LFj1fjGjRuvM6wb1vl1XLduXTXuxnzy5Mmm95nCVPei03ptW/Ob4vsYbJ66e3Ht2rXVuJvXU8zfzuep+57dOjTAWlvNkSc8AAAgPQoeAACQHgUPAABIj4IHAACkd71Ny51wm5DdxqVfRG6zscvdbZydBXNzc9X40aNHq3G30cy9j9t8t8Km5am4jXRu3rmNjG7TYYeblqfm5lfrRnKXyxSbkzvXuunTcTm6eTIL3Jx0c3ioe6srbu1w89qts27jboebtafmctm8eXM1fubMmab36XDT/XUdP368Gu/q92z37t3VeFdrLU94AABAehQ8AAAgPQoeAACQHgUPAABIj4IHAACkN0iXltuJ73Z2u53gS0tL1bjr4uiD2ynv/gn+PXv2VOOt3Rez0G3guDG7uOve6lpr54B7vetwcR0xQ+Un+Q4NN+/cPD148GA1vmXLlmp8yO4tN/db7y2X4yxzc8zFXRfTrHaiuc5PF5/VbjOp/XgLJ6J+8oN7n1n+TrrS1VrLEx4AAJAeBQ8AAEiPggcAAKRHwQMAANKj4AEAAOl12qXlulzc7nK389q9z5BnF7luLLcj3p2L4rom1q9fX40vLCxU40Pm3tpV4HJ0Ll261PT6abkOJtfV13q22yyfBdfV+TobNmzo5H1uhuvydFyHjzMLZxQ5rvPHjXlWuc7bc+fOVeNuTWntYp2F7jQ3f1vHduzYsabXt67LN8J1Z7oOancvulzcOWJuntClBQAAcA0KHgAAkB4FDwAASI+CBwAApEfBAwAA0puqS8t1DLmd+Pv376/G3Q5ut1N7yPOkXCdPa1dBa4fJkOcwuY4H1yXX2l3lzjpau3Zt0/sMxc1H9z3NQheP4+ad67py95ab160dI31w18WNzc27Wb6OjstxVnNxXaytZ7K5Lll37w55/pS7V7o6w839Lg55lqTj5l1rp5jL8cKFC40jquMJDwAASI+CBwAApEfBAwAA0qPgAQAA6VHwAACA9KKUstKfV//Q7bxuPRvEdVm4c2JcfIXurbiBMVRzdGdddcV1LbjzxVZw3RxlrqPjvmfXDeBeP0UuzvVybMrPdRPOz89X425+uc4Q12l3M/NUJkeXi+u0c90Oruunq3PH1MM8devN9u3bq3F3z3XY8dR5ju57duuT64h1ubj3WaF7qtN7sSsR9WGdOHGiGl+hS2vqa+jWQfd7ubi4WI277tbWjt8VdD5PW7Xeu25er9AFVs2RJzwAACA9Ch4AAJAeBQ8AAEiPggcAAKRHwQMAANKb6iyt1vMxHHcm1xRdLlNz73no0KFq3HV0uG4KdzbIkGdmtXLfiesImoVzlWpcJ4DrxnJnhbm46yhYWFioxt18vxlLS0vVuDszq+8OyyHPu2v9Pt29uGnTpmq8w07KqbWe+dR6bpObJ61nXA3FdUbOwjlp7rNcp6Pr0nIdlrOs9Vy71o6zrq4jT3gAAEB6FDwAACA9Ch4AAJAeBQ8AAEiPggcAAKQ3VZdWK7eD23U7uJ34Q3I7613ccV0rs9yl5c6E2bp1azU+qx0drVxXjruGLj5kl4XrVHJdV0eOHKnGXZeL66YYshvLcd+zy911erRe3yG1rhPu9a7bq7ULbChuXG5t6vA8tM65e2gWOsu64n7LXe7uOrpzxLrqcOUJDwAASI+CBwAApEfBAwAA0qPgAQAA6VHwAACA9KKUMvYYAAAAesUTHgAAkB4FDwAASI+CBwAApEfBAwAA0qPgAQAA6VHwAACA9Ch4AABAehQ8AAAgPQoeAACQHgUPAABIj4IHAACkR8EDAADSo+ABAADpUfAAAID0KHgAAEB6FDwAACA9Ch4AAJAeBQ8AAEiPggcAAKRHwQMAANKj4AEAAOlR8AAAgPQoeAAAQHoUPAAAID0KHgAAkB4FDwAASI+CBwAApEfBAwAA0qPgAQAA6VHwAACA9Ch4AABAehQ8AAAgPQoeAACQHgUPAABIj4IHAACkR8EDAADSo+ABAADpUfAAAID0KHgAAEB6oxQ8EfHuiHguIi5FxIsR8ZExxtG3iHhXRFyOiKfGHkvXImJjRHw1Ii5ExPmIeDwi1ow9ri6thhyvSD5XU683EXFXRPxjRLwWEeci4o/GHlPXVsO9uArm6ej5DV7wLE/S45KelXSXpE9Ieioi7h16LAN4QtJ3xx5ET74g6ceS3i5pTtJWSY+OOaAerIYcr0g5V1fJevOEpJ9KukfSxyT9TUTcP+6QOpf6Xsw+T2clvzGe8Nwn6R2SDpVS3iylPCfp25IeHmEsvYmIhyRdlPTNkYfSl02Sni6lXC6lnJf0NUnZFtnVkGP2uZp6vYmI2yX9gaTPlFJeLaV8S9I/KUl+V8l+L6aep5qR/MYoeMLE3jP0QPoSEXdKekzSJ8ceS4+OSHooIm6LiHdKelCTRSiT9Dmugrmafb25V9KbpZQfXBU7o1zFgJT/Xsw+T2civzEKnhc0eTT56Yi4NSI+qMnjydtGGEtfPivpyVLKS2MPpEfPa7Ko/kTSDyV9T9KxMQfUg9WQY/a5mn29uUPSpWtilyT96ghj6VP2ezH7PJ2J/AYveEopP5O0U9KHJZ3X5P8sn9ZkEv/Ci4g5SQ9IOjTyUHoTEbdI+rqkZyTdLuluSeslfX7McXVpleQ4p+RzNft6I+lVSXdeE7tT0isjjKUXq+FezD5PZyW/KKUM+Xn1QUR8R9JiKeVvxx7LzYqIvZL+Qv+/4Nwh6W2S/q2U8r6xxtWliLhb0suS1pVSLi3Hdkr6XCklxSPYVZLjXiWfqzXJ1pvbJV2QdH8p5d+XY1+W9F+llD8fdXAdWQ33Yk2meVozRn5jtaW/NyJ+efnvYz+lyc77o2OMpQdfkvSbmnQSzEn6oqR/lvR74w2pW6WU/5a0JOmRiFgTEeskzWuydyCF1ZCjVsFclXKvN6WU1zR58vFYRNweEb8raYekr4w7su6sknsx9TyVZiO/sf7hwYcl/UiTv9N7v6QPlFLeGGksnSqlvF5KOX/lP00eOV8upbw89tg69lFJH9Lk/7xelPRzSftGHVH3Uue4iuZq2vVm2aOSfkWT/P5e0iOllO+PO6TOpb4Xl2Wfp6PnNxN/pQUAANAnjpYAAADpUfAAAID0KHgAAEB6FDwAACA9Ch4AAJAeBQ8AAEiPggcAAKRHwQMAANKj4AEAAOlR8AAAgPQoeAAAQHoUPAAAID0KHgAAkB4FDwAASI+CBwAApEfBAwAA0qPgAQAA6VHwAACA9Ch4AABAehQ8AAAgPQoeAACQHgUPAABIb811/ry0vNmuXbuq8cXFxWp87dq11fjZs2er8XXr1rUMR5LiBl7TlKNz8eLFanznzp3VuPuuXHwFned44MCBavzgwYPV+KlTp6rxubm5atx9Vytc3+vl2JTf8ePHq/FDhw5V49u2bavG3ffUQ35SR/P09OnT1bjL8dixY02vX0HnOZ48ebIaP3r0aDXuct+7d281Pgv3ostl9+7d1fjmzZur8Y0bNza9/1D3ortX3Hfvrvks/2a4sbn1w62bbp5OofMcW6+Xe73LsavryBMeAACQHgUPAABIj4IHAACkR8EDAADSi1JW3JvUtHEpor4XauvWrdX4vn37qvEdO3a0fOyKQ7qB13SyGdRtTnabttwGSre5cAWd5+iu44YNG6rxI0eOVON79uypxl2O7rtSxxsl3bVym5ndxt39+/dX41Nsgh1sQ+/27durcXePrnBNWk2do9vYun79+mrczVM3755//vlqvHUzvga8F13Dhxuby/HChQvV+FCblt14z5w5U427e87dWy6PIRsI3Lw7d+5cy9t02eQzdY7uszZt2lSNu3vRbdh2azOblgEAAG4QBQ8AAEiPggcAAKRHwQMAANKj4AEAAOld72iJKrdT23E78d2u81nmdpe7Dp+FhYVq3O06d9/tFN1bU3OdEO5oCbez3nVpHT58eJphdcZ9vutIcvm5zib3+iG5eeruudbvxM3fFTqYpuY6Gp3Wf7J/y5Yt1bjrDutDVzm6DkGXu+tAnOJYjam4bqzWjiTXJeTy67AT+C3uXnHdWK1H2bh56o4H6fAoire43yF3vdw91Po719W9yBMeAACQHgUPAABIj4IHAACkR8EDAADSo+ABAADpDdKl5c5acnF3/obbBd9HB5P7LNep5DqSXMeOG7Prjhiys8mN2eXeel7K2Nx37/JeXFysxlvP7+mDmxfu7CTXteK476S1G+hmuK4V1yW3e/fuTj730qVLnbzPjWjtQmntwHHXpXUt71prR6i7F13HUx/dWE7rfGntzHP6uOdauevS+lsyPz/f1ZCqeMIDAADSo+ABAADpUfAAAID0KHgAAEB6FDwAACC9qbq03K5wt8PadbO491m/fn01PuQ5U64Lwp0Z4nJx7+N29M/CjvvWDhB3VozrPBmyi6nGjau1g2nIs5Ycd46O47pi3L3l5ulQZy2txHVSurizffv2avzUqVPV+JDnMPX9/mN3UnZ1D83CuunmhVtXXGdv33OhD65LrvW8zL67BnnCAwAA0qPgAQAA6VHwAACA9Ch4AABAehQ8AAAgvam6tFyXTWvHSOuObHf2iDtr52a4HF3XSlfn97izR4bkzmFyXQjHjx+vxofsqmvhzp9qPQvHdVO0drndjNaOtzNnzjS9fvPmzdX42NdwJa3rgesk2bJlSwejuTGtn+XmsLvnXCel64gdSut5ii4Pt27OQielWzdbu/0iohof8l50a577zVhYWKjG3Zhdx6SrLVq7RXnCAwAA0qPgAQAA6VHwAACA9Ch4AABAehQ8AAAgvam6tBzXOeB2yrd2dfXRjeW480/cmN1ZLm4XueuuGfKcKdf15rhuLKfvc1Gm1Zq3MwudSq3nSblONNflMgtdg46bj0tLS03v476T1nOAbsbWrVubxrBv375eP3coLj/3m+Fe39phOSR3L3a1Dg2p9Zwv9/rWtbOr3xKe8AAAgPQoeAAAQHoUPAAAID0KHgAAkB4FDwAASK/TLq3WM7Zc15XrbHKdUH1wuXR1TtKQHWeO+z7n5+er8cXFxabXz2qHT1fn7rSe4zIkN79ac5yFeeq4c7727NlTjbvc3euHzN2tN67z1Z3d586gOnDgQNPnDsV1Krm8Z+H8ulaug8x1ArtzqVxH3ZDdoq3fc2sXqcuxq+vLEx4AAJAeBQ8AAEiPggcAAKRHwQMAANKj4AEAAOlFKWXsMQAAAPSKJzwAACA9Ch4AAJAeBQ8AAEiPggcAAKRHwQMAANKj4AEAAOn9H9SKvraxKm5JAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x288 with 20 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.pipeline import Pipeline\n",
    "\n",
    "km = KMeans(n_clusters = 10, random_state = seed)\n",
    "pca = PCA(n90)\n",
    "\n",
    "clf = Pipeline(steps = [('PCA', pca), ('k-Means', km)])\n",
    "\n",
    "ydpca = clf.fit_predict(X)\n",
    "\n",
    "ttpca = ytrans(ydpca, y)\n",
    "ydpcat = ttpca[ydpca]\n",
    "digiplot(X[-ndigi:], ydpcat[-ndigi:])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Die Ergebnisse sind nur unwesentlich schlechter als im Fall ohne PCA."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-24T09:28:24.807344Z",
     "iopub.status.busy": "2021-09-24T09:28:24.807021Z",
     "iopub.status.idle": "2021-09-24T09:28:24.813956Z",
     "shell.execute_reply": "2021-09-24T09:28:24.813670Z"
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.99      0.99      0.99       178\n",
      "           1       0.61      0.30      0.40       182\n",
      "           2       0.84      0.83      0.84       177\n",
      "           3       0.87      0.85      0.86       183\n",
      "           4       0.98      0.91      0.94       181\n",
      "           5       0.92      0.75      0.82       182\n",
      "           6       0.97      0.98      0.98       181\n",
      "           7       0.82      0.98      0.89       179\n",
      "           8       0.45      0.58      0.51       174\n",
      "           9       0.57      0.77      0.66       180\n",
      "\n",
      "    accuracy                           0.79      1797\n",
      "   macro avg       0.80      0.79      0.79      1797\n",
      "weighted avg       0.80      0.79      0.79      1797\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(metrics.classification_report(y, ydpcat))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Die Daten mit denen wir hier gearbeitet haben hatten dabei die\n",
    "reduzierte Dimension"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-24T09:28:24.816831Z",
     "iopub.status.busy": "2021-09-24T09:28:24.816366Z",
     "iopub.status.idle": "2021-09-24T09:28:24.822723Z",
     "shell.execute_reply": "2021-09-24T09:28:24.822334Z"
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1797, 20)"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pca.fit_transform(X).shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Zusammenfassung"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- mit SVD können lineare Modelle bzw. hochdimensionale Daten vereinfacht werden\n",
    "\n",
    "- die Berechnung von Singulärwerten und -vektoren kann mit Methoden der linearen Algebra\n",
    "oder der konvexen, restringierten Optimierung erfolgen"
   ]
  }
 ],
 "metadata": {
  "hide_input": false,
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  },
  "nbTranslate": {
   "displayLangs": [
    "fr",
    "en"
   ],
   "hotkey": "alt-t",
   "langInMainMenu": false,
   "sourceLang": "en",
   "targetLang": "fr",
   "useGoogleTranslate": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}