Ensemble_learning_LFW.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Ensemble-learning-LFW.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "mount_file_id": "1ocaKZy1c6ew5DBmyh3Chyn4dqVFhIR4L",
      "authorship_tag": "ABX9TyMlgCO8/gSDK1YiVum0xJWM",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/maede43/ensemble-learning-lfw/blob/main/Ensemble_learning_LFW.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# <font color='orange'><center>**Final Project Of Computational Intelligence**</center></font>\n",
        "### <font color='lightgreen'><center>Maedeh Nadehi</center></font>"
      ],
      "metadata": {
        "id": "WQHhEK1_qmbK"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## <font color='orange'>Ensemble learning</font>\n",
        "***Ensemble learning*** is a general meta approach to machine learning that seeks better predictive performance by combining the predictions from multiple models.\n",
        "<br>\n",
        "The three main classes of ensemble learning methods are :\n",
        "\n",
        ">  *bagging* <br>\n",
        ">  *stacking* <br>\n",
        ">  *boosting* <br>"
      ],
      "metadata": {
        "id": "2-24AmUgbgiZ"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## <font color='orange'>Bagging Ensemble Learning</font>\n",
        "Bootstrap aggregation, or bagging for short, is an ensemble learning method that seeks a diverse group of ensemble members by varying the training data.\n",
        "<br>\n",
        "This typically involves using a single machine learning algorithm, almost always an unpruned decision tree, and training each model on a different sample of the same training dataset. The predictions made by the ensemble members are then combined using simple statistics, such as ***voting*** or ***averaging***.\n",
        "<br>\n",
        "\n",
        "![Bagging-Ensemble.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAOMAAAEVCAYAAAASMlL8AAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAC7GSURBVHhe7Z0HWBRHG8f/ihVRROwReywoqICAYMUA9gKoscReosaKDStWBAvYu7FE40dswV5iL4CCgoqIBbGLXeztvplhj2AHvePmyPt7nntu591yOzf73/ed3SkZVAwQBKFzMirfBEHoGBIjQUgCiZEgJIHESBCSQGIkCEn45NPUU6ej8PrtW7Gc2cAApUoWh6GhoUh/L9du3MTLFy/YMUsoltQTHnFKWUqEn6NFRXMlpT1u3orHo0cPUa5sGcUC8L/vUuxlZM2WDUUKF1KsBJF6PinGkqVK4dbNm8iaNStevnyJ3Llz42hwMIoVLaps8W3wnxo8eDCOh4Vj3949ijX1ZMyYEdnYxc/Pj5MrlzHi4i6LZW0yduxY/P13EMLDwxRLIrXrOMHG2gpTp05VLASRej4bpg4dOhQPHjwQn2fPnmHfvn3CHhp6TFx0CxYswMOHD4Xtzt27mDlrFoI2bcH+/QcQEREh7OHhJzBtuj+ioqKwfMUK5lUeoWXLVhg2zEus57YLFy6KfdetXy9sHP4b/v4BiDx1SmzDf/9D5s2fn3R+aiFu2rwFUWfP4vdly8SH30g4hw8fEee8fPnypGPxc58zZw4WLV6SlA++P//NefMXiPNR52vHzl1ivRqeT27n6z8kJuY8O3d/sf9bJbogiBTBPeOHlChZUuXg4KDy9PRUtWv3i8qsaFHVjZu3VefPX1Axj6QaMmSIqkqVKip3dw/VmzdvVOXKm6tq1qyp6tCxkyqXsbHK29tbxQSpYqGbqnuPnqo6Tk6qDBkyqMJORopj1qpdR/wOt1W1s1f17dtXLAdt2qwKCQlVZc6cWexXq1YtYb98+bLYXg23OTo6qjp16iQ+M2bOEvYqVaxUxYoXF7/Bz2P8hAmqQ4cOJy3zPPXrP0CccwULS5VrvXqqBg0aqqysbVTv3r0T+5cqXVrsz3/DwrKSqmv3HmL5bHS0yBc/N57PatWqiXzzY/H88H1YuCp+i597xYoW4rcIIqV89QFO5iyZmed4hAMH9qF06VIIYV6rKAtX2UWJS5cuISLyFM5Fn8XS35dh2e9LUbBgYr1p7dq1KFOmLBbMn4uFCxcJ26fwHDgAM2bMYMcshosXzmPDhvWoYm0j9ps1a7ay1cfkz58fxYoVEx++rKZVy5bCC9Z2+gmXLl4UtoTHj5nH3o/mzZszjz8Mx8NPIur0Kbi6uqJ+gwY4wcLO06fPiG3btG4t9i/8ww/o0qUzFs6fJ+znos+J7xIlS4l8rvxjlcg3z3/mLFnEurV/BSJ7dkM4ONiz0LUOVq5cIewEkRI+K0YXFxdxUS5dsgTVa9TE4iVLsXXbNtSoUR0ZMmRERQsLGOb4+KHO27dvlCW2/C4xTHv+iTBTTbas2cR3RoN/T0W9/avXr8T3p2jarBnGjBkjPj+3aqlY8dGDJkdHBxZm7oRFxYrihuHoUA2vleM+e/oUr16+EPk0zm0sbHny5BHfWZjAeF2ZeUWR/pBMmQyUpX959eoV2x54cP8+ShQvhlEjRyprCOLrfFaMa/4XiKZuHmjQqAl279yOBvVccePGDRhkNIBRzpwIDQnF40ePUcnSAuYVLdC2bRs0bdoMN65fF/t7eHgwT3cB7uy7a7duwpYSWrRoyepd59C6TRv06tVbsX7MzJmzxfmpP8+fP1fWvM//Av8S51KmTBlYVamC7EysVa2tUMTMDCzsxsVLsaL+l9fUVNnjy/D6ad/+A9Gvbz+UZF6ygnl5vGYi5DRu3Bj3WD3yCRP5tm3bWTRxUNgJIiUYsHqQt7KcRA4jI1haVESJYkXFK4hu3XugY4cOqFihAvMYmXGTibLfgIEwMysCaysrVKpsBVOT3MxbNRUPbSpXroSatWoxL+QI41y50Lp1G6xcsQL9+/dHkSI/oLy5OSpUMIexsTHzujWYNzIRv2lvX41d4CVgU7UqcjHBt2nTFitWLBf7cS+lhu/HbwL8/NQfW1tbsU1Vtm+RIkWQM0cOWFpashtEE7C6HSJOnkChQoUwddpUmDLv5+buLsLY7NmyYeqUqSy8Lggjo5ywsbER2/Hz4cfkIbD6PAsze8OGjZj3zSb+h1mzZ4ljGbLfsmYC59vX/eknnIqMRNmyZTBx4gTx1JcgUsJ399rgTyxLlCghBFC5chVRT+JPVPlhq1WzRwtWh+NPK8+ePYtYVsc0MPg4vEsO37dOndrM07bDdeZlr1y5gnPnor+637fCz/NzoeiX+Nb9COJzaKQLFX8Zvn37VhEqVq9enXlVC2HnXjI4JBjZs2dHo0aNkC9vXmH/Gt+6H0HoM9SfkSAk4auvNgiCSBtIjAQhCSRGgpAEEiNBSAKJkSAkgcRIEJJArzbSAP7e9MCB/UpKP+BNHrt26aKkiLSAxJgGBAQEiCZ9+oQ+nrO+Q2EqQUgCiZEgJIHESBCSQGIkCEkgMRKEJJAYCUISSIwEIQkkRoKQBBIjQUgCiZEgJIHESBCSQGLUMd17/Iq4uDgl9e2sWv3nJ8eO5YNPHzx0CE2aNntvbhC/qdMxxnssvLy8xMxghO4hMeqYnTt3iGEu1cTHx380YQ6f3OdL7fn59r6+k8VoesnhvUVOnz6NGtWrw82tOUaNGiXsfGKfnTu2YbjXMFSxtsbfG/6ddIjQHSRGHcJHM+dC5GPC1qpdB0OHeQnBNGjQUKzv3r0Hmru3gOegQaj7k7MQHZ81i4+AzglcuxaTJk1C33798fr1G6xPNpMXZ+TIERgxYrhY/qVdO5w8eRLjxo/HiuXLsPavv8SUesWKlUBkZKTYhtAtJEYdEhYeBhvmmbjIzsfEYNAgTzHV3q3bt8T6ExER8PWZJOY7yWNigvMXLuJYaChsbe3E+sMHD6KylTWqVKmMrl27MO/nJuyc6HPnYJw7d9KYs1zwbm7umDZtGoI2bUoaoZ1P2vP69WuxTOgWEqMOOcLqcg4shIyKOoufnJ2FcLi3LF68hJhH0pCFnWXK/Ci2fcrS2bNnw/GwMFSpXEnYgkOPwd62qpj3xM6+mrCpOXr0KKpV+9e2Z88eVq9cLWbeCj4arFiBq3GXkZcGiZYCEqMO4cKyZp4tOCQENjbWwsa9pR0TWOSp03j8+LGoK3KP+PDRYzFzNE/z8PLw4cO4d/eOmDXrTFQUrBSBqnn08CFMlcl8+PF79f4Nm4L+ZnVLP4wcNTJpItlDBw/Aqa6zWCZ0C4lRh/Dpz/nEQnzmKkdHR2Hjkwo5OFZHSEgwKleqBDd3D/TrPwArVywX60uWKI4WrVrjCPN8bm4ewlagQH5MmOQjltXwhzncu96+HY8RI0dh8+bNKGpmJqaqa92mLbZs3SrWH2Fe0vknJ2UvQqfwYTcI7eLv768spQw+i/Iv7TuoIiNPKZbUw7yuysvLS0l9mvETJqr27N2rpN4ntedMfD/kGSWFeztz8/JKKvXwuSiRKauS+pi4K1dhYpoPdWrXViyEriExSgifas7P1/e7psHjx5jgnfhe8VMUK2qGXj26KilCBkiMkqKJuR8zZvxy8dL8knJBYiQISSAxEoQkkBgJQhJIjAQhCSRGgpAEEiNBSAKJkSAkgWahSgP8/QOgjVd6T58+Fd+84XimTJnEsqYIP3ECK5Yntocl0gYSox7BG31v3bZV9OK4fu0anj9/JhqYb9gYBDs7W+Q2zgUXFxfRFI5e6OsfJEaJ4UXDPRTvi3j33n38UKgQXFxdRC+Pbdt3YKz3GNFkbuvWbTh15gx6dOuKffsPICw8HJmY3crKCnWd6sDQ0FA5IiEzJEbJ4N2a/tmzF6HHjom0hUVFuPz0U1LP/MDAQNyKv4O+v/UWaTV8+/XrN2LihHFCoHz0gKPBwULIr9+8RTGzImjcuIlogE7ICYlRAngPii2bN+Pm7dvInjUL6tatCxsbm48ais+cPQcF8+dDy5YtFcv7xJy/gIWLFmEc85gfekP1b8RdvUrhrKSQGHXAh16rcKGCYhAq3pPiU/Dt+bCK9eu5JnVC/hw3b8Vj3Phx8Jk4IcmbfsjDhw8pnJUQEmMawYdb3L5jpxg60SBjRlS1tU2RALhwRo0Zg759+uLH0qUU65fh+3DxenoOFL37vwSFs/JAYtQifIS2nUyAt+8+gEkuQ9RxcoI180Ip5QoLKadNmy6GXFSP8pZS+Bg3XiNGolPHDrCoWFGxfh0KZ3UHiVGDcAEcOHQEhw8ewNt371C6VEnmZRqLQaNSy8mICKxatRoTWMjJ3yN+C9zrjZ84Cc51nb4a3n4KCmfTFhLjd5L83V+2LJlRzbEGalZ3+GYBccSrChbODhk8SCMeadaceShcsADc3f8dVzW1UDirfUiMqYT/XZ9691eubFlli+9j6bIVyJhBhY4dOigWzbDyjz/ENx9ZXBNQOKt5SIwp4Gvv/jQBLwbvsePh6GAvLmxtoG4cMJR5XE1C4axmIDF+hpS++9MEXOxew0egc+dOqGRpqVi1w4eNAzQNhbPfDolR4cOL6Gvv/jQFn6Zt/PgJ8PIajkIF0+aC/VLjAE1D4WzK+U+L8Vvf/WmKs9HRmDtvHnx9fNI8pEtJ4wBNQ+Hsl/nPifF73/1piv0HDmD79u2YMH68VsLFlJCaxgGahsLZj0n3YtTkuz9NseZ/gbh77x5+69VTseiOb20coGkonE2nYtTGuz9NMX3GTBQrUuS73vlpmu9tHKBp/qvhbLoQI8+CNt/9aQLugfhsUHw6b4dk8ybKhCYaB2ia/1I4q7diTIt3f5qC3+lHe49D3z6/iTBZZjTdOEDTpOdwVq/EmJbv/jQFb+zt5zdFtDGV8UbxKbTVOEDTpLdwVmox6urdn6YICwvDX2vXieExZKivpgZtNw7QNOkhnJVOjLp+96cJdu7ciefPnyM65jyGDPLU2xBK3TjA3s4ezZs10QtRqtHHcFZnYuTh25W4OFSvXl2ad3+agD+oKVGiBOrUdcaqlfo/1CEX42+9e2P+ggXo3KmTYtUvPhfO8pvL1m3b2Y2mqbKlbtGJGLdt24a2bduhdNmycHV2luLdn6aY7u+PRYsWo2HDBvCdPFmvvMmn4A/KeJ7Wr9+AY6Ehep+f5OHsxdjLWBcYCA8PDyxYMF/nVYnPirF9hw7CrWuaN2/eJA2+yzE2NlaWvo8zZ6KYCBYqqS/Dm6C9Yh5MG/B/UxuRkFHOnOjapYuS+jLaKDv1VaKNvOmq7F68eCEiGU4WJsTs2bKJZU2T4rLjYvwU/v7+ypJ+kJrz1be8cdJz/qjsEqG5NghCEkiMBCEJJEaCkAQSI0FIAomRICSBxEgQkkBiJAhJIDEShCSQGAlCEkiMBCEJ3yRG3p4v5FgYTp06LYa80BT3799Paiv4KfhvxcfHKyntwbvfHDpyVHTn0hRfO3feIDv6XIyS0h78/w0/cVJ8eKNpTcF7RvA8fI64K1dE+RKfJ9Vi5IPu1qhZC//s2oHFSxajWfPmyprvp3v3Hu81Iv+QoKAgDBo8WElph1mzZ2PggP4IPnIYzZu7ib6JmuDipVgM8/JSUu+zd98+uLm5Y/WqP9Cq1c+KVfNwsdR1dsWmoL/x5+pVcHV11ZgghwwZiosXLymp9+nbrz/mzZ2Ldu3aYd/+/YqV+JBUizE09Biq2dth+PDhmBEQAAcHB3HH4/0TJ0yciMm+vkKwnP8FBmL9xr8xdepUXL9+Q6zbyNKcoE1bsGPHDrGPevt49s27UUVGRorjr/5zjbBzWrdpgynT/GFubq5YtEPQps2YOXMWBg0ahOUrViAuLk7YN/wdhBEjRohvzukzZ0QfOX7+vDvOrl27MHr0aDE4MM8Pz1vy/B49egS2trbi4l+yZInYlv9nnLNRZ7F46VKMGzcO95mHSUhIEHZNc+rMWZQsURxjxozBlClT0LhJU9y6dVuc86RJk0Re+DKHlx3PA7fz/4CX4YoVK8U6bt+6bZs4X/X2UWfPsrIpj5iY8yJvy5evENEAz2+lSpaYPHkyho8che1sP+LTpFqMP9V1QmzsZbT6+WeEh5/A0CFDkCtXLnH34/3CzMuXhzcrbC5QfkHnNDJC7OXL8BruBQ93DzGKNYdPd8Yj3ArmFTB8xEgh1vx58wohjhw5Ev3790dERAS2bNueeAEvXgznunVgbW0t9tcWgz0Hsou0iejDly+vKbp16ybO4eTJk8xzd8fihQtFOLl61SqsXLkS7ixPbdv9goePE5A/f36sZh6HezruYVuz/2j+gvmi8/TxY8dQuXIV/Nqzp+hS06VrN3Tq1Fn8Zq9ePVGkcCEcPnwYhtmzISdbrw1sbazw+vUrNG3ajP3WEfTr2wc//FAYAzw94VqvHmrWrIk+v/UWoSz//9+xAnry9BkGsvWNGjfB1GnTxHFGe48VNwxnFxf06NFNeFyjnLnYzeUaBgwcIPLGb0hcvLz/Y5fOnYU4/Sb7oEPHjuIYxMekWoy8A2YQC3P4tNaegzzZxbZQiMWpdm0RivABenkfxWDmQTuxP975p7qs4J4IT2dmVgRGRrmEUKtYVUG9eq5o0KA+LjOxBoeEwKaqDWbPnoPMmTPDz8+P2WNx/ly0KFA+7EZwSKjWRwDgwzMcPLAfvNteVTs7URcqVKAAniY8Fn3pzp6NQv58eRF+MhJTp/ghFxOOI4sOWri7sQv3KUqXLo3joaEYO3YsihUrBtuqVXHz5i2cYNvzsXs2b96MY2z9rJkzEHvp37Bu1erVmME88p/sW1vwISf48blnHD9hPCb7TRFlV9PRAX/88Qf7rBKRyYmTEWjerDnqM4E+fZKAwYOHoFzZMuKmy4Wa2zg3WrVsySIke1G24Wx7q8qWWMo8PtgNluctgt28zrGyU8N71z9+/BjZsmVXLMSHpFqMNlVtxbcjK0A+xfV+VgeYw0RokMkAASxs5WGQtU1VHAsJhr1D4oC4ly5dFGOY8kKuwgqNC7UA8yKc7Tt2wY6JMOz4MVSvURP3HzzEPHbR87Do59ZthDdSw+/G2hwNYCIL0wLXrkWOHDmEZ7C0sMQlFgX0YN6Mh6hjWPhVoFBhcbN5+PA+TExMxE3EnoXtnOPHjsOGee5jx8NhVqSIsIUyW/ly5USn3KfMy9SuXUfkbeiwYSyfc8U2flOnM88bIepx2hzrx7JSJRE6WrEb4YIFC/DPP/8IET5m/+tUFraWK1cWtna2oiyqOSSO7XoyIpKVWSVR5y1ZsqQoQ34z4oSwvJcrVx4hwUdFmcfH32YinyDy16ZtG+YlB4prhAuYD6VpzqIgTT4US2+kWoy1atVk3qyBuFj7sjBnwID+TIAlsYcVrDfzBpu2bEURs6LCi9nbVhVeMCcLYTi8kO2rOQihnmeVfX6BL168SISz3NNYsULv2LE9ujEB9v7tN1bIwUkXp/pi0CYtW/3M6lJT0bt3bxZOdRLjpVSytEAuFmrzG86o0WOYZzdCFKvjWVa0EPuIPCk3navXr6NgwQK4c+e2mFWqKwtxGzZqzNJ3UJnVm0qVKik8DY8S2v/SHvny5cOFCxcxacI43Lt7R4TE3HNqC/5wqEHDhuL/bt+hE4YNHQKzokVxYP8+TPKZjLVr17FzLI0QdrN0YHlSP9nm0VBYeBjsWHny/N64dYvlzwtjx08U9cPjx4/DnkURv/zyC0aw6siAAQOwgdWV85qaomChgvilfXvmXQdzpymELQs8KmjCIgCOj4+PeEPAqxS8fHQCu1N+ki/1Tr53754qNjZW9ebNG8WiUt2+fVvFhKd6/vy5sLOQRNj5Mrdx1Ovq1W8gjsHC06RjqLfn8HX8eMl5/fq16sWLF0rqYzTVW5yfDz+v5L/Pf5fb3r17J84z+bmw+lJSHp4+faqKjDyl6tatu4p5AJEPDt9WvT0/Bj+WOs2PxULhpI/a/iGayp/6f0/+O9zGP+rz5PngsItV5I+jXvdL+w6qCxcvqa5fv550DPX2HH4NfFh2/Nh8e573T6GpvKUWfj7qc+ffvBx5ntTXq6ZI6Tmn2jNyeKhYvHjx9wYn4g8veNiWLVs2YVc/hODL3MZRryteooQ4Bq9TqY+R/KEFX8ePl5xMmTKlyYBB/Hz4eSX/ff673MbrXPw8k59L9uzZk/LAvfit+DuoxyIHPmCxOqTm26q358fgx1Kn+bF42Kv+qO3aQv2/J/8dbuMf9Xmqo5GMGTOK/HHU60xM86FUyRIoXLhw0jGSh9b8Gviw7Pix+fayDZPIz0d97vyblyPPk/p6TWu+SYzfy7w5s5Wl9Ad/4usmydB/2mDG9CnKEqFpdCJGgiA+hsRIEJJAYiQISSAxEoQkkBgJQhJIjAQhCSRGgpAEEiNBSAKJkSAk4bNTwvH+iryLjKZ58+aNaAKmacJPnMCK5SmbnNTfP0D0otA02sobh09Z1qtnTyX1Zajsvg1t5S/FZSdaqKYhmmzoKxvpOW8cyp92oTCVICSBxEgQkkBiJAhJIDEShCSQGAlCEkiMBCEJJEaCkAQSI0FIAomRICQhTcXIBzzm8zTwYezTG3wMTj4osK+vr1hOb/D5RHjZHT16VLGkH/j4sHwAaz4tgy5JUzFeio0VF2y+fO8P5Zce4MP8RUefQ2TkqaShG9MTRkY5sY9drHfvpb9p3fjwjEeOHMWNGzcVi25IUzG6OLvAo0ULlCnzo2JJXzg7O2Po0CFKKn1ha1sVDo6OqO7ooFjSF15ew1CrZg0lpRs+22uDzzClDbd99+5d5M2bOFeDJnF1dUHFChWU1Nfhw/W/ePH5iVm/BW3ljU84061rFyX1dajsvg1t5C81ZfdZMU6b7o+6LbopKfnZu3axmPcjpQz29kXbLr2VlNz8uXQufMek3OPqW9n989cieA4coKS+jj6V3aolczDFe6iS+jL0NJUgJIHESBCSQGIkCEkgMRKEJJAYCUISSIwEIQkkRoKQBBIjQUgCiZEgJOG7xfjq1UvciD2HO1cvJH0e3IhV1n47j+/dVpZ0y73bN97LG/88T3igrE09/P/ix5CBT5Xd954b/7/4cYnU891iPHXsMPwnjsBIz15o28wVfmOHYcEMX2Xtt7E3aDXGD5WjudOi6eNEnjp4NIBXv+5i+cSxYGVt6ji8dyf6tG2EZQtmo5t7XSQ8vKes0Q2XzpwQZTd2aB+0alRX5I1/vpURfTpiQYCPyOPRA/8oViKlfLcYrR2dMGXxWgwe7YNKVaqI5WG+83H1UgxiY84geN92sR1f3rZ+tbCruR9/Q9gunDqmWIAlMybi73WBSkr38LzwPDk41kCfoWPEsk11J5GPPVvX49a1OOEJ+PKBnUHveYVjB3ZhZ1Bgkifdv+UvTJq9HIMnBMDBqT52bt4g7LqiXBV7kZ8xvrNQrnx5scw/PG+Xo0+KPHF4evNfKz9ZdqeOJ/ZN5R6xZKnSGO4zC2P8ZmP9innCTqSc7xbj51g+2xdzpoxH7MWL2LNlHbsDj0J2Q0NMHj0YZyPDEBMVgcE92iFLlixYtmgedv79P7FfE7dWGDFhqliWlUe3r6KjuwvOnzmJu3fi4dnZHffu3MHtmzcx4reOYpvJw/sg9Mh+vH3zBn06uAuRDp+6CKYFCov1EceDUaq0nF3JAlf9Dp9Rg3A17jKOH9wlyswoZ07hNXma34A8u7WGQaZM2LJ+Ddb+Plvkq0v/kaJc5wX4wblxC+VoRErRmhg5XXr1Q+suvbF+1WI4ObsKW2Ura2z4cxm2rluFKlXtRYHaV3PAulVLxPp8ZqXFt+yUZ56kx+BxyJY5I54/e4Z8BQqIz41rl3HraiwO7tmFCpVtxA3I2DQ/jh3crewJTPcehB/LlkVl+1qKRT7a/zoAHXoOxKbAFbB3qC5stnbVRHpb0DqUqVBJ3Eit7Wvgr9X/Tlrz8tkTmBUthv27tykWIqVoVYyZM2cR348fPWYe4hWeJCSgkFkJNGzmgWfPnuPtu3fChkxZ0bbrb2JbfSG7YQ7x/fTpE/BOaDwf/NO6Uy9hz5AxY5KtjktDFCtVVnjHgV1aIl/+Aug9bKLYTlaMjIzE96NHj1heMoh8mOQriCY/d2LLj/mMScLGh6zoMcBL1H+vXb4ACxtHdPccjejTEYnbESlGq2JUU93JFXfv3oNL05a4dO4MLpw/D/sadXAnPl7YDPAOR/f/6zn0iTLmlkhIeIRK1nZwqFUXa1cuQh4Wspma5oFZkR9Qt14jbN+wGip24/EfNwQF85vC0toWESH7Rb1LduxquSL+1i1RTg/u3MLpE6FwqF4TV6/EibzlMTHBro1r8PRJAob2bCdEGRV2GDly5GChbS7lKPLC6/ScyxeiWYgdKZZ5XVkXT4Q1JsY8+Quhdr2mSoqFNLVchI3Tsc9QGOYywcTBPWDCbO7tusKpgRuq1XSCd79OiIu9iL7DxoptOdlymrx3LBlwqFsfefIl5if5+WVny37zV2HlolmY7++DoRP8WfiWFf5LAnFw725Wd+yLVp16w6xkGRgZ54GBYW7s3rldfGIvXRLH0DVGuU1RK9n/bcNCT3VeeTWDRzO87Hh007r7APHQzqNtR5G38NAjGO43DwWLFIen91RMHdUfO7cGIWCpPA/hvkRMdJT4vnf3rqj3cy6e//dBVVpCPf31AOrp/z7U058gCK1CYiQISSAxEoQkkBgJQhJIjAQhCSRGgpAEEiNBSAKJkSAkgcRIEJLwxRY48QmvlZTmePnyhWguliFDBsWiGYrly4levXoqqa9Tv6kHLK1slZRm4N2lMhoYaDxvvM3khv/9oaS+jrbK7g3LX6ZMmZSU5iiazwi9eyU2sE8J2ig7fl1mMsgkehFpknNnIrAxcJWS+jKfFaO2uHDhIjZu3IB7Dx6iVInicHd3h4mJibJWv7lz9y4mTJiIsd5jkDt3bsWaPlizZo34/vnnn8W3vsMv+9DQUARt2iwmt61duw7q1K4p5mrUFWkuxuSkR2FyQXp5eWHqlCnpRpCBgYF49DghVdPSyYhagHwG5mfPX8De3h7167nqVIDJ0akYk5OehPnw4UOMHuONcWO99V6QQZs2ie5vnTsljmCgj4SFh2PLli1ISEgQAmzUsKE0AkyONGJMTnoQZnoQpD4LUS3Al69ew6pKZeYB68HQ0FBZKydSijE5+ixMLshBgwfDx8cH+bQw46824UK8dPkK+vfRj65KnMhTp7CZnTcPqS0tLNC8eTPpBZgc6cWYHH0UJhfkGO+xGDlyhN4Ics+ePYg4dRoD+vVVLPISfe4cNmzYwP7nR3opwOTolRiTo0/CfPbsGbyGj9ALQeqDENUCTHjyFOXLlUPjRg1hbGys8VdKaY3eijE5+iBMLshBQ4dh2JDBKGpmpljlggvxWFg4hg4epFjkQV3G8Xfuwrx8eTRr1jTdvT5KF2JMjszC5IIc6uWFwYMGSSfI4JAQ7N79D0aOGK5YdI8+Vku+h3QnxuTIWJh8aMNhXsPFeD2yCJILce++/fAamvJxdrRF3JWrWLduLe4/eISiRQqjWfPmyJ8vn7I2fZOuxZgcmYSpFmTv3r1RulRJxaobuBB37dqNUaw+qyuu3biJwP+tYSHofZQsboamTZuhQIH8ytr/Dv8ZMSZHBmFyQQ5hdcg+ffroTJAnIyKwavWf8Jvsk+YPP7gAgzZuRNy1myhWpBCaNGuGIoUTh4f8r/KfFGNydClM3iaSP2Xt3r17mguSv5MLDFyL8eO800yIt2/H4++/N+LS5asoXDA/3Dw8/vMCTM5/XozJ0YUw1YLs1KmjeEyfFqSlENUCvHLtBvKYGLP/1APFisr5NFnXkBg/Q1oKkwuy/0BP0RCbv7jWJmejozF33nwETJ8GAwMDxapZHjx4gHXr1uFi7GWYmuRGs2bNUbp0KWUt8TlIjCkgLYTJi2HUaG+0bOmhNUFeuHgJCxcuhM+kiRoXIm9ptHbtWsScv4D8+fKSAL8BEmMq0aYw1YL08HBD5UqVFKtm0IYQuQA3bd4ivG1Ooxxo3rw5ypUtq6wlUguJ8TvQhjB5cQwZ5gV3t+awt7NTrN8HF6J/gD9mBgR8txB5w4UNGzaKXhH5mAckAWoOEqOG0KQweZFMmDgJzs4/fbcgr1y9Cn//AEz2mfTNffjUAuQPfoxz5USjxo21Xrf9L0Ji1AKaEqaPrx/q1K71zYL8HiFyAW7bvh3hJ04ia5bMrA7YjASoZUiMWuZ7hckFaVfVBk5OToolZXAh+kz2FU9NUypE3hBh85YtOH48DNmzZ0PDhg1hbWWlrCW0DYkxDflWYfpOmYqq1lYpFqR6YCz+sOZrffu4ALdt34Hg4GAYMgG6urrC1tZW77sj6SMkRh2RWmH6z5iJShYVvyrIlAiRC3DvvgPYt28vMmfOJMaEIQHqHhKjBKRUmFyQZX/8EQ0a1Fcs78OFOHbsOPj5Tv5IiGoBHj5ymBc6XFxdUM3eXmsv/onUQ2KUjK8JM2DWHJQsXhQNGzRIEhIvwkePHmHoMC/4TvZJ6nTLW/bsYQI8cGC/GIDY1cUFNao7kgAlhcQoMZ8Tpn/ATCxfvhwbNqxHieLF0L5DR5yLicGmvzfC1NQUBw8dxj979ohj1KxZC061a5IA9QASo56QXJiRJ09i27at6NylCwYOHCieePIwtHEzN1iyeqWjg6POR8cmUg+JUYvMnTcPr5hINM0Yb28kPH4sxFajRg0cOnQIL168gEeLFnCoVk3ZSnMY5cyJrkz4hHYhMWqRwd6+aNtFf8Yd/Rxrfp+LyaN1PyRHeoemhCMISSAxEoQkkBgJQhJIjAQhCSRGgpAEEiNBSAKJkSAkgcRIEJJAYiQISSAxEoQkkBh1yJE92zCgQzN0dXPCbJ8RePXq29qx8v0W+X08ldusCdSETZ8gMeqIg7s2448l8zBuxhIsXr8H2Q0NMWFIL2Vt6uCN0U+Ehyupf6nfrJWyROgD1FBci3ypoXiPli4YPGoiSltUVSxA9MkQlKtshzWLZ+DQ3p2i03Cj5i1Rv0VHeP36M0yMjRFzPgZVbB0Rf/0KYmMvoV23PnBwqo/2jWqg1I8/ilHdmnn8DGe39hjQsRn8l20U++bIZYKb1+Jgmr8QxgUswbXLF+A3sj9ym5ggU1ZDDJs0E1myfLrLFTUUTxvIM+qIG9evoWhZSyWVCBfi1Usx2LNjM2av2sKEtAHLF81FwsN7uBwbi9Y9PDF3zXasW7OSiWcG/Ob/ga3rVol937x5jYlzV4l9lsybJULXW7duiXV837Zdf8Oc1Vtx+8ZV3Ll6ATPGD8GA0b4YN2slSpcuzY6zWmxL6A4So44wMcmD+/E3lVQip0IP4GJMFMpa2og091QVK1rg9rXLIm1WsoywFShYCNlzmsDIOA8T3Suxrmx5c7GOf8yKFcPd64n7qClRpoL4zpUrF16/eYsLFy5g3tSxGNK9JcKPBePlyxdiPaE7SIw6wrlJC/y5dK6SAiKOH8WsKeNRvHhxXLsQpViBmJhomOT7+hyGV69cUZbYclwc8v5QXEl9GlPTvOgzbAL8FgaidbcBKFuhsrKG0BUkRh3RquOvePjgPrq510X/9k0wfexgjJ4yD8XLVUbxH8tjUBd39GvfFM4NmsG0QGHkyJFD2TPRu6nhvfA5GTNmxJBuLdG7dX207fyr8JDq7ZLva2SUE5kzGaDP8IkY0acTxvTriHlTRqMo86aEbqEHOFokJT39ed2OPw01yvmvwDjq1xxcVCnlW/Z5kvD4o9/+EHqAkzaQZ9QxXDifEgO3p0ZUnG/Z52tCJNIOEiNBSAKJkSAkgcRIEJJAYiQISSAxEoQkkBgJQhJIjAQhCSRGgpAEEiNBSAI1h9Mi/v4B0PbM3IcPH4ajo6OS0g5ZsmZFr549lRShLUiMek5AQAD69++vpAh9hsJUgpAEEiNBSAKJkSAkgcRIEJJAYiQISSAxEoQkkBgJQhJIjAQhCSRGgpAEEiNBSAKJkSAkgcRIEJJAYiQISSAxEoQkkBgJQhJIjAQhCSRGgpAEEiNBSAKJkSAkgcSox0yaNAleXl4I/GutYiH0GRKjHmNlXRWFChWGu1tzxULoMzQ6nB7z8uVLbN68Be7uboqF0GdIjGnA3HnzxFTh+sSZM1FYtGihkiLSAhJjGqCPY5vSeKxpD9UZCUISSIwEIQkkRoKQBBIjQUgCiZEgJIHEKBlHjh5FHaefUNfZBba2dlixYqWy5mP4g/CdO3ci+lyMYkkZ69evR1xcnJIiZIHEKBFRUVH49ddfsWzZUvyzayf27t2DufPmIjz8BN6+fYvnz5+L7bgI+TJ/6T/MazjymuYRy/wTd+UKnj17JrZLvo96+cWLF/AeNx6mpqbCTsgDiVEipk6bjtGjR6NY0aIinSNHDnTo0BE7du7A0ZBQ9OuX+N7v9Okz6PFrT/z555+4e/cOgjZvgX9AAOo3aIh5c+fCvpoDHj58iB27dou2q5xjx8PE/kuW/o5nT55gy5Ytwk7IA4lRIk6fPo2aNWooqURev36FjBkyIPjIYdja2QpbcEgIrK2tUN68Auq51kPnjh1w/NhxTPbxweTJk1G/nisOHzmK46EhcKheXewTEhKMqra2KFPmRzRs2ACtWrUSdkIeSIySwEPPLFkyM/G9USyJbN60GbVr10ZwcAjsWB2Sc5TVK+3s7BHKxMYFxrnM6oC2tlXFckLCE+ZVjRDGwltrK2thCw4Ohr2dHcLCwmBTNXE7Qi5IjJKQgXk/dzc3jBs/TtTv+Mdv6nSY5jVlwrNDTEwMypcvh5jzF7B+3VpYWlTEyZMnmYe0xvXrN3D+fIyoK965e1d4zmr2tuIhTamSJXDixEns2rkT5ublhQetWjVRwIRckBglom/fvijK6ouOLLR0dnHF61cvsOz338W6liysbNrcDUFBQejcpSsMDQ1RvERJTJw0SYivRYuW6NCxM7p174nly5cha9asaNK0Kfs0ZyHrYbRndU8DAwOUMzeHz2RfcUxCLqiheBqg6UbXvMi4J1V/Dx8+HNVr1kIDVldMCer9vgQ1FE97yDPqIWohqb8LFioMe6W+mBK+JkRCN5AY0wF9fuuNPHnyKClCXyExpgPI06UPSIySwp+M8hf7vFXNhwSHhMLT01NJfZkZM2eKJ7O8MUHkqVOKlZAREqOkeA4ahN69euLdu3dCTBx1k7bDhw+hvLk5Ll6KFS1t1Dx48ACnTkclNYe7f/8+/li1Gm/evEGNmjWxKShI2Ak5ITFKSNTZs3iS8ASWlpYYNmyYaArHmcm83O/LliMiIhJbtm7HooULUM3BUXhP3vi7RYsW+CtwDWrXccKNm7fh5TWcFbAKR4ODYWZWFKfPRInjEHJCYpSQpUuWoHXbtmLZggnyfMw58TJ/Y9Bm9OjeDaEhIQjwnyaavhnlzCm2K1CgIBYvXoz27TugUMGCuHXrJn78sTR69eqF2rVqIZOBwUetewi5IDFKyPGw8KRXFeXKmyOaecoxY8ZghNdQPHr0CMYmJqIxOfeIb1kIysPXiRMnwt/fHxs2rMfx48dRrmwZ0YTOSmkOF3s5Fj8ULiiWCTkhMUrIYyY49asK83JlsWfPXly/dh0uLi4IDj2GakqD8RMnI2BjbY1w9m1mVgQzZswQfSALMM/IW+hER0eLJnCcf3bvRh2numKZkBMSo4TkMjZOeorKRXnjxg1MmTJFpJ8kJIiuUpw7d+6gXoMGqGpthSdPnqJ9x04ICw9HzVq1xXoLSwtMmTpNPNA5yrxk40aJ+xGSwpvDEdqFhY/KUsrw9PRURUaeUj148EC1Zs0aVe/f+ihrVKp3794pS5/mU+vHeI9V7dm7V0mljNSeM/H9kGeUkPbt2+P6jZvYvfsf7N1/AH6+k5U1X3/B/+H6+Ph4lCpVEnVqJ3pLQl5IjBLCX2nUc3WGh4c75s+dI+p/30r+/PnRTnkyS8gNifE/ADWX0w9IjAQhCSRGgpAE6lycBvAxa0JCQpSUfpAla1b06tlTSRFpAYmRICSBwlSCkAQSI0FIAomRICSBxEgQkkBiJAhJIDEShCSQGAlCEkiMBCEJJEaCkALg/6vYyoDEKumWAAAAAElFTkSuQmCC)\n",
        "<br>\n",
        "Many popular ensemble algorithms are based on this approach, including:\n",
        "\n",
        "> Bagged Decision Trees (canonical bagging) <br>\n",
        "> [Random Forest](#cell-1) <font color='lightgreen' size=\"1\"> Which has been used</font> <br>\n",
        "> Extra Trees <br>"
      ],
      "metadata": {
        "id": "iUwyaRQlotOh"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## <font color='orange'>Stacking Ensemble Learning</font>\n",
        "Stacked Generalization, or stacking for short, is an ensemble method that seeks a diverse group of members by varying the model types fit on the training data and using a model to combine predictions.\n",
        "<br>\n",
        "\n",
        "![Stacking-Ensemble.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAOEAAADzCAYAAABwrXHqAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAADFWSURBVHhe7Z0FXFRZG8YfAxW7MEHFBMUEO8DA7o5Vd9eOz1xj1+61W9fa1bULa10D7MLAAiUUFANBsVBSdL7zHi4sBog6M3cG37/Oj3veG3PPnPvc9/RJoRGAYRjVSKn8ZRhGJViEDKMyLEKGURkWIcOoDIuQYVQmwdrRy1eu4MHDAFgWKojSNjaKFXj37h1Spky6dkNDQ3Hrti/KlLZ577yr166jWNEiyJAhg2L5MoKDg+X9xSdP7tzIkye3EtIdHjduwNzcHFmzZFEsQEREBLy8fT6KJ8N8FhLhhwwZOlxjamqqKVuuvCZtunQyTOzdu1fTp19/uZ1UTp4+o0mRIoXm+fPniiUGstG+r2XBggXyGlmzZo37/P7778pe3ZItWzbNrj17lVAMl69c/WQ8GeZzfPKVvXHDeogHGlevXMbf69bB55YPXr58iTlz5uKCqyvc3Nzkm3/Dho2YOXMm/j1wUDkTuH7dXRw3B1u2bMWbN28Uawznzp3D9h07pTf9+++/pSe8edMTh52dcfLUKcydOxe3fX3lsXT91Wv+xFrx/fR9tP9DChYsBPHQx31GjRqFwKAgbBbffePGTXk98ugE3f9fa9fKezt7zlXaiNjvjb3+/fsPsGv3HnmvQugy7Czub+HCRfLasQQGBklb/LjHQvGj+MePD8MkiCLG9+jUubN8q9eyd9BMmjRJ8ygwUHPX/57G0rKwxtzcQrNhwwaNeOA1ZcuW1QwfPlxjYmKicXY5qjl58pT0oP3799cUL1FC00/8jfWETk5OmowZM2p2ir9ErCckj5Y5c2bND127aSpXrqwpZWMj93fs1FleY/DgwdLztGjZStpjofMyifN69uwZ9/H2uSWvSffTtFkzeQ7dz7NnzzTNmrfQNG3aTDNmzBhNzpxmmtu3fTW7du3SZMqUSTNixAhNTjMzzbbtO6SHSye8f8tWrTXVqlUT351d06FjJxlXugZB92NlXVIzbNgwef0///zrPU84dNhwee8DBw6Ucbvr7y/PY5hP8UlPuGH9emzZuh0WFuZYvHgJbCvYImeO7LC3r4UKtrbo0qULpkyZgomTp8gyXerUqfHwwT1s2rwJ9rXrYOnSpTh06DBGC88EpcRJ57Rs2RKtW7WKMRDKvvTp02P93+vwy4hR8PH2lrbdu5wwbtx46Y1at24tbR+SSpS9cotyYOwnTZo08prR0dHSg8+ZPUt6VPKCadOmwfkLF/D06VP8sWIFLC0L4a+/1qJ8+QoQAkPVKlVkvOn8yMhIbN60Eb169UZ4eBg2bdyAjh074v6DB8o3A0LM0tN17txZevf4rFm9CnXr1EUVcc0cOXNi544dyh6G+ZiPRPjw4UMppKJFC8uH8ujRo3j0KEBkSW8DKVKI/ynkce3atceSRYtga1cJWbNmk7a34uGP2QtERUWKawWIc2LCHTp0xA7xMPrfux9jIJR9Oc1iKlNSm6SWAorl7duY7aio97O1sQiPhKlTp8Z9ChUsEHdN2pfaxCQmIO55pRDeb7/9isdPgvFDl85YvXqN0JsG4RHhCAgIgPDC4h47xJ0vvCGdBuHpZEUL/Y1P7L2ZCOHTdeITFRWFFy+ey+v27NFDipFhEuIjEebPnx9R4sFs1ao1hgwZgu4/dke+/OawtiqBLJkzi7KWB/bu3Yt79+8jc+ZM8Pa6KcpHj4R43kpP5+J8GGPHjkX7du2wbPly5arA/PnzUL9BQwwbOlSx/AeJ70Pate8gRPMbfv65Bw4c+Fexvk9gYCDaipdB7Gfpsv++70MaNWqMY+KF0qJ5c2TPngPphQdv1qwZ/O/6I2OmTLLsSS+gpDJr1mxMnz5deMmNaNyooWKNge7Fx8cHadKmFTmGqUjBtaVMIqSaKFC24+jUqRNMTNIgOPgJKleqjGUie0mepXCRogh5+QJ58+VD79694efnB3MLC3Tt2g3m5vnh6OiISuJ4Dw931LK3x+RJk5AqVSrkzZsX1atXR80aNUXW8IXM/mXJkgXVqlVFLjMzFCpUCOXLlRMeJ5W8Dh1brFhR5MmdB6XLlJYiiQgPR/v27ZU7hMx6FihQQLw08sV9ihUtKs+L/T6R3RaCyy6+p5rwcu0RFPQYt2754Keff5Le0LZCBfndbm6XRJa3DQYM6C/uN3XcPdD9WFpaSi9J3xd7n5R9/lG8HNyvX0PXbt3RR/wWqUU8c4qsJ53XrGlTRIgs7R3x+4wfPx4ODvbKXTPMxxjsKIqWoux49cpVtGjZAlu2bMGE8RPQv38/ZS/DJB8MVoThwvPt3rMXr0JCULx4cfYmTLKFxxMyjMpwjQHDqAyLkGFUhkXIMCrDZUI98eeffyE6+tOdDgyVWrXsYWVVQgkxuoJFqCeo+93gwYOVkHGwaNEio7tnY4Szo3qCuvsZ24fRDyxChlEZFiHDqAyLkGFUhkXIMCrDImQYlWERMozKsAgZRmVYhAyjMixChlEZFqHK0MRWNCOcLqE5WWkm9PiTaBFko09YWJhiYdSARagyu/bslRNjfSsXLlyQkxZ/yNWrV+UUj0uXLsP4Cf9NJ0Sz6NWoUUNO6T9u3DjFyqgBi1BlLp53RaXKVaQ3evIkGF5e3rjk5ib3ked68OAhPDxuyE8svvFm9abtkJAQLFiwEK+FV4sPXXPMuPEYPmwYhgwZjD179uDBw4c4e/Ys+g8YKOdLrVypEs5fvKScwagBi1Blzrqel7O+bdy0GZ1/6Ipt27bKaSHPubrighBHw0YN4eS0EwMHDsChQ4fw+vVrdO3+ozyXREpTQ3p5eQnPdgRPHj+W9lj+WLESHdq3lzPF0WfSxAn4+aef0KdPX3FNJxQtUlgeR1lSRj1YhCpCIvK/44ciQgyXLl5A39695BSJNMXiy5BXcD13Dt27d5e2seMn4ODBg7h85SrKlSktz3f38IC1lRVsbGxgWbioEO8QaY9l48ZNaNvmv9nLbUqXxqnTpzFixEiUtLZSrIzasAhVhJZSsy5ZSm5fvnINjZRJhM+dv4CqlSvh2vVraNK4ibQFP3ks5369dPEiqlStJm3Hjx1HlSqV5TJz5cuVkbZYSOBv376Vc6QS9+7dR9u2bbFy1WqsWbNKzslK0N+IcN1WDDGJwyJUkfNCbDQBMk3vSOt5kGBIPJERkXJyZNpPWU3av3r1anTq1Bne3l5ywuWHAQFYsXIlKtjaieNcUbJkyThhEZRtpcmIiYBHgWjeogUWinJj1y6dUax4cWzZuk3uc7t8WdwDT9OvJixCFdG8i0b9+g0QEPAIXbp0kranz56hTetWchGbfPnzy0VsqNxHy77R7OIkxFkzZ2Hx4sVyRvKKdrawKVUK+/fvl8sCxJI5c2aEhsWU9VYKsVJ5sG7dOjI8dcoUufQbQbWmgwbx6HlVoektGN1DS7l9yLt375St9yH7gUOHNSNHjowLx+etEo5v/9S16jdspHnz5s0n95Ht3DlXzdix4xTLx3zqnhntw55QRRKaQoLsWYUnIw8YG45PSiUc3/6paw3s3w9BQUGf3Ee2K1evYqLwkIy6sAgNFKpwsbOtoIS+Dlp1ilbZSoh+ffvIBXsYdWERMozKsAgZRmVYhAyjMixChlEZFiHDqAyLkGFUhkXIMCrDImQYleFVmfREz549Ua9ePSWkHSjlIiLC5VhBXTS6BwcHY+DAgUqI0RUsQj3x5o321iakkRb7/tmPrdu2I3XqVPC7fRvNW7bED106wzyRHjJfCgk7ZUrOLOkaFqERQeL7R4jv+MmT6NypEzZv3oQpk6dg+R9/oE7t2vj3wAHhFdOiixRjPuUsxtBhERoBUVFR2LV7txz4SyPlbSuUx1/r/oZVieKoXq2aFOeQoUOxYP58Oa/MqjV/IjIyUgq1UMECylUYQ4VFaMCQ+Hbv3oMz587JuWFK25SS2cNbIvtJ9hG/DFeOBK5dd8eZM2fQv19fGaZ5Y5YuWy5H15MYC7IYDRYWoQFCI+n3/3sAp06dxs8//yc+Ir7Xo9H48Zk1Zw5aibJhsaJFFUusGJdBo0mB1q3f38cYBixCA4LEt2vXbjkFYe+ePWBtbfVRxciixUtRq1ZNlCv7/pwyRGICJTHS/DI07UXHjh1YjAYEi9AAkOIT2cvzFy6ilxAfzYT2qVrJa9eu4fjxExg8eJBi+Ria9IkmAu7dq6dieZ/4YuzUsSOKFi2i7GHUgkWoIiSE7du346anN7p375ag+AiaKn/EyFFYuGB+gsfE8vvMWbKG1MLcXLF8DInxr7XrEBj0WJQZO4jvtlb2MPqGRagCUnw7dkjP1r9/fxQtUuSzwpo6fYasYClsWUixJAwJduQoEuyCT05tER+qRV25eg0eSzF2lFlgRr+wCPUIzaC2d+8+uLm5of+ApImPOHnyJO7e9Ue3bl0Vy+dxPX8enp5e+OnH7oolcSIiIrFKiJEWj2nevKkoc5ZV9jC6hkWoB0h827bvgI+3DwYO7A8LC4skiY8grzlx8lTMnjnjs17tQ6ZMnYoff/wx0Wzph5BnXL9+A277+ooyYweUZTHqHBahDiHxbRfi8/b2lmtJFChQ4IuF9NuYcRg8aCBy586tWJIOVfiMGj06SdnSD4kQYtzAYtQLLEId8PTpU2zeshWBgY/Qq2fPrxIfQWtPhIaFoU3r/9aT+FLOuZ6XL4Efu3dTLF8GlS+dnHbJ6RHbt2+HinZ2yh5GW7AItQiJb4sQn/89fwwePBj58ub9KvERz549x9z5CzB18sSvvkYsEydOkqM4zM2/vnM3dUDftm27rEwiMdqxGLUGi1ALkPhorT8/kXWjNrx8+fJ9k3AoSag5Ysxvv8pFYL4VypaOHv0rFiyY/82CjhWjp6cnGjduhKpVq37zNb93WITfQFDQY2zeug3BwU8w+H8D5QIs2nggabGWHNmzwdHRUbF8OydPnYaf3x2RLU16DWtikBj37tuH06fPyOxy9erVWIxfCYvwKyDxkVCePHmMwYP+pzXxEbSSLjUVTBw/TusP9dRp00W2tAfyfEUlT0JIMe4VYjzDYvxaWIRfwKNHgcLzbcWL588xcEB/mJmZafWBo6Sgvp8zps9A+vSmilV7hIWFy/Xx586do3WhkBhdjhzFwUOH0LJFCzjY12IxJhEWYRIg8W0h8b14IT0fldN08YBRNzJra2tUqVxJsWifY8dP4PHjx+jQvp1i0S5v3kTD2cUFhw4fRquWLWBfi8X4OViEiXD/wQM4Oe3Gs2dPdSo+4tat29jp5ITRo0YqFt1AyT1x0hT069dHq9nSDyHP6OxyRHjHI2japDFq1az50cgOJgYW4Scg8W3duh1RUZEY0L8/MmfOpNO3OQ3e/WXECMydMwcmJiaKVXeEvHolmy3mzpmtcy/1Jjpajovct+8fNG7UELVrO7AYP4BFGI8HDx5iy7ZtiIyIkGU+WrJaHyxYuAh169aVg3f1xdFjx2XTSru2bRSLbqGxjidPnRJi3I/GjRvKOXF4WbYYWISCO3fuwmnXLlme6de3t97ER1y7fh2nT5+WHlefULKPGz8B/xtIXeJyKVbdQ2K8cPGSbGts2KA+HB3rffdi/K5FSOLbKjxfqlQp0btXL72Kj/iSMYK6ICQkBJMmTcYcPWRLP4TESIOYqW9tw4ZCjPW+XzF+lyL09fXDTqddYkuDPr31L75YZsycJWdPU3OqiUOHDuFlyCu0b9dWseiXWDFSTsS+Vk00athQL+ViQ+K7EqHPrVuyY3XmTJmF5+uB9OnTq1Z9fubsWdzy8ZFDjdSEkn/suAkYRCM1cukvW/oh0W/f4sqVq1i/fr3winWFd/x+xPhdiJCq/zcJ8WXKlBF9evVEhgwZlD3qQNlAaiaYM3umKtnQD6EhV7PnzMXUKZMVi3qQZ7wsxLhhwwY0qF8fdevWQbp06ZS9yZNkLcIbN25gx04n5DIzk6PS1RZfLGPHT5Dzg9IoC0OB5jFNLTwPtekZAiTG6+4eWLt2HRwc7GXzRnIVY7IU4Y2bN7F58xbkFQ85jaMzFPER+/f/K9sFW7VqqVgMA3oMRv/6G0aPHoVsWbMqVvWhbOr16+5SjLWFGBslQzEmKxG6u7vHeL5cZkJ83Q1KfAR1e5s2/XfM+oqpKvRB8NOnmD1rNmbO/F2xGA7R0W9lc86mTZtQ0c4WzZo3RwZRpk8OJAsR0kBTGkxbwqoEOrRvD1NT7Xd+1gbUK2bsmDHIakCe5kP27NmLNGnTyFpKQ4Sm9ff29sGKFSvkWMZmzZsZvRiNWoRx4itRQs4qbcjZFPLQJL56desoFsOEHocx4yZg5C/DDPplQWK86emF1atWoUbNGmjYoAEyZcqk7DUujE6EdLsXLl6UFQmlSlqjTZs2Buv5YqFRGMuWL8PkSZMMMhv6IbQ46Lz5CzB92lTFYriQGP38/LB8+QpUqFAeLVo0NzoxGo0I6TYvCvHRwphUJmjZsqVRFNDpvocNG44pU6cgo4GVURODGs/TpU0np7AwBt69eydnhvuDxGgrxCjKjMYiRoMXYYz4Lons3E6UL19OrjpkTLVjf/71F0qVskHlShUVi3FAv3tMtnS4yJaq06PoayDP6CvE+Ndfa1G0aFG0bt0a2bIZbraaMFgR0m2dc3XF9u075fCX+o6OQnxplb3GAc3psnnrFoz59VfFYlw8efIEi5csFdnoiYrFeCDP6O/vjyVLlqCkeAm2btXKYMVocCKMEd952bHX3r4WGjVsgLRpjUt8BA1qHTlqNH6fMd0o7z8W6mNLk045ODgoFuMivhhLlykrOyPQnECGhMGIkLIRNFnQP/sPyPlJqBbRGB9eigeVXSkLXbNWrU+uI2hM0EM8fMQoNG/WBLWNVIgExePRo0fCsy9DIUtLtG3dSogxh7JXXVQTIa154OXlDRubUlJ8e/bsQ/36jjLrmTZNGuUo48PL2wd16tRGzRo1ZMNychieQwuXduzUEV6eXrBMwqpQhgyJMSAgAEuWLhNxKYxmTZvg6dNglC5dWjlCBUiEatC//wCNo6Oj5pcRozQHDh7SREREKHuMm3V/r9dkzpJFs2LFCo1IcMVq3Hh6emnMzc01q1atUizGj8ixaO7fv6+p51hfU6hQIc2jR4HKHv2ToCekpbhu3fJRQtqFxo/NmjUTaUxMsG7dOqTRgudLmy6dnEwoqRw9elRmHbXNps2bhXe3QRktv1lLliyF/PnzKaHE0UXa3bp1C1evXkW7dtqfpe1L005b8QsLC8NPP/0st21tK2DEiJHQRTPu59IuQREuXLgQFerpZqDnq+AAZMieRzZca6vx2mnTn5g/bZwS+jzDx05By04xCaBV6OfUQUq6Oe/AkCGDlVDi6Crt6FHRRWeDL027+fMXwK6+dl4GFCf6REaEy2FladNpv+PH59IuURHat+6hhAyfTWuWYtbEUUro84yePBsdf+qnhAyfoztWYdjQoUoocZJ72s2bPx912vZSQobP59JO/RGlKmH4nceY74XvVoQMYyiwCBlGZViEDKMyLEKGURkWIcOoDIuQYVSGRcgwKsMiZBiV+WoRrpk3CasXTFdCMWxatQg7N6xWQomzcMoIZetjdm9ZC88r55UQcP3yBZxy2a+EtENiQ0fu3/LAyN6d4O/7X//EJ4EB0hby8rliSRi6d+d9O5TQx8TG/ZanB+aOH4oZo/rD0/2KtOkDXaYdxZvi/zY6GtvXLsOsMYPgsnerspf5FF8tQrcLrti3YxMiIyNkmP5uWL0EHtcuyzBBCUGf+MTaXM+cUiwxRCnXIfxu+SAoKFBun3T+F78O6I77/ndlWFsk1mPm2bNnuHT+LI4e3KtYgH93bcGZk0cRFhqqWN6/51jIRvfu53tLsXz8O1DcKTz5l77o2msAfhr0K8YN7hH3W+qaT6Xd9vWrtZN2It4U/8P7tsvBtH1HTMIJ5wO4fPa4cgTzId+UHa1UtTpcTzjLbfqRK1auKrZiHu9ta5djyM/tMKhbC5w/9q+0Oe/ZggFdmmLs/7oj9PVrafP1volBXVtgWM+O+GPu5PcSnhbrvH7pLLr1/p9i0R6JeUKiWk17nD5yQAkBHpddYVOmrNjSIDjoEUb06oARfbpg+si+CHv1Un5G9u4o7QeEYOkLKC5/Lp4pf4cRvdpLzxfLmzdRGDFlHnJZFEWe/BZyUqLQ16+UvbrnvbQ7dwKVqvyXdjv+/kPe88CuzRNNu7sipzC8h4ib+B3i0k75YR0aNMeg0ZOQ1tQUmlQmiH73LmYH8xHfJMJ6zdrg8D+75fahf3ahlmMTuX3X+zpOuPyL2Ss2YvbKLZgzdZzIxr3AuhWLMP/P7Rg5ZT5ev4p54GaPH4YRE2Zg3uoteBP2CueOHxLWmJSkIS4DRk9BOlPtz1L2ub6jJqaZkDe/Ofz9fOTHLG8BpEpJA3RTyOxYu269MWflJhSxLost61bKT7XaDTBn1VYUK0ViFQ+36wkE+Pti7qrNGDtrOWaOHSLtRDrT9ChToZLcphdWcWsbZM9hJsP64L202+cE+/pN5Tal3fHDMWk3V8QlNu1WL54Tl3axuYGZYwZjyK+T5O8QGRmlpF0MpukzIE3adBgpBBpw9zYKF7NW9jAf8k0iLFi0JB4/vCsTKTI8FDlz5ZF2Hy9PlK5QVT5o6TNmQoECBeFx9hBy584D0wyZkC1nLhQvYSWP9b3tg/kzp2D0wJ/g6X1LTsUeXyI0dMZEpTXO6zdtjaMH9slP09YdFCvg5XkTFWvUgUmatChvawcfdzfc8/GQEwqZiAfPrpLwKiIKt3285dT8FLepvw3BW8370n/7Nhp/LZkFL1EeHD55nmLVD5R2IU+DZNpFR0XALHfM4jR0zzYVqryfdq5HYG5RIC7tihUvEXPsLW+ZdqMG/Ajvmx4xaffB240E2qXnACyfZ/hzmKrFV4swxldpUL1uY8weNwQOypuUMMuTT2ZVCHrQgh4HokBJOwQHP5Fh+jx+HCT358qVG2OnL8DMZevQrUcflC5nJ+3xeRMvi6otPpcdJarXayLf7h5u52Bd1i4uzjlz5Y2L3z1RVs2R10JkKwvLbSLg4UP5N6eZGarWrC3jNnneCjRr+f768HMnj0Z0ZDh++30p0ghBJ0pSbjiJxMajqvDci6aMRDWHBtJCxI9bXNpZV3gv7WLL6x+mXaky5aWdWLVwJi6ccpEvqqzZsiHF2zfKHi2gxd8iNk5EVGSk/BvfphU+c78JijCBYYZxpE9vKi/eqFkruJ0/ixr1GstBkbSOQRm7ajBNa4KZv/0PE4f0QN0mbWBesAjqNGouwzNGD5BzfRC9hvyGySP6YdWcCVi7YrEQcF55jdTxvF+GDOmRVngYbZIikV8mNh6pUqVGcStrFLMpJz1ybJz7DP1VZsXonp02/42uvQahTZce2Ll5HZZMG41/dmxAWvHw1W7YEvf8bmHxlFGYPLw33iiekBaqCfDzwuG9O3DuzElR1mqGPu0b4nFggNz/ST7wMImR1LSr26AJTp84JosRqUzSyDiXtqv6cdoVsETtxq1kHD5Mu/GDf45Lu1x588t4U9rVa9hEZmFXzBqHFQtn46cBw+U5WuELfovPMW/iCJmTCQt9jf91ayFtO9avwcHd2+S2VvjM/X71oN7XIS9F9iSjnMjo1cvnyJQlG6LfvJE1bRlENoa2KatDD3SWbNnlQ6x59xYvnj+Xa8SnFOWrjJljJpWl86PE8RnE9SgbFC7KHKlNUsu3KEFvKLpNKiMmxJcODP118mx0SGBQb/x40Ihr+hXpu+PHmbYjoyLj7pkIDwtFmPikF+EUIt5kfyOOeRUS8t7vQPE1TZ9RlItDhBY04uoxqZQpS9YEJ4bS5qDe+PEIefEMmbNmF2/+t4gID0s47cTv//L5M2FLkWja0TXoupR29GDTb0Llw/Rif0KoOaiXKsPoBZ9KvDheiThnzppNxiFFChplr50X/1cP6v3c25QSIfaBIQEStMgkJWLsdvacZsiaPYdMRCKFSLxsOXLKRI9NRILOzyHKGrEPs6nwFLECJNKkTau1HySWxGIXPx403UHsd8ePM23Hv2eCHjay0QMea6d4fPg7UHzpO8iWLXtO+Zc+CQnwS/mStKO0ICicaNqJvxT+XNrR39i0I+HlMMudqADVhuJM8aX4kQAJioO2n7fESFCEsT9+ciU5xy65p11y46s9obGTnGOX3NMuufEde8Lk+6CyJzQuEhRh8ocfVMYw+G5FyBk2xlD4jj0hwxgG360IOTPKGArsCRlGZRLsMbNgwQLkKBwzGkCbaN69k1XoKbXUMB3LsYN78eey+Uro8/zUbzDqNGqphLQHxY96y2ibZ3fcMXjwICWUOLpKu3dv32o93YgvTbt58+bDrGg5JaQdKG5Uq6xG2iUowpCQELx8+VIJaQ9aCefv9Rvh7e0FR0dHlC9XDnnzxoy++Baoi1X+/PmV0Oe5f/++sqVdli5dhuo1qqNcWe2KIGvWrHLMYVLQRdpdvXYNZ8+eRf9+2l+/40vTTlvxe/QoEFeuXoWzszNsbW3Rrm0bnSxM+7m0S1CEuiYsLBwPHtzH3n/24/HjJ3CsWxtlypRF7ty5lCOME1ome+bMWWjarKnWhagWXl5e2Lx5C8aOHQMTExPFapw8ffoMly+7wfnIUWQT4mjTuhXMzS1iOrWrhGoijE9oaCju3r2LHTudpKesLzxkufLlkCN7TL9GYyNGiDPRrFkzlDVyId67dw8LFy3GtKlTkE6P/Sm1CXlOWlvR5ehxpBJel5b+LlasGDJmNIw+rQYhwvi8fv0at27fxr59+/FO3JqDfU25lLGxCZKE+LsQYnMjFmJgYCDmL1iEiRPGwdRUPU/xNdCL3dPTC4dFVjM09DWaNW0Ka+uSyJIls3KE4WBwIoxPSMgr+Ph4Sw+ZQby1HGrVlA905syG90N+iv+E2FzcdxnFahw8efIEU6bNwLQpk5JcFlWbiIgI3PT0xOnTZ3DHzw+t27SBTamSyJYtZnSEoWLQIowPFcRv3ryJffv3I0vmLKhVswZKlSpl8IKMioqSZcQWLZqLMq9xCPH58xeYOn06xo35TVYqGDL0ovP29sHps+fgefMG6tevj8qVKiJHjv+GYRk6RiPC+NBD4uXlCaddu2GWKzfq1nFAieLFDSaP/yHGJMTXIhv3229jMG7sGJiZ6W/iqS8hOjoavn53cPHSJZw9fQq169ZDrRo1xP3mlDWtxoZRijA+T58+xY0bN+Hk5IQCBQuidu0YQaZP/99gW0MgRogzhRBbGKwQw8PDMXnyFPTt2wcFxW9pSNCUGnfv+uPylSs4dfIk7CpWFC/f2uIlnEu1icC0hdGLMBaKRnDwU7h7eGDvnt1yNrdqVavAysrKYGr1SIhURmzZsiXKlC6tWA2DyMhITJo8Gd26dYNViZjZ1NSG0vRhQADOn78gynmnUbJkSVlzni9fXqNvKolPshFhfOit+eRJMC5cvIijR1xQpFhx1HGwR5EiRXTSGPslxAqxlRAi1foaAlSumjptGjp26ABra/XnBw0MCsK1q1dx8LAzihQujAb1HWFhUUC8TNVNO12RLEUYHxLk48ePcenyFbg4O8NKPGRUqVOsaFHV3qZSiL8LIbZSX4j0+8yaPRt16tRFpYofTzepL54/fy7b8g4eckaOnDnQtHEjmSWmmemSO8lehPGhBy4wMAinz5zByRPHUa68LWrWrC7ftvGnWNQHhiBESnrqh2lrZwcH+1qKVX+8eEE13jdw2OWoCGnQsnkzFBZpYSxNUNriuxJhfGiKP+o7eOLUKVw474oqVarCzs4WloUK6U2QMUL8Ha1bt4aNjY1i1Q+U7KtWrxHeppDI7tVTrLrn9etQWbN95Ogx6f0oW168eDGDb8vTJd+tCONDZSIqhxwVD8aVy26wq1gJVatWQcECBXQuSBLiDCHENnoW4rr1G+T8qG3btNJ5exo1ovv4+OAUNaLfuYMmTZqgtIhrjhwxc5p+77AIP4Amsn344AFOigfG7eIF1BZlpfLly6GAhYXO2qD0LURqX6UVrzp27KAzEdCL7bavL86cOYcbHtfhULsOqlSubLRtebqERZgIVG3/8GEAnF1c4O3tDQcHB5QrVxYW5uZaf3ilEGcIIbalrlalFKv2OXDwEO7e8UPfvn21HgfK4t+5excnTp7CFbdLqFGzFmrVrClHxmhrYuPkCIswiYQLz/Hw4UNZe+fnexuO9eqibNmYsZDaephJ9FRZ01YIkbrkaZszZ87g3DlXDBs2VGveiCq7aGwmdRu7KMrW5cpXQJ3atWVbnr4ru4wVFuFXQGMh/f3vCg95RA71oZ4b5StUQO5cub5ZkDFC/F0Isa1Wheh2+TIOHDiA0aNGfbM46JEJCAjA5StXcezYMXmfDvb2sLAwR5o0aZSjmKTCIvxGqLbPz88Xh51d8OjRIzRq1FB2S8v1Df0uSYhURmynJSHeuHED27Zt/+ZBuVR5dcPDAwcOHYZ5/vxo0KA+ChUsBFNT4xxnaCiwCLXIq1ev4SfKWwcOHpbjIuvWdkDpMqWRM0cO5YikEyvE9u3aye5aXwsNlv5jxUpMmjjhq3oL0cKfXp6e+FeUJWlIUxPxkilcuAgyZkz+jej6gkWoI169orGQPtizd5+ssGjUsIHsEkZDbJKKFOIMIcT2XyfEoKDHQsgzMX3alC/q0E6N6DSO09nlqCgLh8tG9GLFihvkgNjkAItQD7x48QI3bt6UNZOm6UzhWK8OipcogaxZ/ltiLCG+VojBwcGYMnWanJYiKUO8QkPD5ORbx46fEOINQgshPBqJnj3799uIri9YhHrm2bNn8PLyxu49e8QDnkOOhSxWvHiigowTYof2KJmEDtY0p8q4CZMwYdwY8R0JTwtC1/X19ZW9V/z9/eXsdxXKl0POnDm13nzBJAyLUCXoZ6eZv6gL1959/yBPnjyo7WAvs32fKm/FCHEGOnxmpAOVRSdMnIxfR4+UYvoQao/0978nPN5xeArvbF+7NirZ2Ynvz82N6CrBIjQAKAlo6JWHh7ts9iDv1bhRQ1haWr5XliMhThdCjB1yRGXN2EZw2qZeKqN/HYNfhg+Fubm5tBO0j9ryTpw8jQvnz6GeY33Y2lZA/nz5uBHdAGARGhiUHDTS47q7O1xcXJA3X340auCIQoWoKcA0TohVq1QRWUk/DBjQX2ZxadJh8nJdu/6A4iJ7S43o1JZ35qwrzp49I0RnhxrVq6FAAQtuRDcwWIQGDCVNQMAjXLt+Hc6HD8HKuiRq1aopxUhTeNCwH3ch1kWLFgkP+CuGDxuGQYMGy9myDx44gFI2NrAXx5OA1R7MzCQMi9BIIM9GnQHOnnPF5s2bsXuXkyzDrV33NyZOnAA/X185lfyQocNkVpYGxBraPDvMp2ER6pDt23fg2KmzSkh7UDaTmhEIGnlOE90SJErqraPtmk3Nu7dYvmShEmK0DYtQh8ybPx912vZSQsbLpjVLMWviKCXEaBuuk9Yl/HpjkgCLUJdwezeTBFiEDKMyLEKGURkWIcOoDIuQYVSGRcgwKsMiZBiVYREyjMqwCA2M8NDXuHfnNqKj3ygWICoiXNqo/+jnCH39Cq9evlBC70PnPwl8qIQYQ4FFaGBcPHMMHRtVh9u5E4oFOHZwDzo0qIbwsJg+oolBx+532qKE3ofOnzqinxJiDAUWoS75im5r5K1KlioN5/17FAtw5uhB5M2Xj8Y2IToqElddT2Dn2qW4c9tL7qdzrp0/KW1PnzwW4bfSTt6TbCcP7ZXn0flRb6LkPsZwYBHqkq/stlaidAXc8XKXWdKXz58hdapUSEurDadIgQ0rF+LkMReUKFcZcyeOgIebK47s34Ud61fDSthOOu+X1wi4fxeTf+mLQlZl4XXTQ55H59M/xrBgERogqVKlRoVqDrh87qQQ1T+o2aC5skeD/bu2ou/wsbApZ4f23Xrh6L87hQh3osfAX1BK2nrLI48d2ofcefLgoe9NmOXIhr07Nsnz6R9jWLAIDZR6DZvgsMiSnj91BNXrNFSsKaARWU8TZV6Y1KlN8O7tW7wTHjOlYkulTF0RER6BfAWLoEDx0rC0LoehY6cLK3lB9oSGBovQQClRpiLu3b4J0wyZkSaNMjWFKNPVtHfA1rXLRXbTH3u2/o3KDg1Rq35TbN+0TtoO7dkmD61W0x43r1+BWe68ePH0Mc4eEdlUcT5L0PBINVGgbDNahlZAsixlq4SSRkqRFc2Vywx5LQrJaQjtqtdB9py5kDZ1KliXq4hylWvCz8cTZ48eQL3m7VHV3hFFrEoj/NVLnHLZjxqOzWBlbSVFnCtPPjjv3oKIiEj80Hco0mfMjNTitVusZFnl25KG+5WLcHSooYQYbcMj63UIj6xnkgJnRxlGZViEDKMyLEKGURkWoS7h0jaTBFiEuoTbA5gkwCJkGJVhETKMyrAIGUZlWIQMozLcY0aHrFnzJy67x4z50xXPnj9F9mw5lJBueBYchM0b1ikhRtuwCI2chQsXYvDgwUqIMUY4O8owKsMiZBiVYREyjMqwCBlGZViEDKMyLEKGURkWIcOoDIuQYVSGRcgwKsMiZBiVYREyjMqwCI0c7vpr/LAIjZwUKXgODWOHRcgwKsMiZBiVYREyjMqwCBlGZViERg7Xjho/LEIjh2tHjR8WoRGzd+9e7HRywvDhv+Ddu3eKlTE2WIRGTK5cuXDp4iW8fPkSKVNyUhornHJGTOXKlVG0WDH80PUHxcIYIzzloZ4YN248UqdOpYS0x6VLl2Bra6uTsmE9x/qoXq2qEmJ0BYtQTxjj/KA8p6l+4Owow6gMi5BhVIZFqCeMMdfPJRX9wCLUE8bYqM4dAfQDi5BhVIZFaIA8fvwY69ZvwKo1f+Hu3buK9dNQljEkJEQJJQ065/Xr10qIURsWoYHh7OyM5i1aITwsFCapUqB9h444dfqMsvdjXM+fx7z585VQ0nBxccHy5X8oIUZtWIR6IimVHK9evcagwUOwZfNG9O3TBz/++CNWr1qFsWPHyv1/C+8YHh4utzdv2YI7d+5g5u8z4eXpJT3bunV/Y5kQ1+jRo2VXNmL16j/j+pXS+X5+flK0Nz09ERUVJe0JwRUz+oFFqCeSUslx9NhxONjbo1ChQooFKF3aBj7eXlIQs2fPhqmpqbTPnj0H2bJlQ6DIuv7cowfu3buPCRPGo1y5ssiTNz+mTJ2K6OhoLF6yWPYrjT0/e/bsCHr8BD3EOWnSpJHXSgiumNEPLEI9kRSvcveOL4oUKayEYvD390fefPll2dBSEWeQEF6OHDmQJUsWRAjP6FivLi5cvIjBQ4agWtWqaNGiGXxv++LadXdYWVnJc2LPz5QpEyIjIlCjejVpTwz2hPqBRagnkuJVSlhZ47q7e9zDT9lI6jrWuVNHKTJb2wrS7uJyBLYVyuP+/fuwsLCQ17544TysrEvK/VevXIW1tRU8PNxRpXJlaTt0+DAqVaoEn1u3YWNjI22fgz2hfmARGhD16tbBE+HlxogyoMuRoxg2/BcEBQVh0KBBePHiBZ4EP8W+f/ZjvijTVapcBR43PJEqVSo8efIEl4Xw9u3di2PHT2DJkiXo27cfgoODpdfcvWePsC2V57hfv460adPIfYxhwCI0IFKnTg0nJydRrisnvNllNGnSGBs3bpRlt86dOsGmVEmQb1rxx3LUqe2AmjWqoWWL5nj5MkR6z549foaH+3WsXr0aBQpYoHevXihsaYl06dLhzzWrUbVKJdSv7yjLncLNxXwpoz4i8Rg9sGDBAmVL+1y85Kbp37+/EtIeurxn5j/YEyYDihYpjLHjxikhxthgESYDsmbNirx58ighxthgERowVCnj5nZZCb3PmbPn8ObNGyWUMG/fvkVYWJi81vkLFxUrY0iwCA0UEtjo38agcGFLxfIfohiBvn37wsTERLEkzIoVK3HY2QWZM2fGtGnTFCtjSLAIDZRFixejQf36CA0Nw/79+6WNxEfd0m7fvg3z/PmxctVqjBw5SjZREMeOHxfhkZg8eTKeP3+OixcvYtmypXB3d5c9bR49Coxrg2QMBxahAUJZyK1bt6Ftm9ZImTIFlv+xQtoPHz4Mb29vXLlyFYEie1m2bFlkyJgBGzdtwo2bnpg7dy5+7tET+c0tMGHCBBQsVAjR4lqdO3eS52fJmiWuTyljOLAIDRDX8xdQ0c5Wthvmy5cPT58+k/1AZ8yYgbFjxohy4iWMHj0KlStVRNq0aWEuRGdhnh8tW7bGtq1b8O+//8q2wefPX8DayhpFCsd0hQt9HSqPZwwLFqEB4nnzJqxLllJCgGl6U5n1bNqsGczMcuKS22XUq1tX7nN1PS+7sHX5oSsyZkyPbt27o4CFuewd4+bmhooV7eRx5F1DQ1/HdQBnDAcWoQHy7t1bpIvnsYoVKYxVq1bhfwMHyv6kT58+lR24iXv37sHS0hIRkZHIkiUrzrtewLZt2+RoCuo98zLktRz+RFnZRg0byHMYw4JFaIAUKGiJwMBHSggyC0nZUPpLtaZLFi+SdhLkPFEOJFau+AO3b/kgZ84c2LlzJ3KZmaGHKB8Wtiwk+5euXLkKQ4cNk8cyBgZ1m2F0z/z585Wtz/Py5UtNp85dNGFhYZoxY8Zq2rZtpxHZSWXvl7Nx40aNk5OTEko6X3LPzNfDnlBPfMmwIGrTGzJkiGwHrFqtGtatW/tNC74ULGSJVq1aKaGkw0OZ9AOL0ECpVNFO1o42adwI6dOnV6xfB68nYdiwCPWEyHUoW8aDMd6zMcIi1BPGmLXj7Kh+YBHqCarJpAZ3Y/ow+oGXRtMTy5YvR0oj8yxVqlSRo/wZ3cIiZBiV4ewow6gMi5BhVIZFyDAqwyJkGJVhETKMyrAIGUZVgP8DmqH1vCYICjYAAAAASUVORK5CYII=)\n",
        "<br>\n",
        "Many popular ensemble algorithms are based on this approach, including:\n",
        "\n",
        "> [Stacked Models](#cell-2)<font color='lightgreen' size=\"1\"> Which has been used</font> <br>\n",
        "> Blending <br>\n",
        "> Super Ensemble <br>"
      ],
      "metadata": {
        "id": "CTW-DafDkGaS"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## <font color='orange'>Boosting Ensemble Learning</font>\n",
        "Boosting is an ensemble method that seeks to change the training data to focus attention on examples that previous fit models on the training dataset have gotten wrong.\n",
        "<br>\n",
        "\n",
        "![Boosting-Ensemble.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAPkAAAFmCAYAAACvCDyyAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAEGtSURBVHhe7d0FWBTPGwfwr4UCKmAAFiYqIgaIYBfY3YmdmIhdYIuF3Z0/uxNbVEpE0u7EABFb+e8Mg/UXJe6Ou+X9PM897MztHUe8+87szs6kiZGAECJbacVXQohMUZATInMU5ITIHAU5ITJHQU6IzP3x7HrYtev4/PmzKAFFCheCtra2KCXft2/fkDZtWjx7Ho7Xr1+jeLGi4pnE+/2zMuYlzPj7K9Pr1xF4+OgRLEqai5pYrO7Ll68okN9E1BCSwliQ/65goUIxmTJlitHX14/R1dWNyZMnb8y9+/fFs8lz4cKFmCbNmvNtd3f3GPOSFnw7qX7+rHGPyMhI8azyrFu3jn+v3zk4dI5p3qKVKBGS8uJNdyNHjuRZlj3evn2L48eP83qWwVavWYNly5bh6dNnvI4JCQnFokWLsWHjpu+Z9eHDR7yOPa5dv8Hr3WbOhJ+vL06fPgN7+9qYPn0633fHzp0IDr3G35u9B8v2DHvdvHnz4Od3GVu2bOXZ/3c/f1b2yJo1K39/f39/bN36H/+skVHRfN+rgYH8/Vgd+1mY9+/f8+/J6uLen73eV/qcrJ49IiKj+PPsc/7s7PkL/Oe7ffuOqPmBvVfc74R9D0JSwh+b64UKF4ZhzpywsbHBi5cv4enpifPnziOLnj6sLMvC2NgYBgb68PbywsWLl/D48SMpYO3RvEULXA24imw5cuDUCQ+YmhZFzVo1kT59ehw+fBi+Pj6oWq0aHjx4gHHjxiOjVgasWLkKQYFX+ffMpK2DurXtsWTJEkhZHvXq14e5uTkaNWyE1xGvccLDA0ePHUeN6tXEJ/31szK5cuWWgn4EWrRsjYsXzqN123bYsW0bGjRsgBEjRvKm/KjRo3HB8wIyaGlh/769aNS4MZ6Hv0CxoqY4e/YsAq4GoVvXrjh54ji6Sl9XrFwJYyMjNGrUCPPnz5cCfRfeREagW7duaNq0KT58+AAf6WcLC7uGwYMH4230O6xcsRwWFhaoWrUq//1ky54du347QBCiCv/suGpl0MLLFy/h5XUJe/fsxrNnT3Hq5Alpew8MsmXH+vXrMNd9Hho2aYZNGzdi165dOHv6FAKkYP/85QvOnTsHHZ3M2LJ1G3JIwd+mdWvkz18Aw4c5i+/wQ+dOHTFnzhxYWpXDtWvXcPjQIehoa2Pjxg3Y+t/2/+t7x2HnC7Jly8YfWfWyilqgWvUacJ8zGy1aNOfvx3z9+hUnT56Cra0N5kkHkudStj2wfz9q29vBrlYthD9/Lh3QzvJ969arzz9PqVKlpe0GfNskf37cuxubtTNkyCC1LrZg9+7dvAVxQvq9xGHbz58/Q23poFWnbj3sln4vL169Fs8SojrxBnm9evV4s3bNmtWoXrMmz7hfvnxGurTpkC5dOn5iS1s7Ez59+oRPUvBppY99K5YdGVZ36uRJdHZwwNWrAahoWx7ePr78ufhIfWv+NUOG9DwYf/bl80ex9f+qV6+OCRMm8Ee/vn1FLTtAZRBbsQoVLADPCxdgY2uLvfv2o3SZ0ngpmuysS8IC1cXFBQULFOB1ca9nnydHdgO+HR/2O/nZ50+xB6TIyEikT5eWt0wySAdMQlQt3iDfvXsPOnfugkZNmuLo4UOoVqMG6kuZLW3aNFJTuCV69uqFsNBQdOzYCV2kQN4uNYmdh49El84OKFfOGsWKFZea0OXx6PETtGzZggdBpkwZeSCHSxluw4aN4jvFjzXX30l92SZSk7hNm7ai9v/Ffda4BzvD/Se+/gGoIAV4/nz5YGdXCxmkbkQBk7woa2mJO3fu4M2bKB7kP7cG/oZ1dDp37owePXoivXRAqFypsngGqCH9vjJKPyv7+S/7+2P1mrXIoqu4KxSEJFQ66Z/aRWz/oqCU9bJL/Uiz4sWlgO6N7l27IEuWLGjVqjWePX3KsxzL9BYWJVFC6udWqVJVCvoQlC5dmtcb6OtJwdmM17H9XV0nokIFW5gWLSb1Yd/zpnWZMmWQN19eWFtb8+/J+tV58+aVttJIz5WW9i0uvW8V3ncfMGAAlixejO49eiC/ya+Xp9hnNTAw+P6wsbFFZl1dmJc055+fMS1iKh2k6qBixUq868ECfOGixTA2NkLTZs3xROo3s0zu5uYm/TwlpE+An14f+3kKiAwf9zlLmJeQPnt5vHgRjvnSz1ykSGG2K38NO8A1kA5SwUFByCl1U1hTX09Pj7+eEFVS67vQrt+4gZLm5lKfti5v6p4+dRK3bt3mJ/0IIQmj1kHOsEBnfXuGNd9NpKY2ISTh1D7ICSHJ889LaIQQzUZBTojMUZATInMU5ITIHAU5ITJHQU6IzFGQEyJz8V4nZ/dBsxtSNAm7P50NsSWE/BBvkLPx54MGDRIlzaCJn5kQZaPmOiEyR0FOiMxRkBMicxTkhMgcBTkhMkdBTojMUZATInMU5ITIHAU5ITJHQU6IzFGQEyJziQ7ygMBgOA8bLkpJxxYDPHTokCj9wBYhdB42DJu3bMXkqdNEbez+tevWw+UrV9G3n6OoJYT8S6KD3PPcGRgZGYoS+Ppk0dGxK4bG+VPd79gaak+f/VgVNc6gwYMwcOBAtGrZAtu3/cdXC2WBz9YpG+DYD5ZlSiEkJOT7qqeEkL9LdJD7+PiiVOnSOHr8OJ8Hvf+gIfwWTw8PtsBfOEyLFoXT0KGoXr0GDh+NXe64jKXl96AsU9YSAVevYp77PJw4eeaXYGXLBbOVVdjc6mwxQVcXVwwfPgwNGjaCs7MzX1WUyaqn98uyyYSQ+CU6yL18fGBT3hZXAwJQVgrYZYsXomPHDrh16yYuXLzA1wNbMH8+X2d8/bq1fF2ynDly8gUS2RrfmbNkRulSpaCnr4cN61bz+jgrVq5A1249RAmoU6c2XxW1cuVK6NzZQdTGrrT64WP8CyASQn5IVJCzIGX09bLg0iUvdOjYiZdZdi9f3gZ+vr5o2aoVr/siNdnTpsvAF/K3tSnP67y8L0nN7bI84+fOneeXAGdCQ8NgYR476QNr8reS3qttu/Y4c+b0Lxn/wYN7UsbPLkqEkL9JVJBf9r8MG7E44a3bt2BuVoxvBwYFoWRJc1zy8v7eF1+ydAnatGqBkOBglChhzoN08cKFqFCpEnykwLcuZ8X3+1UMD3y2L1s1tbR0QJjnPhdWVuWwefMWvkdkVDS0tDLyAw0h5N8SFeR3pMCuVr06X0zfXApchgV1/vz5eR86/EU4zp8/j8ZNmyFXrtxo3LgRylnbYOGihejn6IisBtlRTuqf6+ro4NDhI9LBIZi/R5z06WPXA58ybRr09PQxZfIkXmbrjrPmP8vua1atQI/u3Xg9ISQB2PRPf+Lu7i62Eub69RsxzVu0FKWkadasecy7d+9E6f+9evU6pkmzFjFfv34VNb9K7GcmJDVIVCb/m9cREWjVsqUoJU2HDh1w5WqgKP2/devXY5nUDfi9L08IiZ/CoqW8dTm0bdtGlJKmRYvmqCBO0v3J4EEDYWSYU5QIIQlBKZEQmaMgJ0TmKMgJkTkKckJkjoKcEJmjICdE5ijICZE5CnJCZC7eVU3btm2LChUqiJLifPjwAZkyZRIlxWJj3x0d+4kSIYSJN8iVxdXVld9wQghRDWquEyJzFOSEyBwFOSEyR0FOiMxRkBMicxTkhMgcBTkhMkdBTojMpchgGJK6sDn5smbNKkqpTxbpZ+/WtasopQAW5KqUGmdUTe2zyNLPn7I/PzXXCZE5CnJCZI6CnBCZoyAnROYoyAmROQpyJTt79hyOHDnC11knJCWoNMjZqqTHjh3DXHf3X9YblzNdXV2cPn0az56HixpCVEulQc6WNw4ICEBgYGCqWbTQysoSderUhZ2dvaghRLVUPuKtd58+GDRwEEqUMBM16mPRosX48uWzKCnOs2fPYWRkKEqKZW9fWy1/lz+bN28eBg0aJEqpT0r//PEGuZ+fH85I/UlFe/Zc+oc3VPw/vJ5eVnTv1k2UkkYT/xk14TNTkKtpkLMPVq15d1FSf5tXL4bbhOGilDQU5MpBQZ6yPz+dXSdE5ijICZE5CnJCZI6CnBCZoyAnROYoyBPg9esInD1/QZSAHTt3ii3A19f3j6PZjh4/LrZ+dfr0mf8b7cfe/19OSa8jJCkoyBPgdUQEVq1Yzrc9PE6gVcuWiI6O5sN0R48dh2wG+vy5OCyIP77/IEq/cps585fRfjt27sKRY8dE6c+eSweRzZs2iRIhiUNBngD6elnxNvodD95p06ahcZOmiIx8g/kLFqBrl87YtWs3HAcMRJeuXXlWZ4OIPn3+BC9vb3R0cEDPXr0wbMRIRERG4dmzZxg1ZhyaNmvOtydNmgi/y/64cfMmevfpy9/n8OHD/PuOGz8BfR37Y/jIkShbtgyvIySxKMgTQF9fXwrqCOzbtx92deogd548fOQeK5e3Lo//tu/AJFcXtGrdBps3b8K5c2dhbJwL48e7YM2qVRg8ZChu37qFoKAglC9fHtOmTIKB9J7pMmghd+48cJs2BU5OThg+bBhcJ4zHkmXLsWfvPun76mHJooX4+OEDKlSoKD4NIYmT5CD/HP0aXz/92iRl5d/r4sNeH58/vU9C31cZWPNaWzsTv3tu8ID+yCYF34qVq9CnT294S9laJ5MWNmzYgJCgQNSsUQOX/a9AT08PxYuZ8ptyot9GoUzpUrh40ROtW7fm7/nw8WP+Ppm0tfn7P3r4CAcO7Ofv07ZNa5w9cxoNGjTk+7IWRMmS5nybkMRKcpA7dmmNPu3qi1IsVp46op8o/V3XVg3E1v9b7DYeF8/8OHG1aPpY/LdxtSiljJCQELRr104Kdm1+++gVfz+0kQLWXAq+tOnSoWrVajh77jxMTPKzGXBRpHAhBAYF49DhoxgydCjK29rCx8dXanZb8mZ79uw5cP3GTehIB4/IqGhkk8o1a9nhzt17yJpVD/nzF8CxY8exU+oKPH36lB8sCEmKZDXXWb804sVTvv3y8R1ejsMyr5/nCQR4/TgrHFd3/9pVUROLlb1OH0V05CtRE4vtP6xHK1z28hQ1ipfQ+9pnz56N7t1i585u2rQZli9fzjNwKQsLdOvaTcroXli5ciUyZ9bF0KHOeBP1FjNmuEkB/w3ZDQxgU94Wffr2lfr3WRDz7StGDBuKwoUK8ttQv335jLVrVkvZ+wzate+Ahg3qY4DUYjAw0EdOQyNMmjyFf19CkiJZQd6oURN4HNrHt48e2IP6onnJgrNv+4YICwrAqWMHMdm5J68fM6AL/C6dx8G9O3Dr5g1et3PtQixdMAcPHz7AkB5t/i/QB42aiMZtu4iS4hUvboap06ZJ2fKZqPmzpk2afM+m7NbOkuY/ms/Vq1dD7969YWSYk+9TtXJFvHzxApMnT8L+/ftRuXJlHtw1pf0YFrxly5bl+zp06sjLefPmgaNjP1Swseb7sANIp44d+HuxByFJlawgr1KzNk4f2cO3/b3Oo5RN7D8xa2oXL1kGHXo7YfA4N9y6cQ2PQr3xJjICvYZOQN/hE5G/QEG+75plC9GsZWsULFQYhYuZSweLXbyeSaeVCXkLJ+xe6W9fv/K7fRL7uHv3Dnbv3o1p06fxS2KKwg4Ee/fsxtIlizF8mLOoJUT1khXkunrZpP5pZgR5nYJJoaKiVmrGv4mUnssuSuD9z0f370FHR0fUAFn19PjXb1L/9cbtuwgLC0U+6T1MS5Tm9YnF+sXsdr7EPtj1bh9vb8xzd6d+L5GlZAU5U6tRS4wdNhB29RuLGqCctQ08TxzE5/dRePLgDu7du4fydVrxE0gvnjzgdaEhwXxfc/OSMCtaBO279cO9G6H49O5Hv14VEhPYq9es5de3mzVvgZ69eie4P/83Xbv3EFv/7+7duxgi9e8JSY4kB3n1GrWQSVsHNezrSX3SGjArWwGGRsYoa1UORiam6Oc8FpOG9cUq9ymYvWILf43rnOVwnzQCm5fNQecusbO4jJmxCB5HD2Nc/44wK2kBq0q1ULK0JXLlysWfZwoXKoRC0iOlLVu2FDt3bMfuXTulvnxxfnacDUndsmUrDhw8xIOeDYa5GhjI61iQsmGs7DnGz+8yf7BhsWzfG7fuIIuuLn/u2HEPXv/+/XteXrR0OTo5OCCzjjYvE5JUNDPMT9jPzJrw8VmwcBEOHjqEgQMGoH69uryubfsO6O/Yn1/jLlfOSgr61zjhcULK0N2l/fpLff3pWLN6NRYvXYGunR3QvXs3PHjwABm1tGBobMzfIyw0FAULFoCJiQl27tyFRQsX4PHjJ9gj9enZwBt20i8+//rM6kATPqMypfTPn+zmemrBMqxjv75YKwXsgQMHMNTZmZ+oK1PKAvv378OVK1f4ErXeXt78bH1uqSXSoH4DNG/aVHp1Ghhm15eez4J2bdugcePGUhfmLry8vGBmVgJr166B/2V/7N2zFy9fxV5dyJ07F7+uXpFGupFkoiBPoFmzZvEx6cbGRlJ2ngFfXz+slgI+f4ECmDF9Gr5+/YpyVlZ49PgxCklZmV03r1ipIm+Wf/nyhTftM2XMyN/ryOFD0nOVcC0sDEUKFYCFRSnpaO8OBynTDxo4gO/DsKGzhoY5RYmQpKEgT6CuXbthwYIF6NS5i7TdBXPmzIatrS22b9+B0WPG8isHmaX+tZlZ7CW/cKlvzp5/9DQcVapWk7KyN9KkSYt+/QfiydOnaCZleDaqLXPmzKhYsaIU4F2wYsVKlCkde3WBtRIKmxbj24QkB/XJf6LMvlM/R0cMHz4CBfKbiBrFoD65+qM+eSpRu3YdhQc4IQlBQa4iTZv8GEdAiCpRkBMicxTkhMgcBTkhMhfv2XU2C8qTyE+ipDjsbjF2M4miXQu6gr3bN4tS0mjiWWA6u67+UvrnjzfIleWilw/u3rmFdm3bihr10Vb6TBUqVBAlxWEDZdIp4cDGpE+fgd+Hrs4oyFP452dBrmqDnJxjpH98UZI/d3d3sZU60c+fsj9/ivTJ2dxo/23bJkqEEGVKkSBnUxx5+fgp5H5sQsjfpdjZdcrmhKhGigU5ZXNCVCNFr5OzbM5WISGEKE+KBjnL5uwebcrmhChPigY506RxI8rmhChRigc5W5iAsjkhypPiQc5QNpevs9IB/MSJE/D0VN5SV+Tv1CLI5ZzNz507h1OnTuHipUuiJnVhi0MeOnQIDx48FDVE1dQiyBmWzQ8cOChK8qGjo8tnd71585aoSV2srCxRp04d1K336wq4RHVUfoPK3wwfMQpuM6aJknw0aNAAmzZv5YseqrNFixbjyxfFrQcX59mz5zAyMhQlxbK3r83XnVNnqe4utL85dPgI/xq3cIGq+fn58W6DorGplY0MFf9PrqeXFd27xa5EowiaeLeYJnxmCvLf9B84GAvnu4uSas2Z646aLeNfm0zd/LdmCaaNHyZKyUdBrhwp/RnVpk8eh2XxuIxOCEk+CnJCZE7tgpyhQCdEcdQ2yI8dPy5KhJDkUMsgZ2rb2/O1vQkhyaO2QV63Tm3spaGuaiEyKhrXb9wQJTYZp7fYAq9nz/8uMChYbP0qIPD/69nijv8S3/uRf1PbIE+bNi2qVa1C2VwN3L93F7NnzeLbZ89fQJVKsWums+Ds07cfdDJp8XIcVu/r4yNKvxo1aqTYisXOvezdt0+U/uzho0eYP2+eKJHEUtsgZxo3bkTZXA2w8ecfPnzk2zPdpqNylap48+YN1m/YAIeOHfj4/N59+mHkqNF4//49rl4NRGHToggKDkbf/gPhOnESFi5ZhojIKP78qNFj0H/gIDx9+gxTp05FSGgYnj0Ph9PQoejn2B8hIaH8ey1bvpwv9bxkyVLYVrDldSTx1DrIKZurhxzZs+Ft9DscOHgIVlZWyG9igtevI7Bp8xbUrGWHxYsXY8nihbC0tMSWrVuxf/8+ZNTKgBEjRmLOzBmws6uFUyc84OV9CbmMDDFl8iS8fPECWbJkhoGBAcaOHoVevfvAaYiT1GKYiYmTJ8PD4wQeP36CxQvnw9vbG+XL24hPQxJLrYOcoWye8rJmzYp376KxcOFCDJIysL6+HuYvWMiz+OnTp6UDQDSGSAF69swZ5M1rAr/L/nzRB4uS5tDW1kZMTBrY2trgakCAlKkH8IP3u/cfoKUV28xn2T0sNBgzZ87EyJGjUKZ0aezavQsdOznw5zNnzozixYrybZJ4ah/kcdmcrbyS0trXtcXGxW6iFKt9vQo4dXiPKP2dU9emYuv/jenXAZ+jX+NWWBCGdG4C5+4tMXFIN3z99EHskbKu+F+Gvb2dlHn1edCdO3cWHTp0gLGxsZTBreDi4oKnUpO7SuXY/nqe3LkQEBSCu/cfYsb0abAqZw0fH19YWZbhzXatjBkRdu06ChUsIB0Q0sPQ0FBquk+Bce7csLGxgYG+Pu7evoULFy/i1atXyJAhA39fknhqH+QMy+bbd+wUpZTz8eNHeBw5IEqA73kPREa8lvqnkaIGeHQrGJ/fR4lSLFbHgvX5s+eiBogKf4SXj++IEhAeHvvcgqmjMXT8dMxatQPZDXPj6H7l/twJvYd/zJgx6Ne3L9+2t7fHnNmzeODVlgI/b948mDZtKm+GM926dYW2bmYMHjQIO7dvw8dPn2BZ1hLNmjblmf1t1Bv079sbefPkkZrsWaSgf4O5c+Zg3LhxKFqkCGpUrwZn52FSK+EUnkhN9qFDnfn7kqTRiCBn2dzaqkyKZ/P00j91wcKFcf/aVV722L8DDZq24tsRL56ie7Ma2P3fRjh2aoYzB7fzwHZsVw//bViDcQO78P2Y5bMnYsrYoVixaC6mjxkoamPNXrkNeQvH3jr55k0EtHV0+baylLUqh9mzZ/OTYH/Tv39/HqBM9erVUblyZb7NsOB3c3NDsaKmfB8WzP5+vli5YhmipIA2MzPjt9m2axe7/h07KFSrVo23CiZOnAgjw5woV64c5kiB3qJFc74Pe27KlCm83LhRA15Hkkbt7kKLD8s4TsNGwH32TFGjeP+6C61N7fIYMnoigi57oWv/ERg/qCvMravws8+RL5/DIIcRGrV2QHTkK/RoVReDRrviwukTcHKZxet6t2uIZVsOoEtzeyxes5W/55SxTtJ7TsaUcc5YtHYbMuga8PqjO9bh2OF9mL0q/ky+eeVC5MkWG3hJNXr0aKnpnIkHUwkpGJ2cnMQzyfdSamZ//PARuaWmu7LQXWj/phGZnGHZ3MbaKsWzuXXF6rh0/gwunvVA+Wq1RS0Q+Spc6lca8W1dvWz49PkzXr58iRy58n2vY33ZiBdP8PnzJ2xdv4I/Chc1Q7oMv15nXjN/Ks5LfV63JZtEzZ+xlVLZP09yHlOnTsPtW7ekrLtC4SuvZs+WTakBThJGY4KcUYelldJpZZKa7EWwfqk77Oo1EbVAYbPS8Ll4lm/fCQuAsZERSpexwmVPj+91b9++hXG+wjyYug0YiQFj3fA2MgLZ9PX4PsyGxW6IlOomzVvFv5eyDRo0kDeNE+Lw4cNo2boNmjRtjnkLFonapGOts1FjxorSr9j36tq9h9SyGMrPvpOk06ggT+lsXqZMGf61UcuOKFHCnGfnXMbGMJICun6LDvw5527NsWzOFIyduYT3ravUrIPBnZtg79Z1sLWtwAN3zEQ3jB3YFYMcGqOoeSnoZM8t9VtL8Nf7+1xCyFV/9Gplzx+nDu3i9epgxsxZ2LJpI/bu2QWvSxf5dWw2uu3ceU/cuHGT78OGuLL+vZe3Dw/O23fuIjQ0dnALG7nGnvP3v8LLodduIDoq9iQle31cPQv+hYsWY9WK5Shnbc2vvZOk05g+eZy4o/+MaVNFjeLQzDB/7zuy0WiMs7Mzv/TF/hbNmrdAkyaNcfrUaXTr1g0hIcE45nESdrVqYN3adejRowc2btyII0cOo36DhmjTpjUCAgJQr159vP8Qe3nwRXg4njx5AmNjI0RFvcWY0aN4PcMGyXTt1p2vtvMn1Cf/N43K5AzL5iXNS8DP77KoIaqyeNFCtG7VCj1798ayZSt4XWupC/VUCtDIN5H4+PkTvxbOLq9VqliJT7Xdu3cv5DA05NfC2Zl3dia+f/8B8Pa6hIuenihVugzc3efyM+wx0kHDR4x5Zy2EXtL3salQMd4AJwmjcUHOtG3TBps2//2kFFGsUWPGITj0Gg/c/Xt2Y/2G9Vi6bBlevnzFs9S3mDSwKW/LJ61kWd7XzxeVKlfm2f7Tp8+4LjXHCxYqxN/r3NmzKFWqNO7cvYvs+npSN8gSnTp1gm2FChgzJnb8e5t2HdClc2d07xI76o0knUYGORuEUcrCgrK5CrVo1gSDBg7kAd2mbTuMlYKxQIGCUsCehtus2bh5/RrfL6te7Ek8by9vfu2b9buLFysGb28vPHr4EMOHD5f63pfRrFns6D/WRNfR0cbkyZMxZ85cFClcBIsWLUJw0FXMnuOOHj178f1I0mlcnzwOa86NGDlSahrOFjXJR33yv/cdWVaOiIiAvr4+7zYx0dHRvBkeV45PTylYJ7hMQDYDAymodUTtD+/evftj/b9Qn/zfNDKTM5TNVY8FcrZs2X4JaDYQ6F8BztjY2vJhrPEFclICnCSMxgY5w26Q2LZ9uygRddaju+IWgSCJo9FBzrK5WfFi3ycZIIT8P40OcoZl87Xr1osSIeR3Gh/kLJubmhambE5IPDT27PrP2Jn2MWPHJ3tFVHZ2/embT6KkON++fkVaBd/8wVwPvoI92zaLUvJpwpnq39HZ9X+TRZAzK1au5KOs1HEZWzYHWsuWrWBomFPUqKe2bduiQoUKoqQ4X6WDnKLvcIvDpplydOwnSuopxQ9ELMjl4NOnTzHDho8UJfVy7vyFmCPHjolS6uPu7i62UqeU/vk1vk8eJ65v/vMiAOqiZMmSCAmmxQFIypBNkDNsrPOKFStFSX2wqY/YFMaEpARZBTnL5vny5VXLbI40svpVEw0iu/+8nj16qGU2T58uDR/7TYiqyS7I2c0S6pjN2R1bt+7cEyVCVEeWbUh1zOZly5ZFwBW6mYaoniyDXB2zeZHChXAtLEyUCFEd2Z4NYtl8w4aNopTy2IHn/WfqkxPVk22Qs6DKmTMH7j94IGpSXlrIYnAhSQQ2RdawYcOwcFHyp7BOKtkGOdO5Sze16ptnzJCOj7NPTeL+ydkSSKlRZwcHZM6SBVWqVhc1qifrINfLogs9AwO1yebFihfnK3mmJuyfnC19XMu+jqhJXViLcubsubAwT7l7KmRzg0p82GT/s2a6YdJEV1GTcthsp36+PnDo1FHUpA4bNm5Eh/btEzRNVErqJB2QyllZiZL6Cw4JwXKppfQvsg9yZtZcd7Ru2QIm+WLXJUspbDAMm5V0/Pjxoka9sH7j/RfRoqT+Hty+ji3rFNcd07RbbRP6eVNFkKtTNh/vMhETXdQzyGm2WnkGuaz75HFY3zxbjhz/XINbFWKQRmwRohqpIsiZLp06YVkC+i/Kpi8dcCIiYxf5I0QVUk2Qs+V5M2bKmOLZvGQpCwQFBYkSIcqXaoKc6d2LLdSXstm8WNFifAkgQlQlVQW5OmTzAvlNcP/RY1EiRPlSVZAz6pDN06ZVzqSGRDn8/a9g9Zo1fPvZ83C0atOWbzPDhg/n68H9jJ1zmTjVTZR+5Tx8hNiKdffu3X8u9fXy1StMd5slSomX6oJcHbJ5OhrDrlG+xcQg4MoVvu3q6grvSxf59kUvHz5MWUtLCydPn8HtO3d5fcy3rxjQtxcfF3He8yIfccmeY0sy371zB94+vvwOSVae4eaGB4+f8tcFBAbz/eMmF2FThrH3ZWu+62TS4nVJkeqCnGHZfPnKlBvTzqZmfi5lBKIZ2KKOb96+48H69OkTlCpTlgf3jBkz4OQ0FC1btcGbyAi4SAcAtsjHylUrESx9dezfH6GhwXB3d8dQZ2f4Xb6Cx48fIywsDF26dMGbqLc4d/YcD+DJU6fhwP59CAy8ihmzZuP27Tvo1asXnj15ghEjhsPSylp8msRLlUHOsjnDml4poXgJc/gHxGYGov70DbLhw/t3cHFxweRJk5FZCvoDRzxQ1LQIPC9cQPTbNzzTs318fX145s2krc2mO+e3PHfo0BGlLUriir8fX5+dDWs2yJYNOXNkRz4TE1SpXAkrli3Fl8+f8PzZU/j5+mKJ1KUcO34C2rVri8JFTGFlWUZ8msRLlUHO9JaOksuWLxcl1bIsa6mRUzRvXr0Yt8J+vfy3Z8sa3L/975tuvn76gFOHdonS/2PPsX2Y4/u2Y4v0vV48UY8bi7Ib6EkZNhDp0qbji3dkzZIZbtMmY+jQoXj14gV69OyFAQMGoH69+mjZsiXevn2L9++iYWxkxF9/8NBhWJe34X3vKlUq82a6diZthF67wW9aYs3yctY2/D1q1bLDiOHD8FBqNeTLk4vv+/ZtFL/RJalSbZAbidVMUiKbsymaNbG5Pn/aBGxcNluUgIe3QjF17FD4+8T2Uf/m2+f32LJmiSj9P/Yc22fhtDG4cz0YpkWLYnD3Nt8DPyWxWYBZoI0bP46Xs2fPjho1avD/oUaNG2Hjxo1wHjYcj6Wm9du30ShcqBBsbcojKCQMQ6Tm/Ib1a2Fdrhzef/yE7FIGDwkN489HS+959Yo/MmfWRZYsuhgzbjzmL1qMosXM0F7K4J2lJv04l0koaW7Ov2+SsbHrqdXTZ89jXCdNFiXVGjfBVWypj9lz5sb434+K95Erd+4YK+vyMb43w3l5kPOomGrVa8S4zFzIyxOmzY6pW69+TI++g2LOB97jdTPnL+V1w0aOjrG2seV1O45djGnVuk1Mo8ZN+DarY895h96PWbF5Ny+zRzlr65izfmHfy78/Rrq6iU+edJu3bIn5+vUr307OSidsBZ+fHT/uEePp6Rnz6tVr/vMnxJcvX8RWrN/f83cJ/byp4gaVv5k4eQpvusdldlWZ4DoJE8aNUavbL/91g0p922Jo69ADhYqaobJdQzh3bwnTklYwzp2b91M9z3hgzBR3eB7bixPHD6PnwBGYOWEY5qzegVMHt2Pb5vVYsmk/OjetCfdV/0FbRxf9OjbGym2H4ejQHIvWbkMGXQPeQnAd7oichkaYumSL+O7/b/PKhciTLenNWGbEiJEoKGXe+fPcERISorAbVFgTfNasmXgTFY2hToNRoEAB8YziJPiGGh7qqRjL5tOmzxAl1Vm3fkPM9Zu3RUk9JCSTHzl/OaZh46Yxe497xvTsNyimz5DRPJO3bdc+ZtXWfd/3lQInZsyUOTFDx03lZZb9WbbedfRcjKGhYUyduvX4I0+evDHrdhz6nsnjXs/2t6tdN2bJ+u3f635/KCKTFy9uFrNk6TKeNTVtzbaEft5U2yePwzK49AdW+TJGpUuX1sgpmo1MTPEi/Dl2/rcRDZu1FrVAFoOciIyM5Nuf30fxa8dZ9fTwPir298rOPDMs45uVMMf05dv4Y+bi1TCz+HHmeOXMsbwfnk4rEyysK/MTUMp09WoA+vTuxfvd/8JOgu3YuRNbtmxV2P8Lu2b+J+wSHTthp4gZh1N9kDO9evbEsuWqHQXHLr/cunlTlDRLhWp2uOzlCZNipUQN0LC1A9YvnYu9W9dgZP8u6NC5B2rY18OFM8exY8MKzJ48hu+XUzpIZJAOAMtnu+K/VfMwffwwpE33I8Ay6mbFlDGD+Vl7dsa9ul1d8YxyJCS447Ru0wbfvsXwz1+nbp1kz9f38NEjrIjnCk+7Dp34QWXE8OH82ntypPo+eRxX14kYOHDg92voqjDWZRImu8SesVUH/+qTP7oVjDyFzXmmfvP6BbLnLoiIF0+lQNGCrl42vh3k74MChYogb+HYOc3Yvpe9LqBwsRKI+fyBBzkT4HVGyu4fUK5iNZ61w+/f+P5cqP9FvIl8g5Jlrfn7xkfVk0ZUqFgJ//23lc8wdPb8BVhblcXqdRvg6+ONjNLBYtq0abjkdUkq++LOvfuwtLTEjZu3EPHqBdatXYOJk6ey/jGuXQvjE1ueO3eWt35sbGywYOFipE2bBgMcHVG8eDF+ec3crBhmzpyJUmXKoI69vfgUP9CkEYnUu3dvlWdzTZuimQU4k0E7Cw9wRj+H8fdAZNuV7Rt9D3CG7WtTvQ5y5Mr3PYiZ0jbVeD0LcObn58zKVuDP/S3AU8KKFcvRp09fdOnSFcWKxn5eNiHJ8qVLkCefCXz8/LB//37eFRsyeBBOeBzD3FkzpKb9a3z8+BGHDu7HMGcntGrVEhs3bOADaMqVs4bTUGe4z5mF6dNn8KBmJ2NZgA8cPAQeJ079McATg4JcMDY2wscPH1XaN0+NUzRrqt279/Dr34cOHkCHDu3Rp18/XAkIwMHDR/nw1t27dqCEWXE8evwEDRs2QFBwsNS8b8sD9uu3r7j/8DGqVqnyfVCLtnYmBEnN8PRaGfH86VOMHDkS48eORtWqVfkNKcx897n8KztAJAcF+U9Unc1T4xTNmso/4CoWLV7CbzS5dfsOChcsiCOHD6NrZwfUq1cfnz99gpGhIdKkScMD+6KnJ8/Sd6Vme34TEz7cNTQsDI+lg8CKlavQvHkLqbvyHnnz5OGXIJ1HjOYj5azKWWGGlNFPnTqF0FDpICB1AzJmzCg+RdJQkP8kLpuziR9VwaxESX7EJ+pv3JhRyJEjB9asXoVMmTLBbcZ0DJaa035+vlLg38GkyZP5aLehQwbz/e3s7GBaWOrSxHzj4zBY0Hfv2QerVq3EnNmzoaeXFUOGOPHmvsv4cVi9Yil69uyJUhYWcHV1wdWrV7Fnzx6slvZPLjrx9ht2p9G2HTvhLP5YysSa6uxkjbpM0UyztSpvttZGjZtg757dCh38RCfekoidOY18/Vol2Zxdvnn3kfrkqcH+fXsVGuCJQUH+Bz179pD6TStESbnY5SdClImC/A9Umc0NstIUzUS5KMjjoapsXqy4GR8cQYiyUJDHIy6bs6GFymRubo4r/po3hp1oDjq7/hfsTPu+ffvQ39FR1CgHu+3UdULKD2+9eOkSLl68JEqKc+DAATRs2FCUFEdLK4NC/zbKPLuuDAn9vBTk/zBs+AhMdHVJ1vQ7/zJy9FhMnzpZlOSFjfEOf/YULVo0FzXqa667O9KmUfxadZ6enqhUqZIoKU769Bng6NhPlP6CBTmJ37Xr12PmzZ8vSsoxYYKL2JKf/gMGfZ95JbVK6fvUqU/+D0VNTfHgwUOl9s3lOkXzpi1b0Kp16xS7Pkxi0W8/AWLPtCtvnnY2RXNQSIgoyQMbzXfF/wqqVq4oakhKoSBPAGVn81KlSiHgir8oycO8hYvQt08fUSIpiYI8gVg2X7NmrSgpVo5sBnil4umnlIndrvv65QsUKhR7zzlJWRTkCcSy+cPHj5V3/3ca+fwpFi9ZAuehzqJEUhoFeSJ0bN8ea9etEyXFion59n2hO03G1vDKqK2t0mm0yN9RkCcCWyLnxo1bSsnmRYsUwX2p36/plixdCsc+vUWJqAMK8kTq0tlBKdm8WLFiCNbwCSQuXbqE0qVKKXXgEEk8CvJEUlY2L2VREkGBV0VJ87CuxqbNW9G+fTtRQ9QFBXkSKCObs+z36s1bUdI8bKJDGviinugvkgTKyuZs7m5NxH4Pnhc8aeCLmqIgT6KOHdtj06ZNoqQYmjpF88pVq9Gju+bMDZfaUJAnEZtV82pgoEKDUhOnaGYDXx4+fMBbN0Q9UZAnQ4f2HRSazTVxiuYFixZh4EDNuQdb1RYvXoLBgwdj/vwFokb1KMiTwcrKUqHZvGiRQrgepjlTQbGBL7o6Oipf212TdO3aBYaGhqhlX1vUqB4FeTIpMpuzKZq/atAUHkuXLUe/vnQTyt+wqyazZs+GWbEfa72pGs0MowBOQ4dixvTpiVoGNz7jXCZhkhqtdBofP7/LfBmfjh07iBrNt1Dqetx/oZrVcxShkLEe+vT+9+hCCnIFYP/wN2/dQpvWrURN0rEVVdgSyrq6uqJGPQ0YOBjz3OfI6rq4pq0gc3LHSjglYKUfaq4rAOubX/TyVsgNJqVKl0FQsHpPILFnz140adyIBr5oCPorKUib1q3x37ZtopR0bIrm4OAgUVI/7CTjmbNnYGdXS9QQdUdBriAVbKzh5eOX7GxeIL8J7t1X37vR2MCXnj16ihLRBBTkCqSobP5NTU+TsGWj7t+/RwNfNAwFuQIpKpung3oG+eJFCzGIBr5oHApyBVNENlfHKZrZajJs3QFjYyNRQzQFBbmCsWx+NTAkWdlcHadoXrBgIfr26y9KRJNQkCtBHfta2LdvvyglXskSJRB0VX0mkAgICIC51A/Xy6Le1+7Jn1GQK0H16tVw5uy5JGdz3lwPfy5KKW/5qjVwcHAQJaJpKMiVhA0WSU42T5NOPSaQOHb8OBrUrU0DXzQY/eWUJLnZnK2umdyz9MnFvv/hI8dQv359UUM0EQW5EiUnm+c3yZviUzSvX78eXRw6ihLRVBTkSsSy+YUkLupvbl4yRadoZuu+BYeEonTp0qKGaCoKciWrX68uDh0+IkoJl9JTNM+bPx8DBtAlMzmgIFcyls2TEuRssoG37z+Jkmo9ffoMbGStSb58ooZoMgpyFUhqNk+TQsNb582fh36OlMWTY9PKhTi4YyPfbl/Xln/18zyBRZNUvxAkBbkKJDXI00t/HVVP0Rwi9cNNTPLTwJdkauvQA3Ubt+Tb63Ye5l/LWFdCnxGT+bYqUZCrSFIC3bSY6qdoXrFyBXp07yZKJKnSaWXiDyaDrgH/+nOdKlGQq0hSgrykublKp2j28DiBalWrKWSuOqI+KMhViAX66dNnROnfihcrits3b4iScrGBL3v37UfTpk1EDZELCnIVSmw2Zxn14+evoqRcmzdv4Qs5EvmhIFexihVsE5XNY5BGbCkPG/hyNTCIT0hJ5IeCXMUaN27Em8UJlVlbiwehMi1eshT9+/cTJSI3FOQqxu7mqla1SoKzeUmLUjzLKsuz5+GIfveOBr7IGAV5CkhMNlf2FM3z58/DAEdHUSJyREGeAhKTzU3y5VXaFM1s4EvevPlgYKAvaogcUZCnEJbNjx4/IUrxYwcEZU3RvHLVShr4kgrQWmgpaMvWrShQsDCf/PFvxo4bh8mTJomSYpw9fwHhz56iRYvmooZcvHQJ3l5eopQ8L1++wqFDhxD19i3s7e1gWqSIeEZxKleuDCsrK1GKHwV5CmIDUJyGjYD77Jmi5s/YAvZt27blc78pihwXLExJ7B6D854XcPrMGX4Hn2HO7KhTuw5MTRUf3IlFQZ7CEpLNT4q+e83q1fjX5Nq4cRNMChRE1coVRQ1Jirv37uPo0SO4f/8h0qaRMmvVKtLvtDK/TVidUJCnsIRkc7bQwlbpYDBw4ABRk3Tsmvv4CS6Y6TZD1JCEYr+7EydPwc/XF1Hv3qOYaWHUq1cPefPkEXuoJwpyNZCQbO7i4io9JohS0s2a647mTZqgUKGCoob8TUBgME6f9MCT5y+QNbMOqlatjooVbDSqm0NBrgYSks3HuUzEJJfxopQ0r19HYPGSJRgzepSoIb97+eoVTp06jcDAQF62sLBAjRrVkT1bNl7WRBTkauK/bdtRpHDheMePT3CdBNcJ40QpaVxdJ0pN/oF0Xfwn7AB74aIXzp49jTdv3yFnNn3Y1a6L0hbmYg/NR0GuJtjZ2REjR2LO7Nmi5ler16xBzZq1+PrlSXH79h3s2rsXzkMGi5rU6+GjRzh8+DBu3bkPba30sCpXDrVq1lC7E2aKQkGuRtauXSs1D0v9MZt7efvgxYsXaFC/nqhJnGHDR2DqlMmpckIIdsLs7PnzOM8Wu5D+2/PmyQ07Ozu1uLylChTkauRv2Tw6Ohrz58/HqFGJ70+zgS/3795Bx44dRI383bhxU8rWh/DiVQRfcrl6tWqoXKliqjzIUZCrmb9l83EukzDJJXH9ctbnHDTYSfYDXyIio+DhcRy+Pj6IkaKaTZ1VX2r1aPIJM0WhIFczf8vm410mYmIiz7Dv3LkLOY2MZTfwhR28fP0u49zZM4iMiuazy7JzFqVLl6JRfL+hIFdDLJtbWlmhlIWFqIk1YcIEjB07NsFNzn+dzNM0bNGH4x4eUlP8Br5+/Qpr63KoUdOOpo/+BwpyNcSCc8zY8XCbMU3UxNry3zbeDLUombDLO0uWLkO1qlVRooSZqNEs7PcQNx6cbRsb5kS9evVTzQkzRaEgV1MrVq5EpYqVfgnQwKBgXLsWhpYtWoia+LGBL7Nmz8KUyaqfzD854k6YPXv+AunTp0P16tVT7QkzRaEgV1N/yuaszs3NDWPGjBE18Zs4eQp69+oFIwXeuaYMrD/N+tX+/peln+8L8uXLqxHjwTUJBbka+1M2T8jJNzbwZfeePRjqNETUqBd//ytS3/o4XkVEIouONuxr10E5K0s6YaYkFORq7E/ZPCGX0YaPGAVX6UCgLiO42HjwQ4cOIywsTGqCp5fFeHBNQkGu5n7P5jNmzODjz+MLYD+/ywgNDU3RgS/s4OTl7ft9PHhuo5yoVqOWrMaDaxIKcjXHAmaC60RMnRw7/dNBKSPmyJEDNuX/fFtqSs34Ejce/P7Dx9BKnxZly1rKejy4JqEg1wCLFy+Bnb0dipqa8tlIzp49C4dOHcWzP+zZsxeZM2eGnV0tUaM8cRMo+Pj44BvSwCRvbjphpqYoyDXAz7O5sJFe46XMPtnVRTwbK7b/Pk7qv08XNYoXHHoNJ44fxeNn4d8nULApX44ub6k5CnINMX/BAtStW5dn8z9NIKGMgS9xEygEBATg48eP/IQZjQfXPBTkGuLnbD5y1GhMnzZVPBN7rZll8Cmi355UcePBjx87yucwy6avB3s7exoPruEoyDVIXDY/ePAgOjl0Ro5sBrx+2vQZ6NatW5IGvrATZkePHsU9qa+fTgpkNoFClarVaDy4jFCQa5C4bF6vfn1eZlM033/wAJs3b8bIESN43b+w97jk5c0nUfj04SNy5TL6Pj84y+SUseWHglzDLFy0CMbGuXHgwD7UqlkLodeuYdyY0X+9VBU3Hvzp83BpPx0+FpzGg6ceFOQaZsuWrWjfvh0MsmXD26goVK1alV+f/jlgWR/91EkP+Pj4Il26dFKWNpX61nbSwcFI7EFSEwpyDcPWE2/QoD5uStn53bto2NhWgK2NjRT47XHy5Ake4DqZtFCzlh2NByccBbkS+Pn54bzU51WWZ8+eY9mypfyy1qdPn3i2LmdtjRrVayB7dsVd3sqSNSu6de0qSkRTUZArwZy57qjZsocoaa7NqxfDbcJwUSKaitpyhMgcBTkhMkdBTojMUZATInMU5ITIHAU5ITJHQU6IzFGQEyJzFOSEyBwFOSEyR0GeAp48uIPJQ3ugT+vaGNGzNUL8L4lnEm/1rHEIv39DlGKdOrQLl04fESWS2lGQq9jn91EY0r0NWnbsjqXbjmHw+BkY59QHES+eij0S5/r1a3gbHS1KscqWrwiLsuVFiaR2dIOKEvztBpVDOzchLNAfTi6zRI2U2W+HIEeufLh/7x7mTR6BdOkz8MkdXOcsw7GDe3lWficFckREBCpWrYnLPhcB6c+2aMthjOzVGp8+f+JTN2XQyoRJ81bjyL4d0NHNzN/b8/g+vHr9Cs+fPZPebwUKFy+JWROG4tHdm/gm/eX7DXdFsZJl+L6/oxtU5IEyuYo9uH8fxiaFRClWrkIlkEE7C2a7OGO46yzMXbsHlta22LJmKSLfvIGhdACYvWY3ipuZwzBPfizafAjppQNBXDO9dec+mLlqFwoUKYZDe7bx17x5E8kfmQ1yYs6aPXDoMwTHD+zEeY8DSJMGmLtuL6YuWAO38U78PYh8UZCrmJGxMd68ChelWLfCgnhznU2BnLdw7JTKhYpb4N6NUL5tUqQ4/5pJRwd58ubj29ramfhXppRFKf61RClLPH70iG/HYQcFRkdbG58/RON6WBh8L3nCuXtLTHDqhcyZs/DniXxRkKtYdft68Di453sfnPXRJzj1xNvI18iaJTNePHnA6+/eDEOeAglbbP/mzZuxX0MDkfsfK5jkM8mP8hWrYtaqHZi2cA2sy9uIZ4hcUZCrmH4OYwwfOxH9OjSBc7fm6NzUDq3ad+EZ3Nl1DkY6dsa4/h1x+thBtOnSB1oZWP88dpLGjBkz8lVBGa2MsZmcfd24cgF/za3rIajftPX317AH22YyZNDir69ZtxHCnzzkZ/X7d26BPCYF+PNEvujEmxIkdGaYz9GvkUE3du70n3399AHptH40xxMivveKT0K+B514kwfK5CkovqBMbIAziQlwJinfg2gmCnJCZI6CnBCZoyAnROYoyAmROQpyQmSOgpwQmaMgJ0TmKMgJkTkKckJkjoa1KsHFS5fg7eUlSsrh6emJSpUqiZJysNtZHR37iRLRVBTkGmrevHkYNGiQKBESP2quEyJzFOSEyBwFOSEyR0FOiMxRkBMicxTkhMgcBTkhMkdBTojMUZATInMU5ITIHAU5ITJHQU6IzFGQEyJzFOQaaNmy5Rg1ejQqVamKb9++iVpC/oyCXAM5OHRCNgMD9OzeDWnT0p+Q/B3dT66htu/chaaNGyGDWNCQkPhQkCtZJwcHlLOyEiXNkCVrVnTr2lWUiKajIFcyTZzBhWadkRfq0BEicxTkhMgcBTkhMkdBTojMUZCrmc+fP+P9+/eiREjyUZCridevI9ChYydUq14dTZu1QPMWLXhdfB4+fIS16zaIUsKw16xbt16USGpBQa4mGjdpigYNG+GCpyeOHjmEWrXsMGz4MP7c48dPvg9fff48HO/evcO69etw794dnvnZ80+fPsPde/f5PsyfXrN02VI8efqjnqQOFORq4NTpM9A3MED7tq1FDdCrZw+cPHmSb9dv0ABfv37l242bNMGjR4+xadMmPJMCmwWzbYUKWLRoIfr164s1a9by/WrXrcO/MvXq18fDR4+wdet/eB7+4vt7kdSBglwNhAQHo3JFW1GK9e7DJz5kNSIyCjo6OnybZe2PHz/C1LQItLV1MG+eO4Kk19arVw+TJk3CnDlzsW37dh7QuYxz8XHt7PXaOtooamoK3cy6mOU2g4bCpjIU5GqABeHvJ9t279yOWjVrwcv7Esrb2PA6L29fPkQ2LnBZsPp4e6Fx48b8+fdSk9zYyAi+vr6wtSnP69jrLcuU5U12w5yGdENLKkR/cTXQoEFD7Ni5CyEhobx84cIFzJo9B6NHj8bN6zdgVrw4z+Izpk+TAr48goKCYG5mxvf1vHBRarI/4v1sd3d3dOjQgbcMTKXMzQ4cM93cUM7aGv4BV2BesiR/DUldKMjVgJFhTmxYvw7Ow4ajWo2aWLFyFQ7s34e8efPAzt4eO3ftgpPTUNjY2qJypUrImTMH7t57AG8fX4S/CMer1xFo2KgxyleoCDu7WrzfvnXbdn7POWsFVJBeZ2xoiNDQUPj7XxHflaQa7AYVojxSdhVbinfn7r2Y5i1aipLiKPMzE9WjTK7B3kVHo7NDJ1Ei5M8oyDVYiRJm30+6ERIfCnI1FS1laUVgJ+zYg6ReFORqaMbMWQgODhalX1WpmrDJGy9evMgH2byJisLgwUNELUmNKMjVzPnz5/E26g0sLCzw8tUrURs7TPVVRCTYND7smvfPZ8lZpvby9sGNGzd5OTIqGu7z5iH6/Udkz5YNV4OCeD1JnSjI1cwEF1c4DXHCzVu3MXCQE6+7/+ABHDp3hreXF16/es2HsLpOdMWmTZv5TSw1atXCsaNHMHbcWMyaOw8hwYF8SOz9u7f469OlTYM3b97wbZL6UJCrkeDQa8iqrw8DA30UL1YUd+/EZuZRo0Zj8qSJCAy8ijZt2/EhrLXt7RH1NgqfpCw+fbobWrVug5q17HEjLARmZuYwNS2Kfn378tczX758EVsktaEgVyN+vj6oKIajxo4vTwM/v8u8bGtri0uXvNCiRQte9vHxhU15G6xduxbz3Odg/769OOlxjI9u+3koLBMeHg596eBBUicKcjXy+tVLGBkZiRKQO28+9B8wANOmTeXlO7dvw6yYKd8OlPrZJUuaY50U5GvXrEGvPv0Qdu0aLC2tcDUgAJZly/L9bt+5i0KFTWnMeipGf3k1YmhohIiISFECihUphDp1asMkXz5+P3i9Bg14sLJbRdnwV5btXaWme5u27TFJ6qOzMfC5jI1RRGqqb968id/IMn/+PAwbFntfOkmlxMg3oiSJGSJ6/fqNmL79+sVIQRxzNTAopmKlyjERb96KZxMv7Nr1mH6O/UUp4WhYq7xQJlcj7D5xkwKF+d1js2bO5FlYL4uueDbxPI57wH3uHFEiqRUFuZoZOcwJurq6WLduLawsLUVt0jg69qUJIggFOSFyR0FOiMxRkBMic7SqqZLNdXdH2jRpREkzVK5cGVYattwyiR8FOSEyR811QmSOgpwQmaMgJ0TmKMgJkTkKckJkjoKcEJmjICdE1oD/ASDmkV8TvtshAAAAAElFTkSuQmCC)\n",
        "<br>\n",
        "To summarize, many popular ensemble algorithms are based on this approach, including:\n",
        "\n",
        "> AdaBoost (canonical boosting) <br>\n",
        "> Gradient Boosting Machines <br>\n",
        "> [Stochastic Gradient Boosting (XGBoost and LightGBM)](#cell-3) <font color='lightgreen' size=\"1\">Which has been used</font>"
      ],
      "metadata": {
        "id": "8Exy-lcknnh-"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## <font color='orange'>Dependencies</font>"
      ],
      "metadata": {
        "id": "H1z0xVqgu0PC"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!unzip \"/content/drive/MyDrive/computational intelligence course/dataset.zip\" -d \"/content/drive/MyDrive/computational intelligence course/dataset/\""
      ],
      "metadata": {
        "id": "rx9DgL9o8abn"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "4lQwwdrPISI8"
      },
      "outputs": [],
      "source": [
        "import pandas as pd\n",
        "import numpy as np\n",
        "from sklearn.preprocessing import StandardScaler\n",
        "from scipy.spatial import distance\n",
        "import lightgbm as lgb\n",
        "from sklearn.decomposition import PCA\n",
        "from sklearn.metrics import confusion_matrix, accuracy_score\n",
        "from sklearn.metrics import accuracy_score\n",
        "from sklearn.model_selection import GridSearchCV\n",
        "from sklearn.pipeline import Pipeline\n",
        "from sklearn.ensemble import RandomForestClassifier\n",
        "from sklearn.ensemble import StackingClassifier\n",
        "from sklearn.tree import DecisionTreeClassifier\n",
        "from sklearn.svm import SVC, LinearSVC\n",
        "from sklearn.linear_model import Perceptron\n",
        "\n",
        "path_dataset = '/content/drive/MyDrive/computational intelligence course/dataset/'"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## <font color='orange'>Load Dataset</font>"
      ],
      "metadata": {
        "id": "sUiSSztouoi7"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def read_features(path):\n",
        "   '''\n",
        "   Returns the list of features of a face that are read from a text file\n",
        "   Parameters:\n",
        "       path (str): path of feature_file.txt\n",
        "   '''\n",
        "   try:\n",
        "     f = open(path, 'r')\n",
        "     lines = f.readlines()\n",
        "     f.close()\n",
        "   except OSError:\n",
        "     print(\"Could not open/read file:\", path)\n",
        "     return\n",
        "\n",
        "   features = []\n",
        "   for line in lines:\n",
        "     line = line.replace('[[', '').replace(']]', '').strip()\n",
        "     feature = list(map(lambda a : float(a), line.split()))\n",
        "     features.extend(feature)\n",
        "   return features\n",
        "\n",
        "\n",
        "def create_df(path):\n",
        "  '''\n",
        "   Returns test/train data frame that contains numerical features of face_1 and face_2,\n",
        "   and subtraction of these numerical features,\n",
        "   and label that indicates whether these two faces belong to one person or not\n",
        "   Parameters:\n",
        "       path (str): path of pairsDevTrain.txt/pairsDevTest.txt\n",
        "  '''\n",
        "  try:\n",
        "      f = open(path, 'r')\n",
        "      lines = f.readlines()\n",
        "      f.close()\n",
        "  except OSError:\n",
        "      print(\"Could not open/read file:\", path)\n",
        "      return\n",
        "\n",
        "  x1 = []\n",
        "  x2 = []\n",
        "  labels = []\n",
        "  n = int(lines[0])\n",
        "  for i in range(1, n+1):\n",
        "    line = lines[i]\n",
        "    parts = line.split()\n",
        "    path1 = path_dataset + \"lfw/\" + parts[0] + \"/\" + parts[0] + \"_\" + str(parts[1]).zfill(4) + \".txt\"\n",
        "    path2 = path_dataset + \"lfw/\" + parts[0] + \"/\" + parts[0] + \"_\" + str(parts[2]).zfill(4) + \".txt\"\n",
        "    x1.append(read_features(path1))\n",
        "    x2.append(read_features(path2))\n",
        "    labels.append(True)\n",
        "\n",
        "  for i in range(n+1, 2*n+1):\n",
        "    line = lines[i]\n",
        "    parts = line.split()\n",
        "    path1 = path_dataset + \"lfw/\" + parts[0] + \"/\" + parts[0] + \"_\" + str(parts[1]).zfill(4) + \".txt\"\n",
        "    path2 = path_dataset + \"lfw/\" + parts[2] + \"/\" + parts[2] + \"_\" + str(parts[3]).zfill(4) + \".txt\"\n",
        "    x1.append(read_features(path1))\n",
        "    x2.append(read_features(path2))\n",
        "    labels.append(False)\n",
        "\n",
        "  x1 = np.array(x1)\n",
        "  x2 = np.array(x2)\n",
        "  sub = x1 - x2\n",
        "  df = pd.DataFrame(columns= ['x1', 'x2', 'subtraction', 'label'])\n",
        "  df.x1 = x1.tolist()\n",
        "  df.x2 = x2.tolist()\n",
        "  df.subtraction = sub.tolist()\n",
        "  df.label = labels\n",
        "  return df"
      ],
      "metadata": {
        "id": "tbc8JSo8CD5q"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "train_df = create_df(path_dataset + \"pairsDevTrain.txt\")\n",
        "test_df = create_df(path_dataset + \"pairsDevTest.txt\")"
      ],
      "metadata": {
        "id": "1TAkG71loZ1z"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "train_df.to_parquet(path_dataset + \"trainset.parquet\", index=False)\n",
        "test_df.to_parquet(path_dataset + \"testset.parquet\", index=False)"
      ],
      "metadata": {
        "id": "L07g5FkgAmdF"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## <font color='orange'>Preprocess & Extract Distance Features</font>"
      ],
      "metadata": {
        "id": "dvhFgmH4u442"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "train_df = pd.read_parquet(path_dataset + \"trainset.parquet\")\n",
        "test_df = pd.read_parquet(path_dataset + \"testset.parquet\")"
      ],
      "metadata": {
        "id": "aU9h88jVsLMD"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "stdScaler = StandardScaler()\n",
        "def normalize(df):\n",
        "  df['x1'] = stdScaler.fit_transform(df['x1'].tolist()).tolist()\n",
        "  df['x2'] = stdScaler.fit_transform(df['x2'].tolist()).tolist()\n",
        "  df['subtraction'] = stdScaler.fit_transform(df['subtraction'].tolist()).tolist()\n",
        "  return df\n",
        "\n",
        "  \n",
        "train_df = normalize(train_df)\n",
        "test_df = normalize(test_df)"
      ],
      "metadata": {
        "id": "hKKut7C_Lr1G"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# def do_pca(df, n=3):\n",
        "#   pca = PCA(n_components=n)\n",
        "#   df['x1'] = pca.fit_transform(df['x1'].tolist()).tolist()\n",
        "#   df['x2'] = pca.fit_transform(df['x2'].tolist()).tolist()\n",
        "#   return df\n",
        "\n",
        "# train_df = do_pca(train_df)\n",
        "# test_df = do_pca(test_df)"
      ],
      "metadata": {
        "id": "wI4DjpE_32wY"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "We use these 3 distance measures as features:\n",
        "<br>\n",
        "\n",
        "![distances.jpg](data:image/jpeg;base64,/9j/4AAQSkZJRgABAQEAYABgAAD/4QAiRXhpZgAATU0AKgAAAAgAAQESAAMAAAABAAEAAAAAAAD/2wBDAAIBAQIBAQICAgICAgICAwUDAwMDAwYEBAMFBwYHBwcGBwcICQsJCAgKCAcHCg0KCgsMDAwMBwkODw0MDgsMDAz/2wBDAQICAgMDAwYDAwYMCAcIDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAz/wAARCADtAsYDASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD9/KKK+J73UfjN+0Z/wUA+N3gjwv8AGzVPhv4Y+G9n4elsrOz8NabqXnNf2kskpZ7iJn+9Dn738dNK4nKx9sZozXy//wAMf/H/AP6Oy8Tf+EHoX/xmj/hj/wCP/wD0dl4m/wDCD0L/AOM0+WPcnmfY+oM0Zr5f/wCGP/j/AP8AR2Xib/wg9C/+M0f8Mf8Ax/8A+jsvE3/hB6F/8Zo5Y9w5n2PqDNGa+X/+GP8A4/8A/R2Xib/wg9C/+M0f8Mf/AB//AOjsvE3/AIQehf8Axmjlj3DmfY+oM0Zr5f8A+GP/AI//APR2Xib/AMIPQv8A4zR/wx/8f/8Ao7LxN/4Qehf/ABmjlj3DmfY+oM0Zr5f/AOGP/j//ANHZeJv/AAg9C/8AjNH/AAx/8f8A/o7LxN/4Qehf/GaOWPcOZ9j6gzRmvl//AIY/+P8A/wBHZeJv/CD0L/4zR/wx/wDH/wD6Oy8Tf+EHoX/xmjlj3DmfY+oM0Zr5f/4Y/wDj/wD9HZeJv/CD0L/4zR/wx/8AH/8A6Oy8Tf8AhB6F/wDGaOWPcOZ9j6gzRmvl/wD4Y/8Aj/8A9HZeJv8Awg9C/wDjNH/DH/x//wCjsvE3/hB6F/8AGaOWPcOZ9j6gzRmvl/8A4Y/+P/8A0dl4m/8ACD0L/wCM0f8ADH/x/wD+jsvE3/hB6F/8Zo5Y9w5n2PqDNGa+X/8Ahj/4/wD/AEdl4m/8IPQv/jNH/DH/AMf/APo7LxN/4Qehf/GaOWPcOZ9j6gzRmvl//hj/AOP/AP0dl4m/8IPQv/jNH/DH/wAf/wDo7LxN/wCEHoX/AMZo5Y9w5n2PqDNGa+X/APhj/wCP/wD0dl4m/wDCD0L/AOM0f8Mf/H//AKOy8Tf+EHoX/wAZo5Y9w5n2PqDNGa+X/wDhj/4//wDR2Xib/wAIPQv/AIzR/wAMf/H/AP6Oy8Tf+EHoX/xmjlj3DmfY+oKK+Qvg3qXxa+Dv7euk/D3xj8XtQ+JWg614UuNZ2Xfh7TtMa3njnWNcNbxKzfL/AHmr69pONioyuFFeGf8ABRH4l+JvhN+yjrmseD9ak8PeIften2VrqK20Vy1r597BAziKVWjb5ZG+8K46z/ZI+P8AcWkcn/DWXiYeYin/AJEPQv8A4zTsK59SZozXy/8A8Mf/AB//AOjsvE3/AIQehf8Axmj/AIY/+P8A/wBHZeJv/CD0L/4zRyx7i5n2PqDNGa+X/wDhj/4//wDR2Xib/wAIPQv/AIzR/wAMf/H/AP6Oy8Tf+EHoX/xmjlj3DmfY+oM0Zr5f/wCGP/j/AP8AR2Xib/wg9C/+M0f8Mf8Ax/8A+jsvE3/hB6F/8Zo5Y9w5n2PqDNGa+X/+GP8A4/8A/R2Xib/wg9C/+M0f8Mf/AB//AOjsvE3/AIQehf8Axmjlj3DmfY+oM0Zr5f8A+GP/AI//APR2Xib/AMIPQv8A4zR/wx/8f/8Ao7LxN/4Qehf/ABmjlj3DmfY+oM0Zr5f/AOGP/j//ANHZeJv/AAg9C/8AjNH/AAx/8f8A/o7LxN/4Qehf/GaOWPcOZ9j6gzRmvl//AIY/+P8A/wBHZeJv/CD0L/4zR/wx/wDH/wD6Oy8Tf+EHoX/xmjlj3DmfY+oM0Zr5f/4Y/wDj/wD9HZeJv/CD0L/4zR/wx/8AH/8A6Oy8Tf8AhB6F/wDGaOWPcOZ9j6gzRmvl/wD4Y/8Aj/8A9HZeJv8Awg9C/wDjNH/DH/x//wCjsvE3/hB6F/8AGaOWPcOZ9j6gzRmvl/8A4Y/+P/8A0dl4m/8ACD0L/wCM0f8ADH/x/wD+jsvE3/hB6F/8Zo5Y9w5n2PqDNGa+X/8Ahj/4/wD/AEdl4m/8IPQv/jNH/DH/AMf/APo7LxN/4Qehf/GaOWPcOZ9j6gzRmvl//hj/AOP/AP0dl4m/8IPQv/jNH/DH/wAf/wDo7LxN/wCEHoX/AMZo5Y9w5n2PqDNGa+X/APhj/wCP/wD0dl4m/wDCD0L/AOM1j+P/ANmT9oLwd4I1jVo/2rPEkz6XZy3axnwJoarJsRmx/qfajlj3DmfY+uKK8v8A2NPiBq3xV/ZP+HfifXLn7drWvaBaX19ceWkfmzSRKzttQKoy3ZVr0bXrh7TQ7yaNtskUDup9CFJFSWWqK+Df2KvhZ8f/ANqX9lHwH8RL79p7xPpd54u0mLUZbSHwjpMkduz5yqsYwSBjqRXqH/DGHx0/6Ou8Xf8AhHaR/wDG6vlS0uZqbavb8j6ior5d/wCGMPjp/wBHXeLv/CO0j/43R/wxh8dP+jrvF3/hHaR/8bpcq7lcz7fkfUVFfLv/AAxh8dP+jrvF3/hHaR/8bo/4Yw+On/R13i7/AMI7SP8A43RyruHM+35H1FRXy7/wxh8dP+jrvF3/AIR2kf8Axuj/AIYw+On/AEdd4u/8I7SP/jdHKu4cz7fkfUVFfLv/AAxh8dP+jrvF3/hHaR/8bo/4Yw+On/R13i7/AMI7SP8A43RyruHM+35H1FRXy7/wxh8dP+jrvF3/AIR2kf8Axuj/AIYw+On/AEdd4u/8I7SP/jdHKu4cz7fkfUVFfLv/AAxh8dP+jrvF3/hHaR/8bo/4Yw+On/R13i7/AMI7SP8A43RyruHM+35H1FRXy7/wxh8dP+jrvF3/AIR2kf8Axuj/AIYw+On/AEdd4u/8I7SP/jdHKu4cz7fkfUVFfLv/AAxh8dP+jrvF3/hHaR/8bo/4Yw+On/R13i7/AMI7SP8A43RyruHM+35H1FRXy7/wxh8dP+jrvF3/AIR2kf8Axuj/AIYw+On/AEdd4u/8I7SP/jdHKu4cz7fkfUVFfLv/AAxh8dP+jrvF3/hHaR/8bo/4Yw+On/R13i7/AMI7SP8A43RyruHM+35H1FRXy7/wxh8dP+jrvF3/AIR2kf8Axuj/AIYw+On/AEdd4u/8I7SP/jdHKu4cz7fkfUVFfLv/AAxh8dP+jrvF3/hHaR/8bo/4Yw+On/R13i7/AMI7SP8A43RyruHM+35H1FRXy7/wxh8dP+jrvF3/AIR2kf8Axutv/gmh8R/F3xG+A+sN428STeLNa0XxPqWkf2nNaQ2r3EUE2xMxxKqA49BRy6XQKWtmj6HoooqSgPSvkv8AZR/5Stftaf8AYP8ABX/pBeV9aHpXyX+yj/yla/a0/wCwf4K/9ILyqjsyJbo+tKKKKksKKKKACiiigAooooA+B/2e/wDgof8AtFfth+Jfih/wrj4V/CttB+HPjbUPBzT654wvLW5untWX97sjsXUKyuv8XrX2V8NvGWpahomk6d4t/wCEf0zx5JpqXup6RpmoNdRW53bWaJnRJHi3/LvZFr8lf+CY3wrg8b+LP2mLqT9pDxd8IWj+NGvJ/ZGl32mwRXH+q/fst1C8m5t237235K+k7bxxoXgn/gvT4a1DUvFFjNpeqfs5rZ2Os3t1EkerSxeIGZysvyo0m1g7Bf71AH3XH420iWHUpV1TTzDo8pgv3Nwu2ydVDFJDn5GCkH5v71czF+0F4a8a/DXxBrng3xF4f8S/2LazyF7G8S5jjljRm2vsPqK/Iz4k+M9L+IfwS/aCvodSjv8A4T65+19plt45vLWdvss3h/yNJS7810/5YNIsSu33dm7+GvsfxF8O/wBmfwb+1dJJ8Nbiz8P/ABIuPAWpefo/g2JIND1DT1ifZLqEVunlb0Zm8p2ZW3bl+b7tAHffsMftlfEL9p/4C/s++L9U0rwSi/EzRLjUfEbx372c9rKiM0QsrVt7SqzD5tz/ACr81dB+3v8At36H+y/+zr8StU8P+JfB918QvB+gXGq2eiXt4jySOi7hvhV1kZfpivz/AP2J/tUej/8ABN1rHd9uXwL4q+z4+9v/ALOl2/8Aj22uOj8DfA/WP+CDHxV13xzpvgW5+OTDXX8R3+qRQt4gj11b6VQvmv8Av0ZVWLainbtxtFAH63+DP2j9DsP2fPBfjDxvr3h/w3/wkmjWN/LJd3S2tv508KOVTe33dzcc121z4/0O08KNr0msaXHoax+ab43KfZ9n97fnbtr8fvFmgeKviH+3b8NdJ1K3+EGpaJb/AAT0R/C9n8TFlbSJJWhT7Y9sio8bT7dmd3zKv3ag8J+CtP8A2Y/2K7PwP4g1L4V/FrQ/Gnxc8jw/aWmrXS+EPCczbneC7dk+eBJPuw7Wj+6tAH67+EPjX4Q8f+FLjXtD8TaHq2i2u4z31pexywRbeu51OBVfQv2g/AvinxfH4f03xh4bvtcmg+0x2FvqMT3Dxf3wgbdivx50kL4X+Ln7dHh+zvPhv9jj+EVpdXVn4Cg+zaLHdbZ1ZvKX5fN2qqsyrXU6V+zf4D+Dn/BO79hXxt4X8JaDofjK68aeGluNds7NItSuluoJ/tCy3CjzHV/4lZmoA/ZCiiigAooooAKKKKACiiigD5g8bf8AKW3wX/2T6+/9Klr6fr5g8bf8pbfBf/ZPr7/0qWvp+qn0Jjuz53/4Km/8mb6v/wBhjRf/AE6WtfQGl/8AINtv+uSfyr5//wCCpv8AyZvq/wD2GNF/9OlrX0Bpf/INtv8Arkn8qOgdSxRRRUlBRRRQAUUUUAFFFFAHzH+19+2/4m+Fnx68G/B34W+D7Dxt8TvGVpPqzJqepNp+maHp8TbWurmVI5ZNrP8AKqqjM1O/Y/8A20fGHxM+P/jj4Q/FPwVYeDfiF4ItLXVFl0jUX1DSNcsbjcEmt5ZIopNysjKyMny15n498b6Z+z9/wW2tdY8Y3EOj6N8Rvh7FpOh6reMIrX7Xa3TyS2vmt8qyMrq23uoqn+3z+2tN8V/g9+0R4P8AgJaXmufFDwH4KS5/4STSIopY7d535s4pl+Zp1hEj7V+7lf4qAPsTw/8AHbwT4r8a3fhvS/Fnh2+8QWX/AB8adb38UlzD9UB3VH43/aB8DfDPxHY6P4i8X+G9F1XUji1s73UYoJ5/91GbJr8q/id4L+Bvhf8A4J4/s3698F7Xw3H8YJPFHhr/AIRzUNMSJfEN9dvcRLqC3bp+9dWgNx56y/L/AHq0P2kPhV4buPjb+094u09Pgf8AF2zuLho/Gnhz4io2meINDaCwRHt9PvXRtsDRLviZdiqzNtb+KgD9MtY+I/iK0/aJ0Dw/a2vhuTwjqOiXuoXd5Nqnl6nHcRSwrEsVts/eQsrvuk3fLtHqN1qP9pb4ez+I9N0dfHHhV9W1hFksbRdThaW8U9Ci7vm/Cvz9+C3i3R/Hn7dn7JOteHrDWtL0HUv2bdZuNOs9Xlae+tbdn0nYkzt8zyKuFZv4vvV8y2v7NHw/8J/8Gs/hL4h6f4P8P2vj6G20zWIvEqWSf2vHdLrkaq63ePNXbGAiqrbVVdv3aAP2s+Ifxo8I/CQWf/CU+JdD8P8A2+Tyrb+0L1Lfz39BuPNdFZ3cWo20c0Ekc0My70dG3KwPcGvyJ+NHg7xV8YP+Ctn7QGn+IrX4F6ktr4e0eLw1b/FAStHDpL27NcS6evlPGv8ApO/zXX5/lT+Gvs7/AIIzfDHWvg3+wvonh3VfH3h/4j2un6hdppGqaNcy3Nnb2PmfurVJJQrOsXzIp/u7aAPrCiiigAooooAKKKKACiiigArlfjn/AMkZ8V/9gi6/9FNXVVyvxz/5Iz4r/wCwRdf+imoA4H/gnd/yYl8I/wDsVdP/APRC1614n/5FrUP+vaT/ANBNeS/8E7v+TEvhH/2Kun/+iFr1rxP/AMi1qH/XtJ/6CaqW5NP4UfPv/BIP/lGR8E/+xXtv619IV83/APBIP/lGR8E/+xXtv619IUT+JhT+FBRRRUlBRRRQAUUUUAcf+0J8Um+B3wD8b+NVtPt7eENAvtaFsX2faPs1vJNs3c43bMZ7Zrx3/glP/wAFD9N/4Ka/sjaZ8SLXST4d1JrufT9V0dpfNfTbiNuFLYGQ8bRyA45WQV2n/BQP/kw342f9iHrn/pvnr8mfhd8b/Ev/AATE/Zz+HmueELG6vG/ar+DXh/T/AA1BCpKQ+OIbS0sITjGF863uIZWJ+8bNs0AfdX7FX/BYez/bR/4KQ/Fz4I6P4Ve08P8Aw30+e8sfEjXO5dcNvepYzmNMY8sXAnUOGOfK9+Pqzw38fvA/jHxjP4d0nxd4d1LXbXd5thbX8clwm3rlAc8d/SvyP/ZI+FWj/wDBP3/gqT8bPCSW+o6rpPws/ZgtDdrp+VvdUkjkS5u5EI586eZ5nyOd8ma4TR9Q0vQpv2FfFnhuP4G+EP8AhJviNoD6XpfhGRp/EsGn3kUgnivrsgPPlZAk2/OZGFAH7Q+Iv2mfh34Rnki1Txt4X0+WG6NjIlxqUUbJOMZjILcMMjj3rxn49/tz3HwY/b9+FHw9ur7w3Y+A/GvhjWdb1HU7x9jxPaCEw7JS4QI3mHOQc8YIr4Z/YM/Zc+Hnxq8bf8FCtW8XeD9B8Taja+OtTtLafUrRLlrOP7PI58neD5TFgCWTDHavPAqj+yT8ONC/aQ+Mf/BPW28e6XZ+LrVfhFrlwbfVYxdQyvEtsIy6NlX2hVwGBHAoA/XHwh8WvC/j/wAKPr2h+INH1bRY92++tbtJbdNvXLg4GPeqXg74/wDgf4hQahJoXi3w9q8ekqXvGtL+OYWqjqX2n5QPU1+U/i/WvCH7LMf7fulL4RjuvA9r4i8PRw+HLCdtN0+KS6ttjlvKAEMJba0m0AEA5q1+zb4es/DP/BZ74P8Ah8TfBM2+vfDnWYtV0n4e2wSxmgCIYo7sYCzEDO0sMkZoA+1rH/gr98H9R+AXxH+IUWro2m/DW9u7G9tftMH2q9aA43Qrvwyv/ASRnFezfDf9p7wX8S/gZpfxEtde0y18MapapdC6uLuNY7fcoJR23bQ65wRnqK/L3wl8C/hnP/wTN/bM0618H+B28Q6P4j8QlbaPSrb7ZZRRsDAQoXeiryUxgDnGK7H4+eMvhv4a/Y7/AGTPBOi+EfAeqR+NLy2+wTajMtr4S069jtN8k1+sI2TSE5wjjlySeaAP0o0T44eDfEvgm48Saf4o0G98P2efP1CG9je2hxwdzg4XqOtQeGv2gfAvjLxjJ4e0nxd4d1LXIVLPYW1/HJcKB1OwHPFfiT8RLhdG+Ev/AAUd8P2up+A7rT7TwroV09v4Kh+z6HHdPtWVooh8qvlQHKjkrX1p+1h+zf4B/ZsvP2JfEPgPwh4f8Ka8fiFpmlSalp1mkN3c21zplz58csoG+UPtBPmFjkZoA/TKiiigAooooAKKKKACvmX/AIJY/wDJGfGv/Y+63/6UV9NV8y/8Esf+SM+Nf+x91v8A9KKpfCyX8SPpqiiipKA9K+S/2Uf+UrX7Wn/YP8Ff+kF5X1oelfJf7KP/ACla/a0/7B/gr/0gvKqOzIluj60oooqSwooooAKKKKACiiigDxnxF/wT2+BfizxBfarqnwi+Huo6lqU73N1c3GhW8ktxKx3M7kr8zFq0/Gv7Fvwl+JHhDQfD+vfDfwdqmieF4/K0exudLieHTU4ykK4/drwPlXj5RXqdFAHEeGv2d/Afg/wXq3hrS/BnhfTvDuttnUNLt9Mhjs7390kX7yILsb93Gicj7qCqvwj/AGXPhx8BNO1Cz8FeCPC/he21f5b2PTtOitxdD+6+1fmHzHhvWvQaKAOQ0T4G+DfDJ8OnTfC+h2LeEYpINE8izjj/ALJR12ukOB8isvB21ynij9hb4N+NfHV54l1j4X+BtU8QagP9Kv7nRoJZrjjG52Zfmb/a+9XrVFAHn3xQ/Zd+HPxr8Nabovi3wR4X8R6Zo4VLC21DTop0slVdoEe5fkG0Y+WnXv7Lvw31H4Vf8ILP4D8JzeDN2f7FOlw/YQ397ytu3d/tda7+igDzXwX+x/8ACv4daZqFjoPw88H6PZ6pZf2dew2ekwxLeW/zfupNq/OvzHhs1vyfBrwnceGNF0STw3oraT4aniudKs/safZ9PliyIniTG1GXcduPu11dFABRRRQAUUUUAFFFFABRRRQB8weNv+Utvgv/ALJ9ff8ApUtfT9fMHjb/AJS2+C/+yfX3/pUtfT9VPoTHdnzv/wAFTf8AkzfV/wDsMaL/AOnS1r6A0v8A5Btt/wBck/lXz/8A8FTf+TN9X/7DGi/+nS1r6A0v/kG23/XJP5UdA6liiiipKCiiigAooooAKKKKAOS+LPwS8H/Hnwz/AGL408L6H4q0rf5i22qWaXMav/eUOPlb3FHwn+B/g74CeGjo3gvwtofhXSmfzGttLskto3f+8wQfM3ua62igDzLwb+x38Kvh/wDEaXxhofw78G6T4ouC7NqlppMMNzuk++Q6r8rN/EV+9UXxR/Ys+Enxs8Xx+IPF/wAN/BviTWowq/bdR0qGeZlX7odmX51Ho2a9SooA5sfC7w2/iTT9ZGg6Suq6TYvpdjdi1RZbW0fZvgRsfLGdifKPl+UVRb4CeCZfhZH4GPhXw+fBsKLGmi/Yo/sKqr7woixt4f5v96uyooA89+MX7LHw1/aEnsZPHHgXwt4rl03i1l1PTorh4B3VWYZC+3Sux8NeGtP8H6FaaXpNjaabptjEIbe0tYVhhgQdFRF+VR9K0aKACiiigAooooAKKKKACiiigArlfjn/AMkZ8V/9gi6/9FNXVVyvxz/5Iz4r/wCwRdf+imoA4H/gnd/yYl8I/wDsVdP/APRC1614n/5FrUP+vaT/ANBNeS/8E7v+TEvhH/2Kun/+iFr1rxP/AMi1qH/XtJ/6CaqW5NP4UfPv/BIP/lGR8E/+xXtv619IV83/APBIP/lGR8E/+xXtv619IUT+JhT+FBRRRUlBRRRQAUUUUAVdc0Sz8TaLeabqFrDe6fqED21zbzIHjnidSrowPBUqSCD1BrBl+CfhCfRvDenSeGdEksfB8sU2hwNZoY9JeJCkbQDGIyqnAK4wK6iigD8/f2M/2rtH8eftQftn/Gvx1rHhzw/8NPhb4hT4fWV/d6daW/2ODTbdDqEk12sfnzrLcSJtjd2A2oEXLYPndp+11+yb410L4c+PP2ZW/Z3kt/EHxi0Lwvr95r/hDWLe8Se58x447GGG1Wa3vX2EwTTolplH3OCvHj/h/wDYx8bfta/sl/8ABQP9nHw/caPD8QdN+OsvjHT7HVXMVrqdncy2epWsMhXlUuI7d1DMSu5sEgAkewfHH4I/tQ/t0+Gvg9rXiz9nDwl8JdT8F/GzwZ4hu9J07xdp2q6h/Y+mi8+13tzdI0cTxR+dEIbeMvKoMnDZqsJFTVHnfTD3vppJU3Obb0fxSTitY2cn7ug8R7tSqod6/wD5KpqnFJbPSL5tpXsvebZf/Zk/4OBv2SfFXxZ+P+gPHp3gvT/C95PqVzqdpomp3reMrGKJRealNDFYBofKkcxsjl3YKXHy5I9p/bS/ai+EP7N37PkkngO+0HR/idJ8J/Enif4WyWGhtdG2srTTGujcRt5LwQwjMLbZ9qSHCgOcrXFfAX9kP4peEP8AgoL+1Z4W8ZfC3SvEH7Pv7R0o1ZvGA8RWoNtnTUs5NPfTiDO7OS37z5UULnLbgB8Tf8EmP2OfiX8d/gZ+1tb6te2fjvVvhP4D1z9m34Z3FtIkMWpRw/bHcrJJJ5QLvLZxiTcqhQAThSa5KinPDW15vYqWm91CSlpumqrpK2l1KUorltbpw7hHEqTty+15bPblcouOuzXs1Uk30tGL15m/r7/glT/wVF/Zx/bf+FnhX4e65448E+LP2gvH3hCyvfHWmTaJ9mPiC+SyWSdWYW6Wk8iKXLRQsxQK/wAqhGx5z/wSi/bE+AP7JH/BJj4W/Hr47ah8P/A/iTXNS1vT4tct/DkNvf37Jql3AI4bbT7fzH2wxxhvLiIAALdSTgfsY/8ABPf9prxH8bP2O9D+JvwZ8G/DHwb+yXo13JJ4psPFVpqkviue4sI4Uto7aJRJbyLJzMX3RuySMsh+TfzXwO/4Ji/tNfsnfs//ALHPxA0X4R6N4++In7Pd94wh1r4c33iuw0+W4i1i9ne3ure+LS2ivGjI5JcsAygAtuC+jXlGU6klaPNJdLpRvPVRWq5tI2u+XSUrx0OHD3+rpSu5LldtnfkneLk9HZ8rbtZtqPxI+5/Dv7aX7Gen/s4+LP2n9N8T/D2HwF4o/wCJb4i8Tx2MizajJ0+yXFr5f2lrghgfIaLzSrA7SCDXzN4x/aY+Bvx0+Mv7HfgX4L3PgHWP2X/ixqPjWTxTpP8AYEEdlcTadpq3KGRbuEXFlJFKxkO3ymKuCdyMM+c6l/wSJ/aL1H4OXnxiHgHwTF8WLz4+2/xtk+EI12Eae1tBE8K6eL9cW32xi5kaU/uiQGyDlD1f7av/AATU+O//AAVl8WfA+7+I/wAK9M+CNhLceOF8VR+F/FNrd3XhmLUNHtrawmvJ43i+3zTXELLKsCMjQ7Y5MKWJ548ybnbWKb5bpq/sk7X2n78pRVrJOCbeqZtTs3yt+61UXNZ/9PFFpbxdowkr3u5WWqOt+OX7R/wj8Jj9mXx98EdP8M3X7PHxj8e/8Ks8ZWEPg2Oys/EsdxHLFZTb7q2jnaO3u4XIeNvLky/LgcfpZ4g+Gnh/xVHo6alo2nXy+H7lLzTBNAr/AGGZFKpJHn7rBWIBHQE1+UP7TMPxc1v9k39hf4KfEvwNpfg/4qR/GfQtPmstOu7S4s73TNCSWV9TtktpCscTW6ROY2CGMll2KNtfr1W0oQjCag7pVJJPvHkpSXprKWj1Xwvayx5pOUeZWbgm12fPUT9dIpXWkrJrRhRRRWJoFFFFABRRRQAV8y/8Esf+SM+Nf+x91v8A9KK+mq+Zf+CWP/JGfGv/AGPut/8ApRVL4WS/iR9NUUUVJQHpXyX+yj/yla/a0/7B/gr/ANILyvrQ9K+S/wBlH/lK1+1p/wBg/wAFf+kF5VR2ZEt0fWlFFFSWFFFB6UANLcUhbIpN2F65zXnnxQ/ar+HvwZuBb+JvFWk6bc94Wl3yr9UXJFZVK0KUeeo7I6sHgsRi6nscNTlOXaKbf4HojPhO9IGwK4f4YftF+B/jIn/FMeJtJ1Ztu7y4Zx5oHrsPP6V3GP8A9dOlUp1I80HdEYrB18NUdLEQcJLo00/xJKKBRWhgFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQB8weNv+Utvgv8A7J9ff+lS19P18weNv+Utvgv/ALJ9ff8ApUtfT9VPoTHdnzv/AMFTf+TN9X/7DGi/+nS1r6A0v/kG23/XJP5V8/8A/BU3/kzfV/8AsMaL/wCnS1r6A0v/AJBtt/1yT+VHQOpYoooqSgooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACuV+Of8AyRnxX/2CLr/0U1dVXK/HP/kjPiv/ALBF1/6KagDgf+Cd3/JiXwj/AOxV0/8A9ELXrXif/kWtQ/69pP8A0E15L/wTu/5MS+Ef/Yq6f/6IWvWvE/8AyLWof9e0n/oJqpbk0/hR8+/8Eg/+UZHwT/7Fe2/rX0hXzf8A8Eg/+UZHwT/7Fe2/rX0hRP4mFP4UFFFFSUFFFR3NzHZW7zTSJDFGCzu7bVUDqST0oBXeiJKK8j1n9vL4P6B4g/su68f6At5u2kJKZIwfeRQUH516T4S8a6P4+0WPUdD1Sx1awm+5cWk6zRt+KkiueliqNRuNOabXZpnpYzJswwlONXFUJwjLZyjKKfo2kmadFFFdB5pg6N8K/C/h3x7rPirT/Deg2PijxFFBBq2sW+nxRX+qRwArCk86qJJVjDEIHYhQTjGa3qKKOlg63AjIrnPhd8HvCPwP8Mtovgrwr4c8H6PJcyXjWGiabDp9q08h3SSmOFVXe55ZsZY8kmujoo63DyCiiigAooooAwdd+FfhfxR460PxRqfhvQdR8TeGFnTR9XutPimvtJWdQk4t5mUvCJFAVwhG4AA5Fb1FFHSweYUUUUAFFFFABRRRQAV8y/8ABLH/AJIz41/7H3W//SivpqvmX/glj/yRnxr/ANj7rf8A6UVS+Fkv4kfTVFFFSUB6V8l/so/8pWv2tP8AsH+Cv/SC8r60PSvkv9lH/lK1+1p/2D/BX/pBeVUdmRLdH1pRRRUlhRRQaAPn3/go3+05c/sx/s/3l/pcirrmqOLOxY/8s2bq/wDwEc1+NPiDX77xRrFxqGpXVxe3t07PLNK7O8jN/Exr9TP+C0PwwvfGf7PFjrVnHJN/wjl3506Ku7EbjaX/AOA1+U9fkvG1as8aqcvgS0P7i+jrl2Ajw/LF0kvbSm1N9dNl6W1+ZreCPHGrfDnxNa6vot9cadqNk4kimhfaVIr9qP2JP2if+Gl/2f8ARvEFwI11Jk8i8ROgmRtr/mRX4g1+rn/BIbwnfeD/AIFww3SyRteO91tf+EOdy/8AjtbcC1631mVJfBY8/wCklluAeUUcXJRVZSsu7Vndemx9kDpRQOlFfqh/FoUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFAHzB42/5S2+C/+yfX3/pUtfT9fMHjb/lLb4L/AOyfX3/pUtfT9VPoTHdnzv8A8FTf+TN9X/7DGi/+nS1r6A0v/kG23/XJP5V8/wD/AAVN/wCTN9X/AOwxov8A6dLWvoDS/wDkG23/AFyT+VHQOpYoooqSgooooAKKKKACijNGaACijNGaACiiigAooooAKKKKACiiigAooooAKKKKACiiigArlfjn/wAkZ8V/9gi6/wDRTV1Vcr8c/wDkjPiv/sEXX/opqAOB/wCCd3/JiXwj/wCxV0//ANELXrXif/kWtQ/69pP/AEE15L/wTu/5MS+Ef/Yq6f8A+iFr1rxP/wAi1qH/AF7Sf+gmqluTT+FHz7/wSD/5RkfBP/sV7b+tfSFfN/8AwSD/AOUZHwT/AOxXtv619IUT+JhT+FBRRRUlBX5uf8Fl/wBrbWIPGsPwy0W8msdOt7dLrVGiba10z8rGSP4QOo9a/SOvyM/4LF/D7UPDP7WE2sTQyf2frtnE9vKfusyDa6/ga+U4yrVqeXP2XVpP0/4eyP2zwBwOBxXFkFjUm4wlKCf86tbTq0rtel+h8nE5Ne5fsH/taa5+zL8Z9LaG8nfw7qtyltqVkzkxOjEKJAOgZc5z6ZFeG1ufDPwzdeMfH+j6bZxtJNc3UYG0fdAYEn8ACa/JcDWq0sRCdD4rq1vy+Z/cHEuBwWMyrEYfMUnScJc19kkm7+TW6fRq5+sv7dv/AAWL+Av/AATo8HQ33xH8ZW8etXkHn2Xh7TgLvVrwEZBEKn92p7PIUU9icVx3/BG3/grxZ/8ABVv4NeOPHc2h6f4I0zRPFUuh6VYz3wkuZbZLa3lEsrHA3s0zcKAAAByQSfkH/gpB/wAGuPhv/goVFN8SvAfjPUPBfxO1GLzL2HWJptR0nVpAuFyWYy2x7ZTegAAEY61h/wDBDb/g3K8N+EfgR480b9qL4VXDeONL8XTQ6dcprt3DbXmn/ZLYpLC1rOkckZkMvzEbgQVOCuB/Qp/lpK19D9mv+E20X/oL6X/4FJ/jR/wm2i/9BfS//ApP8a+P/wDiHo/ZF/6JbN/4U+r/APyVR/xD0fsi/wDRLZv/AAp9X/8AkqgR9gf8Jtov/QX0v/wKT/Gj/hNtF/6C+l/+BSf418f/APEPR+yL/wBEtm/8KfV//kqj/iHo/ZF/6JbN/wCFPq//AMlUAfYH/CbaL/0F9L/8Ck/xo/4TbRf+gvpf/gUn+NfH/wDxD0fsi/8ARLZv/Cn1f/5Ko/4h6P2Rf+iWzf8AhT6v/wDJVAH2B/wm2i/9BfS//ApP8aP+E20X/oL6X/4FJ/jXx/8A8Q9H7Iv/AES2b/wp9X/+SqP+Iej9kX/ols3/AIU+r/8AyVQB9gf8Jtov/QX0v/wKT/Gj/hNtF/6C+l/+BSf418f/APEPR+yL/wBEtm/8KfV//kqj/iHo/ZF/6JbN/wCFPq//AMlUAfZOna/Y6u7LaXtpdMgywhmVyo98GrdfnX8Kv2Hvhr+wL/wWE+Fej/CfRL7wrpfjD4eeJLjWLUaxe3kV/Jb3emCFmWeVxlBLJjGPvmv0UoAKKKKACiiigAr5l/4JY/8AJGfGv/Y+63/6UV9NV8y/8Esf+SM+Nf8Asfdb/wDSiqXwsl/Ej6aoooqSgPSvkv8AZR/5Stftaf8AYP8ABX/pBeV9aHpXyX+yj/yla/a0/wCwf4K/9ILyqjsyJbo+tKKKKksKKKD0oAx/Fvhu18V+H7nTryGO4tbqNo5I3XcrA9q/Or9oj/gkbDb+KLi88M6hNpdnM7P9neLz44/935lZV/76r9KVBps9rHOn7xFb615+PyvC4yPLiI3PpuG+Mc4yCo6mVV3Tb32afqndH5j/AAT/AOCXcen+I7a41Wa41h433KjxLFBu/vMu5t3/AH1WL8Pf+Djz9nX9mD4+eLvg78U9P8YfDfxB4N1a40i61KbT/wC0dKmaJ9isr2+6cbvvfNBtX+9X6nW2nQQHKQxj6CvwV/4K/f8ABuf8Uv2//wDgrV4y8S+A003w/wCD/EHh2DWp9a1LK2ragsTRfZV2/M0kkkW5v7ivuP8ACrVgctw+DhyYeFjPiLizNM9rKvmlZ1Gtuy9ErJH7Yfs5ftUfDn9r3wAnir4Z+MtB8a6B5nlNd6ZciUQSY3bHX7yPtYfKwDc16JX5+/8ABDL45+AvB3wyvP2eh8M7X4D/ABW+G43694KeRnbUA+P+JnbzP811FL8rb2LMu4L93ax/QKu4+dCiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooA+YPG3/KW3wX/wBk+vv/AEqWvp+vmDxt/wApbfBf/ZPr7/0qWvp+qn0Jjuz53/4Km/8AJm+r/wDYY0X/ANOlrX0Bpf8AyDbb/rkn8q+f/wDgqb/yZvq//YY0X/06WtfQGl/8g22/65J/KjoHUsUUUVJQUUUUAFB6UUHpQBGqHGcba5X4t/F3Qfgh4FvfEfiW/j0/S7EbnkILFj2AUcsT6V1QOBX5X/8ABZ/416l4m+Odl4PE0kekaDarceSD8sk0gzvP+6vH414+d5p9Qwjr2u9l6n3Phzwa+Js7p5a5csPik+vKt7eb2PXPE/8AwXW8N2esGPSPA+sahYq23z7i8jtnYeoQB/519Dfsoft2eC/2uLW4TQZLux1WxQPc6beIEnjDfxAqWV191P1xX4m103wg+LOs/BH4h6b4m0G5a31DTZd4/uyL/EjD+JWWvz3A8aYxV08Q7w9PyP6n4l+j1kFTLZrKeanXjHRuTak13v38rW/A/fw880Agiub+FHjeP4geA9M1VP8Al8t0lP8AwJc10eOcV+sQlzR5j+I61KVKcqct1oPoooqjMKKKKACiiigAooooAKKKKACiiigArlfjn/yRnxX/ANgi6/8ARTV1Vcr8c/8AkjPiv/sEXX/opqAOB/4J3f8AJiXwj/7FXT//AEQteteJ/wDkWtQ/69pP/QTXkv8AwTu/5MS+Ef8A2Kun/wDoha9a8T/8i1qH/XtJ/wCgmqluTT+FHz7/AMEg/wDlGR8E/wDsV7b+tfSFfN//AASD/wCUZHwT/wCxXtv619IUT+JhT+FBRRRUlBXk/wC1j+zDoX7TPgKTS9YtRM0eXglX5ZYH/vI3Y/oa9YoIzWdWlCpB06iunumdODxlfCV44nCzcJxd1JOzT7po/JnxZ/wSh1PRddkjg124+y7ztD2G5wPqGAP1wK95/Y//AOCfNj8ONWS8MU1xdNgSXdwnzY/uqOij8z7mvuWfS7e4bMkMbfUVJBaR2y4jRV+grysJw/l+Gq+2o00pd7t29Lt2Ptc88UeKM3wn1DMMXKVN7pKMb/4nGKb9G2iHRdKj0bTYreMbVjUACrVFFeyfAhRRRQAUUUUAFFFFABRRRQAUUUUAfIvxv/5TUfAb/sm3i3/0s0ivrqvkX43/APKaj4Df9k28W/8ApZpFfXVABRRRQAUUUUAFfMv/AASx/wCSM+Nf+x91v/0or6ar5l/4JY/8kZ8a/wDY+63/AOlFUvhZL+JH01RRRUlAelfJf7KP/KVr9rT/ALB/gr/0gvK+tD0r5L/ZR/5Stftaf9g/wV/6QXlVHZkS3R9aUUUVJYUUUUAFFFFABRjFFFAHyh/wUt/4J3H9rTQ9J8ceA9UbwT8evhzuvfBvii12rJ5i5b7Ddf8APW1l+6yt93cW/vKz/wDgmH/wUUX9tPwJqXh3xlpa+Cfjh8P5f7N8ceEp8xz2FwpZVuIlb5ntpdu9HXcvzFc5Wvqyvjr/AIKOf8E2dU/aC8UaP8Xvg34ii+G/7RHguNodJ8QKn+i61at9/T9QRf8AWwMv3WPKsF9toB9i0V8rf8E3v+CjUX7YOm654M8baM/w/wDjn8PXWy8Y+Erlvngfot1bN/y1tZfvI6+tfVNABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQB8weNv+Utvgv/sn19/6VLX0/XzB42/5S2+C/wDsn19/6VLX0/VT6Ex3Z87/APBU3/kzfV/+wxov/p0ta+gNL/5Btt/1yT+VfP8A/wAFTf8AkzfV/wDsMaL/AOnS1r6A0v8A5Btt/wBck/lR0DqWKKKKkoKKKKACiiigCPqM1+ZP/Bab4BX1j8S9P8dWdtNNYahbC0vWRdywunKbv94V+j3jrxnZ/DvwXqmu6kbj7DpNs93cGCJppNiDcdqLyx9hX5CeFP8Ag4g03/go1/wUn8F/ADwP4I+y+A9Yv5rPV9T1+AreXixpI2Et/wDlku5R9/5v9mvLzjLI4/DOg3Y+y4C4uq8NZxTzOmuZLSS7p7/5nzlXUfB/4X33xc8cWek2ccmySTdPKF+WFO7V+jXjv/gkp4Tvtaa4s9PkhRm3bYZ2WP8A75r074A/sM6P8Kwvk2Nvax9WCL8zf7zfxV8Lg+BayrxeJmuTy6n9JcQ/SPy+eXThldKft5LTmtZee7vY9T/Zx0N/D/wz0+3ZdqxxLGo/3RXoD9Kg06xi0u0SGIbVjGBVj7wr9NSsrH8fVKkpyc5bsKKKKZAUUUUAFFFFABRRRQAUUUUAFFFFABXK/HP/AJIz4r/7BF1/6Kauqrlfjn/yRnxX/wBgi6/9FNQBwP8AwTu/5MS+Ef8A2Kun/wDoha9a8T/8i1qH/XtJ/wCgmvJf+Cd3/JiXwj/7FXT/AP0QteteJ/8AkWtQ/wCvaT/0E1Utyafwo+ff+CQf/KMj4J/9ivbf1r6Qr5v/AOCQf/KMj4J/9ivbf1r6Qon8TCn8KCiiipKCiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooA+Rfjf/AMpqPgN/2Tbxb/6WaRX11XyL8b/+U1HwG/7Jt4t/9LNIr66oAKKKKACiiigAr5l/4JY/8kZ8a/8AY+63/wClFfTVfMv/AASx/wCSM+Nf+x91v/0oql8LJfxI+mqKKKkoD0r5L/ZR/wCUrX7Wn/YP8Ff+kF5X1oelfJf7KP8Ayla/a0/7B/gr/wBILyqjsyJbo+tKKKKksKKKKACiiigAooooAKKKKAPjP/gpl/wTz174zeL/AAt8cPg1fW/hn4/fC1Xl0id/ktPE1o3zS6Xe4+/G+PlY/dY++5fQP+Cev/BQvQP26Ph/eLJY3HhH4keE5/7O8XeENQ/d32hXq/fXafmaJsFkf+Ja+jK+K/8Agoj/AME0tS+KPjuz+OXwT1b/AIQb4/8AhKPfa3cbbLHxTEn/AC5X6D5ZFZdyK5+Zd1AH2pRXzV/wTW/4KHaX+378I76efTpvCnxD8H3jaR4w8L3R23Oj3qcPgH5mhbqj/wAVfStABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFAHzB42/5S2+C/+yfX3/pUtfT9fMHjb/lLb4L/AOyfX3/pUtfT9VPoTHdnzv8A8FTf+TN9X/7DGi/+nS1r6A0v/kG23/XJP5V8/wD/AAVN/wCTN9X/AOwxov8A6dLWvoDS/wDkG23/AFyT+VHQOpYoooqSgooooAKKKKAPLf2xv2m9H/ZD/Z68QeOdYUzjTYNllZp80uoXb/JBboO7PIyrXxz+w/8A8EQPCnhb4k+B/wBofxot7ZfHibULnxLrL2bhbNpLzLfY2j9IlYLuH9010XhXxMv/AAVG/wCCh9+fIkk+D37Nep+Um/5oPEniPHDr/C8Vsu7/AIHX3tQAYoxRRQAUUUUAFFFFABRRmjNABRRmjNABRRRQAUUUUAFFFFABXK/HP/kjPiv/ALBF1/6Kauqrlfjn/wAkZ8V/9gi6/wDRTUAcD/wTu/5MS+Ef/Yq6f/6IWvWvE/8AyLWof9e0n/oJryX/AIJ3f8mJfCP/ALFXT/8A0QteteJ/+Ra1D/r2k/8AQTVS3Jp/Cj59/wCCQf8AyjI+Cf8A2K9t/WvpCvm//gkH/wAoyPgn/wBivbf1r6Qon8TCn8KCiiipKCiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooA+Rfjf/ymo+A3/ZNvFv8A6WaRX11XyL8b/wDlNR8Bv+ybeLf/AEs0ivrqgAooooAKKKKACvmX/glj/wAkZ8a/9j7rf/pRX01XzL/wSx/5Iz41/wCx91v/ANKKpfCyX8SPpqiiipKA9K+S/wBlH/lK1+1p/wBg/wAFf+kF5X1oelfJf7KP/KVr9rT/ALB/gr/0gvKqOzIluj60oooqSwooooAKKKKACiiigAooooAKKKKAPiH/AIKCfsFeKtN+KFv+0V+z3Jb6L8afDsHl6lprfu7Hx1Yry1ndL/z0x9x/vK1ev/sCf8FA/CX7evwxfUtJ8zQ/FmiyNZ+JPC98ypqWg3afK8Usf3tu77r/AHWr3+vg3/gpF/wTz8RaF49X9pD9nOJdF+O/hlRLe2ML7LPxtZL/AK2zuovuvIy52v8Ae3baAPvKivCf2Bf27fCn7e/wUt/E2gs+n63p7Cy8R6FdL5V94f1BeJbeaJvmX5t20/xLXu1ABRRRQAUUUUAFFFFABRRRQAUUUUAfMHjb/lLb4L/7J9ff+lS19P18weNv+Utvgv8A7J9ff+lS19P1U+hMd2fO/wDwVN/5M31f/sMaL/6dLWvoDS/+Qbbf9ck/lXz/AP8ABU3/AJM31f8A7DGi/wDp0ta+gNL/AOQbbf8AXJP5UdA6liiiipKCiiigAr5g/wCCmv7SOt/DH4b6P8O/AJ8z4rfF66/sDw9GvzGxRh/pF8/91IItzbv722vpLWdZtdA0e6v7yaO3s7OJpp5ZDhY0UZYn8K+Mf+CdekXX7YX7Q3jX9p7XoZW0jUHk8NfDaOVdq2+jRPtlu0X+FrmVW+b+JFWgD6R/ZN/Zs0P9kj4BeHfAfh9d1potsqz3L/66+uG+aa4kP8TySFmJ969KoooAKKKKACiiigAoPSiigDifjV8cfD3wA8ETa94kvls7KHhV6vK/ZEH8TV8eeI/+C2NtFqbLpfguaSzU/K9xdbXYf7oXivOv+CxvxIvtd/aC07w2ZpBpuh6ck6xfwtNKzZP/AHyq18giv6M4B8MMsxWWU8fmK55VNbXaSXTY/j7xR8as5wmc1csyiXs4UXZuybb6730P14/ZR/4KFeEf2n7n+y1WTRdeVcmxuXU+b/uN/FXvynI69Dzmvwd8G+Lr7wB4psdZ0u4ktb7T5lnhkU/MpWv20+BnjpviH8LtF1aRs3F5aRSyf7zIpNfB+JPA9HIq1Otg5fu6l9OzX6H6l4NeJWJ4nw1Whj1+9pW1WnMn5d9DtBRQOlFfl5+3hRRRQAUUUUAFcr8c/wDkjPiv/sEXX/opq6quV+Of/JGfFf8A2CLr/wBFNQBwP/BO7/kxL4R/9irp/wD6IWvWvE//ACLWof8AXtJ/6Ca8l/4J3f8AJiXwj/7FXT//AEQteteJ/wDkWtQ/69pP/QTVS3Jp/Cj59/4JB/8AKMj4J/8AYr239a+kK+b/APgkH/yjI+Cf/Yr239a+kKJ/Ewp/CgoooqSgooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKAPkX43/APKaj4Df9k28W/8ApZpFfXVfIvxv/wCU1HwG/wCybeLf/SzSK+uqACiiigAooooAK+Zf+CWP/JGfGv8A2Put/wDpRX01XzL/AMEsf+SM+Nf+x91v/wBKKpfCyX8SPpqiiipKA9K+S/2Uf+UrX7Wn/YP8Ff8ApBeV9aHpXyX+yj/yla/a0/7B/gr/ANILyqjsyJbo+tKKKKksKKKKACiiigAooooAKKKKACiiigAooooA+EP27/2IvGXwW+Mr/tKfs3QR2/xJ0+L/AIqvwoD5dj8QLFPmaJ1+6t0q/ck+9Xu37A//AAUD8B/8FDfg+3ibwfcXFvfabL9i13Q75fJ1HQrxf9ZBNF1Xa2cN/Fivea+A/wBu/wDYp8afs7/tASftT/s52K3Hjqztlt/G/gwHba+O9Nj+Zti/wXka5KFfvf71AH35RXj/AOxb+2j4J/bu+B+n+OPBN88ttMTb39hcL5d7o92vEtrcR9UkRsrXsFABRRRQAUUUUAFFFFABRRRQB8weNv8AlLb4L/7J9ff+lS19P18weNv+Utvgv/sn19/6VLX0/VT6Ex3Z87/8FTf+TN9X/wCwxov/AKdLWvoDS/8AkG23/XJP5V8//wDBU3/kzfV/+wxov/p0ta+gNL/5Btt/1yT+VHQOpYoooqSgoorzn9qv9pHw/wDsj/s/+J/iJ4omaLSvDdm1w0afNJdS/digQfxSSOVRR3ZhQB8v/wDBRf4m63+1X8dNC/ZO8C/bYx4rgTVPiNrtsx2+HdCV+bfdjaJ7va8Sr/d3N719neB/BemfDjwfpfh/RbOHT9H0W1isrK1iXbHbwxqFRB9FWvnT/gmB+zh4g+F3wz8QfEb4hQ+X8WvjXqC+J/FEZH/ILVk22mmr/s20GE/32frX1HQAUUUUAFFFFABRRRQAVyuv/Gjwf4SuZYNU8VeGdMmtziWO71SGBoz/ALQZuK4n9rP9uL4b/sU+F7XUPHevLa3mrSfZ9I0Wyhe91jXp+iw2lpEGlmdmwvyrtXd8zLX4U/8ABZH/AIJCftDf8FD/AI92f7Q3hv4VTfD21+Iur6V4cl8L3Wrm51yIP/o8Wr30SDyLVWX7OjxI7NFt3P8AxtQB+j3/AAVP+Edv8VZtN+KHg290/wASaVHD/Z+oTaZcpdRrsLMH3ISvy7mVv7vy18Niv2Q/ZL/Yy8N/so/sf+DfhDpP+k6X4T0tLFrhx815McvNOw9XlZ3x/Dux2ry/4h/8EwvC3iTW5L6LTLdnkbc2yR4lb/gKstftvBHitTyzArAY+m5Rh8LW9uzTsfzV4keBNTOsxlmeVVVCdT41K9r900nv2t8z87vgv8I774ueMLaxtoZPsquv2mfb8sa//FV+xX7N/hg+Evhnp9ntZUiiVIwfRRiuG+DH7HGn/DtYR5MNvDb/AHIoV2rXullZpZWscMa4SMYFfGcc8bVeIMSpJctOGy/V+Z+i+GPhvS4Uwkoylz1qnxv02S8l+JYooor4U/UAooooAKKKKACuV+Of/JGfFf8A2CLr/wBFNXVVyvxz/wCSM+K/+wRdf+imoA4H/gnd/wAmJfCP/sVdP/8ARC1614n/AORa1D/r2k/9BNeS/wDBO7/kxL4R/wDYq6f/AOiFr1rxP/yLWof9e0n/AKCaqW5NP4UfPv8AwSD/AOUZHwT/AOxXtv619IV83/8ABIP/AJRkfBP/ALFe2/rX0hRP4mFP4UFFFFSUFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQB8i/G//lNR8Bv+ybeLf/SzSK+uq+Rfjf8A8pqPgN/2Tbxb/wClmkV9dUAFFFFABRRRQAV8y/8ABLH/AJIz41/7H3W//SivpqvmX/glj/yRnxr/ANj7rf8A6UVS+Fkv4kfTVFFFSUB6V8l/so/8pWv2tP8AsH+Cv/SC8r60PSvkv9lH/lK1+1p/2D/BX/pBeVUdmRLdH1pRRRUlhRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQB+eP7eX7KHjn9hz4u6t+1N+zVpMmo6zIiv8Q/h1Ar/ZfG9qOtxCiK2y+j+9vVfmXdur6x/Yp/bJ8Ift4/s9aH8RPBt1usdUjxdWUzL9r0i6XiW1nVT8ksbZVlr12vw6/wCC337W+j/8EQv2tD48+APijT9L+IPxQt5T4y8AzWzT6RdMyfuNWZFKrDcq3/fxV5oA/cWivzC/4NZv29PFH7cH7DviZvHWvX3iLxn4X8V3cd7e3c/mTXCXJF0h/wBlF81kVR8qrGAK/T0sF6mgAooDZozQAUUUUAFFFFAHzB42/wCUtvgv/sn19/6VLX0/XzB42/5S2+C/+yfX3/pUtfT9VPoTHdnzv/wVN/5M31f/ALDGi/8Ap0ta+gNL/wCQbbf9ck/lXz//AMFTf+TN9X/7DGi/+nS1r6A0v/kG23/XJP5UdA6liiiipKCvhv4iZ/4KJf8ABS228EN/pHwn/Zrltde8QIDug17xPKpexs3/AIWjtYv9Idef3jRKy8V6h/wUs/bI1b9kr4LafZ+B9F/4S34s/EK/Tw94K8Pqx3X164Znmfb0ggiV5Xb5RtTbuG4V1X7BX7Jdr+xb+zZpPg/7c2teILiaXWPE2tyf63XtYum828vH4H35DtX+6iov8NAHtVFFFABRRRQAUU0tt5PAr5Z/aK/4Kf6H4H+IU/w6+FPhvVPjb8WI+JvD/h+VFttJ/wBu/vG/c2q/7x3f7NAH0p4m8Uab4L8PXmratf2el6bp8TXF1eXcywwW8ajLO7t8qqB3NfEU3/BQT4r/APBQDxjrHhf9lPQNN0rwXpsrWV/8ZfFlq8ujM+4o7aNaL/yEpI8H53dIty87lxu2fD3/AATl8Y/tea9Z+K/2sPElt4qt7edbvTvhloTvF4S0tgdyfa/49SlXA5l/dbt22P5q+yNE0Wz8OaRa6fp9rb2NjZxLDb21vEIooUAwqIo4VVA6CgDwP9kv/gm34F/Zb8U3XjW8uNa+JHxb1iPZq3j3xVKL7Wrrd96KFsbLO27LBbqiKuM7vvV9GUUUAFGKKKACiiigAooooAKKKKACiiigArlfjn/yRnxX/wBgi6/9FNXVVyvxz/5Iz4r/AOwRdf8AopqAOB/4J3f8mJfCP/sVdP8A/RC1614n/wCRa1D/AK9pP/QTXkv/AATu/wCTEvhH/wBirp//AKIWvWvE/wDyLWof9e0n/oJqpbk0/hR8+/8ABIP/AJRkfBP/ALFe2/rX0hXzf/wSD/5RkfBP/sV7b+tfSFE/iYU/hQUUUVJQUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFZvjDxbp/gHwnqWuatcrZ6Vo9rJe3lw4JWCGNS7ucc4Cgk/StKmXFvHdwPFKiyRyKVdGG5WB4II7g0Afld8XP8Agsh+zHrH/BVz4O+MrX4zeCZvC+g+BPEum6hqK3o8m0ubi60xoYmPZnWGUgf7Br9H/wBn79orwX+1R8MbPxn8P/EFj4o8LahJJFa6lZMWt7ho3KPsbGG2sCpI4ypHUGv5zP24P+DbLXr7/guDoPw48GWF1Y/CP4pXEnidNShj/daDp0citfwZwQHjZwkYPXz4c9TX9IXwc+D/AIb+APwu0PwZ4P0ez0Hwz4dtEs9PsLSPZFbxr2A7knJJPJJJJJJNAHTUUUUAFFFFABXzL/wSx/5Iz41/7H3W/wD0or6ar5l/4JY/8kZ8a/8AY+63/wClFUvhZL+JH01RRRUlAelfJf7KP/KVr9rT/sH+Cv8A0gvK+tD0r5L/AGUf+UrX7Wn/AGD/AAV/6QXlVHZkS3R9aUUUVJYUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFeV/tD/sT/CP9rPTZLb4kfDnwj4wWSPyvP1LTIpLmNf9ibHmp/wFhXqlBoA+PP2aP+Cc/wAC/wDgkDqfxE+IXgNNQ8J+H/FFlbLqGjzXz3NnG8DSsjQeZuk3v5pXbubd8tfL37QP/BUL4ifFHxXcP4c1SfwnoK/JBawLG0zL/fd2XduP91a+kf8AgtBeX8PwT8OW8eTp8mqk3J/2wjeX/N6/NYGv6K8JODcBXwLzXFwVSUm1FPVK3l3Z/IPjx4g5th80/sTAzdOnBKTadnJvXfsu3fc+oP2fv+CpnxC+GXiu2/4Si+l8XaC3yTQSqkdxGv8AfR1X5mH91vvf7Nfpx8NviHpfxW8F6br+i3C3Wm6nCJ4XHcH19CORivwnr9MP+CPnii7uvgTPptwztb22ozG03dFRtrED/gTPWPi7wfl+FwcMzwkVCd7SS0Tv5dzq8A+Ps1xuYzybH1HUhyuSb1aat13s7/5H2YOlFFFfz2f1oFFFFAHzB42/5S2+C/8Asn19/wClS19P18weNv8AlLb4L/7J9ff+lS19P1U+hMd2fO//AAVN/wCTN9X/AOwxov8A6dLWvoDS/wDkG23/AFyT+VfP/wDwVN/5M31f/sMaL/6dLWvoDS/+Qbbf9ck/lR0DqWKr3V1FY28k0zrHDCpd2Y4VVHUmrFfGn/BT/wCIutfGrxP4T/Za8BahcWXir4wJLP4p1O0P73wv4UiKrfXW7+CWfcLWL1aVj/DUlGF+wPdxft9ftk+Nv2n7h477wT4fWfwH8K43YOotYpWXU9WRf4Gup08pG+80UH91q+6K+KfEf/BJGL4DxRa3+y/4w1D4M+IrSJU/sd5JNQ8M6tgfduLSQtsZv+esTK31r0b9iL9t6++O+ua98O/iFoY8D/GjwOqNrehmTzLe+t24S/sn/wCWts//AH0rfKaAPpCiisfxt450b4b+GLvWvEGp2Oj6Tp8ZluLu8mWKKFR3LNQBsV5Z+09+2J8PP2PvBn9tePPEVrpUcjCO0s0zNfahKfuxQwJmSVz6KtfOmp/8FCfH/wC2jqM/h39mHwrJPpe8wXnxI8SW722h2Y+7utEPz3b+m35fevSP2Yf+CaXhT4KeMf8AhOvF+pal8VPitcDM/ivxDiWa1z1S0h/1drH7IM/7VAHjviDTP2g/+CpktrbeR4k/Zr+CXm7rzz5FTxh4uh4/dBF/48IG/v7vNZW+7X1t+zt+zB4D/ZM+Hdv4X+HvhnTfDejw/M6W6fvbp+8s0rfPLI3d3Zmr0KigAooooAKKKKACiiigAooooAKKKKACiiigAooooAK5X45/8kZ8V/8AYIuv/RTV1Vcr8c/+SM+K/wDsEXX/AKKagDgf+Cd3/JiXwj/7FXT/AP0QteteJ/8AkWtQ/wCvaT/0E15L/wAE7v8AkxL4R/8AYq6f/wCiFr1rxP8A8i1qH/XtJ/6CaqW5NP4UfPv/AASD/wCUZHwT/wCxXtv619IV83/8Eg/+UZHwT/7Fe2/rX0hRP4mFP4UFFFFSUFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABXhX7b/wC2zpn7I3g+Hy4Y9U8SaoCLGyL7VUDrJIeoUfqfxr3WvyN/4KpajdX37aHiRLhpGS3gtY4FJ+VE8hDx9SSfqTX33hvw3hs6zhYfF/BCLm1/NZpW9LvXy0Pyvxi4wxnDnDzxWA0q1JKmpb8t1JuVtm7Rsr9XfpYyfFn/AAUe+LnirxYurf8ACTNp8keRHBaW6LDGpxlQGBODgZyecCvqL9iD/gqnefEnxpaeEviDDZwXmpSCKw1O3Xy45JDwI5FJOCT0YHBJxgcV+dNTaddTWOoQTW7MlxDIrxsp+ZWByCPxr+lM64ByXHYJ4ZUIwdvdlFJOL6O638073P434d8VOI8szGONeKnVTknOE5OUZLqrO9nbZq1j99UcOu5eQaWuT+DHiKbxN4A0+5uP+PhoV8z/AHsDNdZX8XSVnY/0ajK6uFFFFIoK+Zf+CWP/ACRnxr/2Put/+lFfTVfMv/BLH/kjPjX/ALH3W/8A0oql8LJfxI+mqKKKkoD0r5L/AGUf+UrX7Wn/AGD/AAV/6QXlfWh6V8l/so/8pWv2tP8AsH+Cv/SC8qo7MiW6PrSiiipLCiiigAooooAKKKKACiiigAooooAKKKKACiiigAoNFBoA8w/ar+AWn/tHfCC+8O36srN++tpB96GZVOx/1/WvyQ+LH7Nniz4P+JJ9P1DSbiRY32pcRIZI5F/vV+3TfKW/n61yvjD4T6P4yObq3j8zrnbX3nBviBjcgUqdJc1N9H38j8s8QPCjLOKZQr1pOnVjpzrquzXXyPxl8A/ATxL8QNTht7bTbi3hkba086MscdfqV+w38EV+EPw8trVVkWONMBn+9IW+Zm/4E1d1oH7O+haFdLIsKyMvTNd5a2cdjbCONVSNegFTxjx9js/5adZKFOOtl37svw98Lcu4U56tCTqVZaOb7dkuhYooor4U/UAooooA+YPG3/KW3wX/ANk+vv8A0qWvp+vmDxt/ylt8F/8AZPr7/wBKlr6fqp9CY7s+d/8Agqb/AMmb6v8A9hjRf/Tpa19AaX/yDbb/AK5J/Kvn/wD4Km/8mb6v/wBhjRf/AE6WtfQGl/8AINtv+uSfyo6B1ON/aP8Aj94a/Za+Bvij4heLrr7H4f8ACthJf3TjmSTaPliQfxSO21FX+JmUV87/APBLP4F+NNXfxf8AtB/GDS10r4s/GVof+JVncvhXQ7bctjpq/wC18zzS/wB6SX/ZrnfjsP8Ah4b/AMFGtH+Eq/6X8KfgG1t4q8ZbPmg1jX2+bTtMk/hZYF/0p1/vGJTX3EBt+UcCpKHV8g/8FP8A4Ja34cbw5+0R8ObNpviV8G99zLaRL83iLRH/AOP7Tm9WaMb4/wC7JGPWvd/2j/2rPh/+yT4HbxF8RPFWl+GNL3BImupf3ty5OAkUQ+eRuRwqmvkzQNf+PH/BV5bp449c/Z8+BMszRRtJD5firxhb/wB5d3/HnA6/7PmfMfSgDvPGn/BZL4a3Og6Bb/DW31b4s+NvFmnQX+l+G/D0PnzxidFZBcyf6u3+9829uNrVztj/AME9/H37cuqWfiL9qXWbe48P2863en/DLQ5Suj2rK25GvZvvXUi/3eEr6M/ZW/Yz+Gv7Fvw9h8MfDfwrpfhzTYVw7xRKbm6b+/LL952+teqUAZ+geH7Lwpolrpum2tvY2FjEsFvbwoI44UUYVVA6CtCiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACuV+Of/ACRnxX/2CLr/ANFNXVVyvxz/AOSM+K/+wRdf+imoA4H/AIJ3f8mJfCP/ALFXT/8A0QteteJ/+Ra1D/r2k/8AQTXkv/BO7/kxL4R/9irp/wD6IWvWvE//ACLWof8AXtJ/6CaqW5NP4UfPv/BIP/lGR8E/+xXtv619IV83/wDBIP8A5RkfBP8A7Fe2/rX0hRP4mFP4UFFFFSUFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABXxz/wAFOP2KpvjRaxeLPD8Mf/CQWKeXOnT7ZEBwCf7y9vbivsaoryzjvoGjkUMrcEEV62SZ1isqxkMdg5WnH7muqa6p/wBang8ScN4HPcvnluYR5qc/k01s0+jXT7no2j8JNV+GniDRNQa1utG1GK4U7SpgY8/UDFeufsw/sk61418bWN5q1jJbWNtKsiwuvzzsDkZHZc9fWv1A8R/s6aLr16ZjbxqxOeBWt4N+D2k+Dm3w28fmf3sV+mZx4zZnjMHLDUaUabkrOSbb135b7eutump+M8P/AEdsmy/MI43E15Vowd4waSV1qua1+a3bRPrpoXPhh4cbwx4StbVvvKgzXRUirtFLX46f0IFFFFABXzL/AMEsf+SM+Nf+x91v/wBKK+mq+Zf+CWP/ACRnxr/2Put/+lFUvhZL+JH01RRRUlAelfJf7KP/ACla/a0/7B/gr/0gvK+tD0r5L/ZR/wCUrX7Wn/YP8Ff+kF5VR2ZEt0fWlFFFSWFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUYoooAKKKKACiiigAooooA+YPG3/KW3wX/2T6+/9Klr6fr5g8bf8pbfBf8A2T6+/wDSpa+n81U+hMd2fO//AAVN/wCTN9X/AOwxov8A6dLWrn7ef7Wzfsd/sxya5ptn/bHjTXHh0Hwjo6/f1bV7n5LeLb/cVvnc/wAKI1ef/wDBdHx/dfCf/gl78S/FOnpDJfeHVsdSt0l/1bPDewSKG9srX5R/sof8HEWlft0ft+eBPEfjz4b69da94H8Lvp/gXwh4fdbuLWPEdy6pNcgy7fJPlDarMzLHGsrNuo6B1P2U/Yf/AGc9N/YK/ZXWLxRrdm+vX0k3iPxr4jv5xEt9qdw3mXVxJLIflQMdq7j8qqoryvxF/wAFJfFX7WmtXHhj9lPw1H4ujVvJvfiJrUUlv4X0n+95O4CS9lXn5Y/k3Ly1ZujfsCfEj9vL4i6b44/ag1S2tfCunnztJ+Eei3DS6LC3zbJdTlb/AI/J1Vh8u1Y1bdwflr7Y8M+F9N8GaLBpuk2Nppun2q7Yre2iEUUY9lWpKPm39nX/AIJi+Hfh/wCN1+IHxN1i9+MXxXmX5/EHiBFkh0/IbdFZWv8AqreP5j91d2O9fUlFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAVyvxz/5Iz4r/AOwRdf8Aopq6quV+Of8AyRnxX/2CLr/0U1AHA/8ABO7/AJMS+Ef/AGKun/8Aoha9a8T/APItah/17Sf+gmvJf+Cd3/JiXwj/AOxV0/8A9ELXrXif/kWtQ/69pP8A0E1Utyafwo+ff+CQf/KMj4J/9ivbf1r6Qr5v/wCCQf8AyjI+Cf8A2K9t/WvpCifxMKfwoKKKKkoKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAr5l/4JY/8AJGfGv/Y+63/6UV9NV8y/8Esf+SM+Nf8Asfdb/wDSiqXwsl/Ej6aoooqSgr5U8c/sQfEyw/as+IXxM+HPxb0/wZ/wsa30qHUbC78MRant+wW7RRFXeRcffc/d/i9q+q6KcXYmUbnzD/wzx+09/wBHEeHf/CAtf/jtH/DPP7T3/RxHh7/wgLX/AOO19PUU+ZhynzD/AMM8/tPf9HEeHv8AwgLX/wCO0f8ADPP7T3/RxHh7/wAIC1/+O19PUUczDlPmH/hnn9p7/o4jw9/4QFr/APHaP+Gef2nv+jiPD3/hAWv/AMdr6eoo5mHKfMP/AAzz+09/0cR4e/8ACAtf/jtH/DPP7T3/AEcR4e/8IC1/+O19PUUczDlPmH/hnn9p7/o4jw9/4QFr/wDHaP8Ahnn9p7/o4jw9/wCEBa//AB2vp6ijmYcp8w/8M8/tPf8ARxHh7/wgLX/47R/wzz+09/0cR4e/8IC1/wDjtfT1FHMw5T5h/wCGef2nv+jiPD3/AIQFr/8AHaP+Gef2nv8Ao4jw9/4QFr/8dr6eoo5mHKfMP/DPP7T3/RxHh7/wgLX/AOO0f8M8/tPf9HEeHv8AwgLX/wCO19PUUczDlPmH/hnn9p7/AKOI8Pf+EBa//HaP+Gef2nv+jiPD3/hAWv8A8dr6eoo5mHKfMP8Awzz+09/0cR4e/wDCAtf/AI7R/wAM8/tPf9HEeHv/AAgLX/47X09RRzMOU+Yf+Gef2nv+jiPD3/hAWv8A8do/4Z5/ae/6OI8Pf+EBa/8Ax2vp6ijmYcp8w/8ADPP7T3/RxHh7/wAIC1/+O0f8M8/tPf8ARxHh7/wgLX/47X09RRzMOU+Yf+Gef2nv+jiPD3/hAWv/AMdo/wCGef2nv+jiPD3/AIQFr/8AHa+nqKOZhynzT8EP2OviL4b/AGnLX4mfEL4n2Pji+0/RJdEtba18OxaWsaO+8ksjnd83+zX0seRRRSbuOKseK/8ABQX9kSP9u79kTxl8KJ9ak8Pw+LrZLZ79YPPa3CypJ9zcu77mOvevy5+Ff/BnnpPwZ8f6L4o8P/GS8ste8P3kV/YXiaPKslvPE6ujjbc/3lr9sKKOYGrny/H+zv8AtORoq/8ADRHh9toxz4Btf/jtO/4Z5/ae/wCjiPD3/hAWv/x2vp6inzMXIfMP/DPP7T3/AEcR4e/8IC1/+O0f8M8/tPf9HEeHv/CAtf8A47X09RRzMOU+Yf8Ahnn9p7/o4jw9/wCEBa//AB2j/hnn9p7/AKOI8Pf+EBa//Ha+nqKOZhynzD/wzz+09/0cR4e/8IC1/wDjtH/DPP7T3/RxHh7/AMIC1/8AjtfT1FHMw5T5h/4Z5/ae/wCjiPD3/hAWv/x2j/hnn9p7/o4jw9/4QFr/APHa+nqKOZhynzD/AMM8/tPf9HEeHv8AwgLX/wCO0f8ADPP7T3/RxHh7/wAIC1/+O19PUUczDlPmH/hnn9p7/o4jw9/4QFr/APHaP+Gef2nv+jiPD3/hAWv/AMdr6eoo5mHKfMP/AAzz+09/0cR4e/8ACAtf/jtH/DPP7T3/AEcR4e/8IC1/+O19PUUczDlPmH/hnn9p7/o4jw9/4QFr/wDHaP8Ahnn9p7/o4jw9/wCEBa//AB2vp6ijmYcp8w/8M8/tPf8ARxHh7/wgLX/47R/wzz+09/0cR4e/8IC1/wDjtfT1FHMw5T5h/wCGef2nv+jiPD3/AIQFr/8AHaP+Gef2nv8Ao4jw9/4QFr/8dr6eoo5mHKfMP/DPP7T3/RxHh7/wgLX/AOO0f8M8/tPf9HEeHv8AwgLX/wCO19PUUczDlPmH/hnn9p7/AKOI8Pf+EBa//Hao+JP2XP2lPFOgXum3X7Q3h9rW/ge3lA8A2ysUZdp/5a19WUUczDkOJ/Z3+E3/AAoj4F+E/Bf27+0v+EX0m20z7WY/L+0eVGqb9uTjOK6/U7P+0dNuLfdt+0RNHuxnbkEZqeipKPjP4GfsEftCfs7fCLw/4H8M/tIaLb6B4Zs0sbGKb4d280iRL0BdrjLH3NdZ/wAM4ftQf9HLeHv/AA21r/8AJFfUFFVzsz9ml3+9ny//AMM4ftQf9HLeHv8Aw21r/wDJFH/DOH7UH/Ry3h7/AMNta/8AyRX1BRRzP+kP2a8/vZ8v/wDDOH7UH/Ry3h7/AMNta/8AyRR/wzh+1B/0ct4e/wDDbWv/AMkV9QUUcz/pB7Nef3s+X/8AhnD9qD/o5bw9/wCG2tf/AJIo/wCGcP2oP+jlvD3/AIba1/8AkivqCijmf9IPZrz+9ny//wAM4ftQf9HLeHv/AA21r/8AJFH/AAzh+1B/0ct4e/8ADbWv/wAkV9QUUcz/AKQezXn97Pl//hnD9qD/AKOW8Pf+G2tf/kij/hnD9qD/AKOW8Pf+G2tf/kivqCijmf8ASD2a8/vZ8v8A/DOH7UH/AEct4e/8Nta//JFH/DOH7UH/AEct4e/8Nta//JFfUFFHM/6QezXn97Pl/wD4Zw/ag/6OW8Pf+G2tf/kij/hnD9qD/o5bw9/4ba1/+SK+oKKOZ/0g9mvP72fL/wDwzh+1B/0ct4e/8Nta/wDyRR/wzh+1B/0ct4e/8Nta/wDyRX1BRRzP+kHs15/ez5f/AOGcP2oP+jlvD3/htrX/AOSKP+GcP2oP+jlvD3/htrX/AOSK+oKKOZ/0g9mvP72fL/8Awzh+1B/0ct4e/wDDbWv/AMkUf8M4ftQf9HLeHv8Aw21r/wDJFfUFFHM/6QezXn97Pl//AIZw/ag/6OW8Pf8AhtrX/wCSKP8AhnD9qD/o5bw9/wCG2tf/AJIr6goo5n/SD2a8/vZ8v/8ADOH7UH/Ry3h7/wANta//ACRR/wAM4ftQf9HLeHv/AA21r/8AJFfUFFHM/wCkHs15/ez5f/4Zw/ag/wCjlvD3/htrX/5Ir0L9ij9mbUv2VPhFdeH9Y8TR+LtU1DV7vWLvUk08WCyyXD72AiDuFAPoa9eoo5naw1FJ3CiiipKP/9k=)"
      ],
      "metadata": {
        "id": "IsyhL2BbA7eo"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def distances(df):\n",
        "  df['cosine'] = df.apply(lambda y: distance.cosine(y.x1, y.x2), axis=1)\n",
        "  df['euclidean'] = df.apply(lambda y: distance.euclidean(y.x1, y.x2), axis=1)\n",
        "  # df['chebyshev'] = df.apply(lambda y: distance.chebyshev(y.x1, y.x2), axis=1)\n",
        "  df['cityblock'] = df.apply(lambda y: distance.chebyshev(y.x1, y.x2), axis=1) # Manhattan\n",
        "  # df['correlation'] = df.apply(lambda y: distance.correlation(y.x1, y.x2), axis=1)\n",
        "  return df.drop(columns=[\"x1\",\"x2\"])\n",
        "\n",
        "train_df = distances(train_df)  \n",
        "test_df = distances(test_df)"
      ],
      "metadata": {
        "id": "kOVdlgxsNb68"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "train_df.head()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "ZTUMA96ZwhjJ",
        "outputId": "c1d664b5-3fe3-40d3-a000-dece85d1e4f8"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "\n",
              "  <div id=\"df-8652a853-470c-4011-9f3b-a10c9565ff2d\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>subtraction</th>\n",
              "      <th>label</th>\n",
              "      <th>cosine</th>\n",
              "      <th>euclidean</th>\n",
              "      <th>cityblock</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>[0.23113322050990426, -0.6081501268499538, -0....</td>\n",
              "      <td>True</td>\n",
              "      <td>0.409722</td>\n",
              "      <td>20.284064</td>\n",
              "      <td>4.417184</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>[-0.12479491978918901, 0.8076019103984892, -1....</td>\n",
              "      <td>True</td>\n",
              "      <td>0.776713</td>\n",
              "      <td>22.981348</td>\n",
              "      <td>3.610012</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>[-1.4008537813465272, -0.567947699300059, -0.4...</td>\n",
              "      <td>True</td>\n",
              "      <td>0.629385</td>\n",
              "      <td>27.788242</td>\n",
              "      <td>5.730646</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>[0.19247477237824986, 0.4713947405770958, -0.3...</td>\n",
              "      <td>True</td>\n",
              "      <td>0.419443</td>\n",
              "      <td>20.141570</td>\n",
              "      <td>3.559685</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>[0.1473503343898001, 1.4034431743345863, -1.92...</td>\n",
              "      <td>True</td>\n",
              "      <td>0.625576</td>\n",
              "      <td>25.493352</td>\n",
              "      <td>5.074279</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-8652a853-470c-4011-9f3b-a10c9565ff2d')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-8652a853-470c-4011-9f3b-a10c9565ff2d button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-8652a853-470c-4011-9f3b-a10c9565ff2d');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "                                         subtraction  ...  cityblock\n",
              "0  [0.23113322050990426, -0.6081501268499538, -0....  ...   4.417184\n",
              "1  [-0.12479491978918901, 0.8076019103984892, -1....  ...   3.610012\n",
              "2  [-1.4008537813465272, -0.567947699300059, -0.4...  ...   5.730646\n",
              "3  [0.19247477237824986, 0.4713947405770958, -0.3...  ...   3.559685\n",
              "4  [0.1473503343898001, 1.4034431743345863, -1.92...  ...   5.074279\n",
              "\n",
              "[5 rows x 5 columns]"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# def sub_cal(df):\n",
        "#   df[\"subtraction\"] = (np.array(df[\"x1\"].to_list()) - np.array(df[\"x2\"].to_list())).tolist()\n",
        "#   return df\n",
        "\n",
        "# train_df = sub_cal(train_df)\n",
        "# test_df = sub_cal(test_df)"
      ],
      "metadata": {
        "id": "LyaAIqqy7Itq"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# train_df = pd.concat([train_df, pd.DataFrame(train_df[\"subtraction\"].to_list()).add_prefix(\"sub_\")], axis=1)\n",
        "# test_df = pd.concat([test_df, pd.DataFrame(test_df[\"subtraction\"].to_list()).add_prefix(\"sub_\")], axis=1)"
      ],
      "metadata": {
        "id": "u4Q1_PLa8ATr"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# train_df = pd.concat([train_df, pd.DataFrame(train_df[\"x1\"].to_list()).add_prefix(\"x1_\")], axis=1)\n",
        "# train_df = pd.concat([train_df, pd.DataFrame(train_df[\"x2\"].to_list()).add_prefix(\"x2_\")], axis=1)\n",
        "# test_df = pd.concat([test_df, pd.DataFrame(test_df[\"x2\"].to_list()).add_prefix(\"x2_\")], axis=1)\n",
        "# test_df = pd.concat([test_df, pd.DataFrame(test_df[\"x1\"].to_list()).add_prefix(\"x1_\")], axis=1)"
      ],
      "metadata": {
        "id": "cHhpVTiBs9BY"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# train_df = train_df.drop(columns=[\"x1\",\"x2\"])\n",
        "# test_df = test_df.drop(columns=[\"x1\",\"x2\"])"
      ],
      "metadata": {
        "id": "QXqkglZ96Vo7"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# convert True -> 1 and False -> 0\n",
        "train_df[\"label\"] = train_df[\"label\"].astype(int)\n",
        "test_df[\"label\"] = test_df[\"label\"].astype(int)\n",
        "\n",
        "features = train_df.drop(columns=[\"label\",\"subtraction\"]).columns.tolist()\n",
        "\n",
        "x_train = train_df.drop(columns=[\"label\",\"subtraction\"])\n",
        "y_train = train_df[\"label\"]\n",
        "x_test = test_df.drop(columns=[\"label\",\"subtraction\"])\n",
        "y_test = test_df[\"label\"]"
      ],
      "metadata": {
        "id": "NSPPurN0UeBg"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "<a name=\"cell-3\"></a>\n",
        "## <font color='orange'>LightGBM</font>\n",
        "**LightGBM** is a gradient boosting framework that uses tree based learning algorithms."
      ],
      "metadata": {
        "id": "XKxUdVE5xDpM"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "lgb_train = lgb.Dataset(x_train, y_train, feature_name=features)\n",
        "lgb_test = lgb.Dataset(x_test, y_test, feature_name = features)"
      ],
      "metadata": {
        "id": "-Vuy3QhMSbfn"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "params = {\n",
        "'task': 'train'\n",
        ", 'boosting_type': 'gbdt'\n",
        ", 'objective': 'multiclass'\n",
        ", 'num_class': 2\n",
        ", 'metric': 'multi_logloss'\n",
        "}\n",
        " \n",
        "gbm = lgb.train(params, lgb_train, num_boost_round=250, early_stopping_rounds = 15, valid_sets=[lgb_test])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9HK4i5wESjJA",
        "outputId": "d9344006-96bd-44e9-c40c-b8780fcabac8"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[1]\tvalid_0's multi_logloss: 0.628054\n",
            "Training until validation scores don't improve for 15 rounds.\n",
            "[2]\tvalid_0's multi_logloss: 0.574348\n",
            "[3]\tvalid_0's multi_logloss: 0.530532\n",
            "[4]\tvalid_0's multi_logloss: 0.493546\n",
            "[5]\tvalid_0's multi_logloss: 0.462177\n",
            "[6]\tvalid_0's multi_logloss: 0.435419\n",
            "[7]\tvalid_0's multi_logloss: 0.41299\n",
            "[8]\tvalid_0's multi_logloss: 0.392709\n",
            "[9]\tvalid_0's multi_logloss: 0.375311\n",
            "[10]\tvalid_0's multi_logloss: 0.360729\n",
            "[11]\tvalid_0's multi_logloss: 0.348065\n",
            "[12]\tvalid_0's multi_logloss: 0.337075\n",
            "[13]\tvalid_0's multi_logloss: 0.327184\n",
            "[14]\tvalid_0's multi_logloss: 0.318852\n",
            "[15]\tvalid_0's multi_logloss: 0.311178\n",
            "[16]\tvalid_0's multi_logloss: 0.304562\n",
            "[17]\tvalid_0's multi_logloss: 0.298741\n",
            "[18]\tvalid_0's multi_logloss: 0.293579\n",
            "[19]\tvalid_0's multi_logloss: 0.288805\n",
            "[20]\tvalid_0's multi_logloss: 0.284591\n",
            "[21]\tvalid_0's multi_logloss: 0.281561\n",
            "[22]\tvalid_0's multi_logloss: 0.279437\n",
            "[23]\tvalid_0's multi_logloss: 0.277126\n",
            "[24]\tvalid_0's multi_logloss: 0.275355\n",
            "[25]\tvalid_0's multi_logloss: 0.273663\n",
            "[26]\tvalid_0's multi_logloss: 0.27285\n",
            "[27]\tvalid_0's multi_logloss: 0.272205\n",
            "[28]\tvalid_0's multi_logloss: 0.271493\n",
            "[29]\tvalid_0's multi_logloss: 0.271089\n",
            "[30]\tvalid_0's multi_logloss: 0.270585\n",
            "[31]\tvalid_0's multi_logloss: 0.270041\n",
            "[32]\tvalid_0's multi_logloss: 0.270026\n",
            "[33]\tvalid_0's multi_logloss: 0.27039\n",
            "[34]\tvalid_0's multi_logloss: 0.269854\n",
            "[35]\tvalid_0's multi_logloss: 0.269788\n",
            "[36]\tvalid_0's multi_logloss: 0.270046\n",
            "[37]\tvalid_0's multi_logloss: 0.270252\n",
            "[38]\tvalid_0's multi_logloss: 0.270514\n",
            "[39]\tvalid_0's multi_logloss: 0.271002\n",
            "[40]\tvalid_0's multi_logloss: 0.272028\n",
            "[41]\tvalid_0's multi_logloss: 0.272296\n",
            "[42]\tvalid_0's multi_logloss: 0.272878\n",
            "[43]\tvalid_0's multi_logloss: 0.273725\n",
            "[44]\tvalid_0's multi_logloss: 0.273983\n",
            "[45]\tvalid_0's multi_logloss: 0.274538\n",
            "[46]\tvalid_0's multi_logloss: 0.275135\n",
            "[47]\tvalid_0's multi_logloss: 0.27577\n",
            "[48]\tvalid_0's multi_logloss: 0.276355\n",
            "[49]\tvalid_0's multi_logloss: 0.276922\n",
            "[50]\tvalid_0's multi_logloss: 0.277363\n",
            "Early stopping, best iteration is:\n",
            "[35]\tvalid_0's multi_logloss: 0.269788\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "predictions = gbm.predict(x_test)\n",
        " \n",
        "y_pred = []\n",
        "for prediction in predictions:\n",
        "  prediction_class = np.argmax(prediction)\n",
        "  y_pred.append(prediction_class)\n",
        "\n",
        "accuracy = accuracy_score(y_pred, y_test)\n",
        "print(\"accuracy: {:.1f}%\".format(accuracy * 100))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WSgHf-89SmI3",
        "outputId": "7a8892a9-7d68-4559-cc69-8f7f1661bf31"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "accuracy: 89.4%\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<a name=\"cell-1\"></a>\n",
        "## <font color='orange'>Random Forest</font>\n",
        "In random forests, each tree in the ensemble is built from a sample drawn with replacement from the training set. \n",
        "<br>\n",
        "Furthermore, when splitting each node during the construction of a tree, the best split is found either from all input features or a random subset of size `max_features`."
      ],
      "metadata": {
        "id": "XMXvHYzF1uGV"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "steps = [('scaler', StandardScaler()), ('RF', RandomForestClassifier())]\n",
        "pipeline = Pipeline(steps) # define Pipeline object\n",
        "\n",
        "parameters = {'RF__n_estimators':[40, 60, 80, 100, 120],\n",
        "              'RF__criterion':[\"gini\", \"entropy\"],\n",
        "              'RF__max_depth':[2, 3], # weak learner\n",
        "              # 'RF__max_features': [2, 3, 4, 5]\n",
        "              }\n",
        "              \n",
        "grid = GridSearchCV(pipeline, param_grid=parameters, cv=5)"
      ],
      "metadata": {
        "id": "1siVbGH7tiee"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "grid.fit(x_train, y_train)\n",
        "print(\"score = \", grid.score(x_train, y_train)) \n",
        "print(\"best parameters from train data: \", grid.best_params_)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "oXW6HYzrtqr6",
        "outputId": "8e6f35f7-99be-4562-d144-2e27411e793d"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "score =  0.8868181818181818\n",
            "best parameters from train data:  {'RF__criterion': 'gini', 'RF__max_depth': 2, 'RF__n_estimators': 100}\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "y_pred = grid.predict(x_test)\n",
        "\n",
        "accuracy = accuracy_score(y_pred, y_test)\n",
        "print(\"accuracy: {:.1f}%\".format(accuracy * 100))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lUNTSmL6tsic",
        "outputId": "ac14f752-113f-4675-a015-e86ce752d3bf"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "accuracy: 89.4%\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<a name=\"cell-2\"></a>\n",
        "## <font color='orange'>Stacked Model</font>\n",
        "Stacked generalization is a method for combining estimators to reduce their biases. More precisely, the predictions of each individual estimator are stacked together and used as input to a final estimator to compute the prediction. This final estimator is trained through cross-validation."
      ],
      "metadata": {
        "id": "dBv0nSSV3UJh"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "We use 9 base estimators which will be stacked together: \n",
        "*   3 Perceptron\n",
        "*   3 SVC\n",
        "*   3 Decision tree"
      ],
      "metadata": {
        "id": "FCRO0QTbC57J"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "percep1 = Perceptron(tol=1e-3, max_iter=200)\n",
        "percep2 = Perceptron(tol=1e-3, max_iter=200)\n",
        "percep3 = Perceptron(tol=1e-3, max_iter=200)\n",
        "\n",
        "svm1 = SVC(kernel='poly', degree=1)\n",
        "svm2 = SVC(kernel='poly', degree=2)\n",
        "svm3 = SVC(kernel='poly', degree=3)\n",
        "\n",
        "dTree1 = DecisionTreeClassifier(criterion='entropy', max_depth=3)\n",
        "dTree2 = DecisionTreeClassifier(criterion='gini', max_depth=3)\n",
        "dTree3 = DecisionTreeClassifier(criterion='gini', max_depth=4)"
      ],
      "metadata": {
        "id": "8HdCEctc7aVn"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "In this section, we select the best final estimator using grid search"
      ],
      "metadata": {
        "id": "1Ofzw5ieD2Wk"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "estimators = [\n",
        "     ('p1', percep1), ('p2', percep2), ('p3', percep3),\n",
        "     ('svm1', svm1), ('svm2', svm2), ('svm3', svm3),\n",
        "     ('dt1', dTree1), ('dt2', dTree2), ('dt33', dTree3)\n",
        "]\n",
        "\n",
        "steps = [('scaler', StandardScaler()), ('SC', StackingClassifier(estimators=estimators))]\n",
        "pipeline = Pipeline(steps) # define Pipeline object\n",
        "\n",
        "parameters = {'SC__final_estimator':[SVC(), Perceptron(), DecisionTreeClassifier()]}\n",
        "\n",
        "grid = GridSearchCV(pipeline, param_grid=parameters, cv=5)"
      ],
      "metadata": {
        "id": "3o2_EEOD_6in"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "grid.fit(x_train, y_train)\n",
        "print(\"score = \", grid.score(x_train, y_train)) \n",
        "print(\"best parameters from train data: \", grid.best_params_)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Q-xC9_2GAcTC",
        "outputId": "9517f590-f792-479e-c992-a29b75e0e0ff"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "score =  0.8881818181818182\n",
            "best parameters from train data:  {'SC__final_estimator': SVC()}\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "y_pred = grid.predict(x_test)\n",
        "\n",
        "accuracy = accuracy_score(y_pred, y_test)\n",
        "print(\"accuracy: {:.1f}%\".format(accuracy * 100))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "gedTEVYHAmwZ",
        "outputId": "cd44b60b-24be-425d-9241-c6c42f46af08"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "accuracy: 89.0%\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## <font color='orange'>Conclusion</font>\n",
        "\n",
        "\n",
        "> <font color='black'>**LightGBM** <font color='lightgreen'> 89.4%</font><br>\n",
        "> <font color='black'>**Random Forest** </font> <font color='lightgreen'> 89.4%</font><br>\n",
        "> <font color='black'>**Stacked Model** <font> <font color='lightgreen'> 89.0%</font><br>\n",
        "\n"
      ],
      "metadata": {
        "id": "MGzUnOh6EOc1"
      }
    }
  ]
}